Determination of Kinetic Parameters for Heterogeneous Reaction System Employing Discrete Element Methods under HPC Platforms

Poster (2023, June 06)

The complex processes of heterogeneous reactions of granular materials such as occurring during metals-ore reduction or biomass gasification involve numerous physical phenomena. The combination of ... [more ▼]

The complex processes of heterogeneous reactions of granular materials such as occurring during metals-ore reduction or biomass gasification involve numerous physical phenomena. The combination of elevated temperature, complex flow, aggressive atmosphere and heterogeneous chemistry make it difficult to study these industrial processes. One of the most important aspects f heterogeneous reactions is to understand and quantify the evolution of the different transformations. For instance, during metal-oxides reduction processes, it is of high importance to quantify the rate at which the pure metal is formed. Nevertheless, it is almost impossible, by experimental means only, to separately observe, accurately quantify and gain insight into these mingled nonlinear physical and chemical processes. In the last decade, numerical simulation tools for particulate processes, such as the eXtended Discrete Element Method (XDEM), have become indispensable to study complex systems without the need of costly experimental practices. In the past, the XDEM has been employed to predict the reduction of tungsten trioxide (WO 3) in dry hydrogen (H2) atmospheres [1] and reduction of iron ores [2]. In the before-mentioned research works, it was employed kinetic data extracted from literature. On one hand, in these processes the kinetic data differ from each other. This is due to the fact that the experimental data in the literature is interpreted with lumped models and empirical models bonded to the specific experimental conditions. On the other hand, advanced simulation tools, such as XDEM, account for all the influencing phenomena (e.g. species and energy distribution, flow conditions, particles shape, rheological properties) constantly interacting in time and space. In these advanced simulation tools, each particle is treated and solved as individual entities and an accurate prediction of the species formation and transport in time and space is provided. Thus, in such advanced numerical tools, the reaction rate parameters representative of the kinetics alone of the involved chemical reactions must be employed. In this contribution, two XDEM simulation case studies accounting for the industrial reduction of WO 3 are presented. The first case study is employed to determine the reaction rate parameters of the four prevalent reduction steps (WO 3↔WO2.9↔WO2.72↔WO2↔W) upon the H 2 reduction of O3. Where the reaction rates are modeled following an Arrhenius law with two parameters per step i.e. pre-exponential factor and activation energy). The constituted optimization problem of minimization of error of the XDEM simulations vs experimental data, implemented and solved in a High Performance Computing (HPC) cluster, is presented and discussed. The determined parameters are later assessed by comparison to a secondly presented case study. [less ▲]

Installing Scientific Software on HPC with EasyBuild

Scientific Conference (2023, May 19)

Practical Debugging & Performance Engineering for High Performance Computing

Scientific Conference (2023, May 17)

Investigation of OpenFOAM-XDEM momentum coupling results for AWJC Nozzle using preCICE

Scientific Conference (2023, February 14)

The high-speed water jet is the momentum source in an Abrasive Water Jet Cutting Nozzle. This momentum is transferred to the abrasive particles & the air within the nozzle. This leads to turbulent ... [more ▼]

The high-speed water jet is the momentum source in an Abrasive Water Jet Cutting Nozzle. This momentum is transferred to the abrasive particles & the air within the nozzle. This leads to turbulent & complex particle-laden flow in the nozzle. These flow conditions can influence particle impacts on the nozzle, thus influencing erosion. Hence it is imperative that this complex particle-laden flow is captured correctly. The momentum exchange can be directly from the water jet to the particles or indirectly through the airflow. In this work, we investigate these fluid-particle momentum exchanges. Our prototype uses preCICE for volumetric coupling of XDEM (for the particle motion), & OpenFOAM (for the fluid). XDEM uses fluid flow conditions to compute the forces acting on particles. XDEM computes the particle momentum source that is injected into the fluid solver. The results of the coupled simulation align with literature & can be extended to include the FEM component for erosion predictions. [less ▲]

Local Verlet buffer approach for broad-phase interaction detection in Discrete Element Method

E-print/Working paper (2022)

The Extended Discrete Element Method (XDEM) is an innovative numerical simulation technique that extends the dynamics of granular materials known as Discrete Element Method (DEM) by additional properties ... [more ▼]

The Extended Discrete Element Method (XDEM) is an innovative numerical simulation technique that extends the dynamics of granular materials known as Discrete Element Method (DEM) by additional properties such as the thermodynamic state, stress/strain for each particle. Such DEM simulations used by industries to set up their experimental processes are complexes and heavy in computation time. At each time step, those simulations generate a list of interacting particles and this phase is one of the most computationally expensive parts of a DEM simulation. The Verlet buffer method, initially introduced in Molecular Dynamic (MD) (and also used in DEM), allows keeping the interaction list for many time steps by extending each particle neighbourhood by a certain extension range, and thus broadening the interaction list. The method relies on the temporal coherency of DEM, which guarantees that no particles move erratically from one time step to the next. In the classical approach, all the particles have their neighbourhood extended by the same value which leads to suboptimal performances in simulations where different flow regimes coexist. Additionally, and unlike in MD, there is no comprehensive study analysing the different parameters that affect the performance of the Verlet buffer method in DEM. In this work, we propose a new method for the dynamic update of the neighbour list that depends on the particles individual displacement and define a particle-specific extension range based on the local flow regime. The interaction list is analysed throughout the simulation based on the particle's displacement allowing a flexible update according to the flow regime conditions. We evaluate the influence of the Verlet extension range on the execution time through different test cases and analyse empirically the extension range value giving the best performance. [less ▲]

Parallel Multi-Physics Simulation of Biomass Furnace and Cloud-based Workflow for SMEs

in Practice and Experience in Advanced Research Computing (PEARC '22) (2022, July)

Biomass combustion is a well-established process to produce energy that offers a credible alternative to reduce the consumption of fossil fuel. To optimize the process of biomass combustion, numerical ... [more ▼]

Biomass combustion is a well-established process to produce energy that offers a credible alternative to reduce the consumption of fossil fuel. To optimize the process of biomass combustion, numerical simulation is a less expensive and time-effective approach than the experimental method. However, biomass combustion involves intricate physical phenomena that must be modeled (and validated) carefully, in the fuel bed and in the surrounding gas. With this level of complexity, these simulations require the use of High-Performance Computing (HPC) platforms and expertise, which are usually not affordable for manufacturing SMEs. In this work, we developed a parallel simulation tool for the simulation of biomass furnaces that relies on a parallel coupling between Computation Fluid Dynamics (CFD) and Discrete Element Method (DEM). This approach is computation-intensive but provides accurate and detailed results for biomass combustion with a moving fuel bed. Our implementation combines FOAM-extend (for the gas phase) parallelized with MPI, and XDEM (for the solid particles) parallelized with OpenMP, to take advantage of HPC hardware. We also carry out a thorough performance evaluation of our implementation using an industrial biomass furnace setup. Additionally, we present a fully automated workflow that handles all steps from the user input to the analysis of the results. Hundreds of parameters can be modified, including the furnace geometry and fuel settings. The workflow prepares the simulation input, delegates the computing-intensive simulation to an HPC platform, and collects the results. Our solution is integrated into the Digital Marketplace of the CloudiFacturing EU project and is directly available to SMEs via a Cloud portal. As a result, we provide a cutting-edge simulation of a biomass furnace running on HPC. With this tool, we demonstrate how HPC can benefit engineering and manufacturing SMEs, and empower them to compute and solve problems that cannot be tackled without. [less ▲]

An Innovative Partitioning Technology for Coupled Software Modules

in Proceedings of the 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - SIMULTECH (2022, July)

Multi-physics simulation approaches by coupling various software modules is paramount to unveil the underlying physics and thus leads to an improved design of equipment and a more efficient operation ... [more ▼]

Multi-physics simulation approaches by coupling various software modules is paramount to unveil the underlying physics and thus leads to an improved design of equipment and a more efficient operation. These simulations are in general to be carried out on small to massively parallelised computers for which highly efficient partitioning techniques are required. An innovative partitioning technology is presented that relies on a co-located partitioning of overlapping simulation domains meaning that the overlapping areas of each simulation domain are located at one node. Thus, communication between modules is significantly reduced as compared to an allocation of overlapping simulation domains on different nodes. A co-located partitioning reduces both memory and inter-process communication. [less ▲]

Development of an HPC Multi-Physics Biomass Furnace Simulation and Integration in a Cloud-based Workflow

Scientific Conference (2022, June 09)

Biomass combustion offers a credible alternative to reduce the consumption of fossil fuels. To optimize the biomass combustion process and improve the design of biomass furnaces numerical simulation is a ... [more ▼]

Biomass combustion offers a credible alternative to reduce the consumption of fossil fuels. To optimize the biomass combustion process and improve the design of biomass furnaces numerical simulation is a less expensive and time-effective approach as opposed to the experimental method. However, the combustion in a biomass furnace involves intricate physical phenomena that must be modeled (and validated) carefully, in the fuel bed (with particle motion and shrinking, heat transfer, drying, pyrolysis, gasification) and in the surrounding gas (with turbulence, combustion, radiation). With this level of complexity, and to be conducted in a reasonable time, the simulation of industrial biomass furnaces requires the use of High-Performance Computing (HPC) platforms and expertise, which is usually not affordable for manufacturing SMEs. To address this issue, we developed a configurable digital twin of a biomass furnace running on HPC and we designed a cloudified easy-to-use end-to-end workflow. This fully automated workflow, from user input to results analysis, has been integrated into the digital marketplace of the CloudiFacturing EU project and is now directly available to SMEs via a Cloud portal. With this presentation, we want to offer a glance at the internal details and enabling technologies used in our parallel coupled application and scientific workflow. Our parallel simulation tool for biomass furnaces combines OpenFOAM (for the gas phase) parallelized with MPI and XDEM (for the solid wood particles) parallelized with OpenMP. The two libraries are coupled in parallel using an original approach based on the co-located partitioning strategy which has been tailored to minimize communications. As for the cloud workflow, it is based on an all-in-one Singularity image containing all the software, scripts, and data required to prepare the simulation input, execute the computation-intensive simulation, and analyze the results. Finally, we present the lessons learned from the development of this complex workflow and highlight the remaining challenges related to HPC multi-physics coupled simulations. [less ▲]

RESIF 3.0: Toward a Flexible & Automated Management of User Software Environment on HPC facility

in ACM Practice and Experience in Advanced Research Computing (PEARC'21) (2021, July)

High Performance Computing (HPC) is increasingly identified as a strategic asset and enabler to accelerate the research and the business performed in all areas requiring intensive computing and large ... [more ▼]

High Performance Computing (HPC) is increasingly identified as a strategic asset and enabler to accelerate the research and the business performed in all areas requiring intensive computing and large-scale Big Data analytic capabilities. The efficient exploitation of heterogeneous computing resources featuring different processor architectures and generations, coupled with the eventual presence of GPU accelerators, remains a challenge. The University of Luxembourg operates since 2007 a large academic HPC facility which remains one of the reference implementation within the country and offers a cutting-edge research infrastructure to Luxembourg public research. The HPC support team invests a significant amount of time (i.e., several months of effort per year) in providing a software environment optimised for hundreds of users, but the complexity of HPC software was quickly outpacing the capabilities of classical software management tools. Since 2014, our scientific software stack is generated and deployed in an automated and consistent way through the RESIF framework, a wrapper on top of Easybuild and Lmod [5] meant to efficiently handle user software generation. A large code refactoring was performed in 2017 to better handle different software sets and roles across multiple clusters, all piloted through a dedicated control repository. With the advent in 2020 of a new supercomputer featuring a different CPU architecture, and to mitigate the identified limitations of the existing framework, we report in this state-of-practice article RESIF 3.0, the latest iteration of our scientific software management suit now relying on streamline Easybuild. It permitted to reduce by around 90% the number of custom configurations previously enforced by specific Slurm and MPI settings, while sustaining optimised builds coexisting for different dimensions of CPU and GPU architectures. The workflow for contributing back to the Easybuild community was also automated and a current work in progress aims at drastically decrease the building time of a complete software set generation. Overall, most design choices for our wrapper have been motivated by several years of experience in addressing in a flexible and convenient way the heterogeneous needs inherent to an academic environment aiming for research excellence. As the code base is available publicly, and as we wish to transparently report also the pitfalls and difficulties met, this tool may thus help other HPC centres to consolidate their own software management stack. [less ▲]

OpenMP optimisation of the eXtended Discrete Element Method (XDEM)

Report (2021)

The eXtended Discrete Element Method (XDEM) is an extension of the regular Discrete Element Method (DEM) which is a software for simulating the dynamics of granular material. XDEM extends the regular DEM ... [more ▼]

The eXtended Discrete Element Method (XDEM) is an extension of the regular Discrete Element Method (DEM) which is a software for simulating the dynamics of granular material. XDEM extends the regular DEM method by adding features where both micro and macroscopic observables can be computed simultaneously by coupling different time and length scales. In this sense XDEM belongs the category of multi-scale/multi-physics applications which can be used in realistic simulations. In this whitepaper, we detail the different optimisations done during the preparatory PRACE project to overcome known bottlenecks in the OpenMP implementation of XDEM. We analysed the Conversion, Dynamic, and the combined Dynamics-Conversion modules with Extrae/Paraver and Intel VTune profiling tools in order to find the most expensive functions. The proposed code modifications improved the performance of XDEM by ~17% for the computational expensive Dynamics-Conversion combined modules (with 48 cores, full node). Our analysis was performed in the Marenostrum 4 (MN4) PRACE infrastructure at Barcelona Supercomputing Center (BSC). [less ▲]