1. Titles of the Publications Paper (1): Towards Improving Simulations of Flows around Spherical Particles Using Genetic Programming Paper (2): Graph Networks as Inductive Bias for Genetic Programming: Symbolic Models for Particle-Laden Flows ----------------------------------------------------------------------------------------------------------------------- 2. Author Information Julia Reuter Chair of Computational Intelligence, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany julia.reuter@ovgu.de +49 391 67 52756 Hani Elmestikawy Chair of Mechanical Process Engineering, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany hani.elmestikawy@ovgu.de +49 391 67 58783 Sanaz Mostaghim Chair of Computational Intelligence, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany sanaz.mostaghim@ovgu.de +49 391 67 54986 Berend van Wachem Chair of Mechanical Process Engineering, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany berend.vanwachem@ovgu.de +49 391 67 58783 Fabien Evrard Chair of Mechanical Process Engineering, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany fabien.evrard@ovgu.de +49 391 67 52001 Manoj Cendrollu (This author left the university after the first paper was published. His successor is Hani Elmestikawy.) Chair of Mechanical Process Engineering, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, 39106 Magdeburg, Germany manoj.cendrollu@ovgu.de +49 391 67 58783 ----------------------------------------------------------------------------------------------------------------------- 3. Corresponding Author Julia Reuter julia.reuter@ovgu.de ----------------------------------------------------------------------------------------------------------------------- 4. Paper Abstracts Paper 1: The simulation of particle-laden flows is a crucial task in fluid dynamics, requiring high computational cost owing to the complex interactions between numerous particles. Typically, the flow velocity is described with the equations proposed by Stokes. While there is an analytical solution for the Stokes flows around a single spherical particle, the Stokes flows around many particles are still unsolved. In this paper, we study Genetic Programming (GP) for symbolic regressions to explore the potentials of multi-objective GP in recovering analytical expressions for two and, in the future, N particles. We propose a new GP approach containing building blocks to scale up the problem and provide a new benchmark with 22 cases for this application. To identify the strengths and limitations of GP, we generate fully resolved training data from simulations. We compare the results of our algorithm to the superimposition method and a multi-layer perceptron as two baseline methods. The results show that GP can find comparable and sometimes better solutions with smaller failure rates than the two baseline methods. In addition, the produced solutions by GP are explainable and certain function patterns inline with physical laws can be identified across the benchmark problems. Paper 2: High-resolution simulations of particle-laden flows are computationally limited to a scale of thousands of particles due to the complex interactions between particles and fluid. Some approaches to increase the number of particles in such simulations require information about the fluid-induced force on a particle, which is a major challenge in this research area. In this paper, we present an approach to develop symbolic models for the fluid-induced force. We use a graph network as inductive bias to model the underlying pairwise particle interactions. The internal parts of the network are then replaced by symbolic models using a genetic programming algorithm. We include prior problem knowledge in our algorithm. The resulting equations show an accuracy in the same order of magnitude as state-of-the-art approaches for different benchmark datasets. They are interpretable and deliver important building blocks. Our approach is a promising alternative to ``black-box'' models from the literature. ----------------------------------------------------------------------------------------------------------------------- 5. Competition Criteria (D) The result is publishable in its own right as a new scientific result independent of the fact that the result was mechanically created. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. (F) The result is equal to or better than a result that was considered an achievement in its field at the time it was first discovered. ----------------------------------------------------------------------------------------------------------------------- 6. Statement why the results satisfy the criteria (D), (E), (F) When referring to "best" or "better" compared to other approaches, we assess two main quality criteria that are of high importance in the area of Genetic Programming (GP) for Symbolic Regression (SR): a) Accuracy of a symbolic model, assessed by the Coefficient of Determination (R^2) b) Simplicity/Interpretability of a symbolic model, usually assessed by the complexity of the model, as well as the evaluation by the domain experts from fluid mechanics Since the problem addressed in our papers is considerably complex, we give a brief introduction to the simulation of particle-laden flows in Section 9. We recommend to first read Section 9, before proceeding with the subsequent statements. We consider the criteria in order of importance, not in alphabetic order. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. Simulating particle-laden flows, i.e., billions of particles in a fluid flow, is one of the oldest problems in the history of fluid mechanics [1]. Any improvement in understanding and modeling this problem has a considerable impact on the fluid mechanics discipline. Due to its high importance in this area, finding a closure model for the Euler-Lagrange framework by predicting the force induced on a particle, denoted by "F_fluid", has been studied extensively in the past. Early approaches aim for predicting the mean force, until recently the attention was shifted towards more accurate solutions, which also consider the variation around this mean [2]. Recent iterations to model this variation around the mean force include [3-9], of which [3,5] are human-created and [4,6-9] are data-driven. [3,5] are based on human-created equations using correlations from physics, as well as expansions of spherical harmonics. These methods are complex, but interpretable, i.e., the prediction process is traceable and comprehensible. However, they perform poorly on some flow arrangements, which led to the exploration of data-driven approaches, most recently dominated by physics-inspired neural networks (PINN) [8,9]. The increase in accuracy however sacrifices interpretability of the model, as well as the possibility to better understand the underlying interactions. In our work, we aim at achieving similar or higher accuracy as the data-driven approaches (which outperform the human-created models), and maintain a high level of interpretability at the same time. For this, we address a subdomain of the flow problem, the Stokes flow, where Re = 0. We aim to systematically extend the approach to higher flow regimes. Rather than replacing the expert by machine-learned models, we provide the possibility to incorporate domain knowledge in our algorithms to steer the search process. Paper (1) demonstrates how a GP algorithm outperforms a human-created correlation, by including expert knowledge in the function and terminal set of the algorithm. Paper (2) extends the presented approach and incorporates Graph Networks (GN) as inductive bias to reduce the complexity of the problem, which is the main challenge of solving such large-scale problems with GP. We want to state explicitly, that other works try to model the entire parameter space (varying Phi, varying Re), while we approach a subdomain of the problem (varying Phi, Re=0) as a starting point of our research. Within the subdomain of our experiments, considering the two assessment criteria, we are on par with the accuracy of [7,8,9] in terms of R^2. At the same time, the obtained models are considerably simpler, interpretable and in-line with physical laws. Furthermore, in publication (2), we discovered novel building blocks for equations to solve such problems. Using these building blocks, our framework has strong potential to be extended to solve a larger parameter space with varying Re. (F) The result is equal to or better than a result that was considered an achievement in its field at the time it was first discovered. Within the long history of modeling particle-laden flows, the latest achievement is [3], introducing the pairwise interaction assumption between particles. All subsequent publications are based on this assumption [4-9], which confirms that this publication can be seen as the latest achievement in the field. Since then, there have been incremental improvements using various methods. The equations we discovered perform equally well within the subdomain of our experiments compared to the recent advances from the literature. Despite the strong constraints posed on our equations to be in accordance with physical laws, our algorithms identified novel symbolic models that are competitive with results of established methods, such as ANNs. The fact that we identified building blocks for equations to solve such problems is the first step towards interpretable models for particle-laden flows. Considering the large number of preceding attempts to solve this problem and the fact that our publication is the first to obtain such interpretable models, our results show great potential to be extended to higher Reynolds numbers. (D) The result is publishable in its own right as a new scientific result independent of the fact that the result was mechanically created. Both our publications (1) and (2) discovered novel equations that have not yet been reported in any other publication. As we are the first ones to approach this problem with GP, our papers fill the research gap of interpretable models for particle-laden flows. At the same time, they are competitive with state-of-the-art machine learning approaches from the literature, within the subdomain of our experiments. Furthermore, our paper (2) was awarded the best paper award at the 2023 evo* best paper award, which emphasizes the impact of our contribution. ----------------------------------------------------------------------------------------------------------------------- 7. Full Citation Paper 1: J. Reuter, M. Cendrollu, F. Evrard, S. Mostaghim and B. van Wachem, "Towards Improving Simulations of Flows around Spherical Particles Using Genetic Programming," 2022 IEEE Congress on Evolutionary Computation (CEC), Padua, Italy, 2022, pp. 1-8, https://doi.org/10.1109/CEC55065.2022.9870301. Paper 2: Reuter, J., Elmestikawy, H., Evrard, F., Mostaghim, S., van Wachem, B. (2023). Graph Networks as Inductive Bias for Genetic Programming: Symbolic Models for Particle-Laden Flows. In: Pappa, G., Giacobini, M., Vasicek, Z. (eds) Genetic Programming. EuroGP 2023. Lecture Notes in Computer Science, vol 13986. Springer, Cham. https://doi.org/10.1007/978-3-031-29573-7_3 ----------------------------------------------------------------------------------------------------------------------- 8. Prize Money Breakdown If any prize money is granted, it shall be split in the following way: - 75% to Julia Reuter (corresponding author) - 25% to Hani Elmestikawy This decision was made in mutual agreement between all the authors. ----------------------------------------------------------------------------------------------------------------------- 9. A Statement Indicating Why this Entry Could Be the "Best" Particle-laden flows are an essential phenomenon which appears in nature (flow of blood cells in blood plasma), industry (movement of particles in a fluidized bed) etc. They are characterized by two quantities, the Reynolds number "Re" and the particle-volume fraction "Phi". The simulation of particle-laden flows, i.e., the dynamics of billions of particles subject to a fluid flow, is an ever-evolving challenge in engineering sciences [1]. High-resolution simulations of such flows are computationally limited to a scale of thousands of particles. The so-called Euler-Lagrange framework is a promising approach to simulate a large number of particles with sufficiently high accuracy. However, it requires the information about the fluid-induced force "F_fluid" on a particle to be applicable. In other words, if a correct model for "F_fluid" is identified, this problem can be considered to be solved. Various approaches have been presented in the past years to predict "F_fluid", but a perfect model has not been found. In our publication (1), we showed that Genetic Programming (GP) is generally capable to obtain better solutions than a coarse human-designed correlation from fluid mechanics, examined on a flow around two particles. In (2), we demonstrate that GP can be scaled up to larger flow arrangements by using Graph Networks (GN) as inductive bias, which is a well-motivated assumption about the underlying particle interactions. Our approach identifies symbolic models which perform similar to those human-designed and data-driven ones, and include novel building blocks. Yet, there is a long way to fully solve this problem. We humbly express our confidence in the value and potential of our entry, because of the following four reasons: - We make interpretability and discovery of equations that are in-line with physical laws our highest priority, which poses strong constraints on the algorithms that are used. Related works from the field do not have such strong constraints. Initially, GP was seen very critical to be competitive with other machine learning methods to solve the complex problem at hand. We have shown in Paper (1), that GP is generally capable of solving a simple flow arrangement and outperforms a coarse correlation from the fluid mechanics field. Despite the physics constraints, Paper (2) showed that the algorithm can even keep up with the state-of-the-art in the literature to solve the Stokes flow for complex particle arrangements. - We are the first ones to systematically identify symbolic models for this problem with GP, starting with two particles in paper (1) and larger particle arrangements in paper (2). We present equations that are interpretable, concise and in-line with physical laws. Furthermore, we provide an extensive analysis of building blocks which were obtained by the algorithm presented in (2). These building blocks are a promising starting point to extend our approach to more complex flow regimes. - Our algorithm does not only intend to solve the problem at hand, but at the same time helps to understand the underlying particle interactions better. Other data-driven methods such as Artificial Neural Networks (ANN) do not offer such possibility. - Paper (2) was granted the best paper award of the euroGP track at the 2023 evo* conference, which highlights the significance of our contribution to the field of evolutionary computation, and specifically GP. Overall, we have made a significant step towards describing particle-laden flows with symbolic models. The results form a solid and promising basis to approach problems of higher complexity. [1] L. Schiller, and A. Maumann. "Über die grundlegenden Berechnungen bei der Schwerkraftaufbereitung". In: Zeitschrift des Vereines deutscher Ingenieure 77 (12): 318--320 (March 1933) [2] G. Akiki, T. L. Jackson, and S. Balachandar. "Force variation within arrays of monodisperse spherical particles". In: Phys. Rev. Fluids 1, 044202 (August 2016). [3] G. Akiki, T. L. Jackson, and S. Balachandar. “Pairwise interaction extended point-particle model for a random array of monodisperse spheres”. In: Journal of Fluid Mechanics 813 (Feb. 2017), pp. 882–928. [4] S. Balachandar et al. “Toward particle-resolved accuracy in Euler-Lagrange simulations of multiphase flow using machine learning and pairwise interaction extended point-particle (PIEP) approximation”. In: Theoretical and Computational Fluid Dynamics 34.4 (Aug. 2020), pp. 401–428. [5] W. C. Moore and S. Balachandar. “Lagrangian investigation of pseudo-turbulence in multiphase flow using superposable wakes”. In: Physical Review Fluids 4.11 (Nov. 2019), p. 114301. [6] W. C. Moore, S. Balachandar, and G. Akiki. “A hybrid point-particle force model that combines physical and data-driven approaches”. In: Journal of Computational Physics 385 (May 2019), pp. 187–208. [7] A. Seyed-Ahmadi and A. Wachs. “Microstructure-informed probability-driven point-particle model for hydrodynamic forces and torques in particle-laden flows”. In: Journal of Fluid Mechanics 900 (Oct. 2020), A21. [8] A. Seyed-Ahmadi and A. Wachs. “Physics-inspired architecture for neural network modeling of forces and torques in particle-laden flows”. In: Computers & Fluids 238 (Apr. 2022), p. 105379. [9] B. Siddani, S. Balachandar. "Point-particle drag, lift, and torque closure models using machine learning: Hierarchical approach and interpretability". In: Physical Review Fluids 8 (2023). ----------------------------------------------------------------------------------------------------------------------- 10. Method of Evolutionary Computation GP (Genetic Programming) ----------------------------------------------------------------------------------------------------------------------- 11. Publication Date Paper (1): 18 July 2022 (presented at CEC 2022) Paper (2): 23 March 2023 (presented at evo* 2023)