Autoencoders; Data-driven latent spaces; Data-driven post processing Galerkin; Diffusion maps; Reduced dynamics; Reduced order models; Auto encoders; Data driven; Data-driven latent space; Data-driven post processing galerkin; Post-processing; Reduced order modelling; Reduced-order model; Numerical Analysis; Modeling and Simulation; Physics and Astronomy (miscellaneous); Physics and Astronomy (all); Computer Science Applications; Computational Mathematics; Applied Mathematics
Abstract :
[en] This study presents a collection of purely data-driven workflows for constructing reduced-order models (ROMs) for distributed dynamical systems. The ROMs we focus on, are data-assisted models inspired by, and templated upon, the theory of Approximate Inertial Manifolds (AIMs); the particular motivation is the so-called post-processing Galerkin method of Garcia-Archilla, Novo and Titi. Its applicability can be extended: the need for accurate truncated Galerkin projections and for deriving closed-formed corrections can be circumvented using machine learning tools. When the right latent variables are not a priori known, we illustrate how autoencoders as well as Diffusion Maps (a manifold learning scheme) can be used to discover good sets of latent variables and test their explainability. The proposed methodology can express the ROMs in terms of (a) theoretical (Fourier coefficients), (b) linear data-driven (POD modes) and/or (c) nonlinear data-driven (Diffusion Maps) coordinates. Both Black-Box and (theoretically-informed and data-corrected) Gray-Box models are described; the necessity for the latter arises when truncated Galerkin projections are so inaccurate as to not be amenable to post-processing. We use the Chafee-Infante reaction-diffusion and the Kuramoto-Sivashinsky dissipative partial differential equations to illustrate and successfully test the overall framework.
Disciplines :
Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
KORONAKI, Eleni ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
Evangelou, Nikolaos; Department of Chemical and Biomolecular Engineering, Department of Applied Mathematics and Statistics, Whiting School of Engineering, Johns Hopkins University, Baltimore, United States
Martin-Linares, Cristina P.; Department of Mechanical Engineering, Whiting School of Engineering, Johns Hopkins University, Baltimore, United States
Titi, Edriss S.; Department of Mathematics, Texas A & M University, College Station, United States ; Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom
Kevrekidis, Ioannis G. ; Department of Chemical and Biomolecular Engineering, Department of Applied Mathematics and Statistics, Whiting School of Engineering, Johns Hopkins University, Baltimore, United States
External co-authors :
yes
Language :
English
Title :
Nonlinear dimensionality reduction then and now: AIMs for dissipative PDEs in the ML era
The motivation for this work comes in part from initial efforts on reduced modeling of multiphase flows, as part of CML's Thesis. I.G.K. acknowledges partial support from the US AFOSR FA9550-21-0317 and the U.S. Department of Energy SA22-0052-S001. E.D.K. was funded by the Luxembourg National Research Fund (FNR), grant reference 16758846. For the purpose of open access, E.D.K. has applied for a Creative Commons Attribution 4.0 International (CC BY 4.0) license to any Author Accepted Manuscript version arising from this submission. C.M.L. received the support of a 'la Caixa' Foundation Fellowship (ID 100010434), code LCF/BQ/AA19/11720048. The research of E.S.T. was made possible by NPRP grant #S-0207-200290 from the Qatar National Research Fund (a member of Qatar Foundation), and is based upon work supported by King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-2020-CRG9-4336. The work of E.S.T. has also benefited from the inspiring environment of the CRC 1114 “Scaling Cascades in Complex Systems”, Project Number 235221301, Project A02, funded by Deutsche Forschungsgemeinschaft (DFG). For the purpose of open access, E.S.T. has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.The motivation for this work comes in part from initial efforts on reduced modeling of multiphase flows, as part of CML's Thesis. I.G.K. acknowledges partial support from the US AFOSR FA9550-21-0317 and the US Department of Energy SA22-0052-S001. E.D.K. was funded by the Luxembourg National Research Fund (FNR), grant reference 16758846. For the purpose of open access, EDK has applied for a Creative Commons Attribution 4.0 International (CC BY 4.0) license to any Author Accepted Manuscript version arising from this submission. C.M.L. received the support of a “la Caixa” Foundation Fellowship (ID 100010434), code LCF/BQ/AA19/11720048. The research of E.S.T. was made possible by NPRP grant #S-0207-200290 from the Qatar National Research Fund (a member of Qatar Foundation), and is based upon work supported by King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-2020-CRG9-4336. The work of E.S.T. has also benefited from the inspiring environment of the CRC 1114 “Scaling Cascades in Complex Systems”, Project Number 235221301, Project A02, funded by Deutsche Forschungsgemeinschaft (DFG). For the purpose of open access, E.S.T. has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.
Adrover, A., Continillo, G., Crescitelli, S., Giona, M., Russo, L., Construction of approximate inertial manifold by decimation of collocation equations of distributed parameter systems. Comput. Chem. Eng. 26:1 (2002), 113–123.
Akram, Maryam, Hassanaly, Malik, Raman, Venkat, A priori analysis of reduced description of dynamical systems using approximate inertial manifolds. J. Comput. Phys., 409, 2020, 109344.
Alekseenko, S.V., Nakoryakov, V.E., Pokusaev, B.G., Wave formation on vertical falling liquid films. Int. J. Multiph. Flow 11:5 (1985), 607–627.
Anirudh, Rushil, Thiagarajan, Jayaraman J., Bremer, Peer-Timo, Spears, Brian K., Improved surrogates in inertial confinement fusion with manifold and cycle consistencies. Proc. Natl. Acad. Sci. 117:18 (2020), 9741–9746.
Bar-Sinai, Yohai, Hoyer, Stephan, Hickey, Jason, Brenner, Michael P., Learning data-driven discretizations for partial differential equations. Proc. Natl. Acad. Sci. 116:31 (2019), 15344–15349.
Benner, Peter, Gugercin, Serkan, Willcox, Karen, A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57:4 (2015), 483–531.
Brunton, Steven L., Proctor, Joshua L., Kutz, J. Nathan, Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. 113:15 (2016), 3932–3937.
Chang, Hsueh-Chia, Nonlinear waves on liquid film surfaces—i. Flooding in a vertical tube. Chem. Eng. Sci. 41:10 (1986), 2463–2476.
Chang, Hsueh-Chia, Traveling waves on fluid interfaces: normal form analysis of the Kuramoto–Sivashinsky equation. Phys. Rev. A 29:10 (1986), 3142–3147.
Chorin, Alexandre J., Lu, Fei, Discrete approach to stochastic parametrization and dimension reduction in nonlinear dynamics. Proc. Natl. Acad. Sci. 112:32 (2015), 9804–9809.
Coifman, Ronald R., Lafon, Stéphane, Geometric harmonics: a novel tool for multiscale out-of-sample extension of empirical functions. Applied and Computational Harmonic Analysis 21:1 (2006), 31–52.
Coifman, Ronald R., Lafon, Stéphane, Diffusion maps. Appl. Comput. Harmon. Anal. 21:1 (2006), 5–30.
Coifman, Ronald R., Kevrekidis, Ioannis G., Lafon, Stéphane, Maggioni, Mauro, Nadler, Boaz, Diffusion maps, reduction coordinates, and low dimensional representation of stochastic systems. Multiscale Model. Simul. 7:2 (2008), 842–864.
Pérez De Jesús, Carlos E., Graham, Michael D., Data-driven low-dimensional dynamic model of Kolmogorov flow. Phys. Rev. Fluids, 8(4), 2023, 044402.
Dsilva, Carmeline J., Talmon, Ronen, Coifman, Ronald R., Kevrekidis, Ioannis G., Parsimonious representation of nonlinear dynamical systems through manifold learning: a Chemotaxis case study. Appl. Comput. Harmon. Anal. 44:3 (2018), 759–773.
Evangelou, Nikolaos, Dietrich, Felix, Chiavazzo, Eliodoro, Lehmberg, Daniel, Meila, Marina, Kevrekidis, Ioannis G., Double diffusion maps and their latent harmonics for scientific computations in latent space. J. Comput. Phys., 485, 2023, 112072.
Foias, C., Jolly, M.S., Kevrekidis, I.G., Sell, George R., Titi, E.S., On the computation of inertial manifolds. Phys. Lett. A 131:7–8 (1988), 433–436.
Foias, Ciprian, Sell, George R., Temam, Roger, Inertial manifolds for nonlinear evolutionary equations. J. Differ. Equ. 73:2 (1988), 309–353.
Foias, Ciprian, Sell, George R., Titi, Edriss S., Exponential tracking and approximation of inertial manifolds for dissipative nonlinear equations. J. Dyn. Differ. Equ. 1 (1989), 199–244.
García-Archilla, Bosco, Titi, Edriss S., Postprocessing the Galerkin method: the finite-element case. SIAM J. Numer. Anal. 37:2 (1999), 470–499.
García-Archilla, Bosco, Novo, Julia, Titi, Edriss S., Postprocessing the Galerkin method: a novel approach to approximate inertial manifolds. SIAM J. Numer. Anal. 35:3 (1998), 941–972.
García-Archilla, Bosco, Novo, Julia, Titi, Edriss, An approximate inertial manifolds approach to postprocessing the Galerkin method for the Navier-Stokes equations. Math. Comput. 68:227 (1999), 893–911.
Gear, Charles William, Kevrekidis, I.G., Sonday, B.E., Slow Manifold Integration on a Diffusion Map Parameterization. AIP Conference Proceedings, vol. 1389, 2011, American Institute of Physics, 13–16.
Geelen, Rudy, Wright, Stephen, Willcox, Karen, Operator inference for non-intrusive model reduction with quadratic manifolds. Comput. Methods Appl. Mech. Eng., 403, 2023, 115717.
Guermond, J-L., Prudhomme, Serge, A fully discrete nonlinear Galerkin method for the 3D Navier–Stokes equations. Numer. Methods Partial Differ. Equ. 24:3 (2008), 759–775.
Jauberteau, F., Rosier, C., Temam, R., A nonlinear Galerkin method for the Navier-Stokes equations. Comput. Methods Appl. Mech. Eng. 80:1–3 (1990), 245–260.
Jolly, Michael S., Explicit construction of an inertial manifold for a reaction diffusion equation. J. Differ. Equ. 78:2 (1989), 220–261.
Jolly, Michael S., Kevrekidis, I.G., Titi, Edriss S., Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: analysis and computations. Phys. D, Nonlinear Phenom. 44:1–2 (1990), 38–60.
Jolly, M.S., Kevrekidis, I.G., Titi, E.S., Preserving dissipation in approximate inertial forms for the Kuramoto-Sivashinsky equation. J. Dyn. Differ. Equ. 3 (1991), 179–197.
Kang, Wei, Zhang, Jia-Zhong, Ren, Sheng, Lei, Peng-Fei, Nonlinear Galerkin method for low-dimensional modeling of fluid dynamic system using pod modes. Commun. Nonlinear Sci. Numer. Simul. 22:1–3 (2015), 943–952.
Kevrekidis, Ioannis G., Nicolaenko, Basil, Scovel, James C., Back in the saddle again: a computer assisted study of the Kuramoto-Sivashinsky equation. SIAM J. Appl. Math. 50:3 (1990), 760–790 ISSN 00361399 http://www.jstor.org/stable/2101886.
Koronaki, Eleni D., Gakis, Georgios P., Cheimarios, Nikolaos, Boudouvis, Andreas G., Efficient tracing and stability analysis of multiple stationary and periodic states with exploitation of commercial CFD software. Chem. Eng. Sci. 150 (2016), 26–34.
Kramer, Mark A., Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 37:2 (1991), 233–243.
Krischer, K., Rico-Martínez, R., Kevrekidis, I.G., Rotermund, H.H., Ertl, G., Hudson, J.L., Model identification of a spatiotemporally varying catalytic reaction. AIChE J. 39:1 (1993), 89–98, 10.1002/aic.690390110.
Kuramoto, Yoshiki, Tsuzuki, Toshio, Persistent propagation of concentration waves in dissipative media far from thermal equilibrium. Prog. Theor. Phys. 55:2 (1976), 356–369.
Lee, Kookjin, Carlberg, Kevin T., Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys., 404, 2020, 108973.
Lehmberg, Daniel, Dietrich, Felix, Köster, Gerta, Bungartz, Hans-Joachim, Datafold: data-driven models for point clouds and time series on manifolds. J. Open Sour. Softw., 5(51), 2020, 2283, 10.21105/joss.02283.
Linot, Alec J., Graham, Michael D., Deep learning to discover and predict dynamics on an inertial manifold. Phys. Rev. E, 101(6), 2020, 062209.
Linot, Alec J., Graham, Michael D., Data-driven reduced-order modeling of spatiotemporal chaos with neural ordinary differential equations. Chaos, Interdiscip. J. Nonlinear Sci., 32(7), 2022, 073110.
Linot, Alec J., Zeng, Kevin, Graham, Michael D., Turbulence control in plane Couette flow using low-dimensional neural ode-based models and deep reinforcement learning. Int. J. Heat Fluid Flow, 101, 2023, 109139.
Lu, Fei, Lin, Kevin K., Chorin, Alexandre J., Data-based stochastic model reduction for the Kuramoto–Sivashinsky equation. Phys. D, Nonlinear Phenom. 340 (2017), 46–57.
Margolin, Len G., Titi, Edriss S., Wynne, Shannon, The postprocessing Galerkin and nonlinear Galerkin methods—a truncation analysis point of view. SIAM J. Numer. Anal. 41:2 (2003), 695–714.
Martine, Marion, Temam, R., Nonlinear Galerkin methods: the finite elements case. Numer. Math. 57 (1990), 205–226.
Marsden, Jerrold E., Hoffman, Michael J., et al. Elementary Classical Analysis. 1993, Macmillan.
Martin-Linares, Cristina P., Psarellis, Yorgos M., Karapetsas, Georgios, Koronaki, Eleni D., Kevrekidis, Ioannis G., Physics-agnostic and physics-infused machine learning for thin films flows: modeling, and predictions from small data. arXiv preprint arXiv:2301.12508, 2023.
McQuarrie, Shane A., Huang, Cheng, Willcox, Karen E., Data-driven reduced-order models via regularised operator inference for a single-injector combustion process. J. R. Soc. N. Z. 51:2 (2021), 194–211.
Nadler, Boaz, Lafon, Stéphane, Coifman, Ronald R., Kevrekidis, Ioannis G., Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21:1 (2006), 113–127.
Raissi, Maziar, Perdikaris, Paris, Karniadakis, George E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378 (2019), 686–707.
Rico-Martinez, Ramiro, Krischer, K., Kevrekidis, I.G., Kube, M.C., Hudson, J.L., Discrete-vs. continuous-time nonlinear signal processing of cu electrodissolution data. Chem. Eng. Commun. 118:1 (1992), 25–48.
Shan, Shan, Daubechies, Ingrid, Diffusion maps: using the semigroup property for parameter tuning. Theoretical Physics, Wavelets, Analysis, Genomics: An Indisciplinary Tribute to Alex Grossmann, 2022, Springer, 409–424.
Shen, Jie, Long time stability and convergence for fully discrete nonlinear Galerkin methods. Appl. Anal. 38:4 (1990), 201–229.
Shvartsman, Stanislav Y., Kevrekidis, Ioannis G., Nonlinear model reduction for control of distributed systems: a computer-assisted study. AIChE J. 44:7 (1998), 1579–1595.
Sivashinsky, Gregory I., Nonlinear analysis of hydrodynamic instability in laminar flames—i. Derivation of basic equations. Acta Astronaut. 4:11 (1977), 1177–1206.
Sonday, Benjamin, Systematic Model Reduction for Complex Systems Through Data Mining and Dimensionality Reduction. 2011, Princeton University.
Sonday, Benjamin E., Singer, Amit, Gear, C. William, Kevrekidis, Ioannis G., Manifold learning techniques and model reduction applied to dissipative pdes. arXiv preprint arXiv:1011.5197, 2010.
Temam, R., Do inertial manifolds apply to turbulence?. Phys. D, Nonlinear Phenom. 37:1–3 (1989), 146–152.
Theodoropoulos, C., Kevrekidis, I.G., Mountziaris, T.J., Order reduction for nonlinear dynamic models of distributed reacting systems. J. Process Control 10:2–3 (2000), 177–184.
Titi, Edriss S., On approximate inertial manifolds to the Navier-Stokes equations. J. Math. Anal. Appl. 149:2 (1990), 540–557.
Vincent, Pascal, Larochelle, Hugo, Bengio, Yoshua, Manzagol, Pierre-Antoine, Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning, 2008, 1096–1103.
Wahlbin, Lars, Superconvergence in Galerkin Finite Element Methods. 2006, Springer.
Zastrow, Benjamin G., Chaudhuri, Anirban, Willcox, Karen E., Ashley, Anthony S., Henson, Michael C., Data-driven model reduction via operator inference for coupled aeroelastic flutter. AIAA SCITECH 2023 Forum, 2023, 0330.
Zeng, Kevin, Graham, Michael D., Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems. 2023.
Zeng, Kevin, Linot, Alec J., Graham, Michael D., Data-driven control of spatiotemporal chaos with reduced-order neural ode-based models and reinforcement learning. Proc. R. Soc. A, 478(2267), 2022, 20220297.
