Surrogate Modeling; Deep Learning; CNN U-NET; Graph U-Net; Perceiver IO; Finite Element Method
Résumé :
[en] Deep learning surrogate models are being increasingly used in accelerating scientific simulations as a replacement for costly conventional numerical techniques. However, their use remains a significant challenge when dealing with real-world complex examples. In this work, we demonstrate three types of neural network architectures for efficient learning of highly non-linear deformations of solid bodies. The first two architectures are based on the recently proposed CNN U-NET and MAgNET (graph U-NET) frameworks which have shown promising performance for learning on mesh-based data. The third architecture is Perceiver IO, a very recent architecture that belongs to the family of attention-based neural networks–a class that has revolutionised diverse engineering fields and is still unexplored in computational mechanics. We study and compare the performance of all three networks on two benchmark examples, and show their capabilities to accurately predict the non-linear mechanical responses of soft bodies.
Disciplines :
Ingénierie, informatique & technologie: Multidisciplinaire, généralités & autres
Auteur, co-auteur :
DESHPANDE, Saurabh ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
SOSA, Raul Ian ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Physics and Materials Science (DPHYMS)
BORDAS, Stéphane ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
LENGIEWICZ, Jakub ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
Co-auteurs externes :
no
Langue du document :
Anglais
Titre :
Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics
Abadi M. Agarwal A. Barham P. Brevdo E. Chen Z. Citro C. et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems Software available from tensorflow.org.
Abueidda D. W. Koric S. Sobh N. A. Sehitoglu H. (2021). Deep learning for plasticity and thermo-viscoplasticity. Int. J. Plasticity 136, 102852. 10.1016/j.ijplas.2020.102852
Aydin R. C. Braeu F. A. Cyron C. J. (2019). General multi-fidelity framework for training artificial neural networks with computational models. Front. Mater. 6, 61. 10.3389/fmats.2019.00061
Baevski A. Zhou Y. Mohamed A. Auli M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. Adv. Neural Inf. Process. Syst. 33, 12449–12460.
Barrios J. M. Romero P. E. (2019). Decision tree methods for predicting surface roughness in fused deposition modeling parts. Materials 12, 2574. 10.3390/ma12162574
Battaglia P. W. Hamrick J. B. Bapst V. Sanchez-Gonzalez A. Zambaldi V. Malinowski M. et al. (2018). Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261.
Bock F. E. Aydin R. C. Cyron C. J. Huber N. Kalidindi S. R. Klusemann B. (2019). A review of the application of machine learning and data mining approaches in continuum materials mechanics. Front. Mater. 6, 110, 10.3389/fmats.2019.00110
Bronstein M. M. Bruna J. Cohen T. Veličković P. (2021). Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478.
Brown T. Mann B. Ryder N. Subbiah M. Kaplan J. D. Dhariwal P. et al. (2020). Language models are few-shot learners. Adv. neural Inf. Process. Syst. 33, 1877–1901.
Brunet J.-N. Mendizabal A. Petit A. Golse N. Vibert E. Cotin S. (2019). “Physics-based deep neural network for augmented reality during liver surgery,” in Medical image computing and computer assisted intervention – miccai 2019. Editors Shen D. Liu T. Peters T. M. Staib L. H. Essert C. Zhou S. et al. (Cham: Springer International Publishing), 137–145.
Bui H. P. Tomar S. Courtecuisse H. Cotin S. Bordas S. P. A. (2018). Real-time error control for surgical simulation. IEEE Trans. Biomed. Eng. 65, 596–607. 10.1109/TBME.2017.2695587
Butler K. T. Davies D. W. Cartwright H. Isayev O. Walsh A. (2018). Machine learning for molecular and materials science. Nature 559, 547–555. 10.1038/s41586-018-0337-2
Capuano G. Rimoli J. J. (2019). Smart finite elements: A novel machine learning application. Comput. Methods Appl. Mech. Eng. 345, 363–381. 10.1016/j.cma.2018.10.046
Chen A. I. Balter M. L. Maguire T. J. Yarmush M. L. (2020). Deep learning robotic guidance for autonomous vascular access. Nat. Mach. Intell. 2, 104–115. 10.1038/s42256-020-0148-7
Choi H. Crump C. Duriez C. Elmquist A. Hager G. Han D. et al. (2021). On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward. Proc. Natl. Acad. Sci. 118, e1907856118. 10.1073/pnas.1907856118
Choudhary K. DeCost B. Chen C. Jain A. Tavazza F. Cohn R. et al. (2022). Recent advances and applications of deep learning methods in materials science. npj Comput. Mater. 8, 59–26. 10.1038/s41524-022-00734-6
Cotin S. Delingette H. Ayache N. (1999). Real-time elastic deformations of soft tissues for surgery simulation. IEEE Trans. Vis. Comput. Graph. 5, 62–73. 10.1109/2945.764872
Courtecuisse H. Allard J. Kerfriden P. Bordas S. P. Cotin S. Duriez C. (2014). Real-time simulation of contact and cutting of heterogeneous soft-tissues. Med. image Anal. 18, 394–410. 10.1016/j.media.2013.11.001
De Vivo M. Masetti M. Bottegoni G. Cavalli A. (2016). Role of molecular dynamics and related methods in drug discovery. J. Med. Chem. 59, 4035–4061. 10.1021/acs.jmedchem.5b01684
Dennler C. Bauer D. E. Scheibler A.-G. Spirig J. Götschi T. Fürnstahl P. et al. (2021). Augmented reality in the operating room: A clinical feasibility study. BMC Musculoskelet. Disord. 22, 451. 10.1186/s12891-021-04339-w
Deshpande S. Bordas S. P. A. Lengiewicz J. (2022b). MAgNET: A graph U-net architecture for mesh-based simulations. arXiv. 10.48550/ARXIV.2211.00713
Deshpande S. Lengiewicz J. Bordas S. P. (2022a). Probabilistic deep learning for real-time large deformation simulations. Comput. Methods Appl. Mech. Eng. 398, 115307. 10.1016/j.cma.2022.115307
Devlin J. Chang M.-W. Lee K. Toutanova K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
El Haber G. Viquerat J. Larcher A. Ryckelynck D. Alves J. Patil A. et al. (2022). Deep learning model to assist multiphysics conjugate problems. Phys. Fluids 34, 015131. 10.1063/5.0077723
Elouneg A. Bertin A. Lucot Q. Tissot V. Jacquet E. Chambert J. et al. (2022). In vivo skin anisotropy dataset from annular suction test. Data Brief 40, 107835. 10.1016/j.dib.2022.107835
Flaschel M. Kumar S. De Lorenzis L. (2021). Unsupervised discovery of interpretable hyperelastic constitutive laws. Comput. Methods Appl. Mech. Eng. 381, 113852. 10.1016/j.cma.2021.113852
Friesner R. A. (2005). Ab initio quantum chemistry: Methodology and applications. Proc. Natl. Acad. Sci. 102, 6648–6653. 10.1073/pnas.0408036102
Gholamalizadeh T. Moshfeghifar F. Ferguson Z. Schneider T. Panozzo D. Darkner S. et al. (2022). Open-full-jaw: An open-access dataset and pipeline for finite element models of human jaw. Comput. Methods Programs Biomed. 224, 107009. 10.1016/j.cmpb.2022.107009
Guo X. Li W. Iorio F. (2016). “Convolutional neural networks for steady flow approximation,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (New York, NY, USA: Association for Computing Machinery), 481–490. 10.1145/2939672.2939738
Hauseux P. Nguyen T.-T. Ambrosetti A. Ruiz K. S. Bordas S. P. A. Tkatchenko A. (2020). From quantum to continuum mechanics in the delamination of atomically-thin layers from substrates. Nat. Commun. 11, 1651. 10.1038/s41467-020-15480-w
Jaegle A. Borgeaud S. Alayrac J.-B. Doersch C. Ionescu C. Ding D. et al. (2022). “Perceiver IO: A general architecture for structured inputs and outputs,” in International conference on learning representations.
Jha D. Ward L. Paul A. Liao W.-k. Choudhary A. Wolverton C. et al. (2018). Elemnet: Deep learning the chemistry of materials from only elemental composition. Sci. Rep. 8, 1–13. 10.1038/s41598-018-35934-y
Kim Y. Mishra S. Jin S. Panda R. Kuehne H. Karlinsky L. et al. (2022). “How transferable are video representations based on synthetic data?,” in Thirty-sixth conference on neural information processing systems datasets and benchmarks track.
Kingma D. P. Ba J. (2014). Adam: A method for stochastic optimization. arXiv. 10.48550/ARXIV.1412.6980
Korelc J. (2002). Multi-language and multi-environment generation of nonlinear finite element codes. Eng. Comput. 18, 312–327. 10.1007/s003660200028
Krokos V. Bordas S. P. A. Kerfriden P. (2022a). A graph-based probabilistic geometric deep learning framework with online physics-based corrections to predict the criticality of defects in porous materials. 10.48550/ARXIV.2205.06562
Krokos V. Bui Xuan V. Bordas S. P. A. Young P. Kerfriden P. (2022b). A bayesian multiscale cnn framework to predict local stress fields in structures with microscale features. Comput. Mech. 69, 733–766. 10.1007/s00466-021-02112-3
Le T. A. Baydin A. G. Zinkov R. Wood F. (2017). Using synthetic data to train neural networks is model-based reasoning. In 2017 international joint conference on neural networks (IJCNN) (IEEE), 3514–3521.
Liu Y. Zhao T. Ju W. Shi S. (2017). Materials discovery and design using machine learning. J. Materiomics 3, 159–177. 10.1016/j.jmat.2017.08.002
Loshchilov I. Hutter F. (2017). Decoupled weight decay regularization. arXiv. 10.48550/ARXIV.1711.05101
Mao Z. Jagtap A. D. Karniadakis G. E. (2020). Physics-informed neural networks for high-speed flows. Comput. Methods Appl. Mech. Eng. 360, 112789. 10.1016/j.cma.2019.112789
Mazier A. Ribes S. Gilles B. Bordas S. P. (2021). A rigged model of the breast for preoperative surgical planning. J. Biomechanics 128, 110645. 10.1016/j.jbiomech.2021.110645
McFall K. Mahan J. (2009). Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Trans. neural Netw. 20, 1221. 1233. 10.1109/tnn.2009.2020735
Mendizabal A. Márquez-Neila P. Cotin S. (2019). Simulation of hyperelastic materials in real-time using deep learning. Med. Image Anal. 59, 101569. 10.1016/j.media.2019.101569
Mianroodi J. R. H Siboni N. Raabe D. (2021). Teaching solid mechanics to artificial intelligence—A fast solver for heterogeneous materials. Npj Comput. Mater. 7, 99–10. 10.1038/s41524-021-00571-z
Odot A. Haferssas R. Cotin S. (2022). Deepphysics: A physics aware deep learning framework for real-time simulation. Int. J. Numer. Methods Eng. 123, 2381–2398. 10.1002/nme.6943
Oishi A. Yagawa G. (2017). Computational mechanics enhanced by deep learning. Comput. Methods Appl. Mech. Eng. 327, 327–351. 10.1016/j.cma.2017.08.040
Paszke A. Gross S. Massa F. Lerer A. Bradbury J. Chanan G. et al. (2019). “Pytorch: An imperative style, high-performance deep learning library,” in Advances in neural information processing systems (Curran Associates, Inc.), 32, 8024–8035.
Pfaff T. Fortunato M. Gonzalez A. Battaglia P. (2021). “Learning mesh-based simulation with graph networks,” in International conference on learning representations.
Pfeiffer M. Riediger C. Weitz J. Speidel S. (2019). Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks. Int. J. Comput. Assisted Radiology Surg. 14, 1147–1155. 10.1007/s11548-019-01965-7
Rupp M. Tkatchenko A. Müller K.-R. von Lilienfeld O. A. (2012). Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301. 10.1103/PhysRevLett.108.058301
Rus D. Tolley M. T. (2015). Design, fabrication and control of soft robots. Nature 521, 467–475. 10.1038/nature14543
Samaniego E. Anitescu C. Goswami S. Nguyen-Thanh V. Guo H. Hamdia K. et al. (2020). An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Comput. Methods Appl. Mech. Eng. 362, 112790. 10.1016/j.cma.2019.112790
Schleder G. R. Padilha A. C. Acosta C. M. Costa M. Fazzio A. (2019). From DFT to machine learning: Recent approaches to materials science–a review. J. Phys. Mater. 2, 032001. 10.1088/2515-7639/ab084b
Schmidt J. Marques M. R. Botti S. Marques M. A. (2019). Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mater. 5, 83–36. 10.1038/s41524-019-0221-0
Schütt K. Kindermans P.-J. Sauceda Felix H. E. Chmiela S. Tkatchenko A. Müller K.-R. (2017). “Schnet: A continuous-filter convolutional neural network for modeling quantum interactions,” in Advances in neural information processing systems. Editors Guyon I. Luxburg U. V. Bengio S. Wallach H. Fergus R. Vishwanathan S. et al. (Curran Associates, Inc.), 30.
Schütt K. T. Arbabzadah F. Chmiela S. Müller K. R. Tkatchenko A. (2017). Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 8, 13890–13898. 10.1038/ncomms13890
Strönisch S. Meyer M. Lehmann C. (2022). “Flow field prediction on large variable sized 2d point clouds with graph convolution,” in Proceedings of the platform for advanced scientific computing conference (New York, NY, USA: Association for Computing Machinery). PASC ’22. 10.1145/3539781.3539789
Unke O. T. Chmiela S. Sauceda H. E. Gastegger M. Poltavsky I. Schütt K. T. et al. (2021). Machine learning force fields. Chem. Rev. 121, 10142–10186. 10.1021/acs.chemrev.0c01111
Varrette S. Bouvry P. Cartiaux H. Georgatos F. (2014). Management of an academic hpc cluster: The ul experience. 10.1109/HPCSim.2014.6903792
Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A. N. et al. (2017). Attention is all you need. Adv. neural Inf. Process. Syst. 30.
Vijayaraghavan S. Wu L. Noels L. Bordas S. P. A. Natarajan S. Beex L. A. A. (2021). Neural-network acceleration of projection-based model-order-reduction for finite plasticity: Application to RVEs. arXiv. 10.48550/ARXIV.2109.07747
Vlassis N. N. Ma R. Sun W. (2020). Geometric deep learning for computational mechanics part i: Anisotropic hyperelasticity. Comput. Methods Appl. Mech. Eng. 371, 113299. 10.1016/j.cma.2020.113299
Voulodimos A. Doulamis N. Doulamis A. Protopapadakis E. (2018). Deep learning for computer vision: A brief review. Computational intelligence and neuroscience, 2018.
Weerasuriya A. U. Zhang X. Lu B. Tse K. T. Liu C. (2021). A Gaussian process-based emulator for modeling pedestrian-level wind field. Build. Environ. 188, 107500. 10.1016/j.buildenv.2020.107500
Wirtz D. Karajan N. Haasdonk B. (2015). Surrogate modeling of multiscale models using kernel methods. Int. J. Numer. Methods Eng. 101, 1–28. 10.1002/nme.4767
Xu K. Ba J. Kiros R. Cho K. Courville A. Salakhudinov R. et al. (2015). “Show, attend and tell: Neural image caption generation with visual attention,” in International conference on machine learning (PMLR), 2048–2057.
Zakutayev A. Wunder N. Schwarting M. Perkins J. D. White R. Munch K. et al. (2018). An open experimental database for exploring inorganic materials. Sci. data 5, 1–12. 10.1038/sdata.2018.53
