AI & Art; XAI; agents; Neural Style Transfer; Cultural Heritage
Abstract :
[en] The advent of Artificial Intelligence (AI) has brought about significant changes in our daily lives with applications including industry, smart cities, agriculture, and telemedicine. Despite the successes of AI in other "less-technical" domains, human-AI synergies are required to ensure user engagement and provide interactive expert knowledge. This is notably the case of applications related to art since the appreciation and the comprehension of art is considered to be an exclusively human capacity. This paper discusses the potential human-AI synergies aiming at explaining the history of art and artistic style transfer. This work is done in the context of the "Smart Photobooth" a project which runs within the AI & Art pavilion. The latter is a satellite event of Esch2022 European Capital of Culture whose main aim is to reflect on AI and the future of art. The project is mainly an outreach and knowledge dissemination project, it uses a smart photo-booth, capable of automatically transforming the user's picture into a well-known artistic style (e.g., impressionism), as an interactive approach to introduce the principles of the history of art to the open public and provide them with a simple explanation of different art painting styles. Whereas some of the cuttingedge AI algorithms can provide insights on what constitutes an artistic style on the visual level, the information provided by human experts is essential to explain the historical and political context in which the style emerged. To bridge this gap, this paper explores Human-AI synergies in which the explanation generated by the eXplainable AI (XAI) mechanism is coupled with insights from the human expert to provide explanations for school students as well as a wider audience. Open issues and challenges are also identified and discussed.
Disciplines :
Computer science
Author, co-author :
Van Der Peijl, Egberdien; AI-Robolab/ICR, University of Luxembourg, Computer Science and Communications, Esch-sur-Alzette, Luxembourg
Mualla, Yazan; CIAD, Univ. Bourgogne Franche-Comté, UTBM, Belfort, France
BOURSCHEID, Thiago Jorge ; University of Luxembourg > Faculty of Science, Technology and Medicine > Department of Computer Science > Team Christoph SCHOMMER
Elias, Yolanda; Dpto. Dibujo, University of Seville, C/Larana n°3, Spain
KARPATI, Daniel ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
NOUZRI, Sana ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
External co-authors :
yes
Language :
English
Title :
Toward XAI & Human Synergies to Explain the History of Art: The Smart Photobooth Project
Publication date :
01 June 2021
Event name :
Third International Workshop, EXTRAAMAS 2021, Virtual Event
Event date :
May 3–7, 2021
Main work title :
Explainable and Transparent AI and Multi-Agent Systems
Images of artworks in the public domain. https://www.metmuseum.org/about-the-met/policies-and-documents/image-resources
Alpers, S.: Style is what you make it: the visual arts again. In: Lang, B. (ed.) The Concept of Style (Revised and Expanded Edition. Cornell Univ. Pr., Ithaca and London, 1979, 1987), pp. 137–162 (1979)
Anjomshoae, S., Najjar, A., Calvaresi, D., Främling, K.: Explainable agents and robots: results from a systematic literature review. In: 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, 13–17 May 2019, pp. 1078–1088. International Foundation for Autonomous Agents and Multiagent Systems (2019)
Baudrillard, J.: The Conspiracy of Art. Manifestos, Interviews, Essays. Semio-text(e) Foreign Agents Series (2003)
Bouchard, A.: Gamification in the arts (2013)
Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations (2018)
Brownlee, J.: How to develop a CycleGAN for image-to-image translation with Keras, August 2019
Calegari, R., Ciatto, G., Omicini, A.: On the integration of symbolic and sub-symbolic techniques for XAI: a survey. Intell. Artifi. 14(1), 7–32 (2020)
Carrozzino, M., Bergamasco, M.: Beyond virtual museums: experiencing immersive virtual reality in real museums. J. Cult. Herit. 11(4), 452–458 (2010)
Cetinic, E., Lipic, T., Grgic, S.: Learning the principles of art history with convolutional neural networks. Pattern Recogn. Lett. 129, 56–62 (2020)
Cetinic, E., She, J.: Understanding and creating art with AI: review and outlook. arXiv preprint arXiv:2102.09109 (2021)
Coelho, A., Cardoso, P., van Zeller, M., Santos, L., Raimundo, J., Vaz, R.: Gami-fying the museological experience (2020)
Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining “gamification”. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9– 15 (2011)
Díaz-Rodríguez, N., Pisoni, G.: Accessible cultural heritage through explainable artificial intelligence. In: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, pp. 317–324 (2020)
Dicheva, D., Dichev, C., Agre, G., Angelova, G.: Gamification in education: a systematic mapping study. J. Educ. Technol. Soc. 18(3), 75–88 (2015)
Elgammal, A., Liu, B., Kim, D., Elhoseiny, M., Mazzone, M.: The shape of art history in the eyes of the machine. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Frid, E., Lindetorp, H., Hansen, K.F., Elblaus, L., Bresin, R.: Sound forest: evaluation of an accessible multisensory music installation. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2019)
Gammel, I.: Baroness Elsa, Gender, Dada and Everyday Modernity. A Cultural Biography. MIT Press, Cambridge (2003)
d’Avila Garcez, A.S., Broda, K.B., Gabbay, D.M.: Neural-Symbolic Learning Systems: Foundations and Applications. Springer, London (2002). https://doi.org/10. 1007/978-1-4471-0211-3
Goodfellow, I.J., et al.: Generative adversarial networks, June 2014
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. (CSUR) 51(5), 1–42 (2018)
Hamari, J.: Transforming homo economicus into homo ludens: a field experiment on gamification in a utilitarian peer-to-peer trading service. Electron. Commer. Res. Appl. 12(4), 236–245 (2013)
Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work?-a literature review of empirical studies on gamification. In: 2014 47th Hawaii International Conference on System Sciences, pp. 3025–3034. IEEE (2014)
Hamari, J., Lehdonvirta, V.: Game design as marketing: how game mechanics create demand for virtual goods. Int. J. Bus. Sci. Appl. Manag. 5(1), 14–29 (2010)
Harbers, M., van den Bosch, K., Meyer, J.-J.: A methodology for developing self-explaining agents for virtual training. In: Dastani, M., El Fallah Segrouchni, A., Leite, J., Torroni, P. (eds.) LADS 2009. LNCS (LNAI), vol. 6039, pp. 168–182. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13338-1 10
Hoffman, R.R., Mueller, S.T., Klein, G., Litman, J.: Metrics for explainable AI: challenges and prospects. arXiv preprint arXiv:1812.04608 (2018)
Huotari, K., Hamari, J.: Defining gamification: a service marketing perspective. In: Proceeding of the 16th International Academic MindTrek Conference, pp. 17–22 (2012)
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., Song, M.: Neural style transfer: a review, October 2018. arXiv: 1705.04058
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks, March 2019
Kostoska, G., Baez, M., Daniel, F., Casati, F.: Virtual, remote participation in museum visits by older adults: a feasibility study. In: 8th International Workshop on Personalized Access to Cultural Heritage (PATCH 2015), ACM IUI 2015, pp. 1–4 (2015)
Kostoska, G., Vermeeren, A.P.O.S., Kort, J., Gullström, C.: Video-mediated participation in virtual museum tours for older adults. In: 10th International Conference on Design & Emotion, 27–30 September 2016, Amsterdam. The Design & Emotion Society (2016)
Kulkarni, R., Gaikwad, R., Sugandhi, R., Kulkarni, P., Kone, S.: Survey on deep learning in music using GAN. Int. J. Eng. Res. Technol. 8(9), 646–648 (2019)
Lamas, A., et al.: MonuMAI: dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification. Neurocomputing 420, 266–280 (2021)
Lang, B.: The Concept of Style. Cornell University Press, Ithaca (1987)
Latour, G.: Guernica, histoire secrète d’un tableau. Média Diffusion (2013)
Negrevergne, B., Lecoutre, A., Yger, F.: Recognizing art style automatically with deep learning. In: Proceedings of Machine Learning Research, no. 77, pp. 327–342 (2017)
Mao, H., Cheung, M., She, J.: DeepArt: learning joint representations of visual arts. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1183–1191 (2017)
Mualla, Y.: Explaining the behavior of remote robots to humans: an agent-based approach. Ph.D. thesis, University of Burgundy-Franche-Comté, Belfort, France (2020). 2020UBFCA023
Mualla, Y., Kampik, T., Tchappi, I.H., Najjar, A., Galland, S., Nicolle, C.: Explainable agents as static web pages: UAV simulation example. In: Calvaresi, D., Najjar, A., Winikoff, M., Främling, K. (eds.) EXTRAAMAS 2020. LNCS (LNAI), vol. 12175, pp. 149–154. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51924-7 9
Mualla, Y., Najjar, A., Kampik, T., Tchappi, I., Galland, S., Nicolle, C.: Towards explainability for a civilian UAV fleet management using an agent-based approach. arXiv preprint arXiv:1909.10090 (2019)
Mualla, Y., Tchappi, I., Najjar, A., Kampik, T., Galland, S., Nicolle, C.: Human-agent explainability: an experimental case study on the filtering of explanations. In: Proceedings of the 12th International Conference on Agents and Artificial Intelligence-Volume 1: HAMT, pp. 378–385. INSTICC, SciTePress (2020)
Xin Ning, Fangzhe Nan, Shaohui Xu, Lina Yu, and Liping Zhang. Multi-view frontal face image generation: A survey. Concurrency and Computation: Practice and Experience, page e6147, 2020
Perry, L.C.: Reminiscences of Claude Monet from 1889 to 1909. Am. Mag. Art XVIII(3), 123 (1927)
Raento, P., Watson, C.J.: Gernika, Guernica, Guernica?: contested meanings of a Basque place. Polit. Geogr. 19(6), 707–736 (2000)
Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55(1), 68 (2000)
Seaborn, K., Fels, D.I.: Gamification in theory and action: a survey. Int. J. Hum.-Comput. Stud. 74, 14–31 (2015)
Singh, R., et al.: Directive explanations for actionable explainability in machine learning applications. arXiv preprint arXiv:2102.02671 (2021)
Stein, G.: Autobiography of Alice Toklas 1907–1914. The Library of America (1933)
Tan, W.R., Chan, C.S., Aguirre, H.E., Tanaka, K.: Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3703–3707. IEEE (2016)
Toshpulatov, M., Lee, W., Lee, S.: Generative adversarial networks and their application to 3D face generation: a survey. Image Vis. Comput. 108, 104119 (2021)
Trotta, R., Hajas, D., Camargo-Molina, J.E., Cobden, R., Maggioni, E., Obrist, M.: Communicating cosmology with multisensory metaphorical experiences. J. Sci. Commun. 19(2) (2020)
Wölfflin, H.: Principles of Art History (1915)
Zhang, S., Han, Z., Lai, Y.-K., Zwicker, M., Zhang, H.: Stylistic scene enhancement GAN: mixed stylistic enhancement generation for 3D indoor scenes. Vis. Comput. 35(6), 1157–1169 (2019). https://doi.org/10.1007/s00371-019-01691-w
Zhao, Y., Wu, S., Reynolds, L., Azenkot, S.: The effect of computer-generated descriptions on photo-sharing experiences of people with visual impairments. In: Proceedings of the ACM on Human-Computer Interaction, vol. 1, no. CSCW, pp. 1–22 (2017)
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks, August 2020
