When arts and science meet: Digital technology in artistic research

Caruso, Giusy; NIJS, Luc

doi:10.1386/jmte_00019_1

Download

Article (Scientific journals)

When arts and science meet: Digital technology in artistic research

Caruso, Giusy; NIJS, Luc

2021 • In Journal of Music, Technology and Education, 13 (2-3), p. 117 - 140

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/67237

DOI
10.1386/jmte_00019_1

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Caruso & Nijs (2021) - When arts and science meet- Digital technology in artistic research.pdf

Author postprint (2.93 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

art-science; artistic research; embodiment; music improvisation; music performance; technology

Abstract :

[en] In recent decades, advancements in digital technologies have become a rich source of inspiration for artists, who seek to leave the trodden paths and find novel ways of expression. In addition, digital technologies are increasingly implemented in the development of artistic skills, providing new means to develop the artists’ reflection on their own development. As such, they hold great potential to shape artistic research. Moreover, digital technologies offer possibilities to capture the learning process based on quantitative measurement, thereby becoming a potential interface between artistic and scientific approaches to investigating artistic growth. This contribution presents two artistic projects illustrating the potentialities of the art– science encounter. Embedded in the research paradigm of embodied music cognition, both projects explore the role of the body in music performance (interpretation and improvisation). The first project investigates the relation between gesture and interpretative intentions in a contemporary piano composition. The second project concerns the development of one’s musical language through kinemusical improvisation. A mixed methodology and the use of technology as ‘an augmented mirror’ to monitor artistic practice were applied. Both projects illustrate how the implementation of digital technologies may boost the evolution in artistic research and facilitate novel approaches to music teaching and learning.

Disciplines :

Performing arts

Author, co-author :

Caruso, Giusy ; Department of Art History, Musicology and Theatre Studies, Institute for Psychoacoustics and Electronic Music (IPEM), Ghent University, Ghent, Belgium

NIJS, Luc ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Education and Social Work (DESW) > Institute of Musicology and Arts ; Royal Conservatory of Antwerp, CORPoREAL, Belgium

External co-authors :

yes

Language :

English

Title :

When arts and science meet: Digital technology in artistic research

Publication date :

2021

Journal title :

Journal of Music, Technology and Education

ISSN :

1752-7066

eISSN :

1752-7074

Publisher :

Intellect Ltd.

Volume :

Issue :

2-3

Pages :

117 - 140

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://intellectdiscover.com/content/journals/10.1386/jmte_00019_1?crawler=true&mimetype=application/pdf

Available on ORBilu :

since 11 January 2026

Statistics

Number of views

10 (0 by Unilu)

Number of downloads

3 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

Baets, Th. De and Nijs, L. (2015), ‘Onderzoek door muziekdocenten, is dat wel wetenschap?’, Cultuur + Educatie, 42:15, pp. 8–19.
Bernardi, V. Di (2018), ‘Corpi e visioni nell’era digitale’, in V. Di Bernardi and L. G. Monda (eds), Immaginare la danza. Corpi e visioni nell’era digitale, Bologna: Massimilano Piretti, pp. 5–16.
Bishop, L. and Goebl, W. (2016), ‘Music and movement: Musical instruments and performers’, in R. Ashley and R. Timmers (eds), Routledge Companion to Music Cognition, London: Routledge, pp. 349–61.
Bishop, L. and Goebl, W. (2018), ‘Communication for coordination: Gesture kinematics and conventionality affect synchronization success in piano duos’, Psychological Research, 82:6, pp. 1177–94.
Bleeker, M. (2017), ‘What if this were an archive? Abstraction, enactment and human implicatedness’, in M. Bleeker (ed.), Transmission in Motion: The Technologizing of Dance, London: Routledge, pp. 199–214.
Borgdorff, H. A. (2012), The Conflict of the Faculties: Perspectives on Artistic Research and Academia, Leiden: Leiden University Press.
Bransford, J., Brown, A. and Cocking, R. (eds) (1999), How People Learn: Brain, Mind, Experience, Washington, DC: National Academy Press.
Bremmer, M. and Nijs, L. (2020), ‘The role of the body in instrumental and vocal music pedagogy: A dynamical systems theory perspective on the music teacher’s bodily engagement in teaching and learning’, Frontiers in Education, 5:79, https://doi.org/10.3389/feduc.2020.00079. Accessed 5 October 2020.
Buck, B., MacRitchie, J. and Bailey, N. J. (2013), ‘The interpretive shaping of embodied musical structure in piano performance’, Empirical Musicology Review, 8:2, pp. 92–119.
Caruso, G. (2015), ‘Music performance analysis using the motion capture System: EXPERIMENT REPORTAGE‘, YouTube, 11 January 2017, https://www.youtube.com/watch?v=tOmF4YeBgvY. Accessed 18 October 2020.
Caruso, G. (2018), ‘Mirroring the intentionality and gesture of a piano performance: An interpretation of 72 Etudes Karnatiques pour piano’, Ph.D. thesis, Ghent: Ghent University Press.
Caruso, G. (2019), ‘Un’indagine analitico-performativa sui 72 studi carnatici per pianoforte di Jacques Charpentier (1933–2017)’, Rivista di Analise e Teoria Musicale, 25:2, pp. 91–137.
Caruso, G., Coorevits, E., Nijs, L. and Leman, M. (2016), ‘Gestures in contemporary music performance: A method to assist the performer’s artistic process’, Contemporary Music Review, 35:4–5, pp. 402–22.
Caruso, G., Nijs, L. and Leman, M. (2021), ‘“My avatar and me”: Technology-enhanced mirror in monitoring music performance practice’, in R. Hepworth-Sawyer (ed.), Innovation in Music: Future innovations, London: Routledge, pp. 355–69.
Chaffin, R. and Imreh, G. (2002), ‘Practicing perfection: Piano performance as expert memory’, Psychological Science: A Journal of the American Psychological Society, 13:4, pp. 342–49.
Chaffin, R., Imreh, G., Lemieux, A. F. and Chen, C. (2003), ‘“Seeing the big picture”: Piano practice as expert problem solving’, Music Perception, 20, pp. 461–85.
Coessens, K., Crispin, D. and Douglas, A. (2009), The Artistic Turn: A Manifesto, Leuven: Leuven University Press.
Cook, N. (1999), ‘Analysing performance and performing analysis’, in N. Cook and M. Everist (eds), Rethinking Music, Oxford: Oxford University Press, pp. 239–61.
Coorevits, E., Moelants, D., Maes, P. J. and Leman, M. (2019), ‘Exploring the effect of tempo changes on violinists’ body movements’, Musicae Scientiae, 23:1, pp. 87–110.
Coorevits, E., Moelants, D., Östersjö, S. and Gorton, D. (2015), ‘Decomposing a composition: On the multi-layered analysis of expressive music performance’, in M. Aramaki, R. Kronland and S. Ystad (eds), Lecture Notes in Computer Science (LNCS): Post-Proceedings of the 11th International Symposium on CMMR, Plymouth, New York, 16–19 June, New York: Springer, pp. 171–189.
Dahl, S. and Friberg, A. (2004), ‘Expressiveness of musician’s body movements in performances on marimba’, in A. Camurri and G. Volpe (eds), GestureBased Communication in Human–Computer Interaction, New York: Springer, pp. 479–86.
Davidson, J. W. (2005), ‘Bodily communication in musical performance’, in D. Miell, R. Macdonald and D. J. Hargreaves (eds), Musical Communication, Oxford: Oxford University Press, pp. 215–28.
Davidson, J. W. (2007), ‘Qualitative insights into the use of expressive body movement in solo piano performance: A case study approach’, Psychology of Music, 35:3, pp. 381–401.
Demos, A. P. and Chaffin, R. (2017), ‘Recurrence, mixed models, and surrogates’, in M. Lesaffre, P. J. Maes and M. Leman (eds), The Routledge Companion to Embodied Music Interaction, London: Routledge, pp. 341–49.
Demos, A. P., Chaffin, R. and Logan, T. (2018), ‘Musicians body sway embodies musical structure and expression: A recurrence-based approach’, Musicae Scientiae, 22:2, pp. 244–63.
Desmet, F., Nijs, L., Demey, M., Lesaffre, M. and Leman, M. (2012), ‘Assessing a clarinet player’s performer gestures in relation to locally intended musical targets’, Journal of New Music Research, 41:1, pp. 31–48.
Doğantan-Dack, M. (2015), Artistic Practice as Research in Music: Theory, Criticism, Practice, Farnham: Ashgate.
Dourish, Paul (2004), Where the Action Is: The Foundations of Embodied Interaction, Cambridge, MA: MIT Press.
Dries, K. (2017), ‘“Voyages”: Dualism in artistic research and performance of a saxophone player. An Interdisciplinary approach’, Ph.D. thesis, Brussels: VUB.
Duval, E., Sharples, M. and Sutherland, R. (eds) (2017), Technology Enhanced Learning, New York: Springer.
Edler, A. (1991), Schumann e il suo tempo (Robert Schumann und Sein Zeit) (trans. L. Dallapiccola), Turin: E.D.T.
Gallagher, S. (1986), ‘Body image and body schema: A conceptual clarification’, Journal of Mind and Behaviour, 7:4, pp. 541–54.
Godøy, R. I. and Leman, M. (2010), Musical Gestures: Sound, Movement, and Meaning, London: Routledge.
Goebl, W., Dixon, S. and Schubert, E. (2014), ‘Quantitative methods: Motion analysis, audio analysis, and continuous response techniques’, in D. Fabian, R. Timmers and E. Schubert (eds), Expressiveness in Music Performance: Empirical Approaches Across Styles and Cultures, Oxford: Oxford University Press, pp. 221–39.
Goffman, E. (1974), ‘The frame’, in C. Counsell and L. Wolf (eds), Performance Analysis: An Introductory Course Book, London: Routledge, pp. 24–30.
Guichard, C. (2004), Marie Jaëll: The Magic Touch. Piano Music by Mind Training, New York: Algora Publishing.
Haak, M. J. Van den and Jong, M. D. De (2003), ‘Exploring two methods of usability testing’, in Proceedings of Concurrent versus Retrospective ThinkAloud Protocols. Professional Communication Conference (IPCC), Orlando, FL, 21–24 September, Orlando: IEEE International, pp. 285–87.
Himonides, E. (2012), ‘The misunderstanding of music-technology-education: A meta perspective’, in G. E. McPherson and G. F. Welch (eds), The Oxford Handbook of Music Education, vol. 2, Oxford: Oxford University Press, pp. 433–56.
Hospelhorn, E. and Radinsky, J. (2017), ‘Method for analyzing gestural communication in musical groups’, Discourse Processes, 54:7, pp. 504–23.
Hubrich, S. G. B. (2016), ‘The performer’s body in creative interpretations of repertoire music’, Arts and Humanities in Higher Education, 15:3–4, pp. 337–52.
Jensenius, A. R. (2007), ‘Action – sound: Developing methods and tools to study music-related body movement’, Ph.D. thesis, Oslo: University of Oslo.
Klein, J. (2010), ‘The other side of the frame. Artistic experience as felt framing: Fundamental principles of an artistic theory of relativity’, in S. Flach and J. Söffner (eds), Habitus in Habitat II: Other Sides of Cognition, Bern: Verlag Peter Lang, pp. 121–37.
Leman, M. (2007), Embodied Music Cognition and Mediation Technology, Cambridge, MA: MIT Press.
Leman, M. (2016), The Expressive Moment: How Interaction (with Music) Shapes Human Empowerment, Cambridge, MA: MIT Press.
Lesaffre, M., Maes, P. J. and Leman, M. (2017), The Routledge Companion to Embodied Music Interaction, London: Routledge.
McPershon, A. (2016), ‘Magnetic resonator piano project’, iRCAM, 18 June, https://medias.ircam.fr/x57c686. Accessed 15 October 2020.
Merleau-Ponty, M. (1945), Phénoménologie de la perception, Paris: Gallimard.
Miranda, E. (2015), ‘Biocomputer music’, Vimeo, 10 November, https://vimeo. com/111409050. Accessed 15 October 2020.
Motion Capture Optitrack (n.d.), NaturalPoint, Inc. DBA OptiTrack, http://optitrack.com/. Accessed 14 June 2021.
Neuhaus, Henry (1973), The Art of Piano Playing, New York: Praeger Publishers Inc.
Newell, K. M. (2003), ‘Change in motor learning: A coordination and control perspective’, Motriz Rio Claro, 9:1, pp. 1–6.
Nijs, L. (2017), ‘The merging of musician and musical instrument: Incorporation, presence and the levels of embodiment’, in M. Lesaffre, P. J. Maes and M. Leman (eds), The Routledge Companion to Embodied Music Interaction, London: Routledge, pp. 49–57.
Nijs, L. (2019), ‘Moving together while playing music: Promoting involvement through student-centred collaborative practices’, in S. Gies and J. H. Sætre (eds), Becoming Musicians: Student Involvement and Teacher Collaboration in Higher Music Education, Oslo: The Norwegian Academy of Music, pp. 239–60.
Nijs, L., Lesaffre, M. and Leman, M. (2013), ‘The musical instrument as a natural extension of the musician’, in M. Castellengo and H. Genevois (eds), Music and Its Instruments, Sampzon: Editions Delatour France, pp. 467–84.
Norgaard, M. (2014), ‘How jazz musicians improvise: The central role of auditory and motor patterns’, Music Perception: An Interdisciplinary Journal, 31:3, pp. 271–87.
Owens, T. (1974), ‘Charlie Parker: Techniques of improvisation’, unpublished doctoral dissertation, Los Angeles: University of California.
Palmer, C. and Hutchins, S. (2006), ‘What is musical prosody?’, Psychology of Learning and Motivation, 46, pp. 245–78.
Perception Neuron MOCAP (n.d.), Noitom, https://neuronmocap.com. Accessed 14 June 2021.
Pitozzi, E. (2014), ‘The perception is a prism: Body, presence, and technologies’, Brazilian Journal on Presence Studies, 4:2, pp. 174–204.
Polanyi, M. (1966), The Tacit Dimension, Garden City, NY: Anchor Books.
Prins Stefan (2019), ‘Piano Hero‘, YouTube, 5 January, https://www.youtube. com/watch?v=6N8qK8KA3bY. Accessed 14 June 2021.
Rattalino, P. (1982), Storia del Pianoforte, Milano: Il Saggiatore.
Reynolds, R. (1925), ‘On playing the “pianola”’, in P. A. Scholes (ed.), The Appreciation of Music by Means of the ‘Pianola’, Oxford: Oxford University Press, http://www.musanim.com/Tapper/OnPlayingThePianola.html. Accessed 18 October 2020.
Rink, J. (2015), ‘The (f)utility of performance analysis’, in M. Doğantan-Dack (ed.), Artistic Practice as Research in Music: Theory, Criticism, Practice, Farnham: Ashgate, pp. 127–47.
Sani, N. (2015), ‘Chemical free’, YouTube, 21 December, https://www.youtube. com/watch?v=VHxKyVMD6pU. Accessed 15 October 2020.
Schacher, J. C., Järveläinen, H., Strinning, C. and Neff, P. (2015), ‘Movement perception in music performance: A mixed methods investigation’, in Proceedings of the International Conference on Sound and Music Computing (SMC’15), Maynooth, Ireland, 25 June–1 August, pp. 132–39.
Stroppa, M. ([1982] 2010), ‘Traiettoria…deviata’, YouTube, 15 July, https://www. youtube.com/watch?v=Nn2FqP5MXkw. Accessed 15 October 2020.
Sutil, N. S. (2015), Motion and Representation: The Language of Human Movement, Cambridge, MA: MIT Press.
Tanaka, A. (2010), ‘Suspension’, YouTube, 30 November, https://www.youtube. com/watch?v=02jPgh4Cvxo. Accessed 15 October 2020.
Tanaka, A. (2014), ‘Creative applications of interactive mobile music’, in S. Gopinath and J. Stanyek (eds), The Oxford Handbook of Mobile Music Studies, vol. 2, Oxford: Oxford University Press, pp. 470–86.
Taylor, G. D. (2014), When the Machine-Made Art: The Troubled History of Computer Art, New York: Bloomsbury Publishing.
Waddell, G. and Williamon, A. (2019), ‘Technology use and attitudes in music learning’, Frontiers in ICT, 6:11, pp. 1–14.
Wanderley, M. M., Vines, B. W., Middleton, N., McKay, C. and Hatch, W. (2005), ‘The musical significance of clarinetists’ ancillary gestures: An exploration of the field’, Journal of New Music Research, 34:1, pp. 97–113.
Wilson, M. (1985), Drawing with Computers: The Artist’s Guide to Computer Graphics, New York: Perigee Books.
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A. and Sloetjes, H. (2006), ‘ELAN: A professional framework for multimodality research’, in N. Calzolari, K. Choukri, A. Gangemi, B. Maegaard, J. Mariani, J. Odijk and D. Tapias (eds), Proceedings of LREC 2006, Genova, Italy, 22–28 May, Genova: European Language Resources Association (ELRA), pp. 1556–59.
Wooley, N. (2020), ‘Built from the root, ever-changing at the bloom’, in A. Hamilton and L. Pearson (eds), The Aesthetics of Imperfection in Music and the Arts: Spontaneity, Flaws and the Unfinished, New York: Bloomsbury Publishing, pp. 45–52.