In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen. How can humans and machines be creative together?

Figure: Brain wave generated poetry

Research result of the month: Brain Poetry

This series presents the researchers behind the latest scientific achievements, gives glimpses of scientists in action, and discusses the societal and industrial relevance of the results without forgetting the big picture.

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen.

Tell a bit about the background of the work?

I started working on computational creativity in the summer 2011 as a trainee / research assistant in the Discovery group of prof. Hannu Toivonen. This work concentrated on linguistic creativity and especially poetry generation. The idea was to develop new methods that could be used to produce poetry automatically. Later in 2013, we developed an installation that measures electroencephalography (EEG) signals, i.e. brain waves, and produces poetry in different styles based on these measurements. This work was done by the Brains on Art art-science co-operation crew, which I'm a member of, and the poetry generation methods used in this work have been developed in the Discovery group. A scientific paper describing the installation, its cultural impact, and more generally use of personal biosignal measurements in combination with methods of computational creativity will be submitted to the most important computational creativity conference (ICCC) in the end of February. Earlier work on poetry generation methods has been previously published in the same conference (see, e.g., Toivanen et al. 2012, 2013).

Brain Poetry has been exhibited in many places both domestically and internationally including, for instance, Kiasma the museum of contemporary art in Helsinki and the media museum Rupriikki in Tampere. The most notable of these appearances is the Frankfurt International Book Fair in October 2014 where almost 5000 unique poems were generated and there was a constant queue to the installation for a whole week. The Frankfurt International Book Fair is the world's largest trade fair for books and around 100 000 people visited the Guest of Honour Pavilion of Finland where Brain Poetry was exhibited along with two other installations. Brain Poetry was also demonstrated in a prime time Finnish TV show Prisma Studio in February 2015. In addition, Brain Poetry has been covered in many electronic and print media articles in Finland and internationally.   

What got you started in this research topic?

Back in 2011, prof. Hannu Toivonen was interested in starting new research on computational creativity and this sounded very interesting to me. During that time I was still doing my MSc degree in Cognitive Science but I was very much into Computer Science and Mathematics, as well. Computational creativity seemed to be an exciting research area in the intersection of these fields. In 2012, I started working on my Computer Science PhD degree supervised by Hannu Toivonen. Since then, I have been researching different topics related to computational creativity and especially to linguistic creativity.

Can you outline the key scientific insights in the article/demo?

Human connection is very important in much of the arts. We think that this is also one of the main reasons why algorithmically created artefacts are in many cases seen as irrelevant as people are assessing the essence and quality of them. However, people are usually very interested in themselves and hence, if methods for computationally generating art are linked to real characteristics of people the results seem much more interesting. We propose using personal biosignal measurements, like EEG, to provide more context and inspiration for computationally creative systems. Also, applications bridging EEG measurements and language are quite rare though converting EEG into music or visual arts has been done in many occasions. Brain Poetry installation / demo was built to study, for instance, these aspects. Different biosignals, like EEG, Electrocardiography (ECG), and facial electromyography (EMG) provide attractive possibilities for measuring objective data of profound human qualities, thinking and feeling, and then creating artefacts that reflect these qualities.

Figure: Brain poetry in action

Can you elaborate on the applications of this result?

People seem to have lots of fun when reading brain poems, so the poetry generation that's linked to their personal biosignals seems to have some application value of its own. More generally, the poetry generation methods have applications, for instance, in pedagogical tools and games for learning about language and literature. Some work on this area has already been carried out by Anna Kantosalo et al. (2014a, 2014b).

What are the future directions of this research?

Pedagogical tools using the developed methods seem promising. Also, measurement of different kind of biosignals for creative applications can be expanded a lot. For instance, one idea could be to measure the tension of facial muscles with EMG to produce happy poems for people who smile and sad poems for those who frown, or vice versa. I'm also interested in generation of songs where the system generates both the music and lyrics. There are a multitude of different research paths.

References

Anna Kantosalo, Jukka M. Toivanen, Ping Xiao, and Hannu Toivonen: From isolation to involvement: Adapting machine creativity software to support human-computer co-creation. In Proceedings of the Fifth International Conference on Computational Creativity. 2014a.

Anna Kantosalo and Sirpa Riihiaho: Let's play the feedback game. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, pp. 943-946. ACM, 2014b.

Jukka Toivanen, Hannu Toivonen, Alessandro Valitutti, and Oskar Gross: Corpus-based generation of content and form in poetry. In Proceedings of the Third International Conference on Computational Creativity. 2012.

Jukka M. Toivanen, Matti Järvisalo, and Hannu Toivonen: Harnessing constraint programming for poetry composition. In International Conference on Computational Creativity, pp. 160-167. 2013.

Links

Discovery research group: https://www.cs.helsinki.fi/en/discovery

Brains on art: https://www.facebook.com/BrainsOnArt , https://brainsonart.wordpress.com/

International Conference on Computational Creativity: http://computationalcreativity.net/iccc2015/

Prisma Studio on Computational Creativity (Finnish only): http://areena.yle.fi/tv/2466699