Brain poetry

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen. How can humans and machines be creative together?

Figure: Brain wave generated poetry

 

Research result of the month: Brain Poetry

This series presents the researchers behind the latest scientific achievements, gives glimpses of scientists in action, and discusses the societal and industrial relevance of the results without forgetting the big picture.

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen.

 

Tell a bit about the background of the work?

I started working on computational creativity in the summer 2011 as a trainee / research assistant in the Discovery group of prof. Hannu Toivonen. This work concentrated on linguistic creativity and especially poetry generation. The idea was to develop new methods that could be used to produce poetry automatically. Later in 2013, we developed an installation that measures electroencephalography (EEG) signals, i.e. brain waves, and produces poetry in different styles based on these measurements. This work was done by the Brains on Art art-science co-operation crew, which I'm a member of, and the poetry generation methods used in this work have been developed in the Discovery group. A scientific paper describing the installation, its cultural impact, and more generally use of personal biosignal measurements in combination with methods of computational creativity will be submitted to the most important computational creativity conference (ICCC) in the end of February. Earlier work on poetry generation methods has been previously published in the same conference (see, e.g., Toivanen et al. 2012, 2013).

Brain Poetry has been exhibited in many places both domestically and internationally including, for instance, Kiasma the museum of contemporary art in Helsinki and the media museum Rupriikki in Tampere. The most notable of these appearances is the Frankfurt International Book Fair in October 2014 where almost 5000 unique poems were generated and there was a constant queue to the installation for a whole week. The Frankfurt International Book Fair is the world's largest trade fair for books and around 100 000 people visited the Guest of Honour Pavilion of Finland where Brain Poetry was exhibited along with two other installations. Brain Poetry was also demonstrated in a prime time Finnish TV show Prisma Studio in February 2015. In addition, Brain Poetry has been covered in many electronic and print media articles in Finland and internationally.   

 

What got you started in this research topic?

Back in 2011, prof. Hannu Toivonen was interested in starting new research on computational creativity and this sounded very interesting to me. During that time I was still doing my MSc degree in Cognitive Science but I was very much into Computer Science and Mathematics, as well. Computational creativity seemed to be an exciting research area in the intersection of these fields. In 2012, I started working on my Computer Science PhD degree supervised by Hannu Toivonen. Since then, I have been researching different topics related to computational creativity and especially to linguistic creativity.

 

Can you outline the key scientific insights in the article/demo?

Human connection is very important in much of the arts. We think that this is also one of the main reasons why algorithmically created artefacts are in many cases seen as irrelevant as people are assessing the essence and quality of them. However, people are usually very interested in themselves and hence, if methods for computationally generating art are linked to real characteristics of people the results seem much more interesting. We propose using personal biosignal measurements, like EEG, to provide more context and inspiration for computationally creative systems. Also, applications bridging EEG measurements and language are quite rare though converting EEG into music or visual arts has been done in many occasions. Brain Poetry installation / demo was built to study, for instance, these aspects. Different biosignals, like EEG, Electrocardiography (ECG), and facial electromyography (EMG) provide attractive possibilities for measuring objective data of profound human qualities, thinking and feeling, and then creating artefacts that reflect these qualities.

Figure: Brain poetry in action

 

Can you elaborate on the applications of this result?

People seem to have lots of fun when reading brain poems, so the poetry generation that's linked to their personal biosignals seems to have some application value of its own. More generally, the poetry generation methods have applications, for instance, in pedagogical tools and games for learning about language and literature. Some work on this area has already been carried out by Anna Kantosalo et al. (2014a, 2014b).

 

What are the future directions of this research?

Pedagogical tools using the developed methods seem promising. Also, measurement of different kind of biosignals for creative applications can be expanded a lot. For instance, one idea could be to measure the tension of facial muscles with EMG to produce happy poems for people who smile and sad poems for those who frown, or vice versa. I'm also interested in generation of songs where the system generates both the music and lyrics. There are a multitude of different research paths.

 

References

Anna Kantosalo, Jukka M. Toivanen, Ping Xiao, and Hannu Toivonen: From isolation to involvement: Adapting machine creativity software to support human-computer co-creation. In Proceedings of the Fifth International Conference on Computational Creativity. 2014a.

Anna Kantosalo and Sirpa Riihiaho: Let's play the feedback game. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, pp. 943-946. ACM, 2014b.

Jukka Toivanen, Hannu Toivonen, Alessandro Valitutti, and Oskar Gross: Corpus-based generation of content and form in poetry. In Proceedings of the Third International Conference on Computational Creativity. 2012.

Jukka M. Toivanen, Matti Järvisalo, and Hannu Toivonen: Harnessing constraint programming for poetry composition. In International Conference on Computational Creativity, pp. 160-167. 2013.

 

Links

Discovery research group: https://www.cs.helsinki.fi/en/discovery

Brains on art: https://www.facebook.com/BrainsOnArt , https://brainsonart.wordpress.com/

International Conference on Computational Creativity: http://computationalcreativity.net/iccc2015/

Prisma Studio on Computational Creativity (Finnish only): http://areena.yle.fi/tv/2466699

 

Created date

24.03.2015 - 10:23

Brain poetry

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen. How can humans and machines be creative together?

Kjell Lemström to be new Head of Studies

Since Jaakko Kurhila left the department to head the Open University, we had to find a new university lecturer to act as head of studies in short order. We received a total of 28 applications. Out of these, and after a preliminary qualification round, evaluations, interviews, and a department council hearing, Kjell Lemström (KL) was elected for the post. He started working as the department's Head of Studies on 2 March 2015, so the Head of the department (JP) conducted the following induction interview that very week.

This is by no means the first time Kjell has been employed by the department. He defended his thesis on ‘String Matching Techniques for Music Retrieval’ in 2000, and has held numerous teaching and research positions both before and after that, until he transferred to the Laurea University of Applied Sciences in 2011 (luckily, that was only temporary).

Head of Studies Jaakko Kurhila to head Open University

The Head of Studies at the department, University Lecturer Jaakko Kurhila, has been elected to the post of director of the Open University at the University of Helsinki. It was a tough race: all in all, 39 applicants sought the post, some of them through the Mercuri Urval headhunting process. After a consultant evaluation, interviews, and aptitude assessments, the preparatory committee for the post, the steering committee for the Open University, and the rector of the university came to a unanimous decision to select Jaakko, and the contract is already being drawn up.

Being selected from this prestigious group of applicants, and after such a thorough process, is indisputable proof of the qualifications of Jaakko and the high esteem the academic community has for him. The department extends its warmest congratulations to Jaakko for this career development and is proud of the success of its protégé.

Bridging the Gap Between Research and Standardization

In the fourth research result of the month, we report a joint work between the UH NODES group and the Cambridge NetOS group, lead by Prof. Sasu Tarkoma and Prof. Jon Crowcroft, respectively. Their work recently received the best paper award "Best of CCR" from ACM SIGCOMM.