Brain poetry

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen. How can humans and machines be creative together?

Figure: Brain wave generated poetry

 

Research result of the month: Brain Poetry

This series presents the researchers behind the latest scientific achievements, gives glimpses of scientists in action, and discusses the societal and industrial relevance of the results without forgetting the big picture.

In the latest research result of the month section, we interview PhD student Jukka Toivanen about his recent work on brain poetry in the Discovery group led by professor Hannu Toivonen.

 

Tell a bit about the background of the work?

I started working on computational creativity in the summer 2011 as a trainee / research assistant in the Discovery group of prof. Hannu Toivonen. This work concentrated on linguistic creativity and especially poetry generation. The idea was to develop new methods that could be used to produce poetry automatically. Later in 2013, we developed an installation that measures electroencephalography (EEG) signals, i.e. brain waves, and produces poetry in different styles based on these measurements. This work was done by the Brains on Art art-science co-operation crew, which I'm a member of, and the poetry generation methods used in this work have been developed in the Discovery group. A scientific paper describing the installation, its cultural impact, and more generally use of personal biosignal measurements in combination with methods of computational creativity will be submitted to the most important computational creativity conference (ICCC) in the end of February. Earlier work on poetry generation methods has been previously published in the same conference (see, e.g., Toivanen et al. 2012, 2013).

Brain Poetry has been exhibited in many places both domestically and internationally including, for instance, Kiasma the museum of contemporary art in Helsinki and the media museum Rupriikki in Tampere. The most notable of these appearances is the Frankfurt International Book Fair in October 2014 where almost 5000 unique poems were generated and there was a constant queue to the installation for a whole week. The Frankfurt International Book Fair is the world's largest trade fair for books and around 100 000 people visited the Guest of Honour Pavilion of Finland where Brain Poetry was exhibited along with two other installations. Brain Poetry was also demonstrated in a prime time Finnish TV show Prisma Studio in February 2015. In addition, Brain Poetry has been covered in many electronic and print media articles in Finland and internationally.   

 

What got you started in this research topic?

Back in 2011, prof. Hannu Toivonen was interested in starting new research on computational creativity and this sounded very interesting to me. During that time I was still doing my MSc degree in Cognitive Science but I was very much into Computer Science and Mathematics, as well. Computational creativity seemed to be an exciting research area in the intersection of these fields. In 2012, I started working on my Computer Science PhD degree supervised by Hannu Toivonen. Since then, I have been researching different topics related to computational creativity and especially to linguistic creativity.

 

Can you outline the key scientific insights in the article/demo?

Human connection is very important in much of the arts. We think that this is also one of the main reasons why algorithmically created artefacts are in many cases seen as irrelevant as people are assessing the essence and quality of them. However, people are usually very interested in themselves and hence, if methods for computationally generating art are linked to real characteristics of people the results seem much more interesting. We propose using personal biosignal measurements, like EEG, to provide more context and inspiration for computationally creative systems. Also, applications bridging EEG measurements and language are quite rare though converting EEG into music or visual arts has been done in many occasions. Brain Poetry installation / demo was built to study, for instance, these aspects. Different biosignals, like EEG, Electrocardiography (ECG), and facial electromyography (EMG) provide attractive possibilities for measuring objective data of profound human qualities, thinking and feeling, and then creating artefacts that reflect these qualities.

Figure: Brain poetry in action

 

Can you elaborate on the applications of this result?

People seem to have lots of fun when reading brain poems, so the poetry generation that's linked to their personal biosignals seems to have some application value of its own. More generally, the poetry generation methods have applications, for instance, in pedagogical tools and games for learning about language and literature. Some work on this area has already been carried out by Anna Kantosalo et al. (2014a, 2014b).

 

What are the future directions of this research?

Pedagogical tools using the developed methods seem promising. Also, measurement of different kind of biosignals for creative applications can be expanded a lot. For instance, one idea could be to measure the tension of facial muscles with EMG to produce happy poems for people who smile and sad poems for those who frown, or vice versa. I'm also interested in generation of songs where the system generates both the music and lyrics. There are a multitude of different research paths.

 

References

Anna Kantosalo, Jukka M. Toivanen, Ping Xiao, and Hannu Toivonen: From isolation to involvement: Adapting machine creativity software to support human-computer co-creation. In Proceedings of the Fifth International Conference on Computational Creativity. 2014a.

Anna Kantosalo and Sirpa Riihiaho: Let's play the feedback game. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, pp. 943-946. ACM, 2014b.

Jukka Toivanen, Hannu Toivonen, Alessandro Valitutti, and Oskar Gross: Corpus-based generation of content and form in poetry. In Proceedings of the Third International Conference on Computational Creativity. 2012.

Jukka M. Toivanen, Matti Järvisalo, and Hannu Toivonen: Harnessing constraint programming for poetry composition. In International Conference on Computational Creativity, pp. 160-167. 2013.

 

Links

Discovery research group: https://www.cs.helsinki.fi/en/discovery

Brains on art: https://www.facebook.com/BrainsOnArt , https://brainsonart.wordpress.com/

International Conference on Computational Creativity: http://computationalcreativity.net/iccc2015/

Prisma Studio on Computational Creativity (Finnish only): http://areena.yle.fi/tv/2466699

 

Created date

24.03.2015 - 10:23

Inter-university research and training centre on information security

The University of Helsinki and Aalto University have set up a joint research centre focusing on information security. The new centre, HAIC (Helsinki-Aalto Centre for Information Security), will coordinate the Master’s-level security education between the university and Aalto, with links to research and doctoral education.

The idea is to build bridges to the industries and gain their support for the education, and e.g. grants for MSc students coming from outside the EU, the head of the Department of Computer Science, Sasu Tarkoma, says.

Computer science undergraduate Petteri Timonen awarded in US science competition

Petteri Timonen, 19, came second in his category of the Intel International Science and Engineering Fair (ISEF) in Phoenix, Arizona.

 

On Friday, 15 May, Timonen, who is studying computer science at the University of Helsinki, was awarded a grant worth 1500 USD, some 1330 euros, in the Systems Software category of the Intel ISEF science competition.
 
As his entry, Timonen submitted a software tool he developed for Finland’s Red Cross to make mobile blood runs around the country as cost-effective as possible. Timonen implemented his tool in cooperation with the Blood Service.

The tool has gained international attention, as no tool like it seems to have been developed anywhere else. Timonen has also negotiated with the American Red Cross by email.

Renewed Carat App Gives a Smart Boost to Battery

 
The Carat Project Team at the University of Helsinki, Department of Computer Science, has published a new version of the popular mobile energy-awareness application.

After launch in June 2012, Carat has helped over 850,000 users, of which 41 per cent have been Android and 59 per cent iOS users, respectively. The new user interface follows modern application design guidelines and presents battery information in a more intuitive and easy to use manner.

- In addition to the new user interface, we have increased the accuracy of the energy saving recommendations of Carat, says Professor Sasu Tarkoma, the leader of this research done at the university.

The user interface features the number of energy intensive applications (Hogs), energy anomalies (Bugs) and user recommendations (Actions) at a glance on the main screen as well as global energy statistics for the device community.

Cover Song Identification Using Compression-based Distance Measures

M.Sc. Teppo E. Ahonen will defend his doctoral thesis Cover Song Identification Using Compression-based Distance Measures on Friday the 1st of April 2016 at 12 o'clock in the University of Helsinki Exactum Building, Auditorium CK112 (Gustaf Hällströminkatu 2b).  His opponent is Academy Professor Petri Toiviainen (University of Jyväskylä) and custos Professor Esko Ukkonen (University of Helsinki). The defence will be held in Finnish.

Measuring similarity in music data is a problem with various potential applications. In recent years, the task known as cover song identification has gained widespread attention. In cover song identification, the purpose is to determine whether a piece of music is a different rendition of a previous version of the composition. The task is quite trivial for a human listener, but highly challenging for a computer.