Jinwoo Lee
Jinwoo Yi
Jinwoo Yi
dfsdfsdfsdfsNews
(Dr.Cha, Joonwoo, Sooyoung, Me, and Heehwan from the left)
My colleagues and I were awarded the grand prize at the 2024 AI × Art Hackathon at Seoul National University! The event focused on exploring the aesthetic and ethical implications of creative activities using generative models, as well as contemplating future directions in this field. We proposed a program that captures EEG data while individuals recall cherished memories and emotions they wish to express as art. This data, along with accompanying writings, sketches, and music, is processed by generative models to create a short music video, blending personal experiences with AI-generated content.
In contemporary art, where autobiographical works are increasingly prevalent, we noticed that many individuals struggle to create what they wish to express due to a lack of technical skills. In this context, generative models can offer aspiring creators exceptional technical capabilities. At the same time, by inputting human memories and emotions into generative models, we sought to address the societal question: "What should we create with these models?" Through the interaction between humans and machines, our goal was to align the 'hallucinations' of machines with those of humans.
In this process, we integrated an EEG-emotion decoder, developed for research purposes, into our framework. This allowed us to extract an "emotional sequence" embedding (e.g., a memory transitioning from pain to nostalgia) from the EEG data recorded while recalling memories, which we used as a generative model prompt. The prompt, combined with the creator's writings and sketches, was input into the generative model to produce a short video based on the creator's sketch as a seed. Additionally, we reintroduced the music the creator associated with the memory, embedding it with the emotional sequence, which resulted in an emotionally resonant, rearranged soundtrack for the video.
Although this is a protocol developed during a brief 9-hour hackathon and still has room for improvement, we plan to update it extensively over the next three weeks. We aim to create an interactive booth where attendees can experience it firsthand at the "Art Diffusion 2024" event, which will be held at the Seoul National University Museum of Art on October 30th. We appreciate your interest and support!
Thanks for my colleagues, Joonwoo, Sooyoung, Heehwan, and our supervisor, Dr. Jiook Cha!
↓ Here’s our framework and prototype!
© 2024 Jinwoo Lee. All rights are reserved.