Together with Prof. Karcher Morris and Postdoctoral scholar Jon Paden, we have been awarded a $45,000 grant from the UC San Diego Course Development and Instructional Improvement Program (CDIIP) to develop and pilot imagination for engineers within STEM curricula. This builds on work I have done as a lecturer in Data Science, Electrical and Computer Engineering/ML for the Arts, bridging cultivate of human imagination within STEM education, and focused on imagination as a driver of engagement, retention, and broadening the scope of STEM disciplines. The modules and resources we develop (and publish) will be shaped with an eye towards broad applicability across diverse educational fields.

I gave a workshop with faculty and graduate students from the University of Chicago Digital Media Workshop and Poetry & Poetics Workshop on Machine Imagination: Text to Image Generation with Neural Networks.

Description: With recent advancements in machine learning techniques, researchers have demonstrated remarkable achievements in image synthesis (BigGAN, StyleGAN), textual understanding (GPT-3), and other areas of text and image manipulation. This hands-on workshop introduces state-of-the-art techniques for text-to-image translation, where textual prompts are used to guide the generation of visual imagery. Participants will gain experience with Open AI’s CLIP network and Google’s BigGAN, using free Google Colab notebooks which they can apply to their own work after the event. We will discuss other relationships between text and image in art and literature; consider the strengths and limitations of these new techniques; and relate these computational processes to human language, perception, and visual expression and imagination. Please bring a text you would like to experiment with!

Workshop link here: https://github.com/roberttwomey/machine-imagination-workshop

I’m excited that our workshop, Measuring Computational Creativity: Collaboratively Designing Metrics to Evaluating Creative Machines will be featured at ISEA2020 – Why Sentience? in Montreal in October. Eunsu Kang, Jean Oh, and I, together with ISEA participants, will develop metrics to assess computational creativity. We will address questions including:

How do we make a creative machine? Creativity is not a sudden burst out of blank space. It involves “a multitude of definitions, conceptualizations, domains, disciplines that bear on its study, empirical methods, and levels of analysis, as well as research orientations that are both basic and applied – and applied in varied contexts.” From Newell, Shaw, and Simon’s insights on computational creativity to Boden’s definitions such as combinational creativity, exploratory creativity, and transformational creativity, defining what kind of creativity, which is appropriate for the specific task of a machine, would be a sensible first step to build a creative algorithm/machine. Yet some questions remain. Can we computationally model ambiguity? Would a novelty search result in valuable discoveries? Where is the threshold between randomness and creativity? Last but not least, how do we evaluate the creativity of an algorithm? This workshop is a first attempt to establish evaluation metrics assessing computational creativity in our current international Arts and Machine Learning (ML) research renaissance.

You can read more about the preliminary programming here: http://isea2020.isea-international.org/preliminary-programming/

I’m thrilled to offer my new course through the Halicioglu Data Science Institute at UC San Diego. It’s been in the works for about a year now.

This course addresses the intersection of data science and contemporary arts and culture, exploring four main themes of authorship, representation, visualization, and data provenance. The course is not solely an introducing to data science techniques, nor merely an arts practice course, but explores significant new possibilities for both fields arising from their intersection. Students will examine problems from complementary perspectives of artist-researchers and data scientists.

Read more here: dsc160.roberttwomey.com

How can data science and the arts and humanities learn from one another?

Two days of events February 7-8 considering the growing digitization of the cultural record and the explosion of new data generation, collection, and analysis practices create a new state of cultured data: culture as data, and data as a driver of culture. Our symposium examines this emerging condition, considering both how analytic techniques enable new understandings of culture, and how the proliferation of data in everyday life changes how culture is produced, distributed, and influenced. In these panels, we wrestle with new modes of scholarship and cultural production enabled by data-forward analysis methods, and consider perspectives from the arts and humanities for data science practice. What can these disciplines teach one another about their possibilities and limits towards realizing a more just, informed, and culturally-rich future?

With 200 RSVPs for both days, and a robust and diverse turnout, the event was a success!

Day 1 Talks @ Atkinson Hall, UC San Diego: https://youtu.be/3qBd5t0iV8c?t=1365

Stay tuned for complete archives of the talks and performances on the website: cultureddata.net

This Friday I’m speaking to the Electronic Media Studio: Introduction to Interactivity students at CMU.

I’ll give a talk followed by a workshop and demo exploring artistic applications of smart home technologies. Topics include wireless sensing, computer vision, and machine listening to create narratives of inhabited space. We consider the creative possibilities and personal consequences of life with ubiquitous sensing, perceiving machines.

Workshop: https://github.com/roberttwomey/housemachine-workshop

Course website: http://cmuems.com/2017/60210c/machines-for-living-by-robert-twomey/

I’m thrilled that our paper on Rover has been accepted to SIGGRAPH 2017. See you in LA!