Dances with Robots: VR

For ON DISPLAY GLOBAL 2024 I collaborated with Hortense Gerardo (APT Program), Amy Eguchi, and Joe Unger and Trisha WIlliams (Maveric Studio) for a new performance with robots DANCES WITH ROBOTS: VR. (Ghost Ride the Headset)

This project features:

CRANE, an xArm7, a co-robotic arm from UFACTORY with a portable Robotics and Performative AI Workstation, and a custom creative robotics software stack. This 7 degree of freedom (DOF) articulated robot arm has integrated force and temperature sensing and interactive programming modes. CRANE was operated in the VR space by ROBERT TWOMEY. CRANE is represented in the VR space by a floating, legless, avatar.

PEPPER a social humanoid robot from United Robotics Group (former Sotfbank Robotics), and is capable of recog- nizing faces, facial expressions/basic human emotions. Pepper was optimized for human interaction and is able to engage with peo- ple through various sensor inputs (touch, voice/sound, and vision). It can also carry out prescripted conversations based on human inputs using voice recognition technology. PEPPER was operated in the VR space by AMY EGUCHI. PEPPER is represented in the VR space by a small, humanoid robot.

HORTENSE GERARDO, Director of the Anthropology, Performance, and Technology (APT) Program at the University of California, San Diego. HORTENSE is represented in the VR space as a human, female avatar.

The VR space, Grove of the Matriarchs, was designed by Trisha Williams of ORIGAMI AIR, and recorded at the MAVERIC Studio on November 7, 2024.

Documentation

I was an invited participant in the 2024 World Imagination Network meetup at ASU’s Center for Science and the Imagination October 23-25 2024. Over the course of three days we did intense workshopping of scholarship, development plans, and curricula and toolkits in the area of imagination studies. Guests included Robert Alexander, Stephen Asma, Bob Beard, Ron Beghetto, Fabrice Guerrier, Amy Kind, Peter Langland-Hassan, Karcher Morris, Brendan Bo O’Connor, Robert Sinclair, Laurne Keeler, Clinton Tolley, Cassi Vietan. Thank you Ed Finn, Ruth Wylie, Bob Beard, Nina Miller, and Linsey Wilt from CSI for hosting us! Nice to see our sister institution for the Arthur C. Clarke Center for Human Imagination.

Researchers at Scripps Oceanography have been eavesdropping on marine soundscapes for decades to study animals and the threats they face. High-frequency Acoustic Recording Packages (HARPs) use underwater microphones, or hydrophones, to capture everything from whale songs and dolphin clicks to passing ships and military sonar.

Inspired by this work, Ash Eliza Smith and Robert Twomey invite us into an interactive audio experience that blends storytelling with a guided sound bath. Inside a phone booth, participants embark on an underwater journey, tuning in to both animal and human-made sounds—and responding with their own vocalizations. Outside the booth, people’s movements are tracked, creating visual trails that echo the migratory patterns of sea life.

The project poses thought-provoking questions: What if the animals we listen to could talk back? What if a fish could text you about ocean noise? Could a phone call connect us to an “internet of animals,” and what would that mean? This work brings ocean acoustics and the idea of an “internet of animals” to life, transforming data beyond human perception into an immersive sensory experience.

Fish Phone Booth is a project of Ash Eliza Smith + Robert Twomey (A Speculative Devices x Cohab Labs Production). Production, Graphic and Experience Design support: Sam Bendix. Developer, Design support: Reid Brockmeier. Dramaturgy: Micha? Stankiewicz. Prototyped in support from the La Jolla Playhouse Without Walls Festival, Worlds In Play and The Johnny Carson Center for Emerging Media Arts.

https://aquarium.ucsd.edu/newsroom/embodied-pacific-ocean-unseen

Wonderful to be at SIGGRAPH 2024 as a viewer/consumer/reviewer and not a presenter. I really enjouyed it. So much food for thought, so many inspiring contributions.

We took a crew of Johnny Carson Center for Emerging Media Arts students, it was amazing! A really good mix of influences I think.

I had a tremendous time with small group of diverse artists, computer scientists, and educators designing standards and structures for joint Arts / CS education. This grew out of an NSF CUE proposal.

Thank you Erik Brunvand, Erin Parker, Bill Manaris, Renee McCauley and Roger Eastman for including me with the group!

A few fun images from (1) the session at Saltgrass Studios printmaking shop, and (2) a data visualization cyanotype exercise with Renee and Courtney Starrett on day 2.