Devs 4 – Exploration

Group 2 (Mona, Ricky, Shipra, Zaheen)


Description + Process

For this Devs, we wanted to explore the relationship between a human and an orchestra conductor in the form of a robot. A conductor during a performance serves as a messenger for the composers. Their responsibility is to understand the music and convey it through gestures (hand movements) so transparently that the musicians in the orchestra understand it perfectly.

As a starting point, we wanted to experiment with one aspect of this and then move ahead. Therefore, we tried to understand the hand movements of a conductor, how that reflects in the music composition, and whether the robot will be able to mimic those movements.
A case study we looked at while exploring was ABB’s Robot YuMi, wherein a robot’s performance was developed by capturing the movements of maestro Andrea Colombini through a process known as lead-through programming. The robot’s arms were guided to follow the conductor’s motions meticulously; the movements were then recorded and further fine-tunes in the software. Taking inspiration from this example, we wanted to see how a robot could be in charge of conducting a performance.


Final Project Video

Final Video


We worked with State Machines to create the different instances. We also developed each of the states to perform an action on the laptop and an action/ movement for the robot. We worked with 3 states; Pause, Low, and High. The pause state would be nil/ zero state, no action happens and no audio is played. The low state plays the audio at a lower volume and lower sleep, and the robot does smaller slower movements. The high state plays the audio at a faster phase and at a higher volume. The actions/ movements of the robot also change.




Further Steps

Moving forward, we intend to work on:

  1. Creating a performance piece that will include the robot controlling the music composition. This might be in relation to how the volume changes, the pitch, and the speed of the music composition.
  2. Designing a tool – a Baton, that will be provided to the robot. We want to explore the way we can manipulate size in comparison to the robot by providing a miniature-size tool.
  3. Creating a storyboard to choreograph the scene and props, while developing the character of the robot.

Learnings/ Takeaways

This project helped us in understanding the first few steps of playing with sending data and state machines. This opened up various opportunities for exploration, we narrowed down on this topic as we are working to develop this further for Project 2. The intent was to be able to mimic the movements of a conductor while synchronising the movements to a classical sound piece. After understanding the basics of working using this method, we were able to create a program on processing to send mouse data and trigger the volume levels and speed of the soundtrack.



Leave a Reply

Your email address will not be published. Required fields are marked *