By: Victoria Gottardi, Yifan Xu, and Firaas Khan

What are We Investigating? 

For Devs 2, we wanted to experiment with setting up some objects of different heights on the table, and try to discover how the robot arm could be capable of filming these objects to our precise measurements. We also wanted to experiment with how the gripper moves while holding the camera. For project one, we are planning on creating an advertisement, so we must be very strategic with our shots. We want to make sure the robot gives us the most affordances when creating really clean and usable shots. The positioning and movement of the phone and gripper will be crucial to getting what we’ll need.

The Process

Here is the link to the first attempt

Here is the link to the second attempt

Code

Here is the link to attempt one’s code

Here is the link to attempt two’s code

What We Learned

For this experiment, our main takeaway was that the gripper does not spin 360 degrees, so in order to move it in the correct position, you must make sure it is not fully turned one way to the max. We also learned that spinning the gripper too much can actually take away from the shot and potentially be visually nauseating to the viewer. In these experiments, we also played with the time of each waypoint, we discovered that the best length of time was 3-4 seconds and it appears that this amount of time could capture our desired items effectively. However, we think the length of time of each waypoint might also depend on what we are planning to show with the shot, and how we want the viewer to view and understand the shot.

Description of what you are investigating

As we talked about in Dev1, we have had an obvious clue since that time, we want to make an experimental film which is based on robot technique. After so many discussions, we gradually defined the concept of our film. Before the first test, we got some feedback, such as some environment setup and shooting angles, so we pay more attention this time.

In this section, we are going to know how to control the camera which can maintain our script of the story. And we also want to try how people can catch the movement of the camera and collaborate with it. So we decide people/cameras follow each other’s movement to shoot a suitable video.

Video

https://vimeo.com/manage/videos/795793529

What is it you are trying to figure out?

In the previous exercise, we moved the robot’s arm in a circular direction, and the desired subject (plate and candle) was in a fixed place.  But at this stage, we implemented the zigzag movement for the robot arm, which was in four movements to the right and left.  Also, the subjects intended for filming were moving, we also considered different speeds and stops at some points to coordinate the movement of the robot’s arm and determined the coordination between the robot’s movement and the hand’s movement.
Since the robot was not specifically designed for filming, the filming process was more challenging and time-consuming.
Therefore, our first challenge was to control and coordinate the movement of the hand and time and the movement of the robot in different directions.
Also, this project had to be filmed within the boundaries of the table with environmental restrictions, so the final sequence where the crumpled paper was thrown should not be outside the boundaries of the table and we repeated it many times to achieve the desired result.

Processing

1. Drawing the step of robot movement before we shot the video.

We became more organized this time, we wrote down the script and drew the camera movement before we do the shooting. We think it is really a good way to do our plan on a robot camera. Because we should set up the movement in the programme, so we should know every move is what we want.

2. Showing the paper: We decided to show the paper from far to close, which can get more feeling about the story will begin.

3. Write down some words: We set up 4 movements like Z, and the actor just follows the movement to write down the 4 words.

4. Close-up shot. To show the paper ball in order to let the audience know the emotion of the actor.

Describe what you learned

In this teamwork, According to the specific details of the video and filming project, we designed the movement path and waypoints. Filming with the help of a robot arm can be a unique experience because the robot moves along the specified path with high precision and control without vibration, which is difficult to achieve with human camera operators.

1. Improving camera movement: In this exercise, we were able to set smooth, accurate and repeatable camera movements, which were easier to take complex shots.  This is especially useful in film and video production where the ability to control camera movement is essential.
2. Advanced Versatility: We programmed the path programming and precise selection of waypoints to perform certain movements and move it along the filming path with precise timing, which is difficult to do with traditional camera setups.  This versatility is especially useful for shooting videos that require fast, smooth and precise camera movement. It can also save time and effort and reduce the risk of error or mistake because a human operator is not required.
Overall, the experience of using a robot for filming was very interesting for us, and this path has the potential to revolutionize the film and video production industry and offers a wide range of advantages that can improve the quality, efficiency and safety of filming.  to improve

How did it go?

In the previous exercise, we moved the robot’s arm in a circular direction, and the desired subject (plate and candle) was in a fixed place.  But at this stage, we implemented the zigzag movement for the robot arm, which was in four movements to the right and left.  Also, the subjects intended for filming were moving, we also considered different speeds and stops at some points to coordinate the movement of the robot’s arm and determined the coordination between the robot’s movement and the hand’s movement.
Since the robot was not specifically designed for filming, the filming process was more challenging and time-consuming.
Therefore, our first challenge was to control and coordinate the movement of the hand and time and the movement of the robot in different directions.
Also, this project had to be filmed within the boundaries of the table with environmental restrictions, so the final sequence where the crumpled paper was thrown should not be outside the boundaries of the table and we repeated it many times to achieve the desired result.

Group 3: Wentian, Nicky, Shipra, Gavin

Idea

For Dev 2, we wanted to test out two specific things – the camera’s range of movement and the ability to program using Mimic within Maya. We took into consideration the visuals and images that we hoped to eventually achieve for Project 1 and made it a worthwhile experiment for Dev 2. We knew that we wanted the robot arm to zoom and pan across an actor at a desk, so we used that setup to create keyframes in Mimic for Maya, and then recorded the resulting animation. By doing so, we investigated the capacity for Mimic to help with the preproduction stage of the project, by helping us to visualize the final output of shot sizes and angle. Furthermore, we were able to see how intuitive Mimic can be for programming the robot, as opposed to the manual input that we’ve used for prior testing.

Process and Results

After settling the thoughts, we started exploring and programming in Mimic. In the beginning, it took us a while to learn all the fundamental terminologies and techniques so that we could have a basic understanding of how to create our intended animation. Then we built some essential setups in the Mimic such as the robot arm, table, and background. After that, we embedded the camera from Maya into the robot arm and adjusted it to the proper position. Considering our initial idea involved an actor in that scenario, we also downloaded a human model from Sketchfab and placed it in front of the camera, which is not functional but just to fit into the scene we created. Finally, we programmed the robot arm with certain movements inspired by our early ideation, and it turned out to work quite well.

Learnings & Outcomes

This process helped in understanding the relationship between maya, the robot arm and the scene. It was a step towards familiarizing ourselves with the tools and controls. The intent was to be able to test the shots/ scenes using Maya as a preliminary trial for Project 1. The outcomes are two videos, one displaying the scene, the actor and the robot arm holding the phone camera; The other from the camera’s point of view, displaying the shot. This helped in understanding timing, movement, and angles. This exercise was able to add value to our Project 1 prototyping process.

3D Model:

“SF_Girl” by Stan, February 10, 2023

https://sketchfab.com/3d-models/sf-girl-a219070284e64ddd8a51f69c207cc81

Video Link:

https://drive.google.com/file/d/1mQIW6Zh_D0KTh51CO6dtcA8hf1Mf6JiR/view?usp=sharing

https://drive.google.com/file/d/1saL-Pwhs0dbql0qG0pFilpfRaSnBBCj3/view?usp=sharing

Group: Dorothy Choi, Jiamin Liu, Mona Safari, Zaheen Sandhu

Description of what we are investigating:

In continuation from Dev 1, our theme is creating a simple, cinematic nature scene that involves spatial and temporal changes, such as day to night transitions and playing with the weather. This time we decided to explore a bit more with some characters for our story and the environment. We wanted to play around with light and shadow, and make use of different materials that produce shadows in an exciting way. To do this, we used everyday objects that we had handy with us, such as a blue water bottle and a flashlight on a mobile phone to give the watery effect on a grey back board.

What we learned / How it went:

In this group work, continuing from Dev 1, we were able to further develop details to the nature scene for filming, and movements of the robotic arm for different filming positions.

1. Learned to manipulate different speeds of the robot arm to capture desired video effects, different frames of reference and focus points.
2. Learned that random exploration, or just ‘trying things out’ with both the robot arm and the nature scene helps ideate and create interesting scenes to film. For example: changing the order of objects in which they are placed, or testing different focus points for lights and shadows and how it affects the scene (e.g. light from the top puts a spotlight on the characters, but the shadow of the robot arm hinders the scene), and how the robot arm can impact the filming process (e.g. producing a consistent water effect required the producers to move in sync with the speed of the robot arm).
3. Played with light and shadow, and learned how it can impact a scene in terms of its ambiance, storytelling and potential message for the audience.

Our Challenges:

One of the main challenges that we faced while we were setting up our scene was having the surrounding environment in the video as well. We noticed that it was tough to be able to record the scene and not have the immediate environment exposed to the camera. To tackle this, we decided to create walls of thick sheets that would solve this purpose. While this did in fact solve our main challenge, it also acted like a bonus element for our project. We wanted to play with light and shadow and the sheet acted as a great background for that purpose.

Our Tests:

• Devs 2 – Explorations

Description of what you are investigating

In this mission, we study the control of the robotic arm and establish the operating range to carry out the automatic shooting. The controllability of the robotic arm has inspired us to use it in experimental video production. An experimental film is a film with a single objective language, without narration or subversion of traditional storytelling techniques. It mainly creates short films and closely relates to surrealism, expressionism, and avant-garde art. The methods used in the experimental film include defocusing, distortion, staining, repetition and quick editing of the image. Furthermore, unsynchronized sound and image, voice variations, grotesque characters and vague themes are also common styles in experimental movies.

To achieve a unique camera lens in experimental film, we think robotic arms are a good starting point for research. So we would like to test this technique on our first mission.

Video

https://vimeo.com/manage/videos/795793529

Processing

For our project, there is a frame which we will use in the edited video to show whether the candle was lit or extinguished. It is essential for this Dev1 to be familiar with the robot arm because the robot arm can complete precisely the same movement every time. Then we can have 3 similar frames, but the candles are different.

1. Turn around the empty plate. First, we marked 6 points around the plate. Then we adjusted the angle of the robot arm by hand at the first point and set it to be Waypoint 1. Besides, we moved the robot arm to each of the remaining 5 points via the touch screen to control the robot arm and turned it back to the first point. This makes the camera track around the plate 360 degrees. Thus we set up the robot arm’s trajectory and started recording the first frame – the camera circling the empty plate.

2. Three candles on the plate. We put three candles into the plate when the robot arm was circling the second time.

3. Lit three candles on the plate. We controlled the robot to stop at Waypoint2, Waypoint4 and Waypoint6 by the touch screen. Then another person can have time to light the candles.

Reference

1. YouTube. (2006). YouTube. Retrieved February 3, 2023, from https://www.youtube.com/watch?v=PJw9az9vAf4.
2. Matt. (2022, September 25). What is experimental film: The Essential Guide • Filmmaking Lifestyle. Filmmaking Lifestyle. Retrieved February 3, 2023, from https://filmlifestyle.com/what-is-experimental-film/

Dev 1 assignment

Group: Dorothy Choi, Jiamin Liu, Mona Safari, Zaheen Sandhu

Description of what we are investigating:

From researching creative inspirations that we found from Behance, Instagram, and the web, we were inspired by cinematic themes in nature, miniatures, and light imagery. Our brainstorming led us to investigate how to create a simple, cinematic nature scene (e.g. animals and a cottage at the top of a hill in the countryside, near the ocean) that involves spatial and temporal changes, such as day-to-night transitions and playing with the weather (e.g. inducing fog, precipitation). We used Miro as a collaborative space to note ideas and resources that would be helpful in creating this scene. Additionally, we experimented with the Robot arm to get a sense of how it can be used to film the scene. For example, the Robot arm could be useful in holding a phone to film the scene from a panoramic aerial view, panning over the scenic transitions. We are also thinking of minimizing post-production, focusing most work on the production process.

What we learned / How it went:

In this group work, we figured out the technical principles of the robot arms from zero to one, and complete the design of how to do interaction based on the technology of the robot arms.

1. Learned how the robotic arm works and used a robot arm to manipulate the behavioural trajectory of the robotic arm movement in the course.
2. Searching the internet for works related to robotic arms helped us to understand the possibilities of human-computer interaction, and also helped us to open our eyes and think in a broader way.
3. Learned from each other’s ideas in group discussions and brainstorming sessions with team members from different backgrounds.
4. Several techniques are tested for light effects, such as using a shiny surface to make reflections on objects.
5. For the project, we tested the possibility of changing the object’s position simultaneously with camera movements. Moving the objects and using a hand in the scene makes it feel like a theatre scene, which makes our set feel more alive!

Our Challenges:

1. As part of the test, we used the flashlight on our cellphones to see if it was able to make a sharp contrast on the object. This was not successful on the day, so we will need to work on that for the next session.
2. In order to make the scene look more detailed, we need to experiment with different backgrounds to see if we can create a blurry background, but the space we have is not enough to experiment with different sets, so we’ll need to figure out how to accomplish this.

Our Tests:

• Behind the Scene Test!
• Explorations

Resources:

• Felix Hernandez Dreamphography – Ciao Papa.
• Y S Unsplash Photo
• Ochir-Erdene Oyunmedeg
• Marie Sentier – MODELISME
• Ali Taha Y. – Miniature
• Hardi Saputra – Miniature in Nature

Devs 1 – By: Firaas Khan, Victoria Gottardi, and Yifan Xu

What are we investigating?

For this first assignment, our main goal was to experiment with the robot arm and a camera, since our first assignment is going to be between us and the robot holding the camera. We want to investigate how the robot arm moves with a phone and phone stand attached to the gripper, what the limitations of the movement could be, other capabilities of the robot arm movement that we might not have expected, and what a short video might look like filmed by the robot arm. We figured this would be good practice for project 1, to give us a better idea of what we would be potentially working with before we got too deep into the storyboarding process for our final video.

Process and Results

Final Video Link

What we learned

We first built a program that had the robot clutch the iPhone while recording a brief video introducing the computer that was on the table from all sides, in order to verify the hypothesis that a smart robot is capable of spinning 360 degrees and shooting while in motion. But it didn’t turn out that way. Due to space limitations and the length of its arm, which made it simple for things to collide with it, the robot was unable to complete the 360-degree rotation assignment. At that point, we noticed that when we built up the project, the space and the robot’s structure had not been taken into account. We updated our software to a successful half-circle spin after carefully observing and measuring the area and the robot’s construction. To sum up, through our initial interaction with the robot, we have discovered that the first thing we need to keep in mind when beginning a new program is the robot’s entire environment.

Link to Code