Description of what you are investigating

As we talked about in Dev1, we have had an obvious clue since that time, we want to make an experimental film which is based on robot technique. After so many discussions, we gradually defined the concept of our film. Before the first test, we got some feedback, such as some environment setup and shooting angles, so we pay more attention this time.

In this section, we are going to know how to control the camera which can maintain our script of the story. And we also want to try how people can catch the movement of the camera and collaborate with it. So we decide people/cameras follow each other’s movement to shoot a suitable video.

Video

https://vimeo.com/manage/videos/795793529

What is it you are trying to figure out?

In the previous exercise, we moved the robot’s arm in a circular direction, and the desired subject (plate and candle) was in a fixed place.  But at this stage, we implemented the zigzag movement for the robot arm, which was in four movements to the right and left.  Also, the subjects intended for filming were moving, we also considered different speeds and stops at some points to coordinate the movement of the robot’s arm and determined the coordination between the robot’s movement and the hand’s movement.
Since the robot was not specifically designed for filming, the filming process was more challenging and time-consuming.
Therefore, our first challenge was to control and coordinate the movement of the hand and time and the movement of the robot in different directions.
Also, this project had to be filmed within the boundaries of the table with environmental restrictions, so the final sequence where the crumpled paper was thrown should not be outside the boundaries of the table and we repeated it many times to achieve the desired result.

Processing

1. Drawing the step of robot movement before we shot the video.

We became more organized this time, we wrote down the script and drew the camera movement before we do the shooting. We think it is really a good way to do our plan on a robot camera. Because we should set up the movement in the programme, so we should know every move is what we want.

2. Showing the paper: We decided to show the paper from far to close, which can get more feeling about the story will begin.

3. Write down some words: We set up 4 movements like Z, and the actor just follows the movement to write down the 4 words.

4. Close-up shot. To show the paper ball in order to let the audience know the emotion of the actor.

Describe what you learned

In this teamwork, According to the specific details of the video and filming project, we designed the movement path and waypoints. Filming with the help of a robot arm can be a unique experience because the robot moves along the specified path with high precision and control without vibration, which is difficult to achieve with human camera operators.

1. Improving camera movement: In this exercise, we were able to set smooth, accurate and repeatable camera movements, which were easier to take complex shots.  This is especially useful in film and video production where the ability to control camera movement is essential.
2. Advanced Versatility: We programmed the path programming and precise selection of waypoints to perform certain movements and move it along the filming path with precise timing, which is difficult to do with traditional camera setups.  This versatility is especially useful for shooting videos that require fast, smooth and precise camera movement. It can also save time and effort and reduce the risk of error or mistake because a human operator is not required.
Overall, the experience of using a robot for filming was very interesting for us, and this path has the potential to revolutionize the film and video production industry and offers a wide range of advantages that can improve the quality, efficiency and safety of filming.  to improve

How did it go?

In the previous exercise, we moved the robot’s arm in a circular direction, and the desired subject (plate and candle) was in a fixed place.  But at this stage, we implemented the zigzag movement for the robot arm, which was in four movements to the right and left.  Also, the subjects intended for filming were moving, we also considered different speeds and stops at some points to coordinate the movement of the robot’s arm and determined the coordination between the robot’s movement and the hand’s movement.
Since the robot was not specifically designed for filming, the filming process was more challenging and time-consuming.
Therefore, our first challenge was to control and coordinate the movement of the hand and time and the movement of the robot in different directions.
Also, this project had to be filmed within the boundaries of the table with environmental restrictions, so the final sequence where the crumpled paper was thrown should not be outside the boundaries of the table and we repeated it many times to achieve the desired result.

Group 3: Wentian, Nicky, Shipra, Gavin

Idea

For Dev 2, we wanted to test out two specific things – the camera’s range of movement and the ability to program using Mimic within Maya. We took into consideration the visuals and images that we hoped to eventually achieve for Project 1 and made it a worthwhile experiment for Dev 2. We knew that we wanted the robot arm to zoom and pan across an actor at a desk, so we used that setup to create keyframes in Mimic for Maya, and then recorded the resulting animation. By doing so, we investigated the capacity for Mimic to help with the preproduction stage of the project, by helping us to visualize the final output of shot sizes and angle. Furthermore, we were able to see how intuitive Mimic can be for programming the robot, as opposed to the manual input that we’ve used for prior testing.

Process and Results

After settling the thoughts, we started exploring and programming in Mimic. In the beginning, it took us a while to learn all the fundamental terminologies and techniques so that we could have a basic understanding of how to create our intended animation. Then we built some essential setups in the Mimic such as the robot arm, table, and background. After that, we embedded the camera from Maya into the robot arm and adjusted it to the proper position. Considering our initial idea involved an actor in that scenario, we also downloaded a human model from Sketchfab and placed it in front of the camera, which is not functional but just to fit into the scene we created. Finally, we programmed the robot arm with certain movements inspired by our early ideation, and it turned out to work quite well.

Learnings & Outcomes

This process helped in understanding the relationship between maya, the robot arm and the scene. It was a step towards familiarizing ourselves with the tools and controls. The intent was to be able to test the shots/ scenes using Maya as a preliminary trial for Project 1. The outcomes are two videos, one displaying the scene, the actor and the robot arm holding the phone camera; The other from the camera’s point of view, displaying the shot. This helped in understanding timing, movement, and angles. This exercise was able to add value to our Project 1 prototyping process.

3D Model:

“SF_Girl” by Stan, February 10, 2023

https://sketchfab.com/3d-models/sf-girl-a219070284e64ddd8a51f69c207cc81

Video Link:

https://drive.google.com/file/d/1mQIW6Zh_D0KTh51CO6dtcA8hf1Mf6JiR/view?usp=sharing

https://drive.google.com/file/d/1saL-Pwhs0dbql0qG0pFilpfRaSnBBCj3/view?usp=sharing

Group: Dorothy Choi, Jiamin Liu, Mona Safari, Zaheen Sandhu

Description of what we are investigating:

In continuation from Dev 1, our theme is creating a simple, cinematic nature scene that involves spatial and temporal changes, such as day to night transitions and playing with the weather. This time we decided to explore a bit more with some characters for our story and the environment. We wanted to play around with light and shadow, and make use of different materials that produce shadows in an exciting way. To do this, we used everyday objects that we had handy with us, such as a blue water bottle and a flashlight on a mobile phone to give the watery effect on a grey back board.

What we learned / How it went:

In this group work, continuing from Dev 1, we were able to further develop details to the nature scene for filming, and movements of the robotic arm for different filming positions.

1. Learned to manipulate different speeds of the robot arm to capture desired video effects, different frames of reference and focus points.
2. Learned that random exploration, or just ‘trying things out’ with both the robot arm and the nature scene helps ideate and create interesting scenes to film. For example: changing the order of objects in which they are placed, or testing different focus points for lights and shadows and how it affects the scene (e.g. light from the top puts a spotlight on the characters, but the shadow of the robot arm hinders the scene), and how the robot arm can impact the filming process (e.g. producing a consistent water effect required the producers to move in sync with the speed of the robot arm).
3. Played with light and shadow, and learned how it can impact a scene in terms of its ambiance, storytelling and potential message for the audience.

Our Challenges:

One of the main challenges that we faced while we were setting up our scene was having the surrounding environment in the video as well. We noticed that it was tough to be able to record the scene and not have the immediate environment exposed to the camera. To tackle this, we decided to create walls of thick sheets that would solve this purpose. While this did in fact solve our main challenge, it also acted like a bonus element for our project. We wanted to play with light and shadow and the sheet acted as a great background for that purpose.

Our Tests:

• Devs 2 – Explorations

Description of what you are investigating

In this mission, we study the control of the robotic arm and establish the operating range to carry out the automatic shooting. The controllability of the robotic arm has inspired us to use it in experimental video production. An experimental film is a film with a single objective language, without narration or subversion of traditional storytelling techniques. It mainly creates short films and closely relates to surrealism, expressionism, and avant-garde art. The methods used in the experimental film include defocusing, distortion, staining, repetition and quick editing of the image. Furthermore, unsynchronized sound and image, voice variations, grotesque characters and vague themes are also common styles in experimental movies.

To achieve a unique camera lens in experimental film, we think robotic arms are a good starting point for research. So we would like to test this technique on our first mission.

Video

https://vimeo.com/manage/videos/795793529

Processing

For our project, there is a frame which we will use in the edited video to show whether the candle was lit or extinguished. It is essential for this Dev1 to be familiar with the robot arm because the robot arm can complete precisely the same movement every time. Then we can have 3 similar frames, but the candles are different.

1. Turn around the empty plate. First, we marked 6 points around the plate. Then we adjusted the angle of the robot arm by hand at the first point and set it to be Waypoint 1. Besides, we moved the robot arm to each of the remaining 5 points via the touch screen to control the robot arm and turned it back to the first point. This makes the camera track around the plate 360 degrees. Thus we set up the robot arm’s trajectory and started recording the first frame – the camera circling the empty plate.

2. Three candles on the plate. We put three candles into the plate when the robot arm was circling the second time.

3. Lit three candles on the plate. We controlled the robot to stop at Waypoint2, Waypoint4 and Waypoint6 by the touch screen. Then another person can have time to light the candles.

Reference

1. YouTube. (2006). YouTube. Retrieved February 3, 2023, from https://www.youtube.com/watch?v=PJw9az9vAf4.
2. Matt. (2022, September 25). What is experimental film: The Essential Guide • Filmmaking Lifestyle. Filmmaking Lifestyle. Retrieved February 3, 2023, from https://filmlifestyle.com/what-is-experimental-film/

Dev 1 assignment

Group: Dorothy Choi, Jiamin Liu, Mona Safari, Zaheen Sandhu

Description of what we are investigating:

From researching creative inspirations that we found from Behance, Instagram, and the web, we were inspired by cinematic themes in nature, miniatures, and light imagery. Our brainstorming led us to investigate how to create a simple, cinematic nature scene (e.g. animals and a cottage at the top of a hill in the countryside, near the ocean) that involves spatial and temporal changes, such as day-to-night transitions and playing with the weather (e.g. inducing fog, precipitation). We used Miro as a collaborative space to note ideas and resources that would be helpful in creating this scene. Additionally, we experimented with the Robot arm to get a sense of how it can be used to film the scene. For example, the Robot arm could be useful in holding a phone to film the scene from a panoramic aerial view, panning over the scenic transitions. We are also thinking of minimizing post-production, focusing most work on the production process.

What we learned / How it went:

In this group work, we figured out the technical principles of the robot arms from zero to one, and complete the design of how to do interaction based on the technology of the robot arms.

1. Learned how the robotic arm works and used a robot arm to manipulate the behavioural trajectory of the robotic arm movement in the course.
2. Searching the internet for works related to robotic arms helped us to understand the possibilities of human-computer interaction, and also helped us to open our eyes and think in a broader way.
3. Learned from each other’s ideas in group discussions and brainstorming sessions with team members from different backgrounds.
4. Several techniques are tested for light effects, such as using a shiny surface to make reflections on objects.
5. For the project, we tested the possibility of changing the object’s position simultaneously with camera movements. Moving the objects and using a hand in the scene makes it feel like a theatre scene, which makes our set feel more alive!

Our Challenges:

1. As part of the test, we used the flashlight on our cellphones to see if it was able to make a sharp contrast on the object. This was not successful on the day, so we will need to work on that for the next session.
2. In order to make the scene look more detailed, we need to experiment with different backgrounds to see if we can create a blurry background, but the space we have is not enough to experiment with different sets, so we’ll need to figure out how to accomplish this.

Our Tests:

• Behind the Scene Test!
• Explorations

Resources:

• Felix Hernandez Dreamphography – Ciao Papa.
• Y S Unsplash Photo
• Ochir-Erdene Oyunmedeg
• Marie Sentier – MODELISME
• Ali Taha Y. – Miniature
• Hardi Saputra – Miniature in Nature

Devs 1 – By: Firaas Khan, Victoria Gottardi, and Yifan Xu

What are we investigating?

For this first assignment, our main goal was to experiment with the robot arm and a camera, since our first assignment is going to be between us and the robot holding the camera. We want to investigate how the robot arm moves with a phone and phone stand attached to the gripper, what the limitations of the movement could be, other capabilities of the robot arm movement that we might not have expected, and what a short video might look like filmed by the robot arm. We figured this would be good practice for project 1, to give us a better idea of what we would be potentially working with before we got too deep into the storyboarding process for our final video.

Process and Results

Final Video Link

What we learned

We first built a program that had the robot clutch the iPhone while recording a brief video introducing the computer that was on the table from all sides, in order to verify the hypothesis that a smart robot is capable of spinning 360 degrees and shooting while in motion. But it didn’t turn out that way. Due to space limitations and the length of its arm, which made it simple for things to collide with it, the robot was unable to complete the 360-degree rotation assignment. At that point, we noticed that when we built up the project, the space and the robot’s structure had not been taken into account. We updated our software to a successful half-circle spin after carefully observing and measuring the area and the robot’s construction. To sum up, through our initial interaction with the robot, we have discovered that the first thing we need to keep in mind when beginning a new program is the robot’s entire environment.

Link to Code

Dev 1
Group 3: Wentian, Nicky, Shipra, Gavin

Idea
For Dev 1, we wanted to experiment with the robot arm’s spatial movement. Primarily, the goal was to test how the robot arm could handle a 360 degree encircling of an object.  We attached a phone to the gripper, utilizing the phone’s camera to take photos/videos of the object. Additionally, we also wanted to explore shooting angles such as a top-down perspective. In these experiments, we are trying to figure out how fluid the robot arm can be, after we’ve programmed a series of moving commands into the console. We observed the speed of the arm transitioning to different points, whether it would collide with the object, and the quality of the photographs/videos taken on the phone. This is prep work for our Project 1, where the camera’s perspective will be crucial. This will also be very useful for Dev 2, where we plan to use the robot to 3D scan an object.

Process
First, we placed our main character “Dead Panda” on the central area of the table beside the robot arm. To prevent the arm from reaching the maximum moving distance to the bottom, we put some wood blocks under the object to increase its height. Then we started to position the beginning waypoint of the robot arm and adjust the shooting angle of our camera. After all the initial preparation, we started our first attempt by inputting four waypoints that were programmed by each of us. Unfortunately, our first attempt failed. The gripper kept doing full rotations during the demonstration – a potential hazard for the object – because we were manually changing the positions and angles of the gripper. After another failed attempt, we realized we needed to control the gripper and arm on the pad rather than direct operation on the robot to resolve this issue. Although the camera collided with the object several times in movement due to inappropriate positioning of waypoints, we eventually accomplished the task and got a smooth video of an encircling shot of our main object.

Documentation

VIDEO OF DEV 1 <— Click

Learnings & Outcomes
While going through this process of trial and error while working with the robot arm, we were faced with a few of the aforementioned challenges. This process helped in understanding the limitations of working with the said technology. We realized the importance of working towards smaller goals helped in understanding the movements and familiarizing ourselves with the controls. The first step towards Project 1, resulted in a short 360 degree video of a small object placed on the table. The intent was to be able to create a smooth transition between camera angles and positions. We achieved that to an extent by working as a team art directing the scene. This exercise was able to add value to our Project 1 Ideation process.