KARATE – Experiment 5 – Book++


Team members: Dimitra Grovestine (3165616), Salisa Jatuweerapong (3161327), Siyue Liang (3165618)

January 17, 2019



Our group picked the book Karate: Questions And Answers.  It introduces the history, development, impacts and the essential practices of karate.


KARATE is a virtual karate simulator, inspired by Donald S. Bitanga’s non-fiction book, Karate: Questions and Answers. This hybrid game, installation, and performance piece prompts a meditative approach to the typically fast-paced sport.

The physical set-up consists of a modified bo staff (attached phone holder) and a single projection on the wall depicting a traditional Japanese dojo. Inside this virtual space lives our Karate Sensei, who bows before demonstrating a standard set of practice moves (katas). In front of the sensei is a floating virtual bo staff, which is mapped to the real-world bo staff. The player, standing in front of the wall projection, uses the physical bo staff to follow along and mimic Sensei’s movements in order to “learn their inner self and find peace within.”

The simulator is powered by Unity, and takes input from a smartphone gyroscope. 

*some Karate facts presented as truths should be taken with a grain of salt



Siyue was inspired by this video to create an experience with a staff and a phone’s built in gyroscope. Salisa was reminded of DF Grad’s River Styx, and we pulled up their documentation  in hopes of some pointers on their technical workflow. We found the extraction and duplication of an object from the virtual world into the physical interesting, especially when the object was able to interact within both realms, and wanted to use it in our project.


Our initial idea was to create a two-player fighting game using Unity–though we would put an interesting twist on it by having you fight your opponent in a virtual space (the wall in between, refer to diagram ). This would allow us to detect collisions and put in additional gameplay mechanics. After some experimentation and feedback from Nick, we found that with the lag between data and visuals, it would be difficult to make a satisfying fighting game. As well, the lack of haptic feedback despite on-screen collisions could perhaps cause confusion during gameplay.

This led us to shift into an alternate mindset for Karate: its meditative rhythm and all practitioners’ learnt affinity for peace, dedication, and hard work. As well, instead of having the virtual avatar be linked to Player 2’s movements, we decided to instead have a pre-animated master: presenting, Gichin Funakoshi.

Gichin Funakoshi, founder of Shotokan karate, is generally credited with having introduced and popularized karate on the main islands of Japan. In addition many Okinawans were actively teaching, and are thus also responsible for the development of karate on the main islands.

This also led to our creation of the intro and outro texts, the former which Dimitra wrote (paraphrasing the book) and the latter which is a poem lifted from the opening pages of the text. We focused less on gameplay and more on a single narrative. While the Sensei was unnamed in the final game, he will be Funakoshi in our hearts.

The last major iteration we had before the version we demoed in class is technically very similar, but much more performative in nature. It follows the original two-projection set-up (first paragraph). Here is the discarded description for that installation:

The set-up is two projections on either side of a standing wall, projecting a first person view of a traditional dojo on either side. With that, the wall is transformed into a portal to virtual space, where we situate the other half of our installation. In the physical world, two participants stand on either side of the wall and given one bo staff each that have been modified to hold a smartphone.

The two participants are told that one will be the “master” and one will be the “student”, but this information is not disclosed to the audience. (The “master” would likely be the one facilitating the experience). While their physical bodies are not visible to each other, they are told that the Sensei depicted on the projection screen is the avatar of the “master”. The “student” will learn the katas from the “master”. The audience watches this Karate lesson, but as they are free (and encouraged) to roam around the room and watch both participants, they soon realize that something is off– both performers appear to be following along to the “other”.

It is revealed that both particpants have been “students” and are following along to a pre-animated, virtual avatar that is the “master”. There are many threads of themes this runs along, the main one being a perceived superiority of virtual forms. The master was an “AI” all along; there is no real Karate master (this is effective when the lesson intro text promises to turn you into a Karate master in 1 minute. Just like the Youtube tutorials!). This could be followed into a commentary on the obsoleteness of physical texts, when it has been reduced to a projector stand and a virtual avatar holds all its knowledge.

This installation was inspired by the prompt of creating different levels of participation: the virtual avatar, the two real participants, and then the audience are all on different levels of perception. It is not till the end that the mystery is understood and the levels are unpacked. Like the Wizard of Oz! (but not really). 


We chose to create our experience in a Unity environment using smartphones as a wireless sensor (gyroscope). While all three of us had minimal experience with Unity, we were all interested in learning it and using it for future projects. Given the simple functionality we needed the PubNub part to have (access phone gyroscope and send values to Unity), we figured we would be able to do that and then dedicate time to learning Unity.

Another big reason we chose to use Unity was that Siyue and Salisa were both interested in doing 3D modelling and animation for this project. Based on past experiences, p5.js’ 3D capabilities have been disappointing and limited; our choice to use Unity allowed us to model and animate on Blender and export straight into Unity.

The learning curve was steep but overall worth it.



Weird bugs galore! That pretty much comes with the Digital Futures territory, though. With lots of help from Nick, we were able to resolve most of our networked code issues.

Something we overlooked that ended up taking a lot of time to work around was mapping the physical staff to the virtual staff. We had to adjust and tinker around with some of the gyroscope readings.

One issue that we are still unable to resolve is the floating bo staff in the final game. When the Blender animation file of the Sensei character holding his bo staff is imported into Unity, the bo staff does not properly retain its animation. Instead, it seems to have a different origin point and a different pivot point. Originally when the issue appeared, we believed it to be an origin point problem, which Nick suggested we fix by exporting the staff and the character separately. Unfortunately while this solved the origin problem it did not fix the pivot problem, which is why the stick appears to be controlled by the Force. When Salisa downloaded a free open source model/rig for the Sensei character, she modeled & rigged a bo staff separately which she then parented to a hand much like a third joint. It seems likely that the issue stemmed from this method of joining the rigs, but we’re still unsure at exactly what point (tutorials were followed).

The good news is that this happy accident makes our outro poem very literal; the first line is: “My hands are empty.” Literally. 



A UI/UX inspiration for us was Beat Sabre: how it felt to hold a stick in virtual space and have a satisfying time swinging it. This was also where we realized the necessity of haptic feedback (one of Beat Sabre’s greatest accomplishments, in Salisa’s opinion, is their haptic feedback). 

In a way, this alternative game controller was also inspired by previous projects, including Salisa’s Naruto Glove project, which was in turn inspired by D.VA Controllers. This extraction and replication of in-game objects to use as controllers is a common theme and explores a way to have a physical connection to a virtual space.

We were also inspired by the traditional graphics within the text. Siyue image-traced photos of the text on Illustrator then transferred them to Blender to turn them into 3D shapes that appear in our final project. The colour scheme of the UI was also originally taken from the text’s colour scheme, but it was not as pretty as we’d hoped.

As well, we binged a number of Karate master videos on Youtube.



  • Blender animation import to Unity glitch fix
  • Background music! It’s an immersive meditation experience. Simone pointed that out.
    • A change in setting (soft morning light, dimmer room, carpet flooring) could increase the immersion. As well, a larger screen.
  • Two-handed Blender rig for animation

CODE: https://github.com/haizaila/AtelierII 

YOUTUBE ( video & photo documentation ) : https://www.youtube.com/watch?v=M4QX9tMC1k8&










http://www.karatebyjesse.com/slow-karate-training/  (Design decision)

“Level Changes” : https://upload.wikimedia.org/wikipedia/commons/7/78/Gichin_Funakoshi_-_Heian_Nidan_%28Yoi%29.png



Leave a Reply