Kinect and Digital Media Arts

Lee, H. Kim, J. & Lee, W. (2014) Interactive Digital Art based on user’s Physical Effort with Sensor Technology. Chaung-Ang University. China.

Three types of interaction:

  • “The audiences are able to watch themselves directly so they can build up their expanded ego and they manipulate the digital environment through the feature of themselves in a virtual space.” [Lee, Kim, Lee. 2014:3]
  • “For the second type of art work is the audiences comes up as abstraction shapes like silhouette or shadow in art works.” [Lee, Kim, Lee.2014:3]
  • “The last type is the audience’s body does not reflect on an art work directly, however, the art work react on the audience’s movement in a real-time to develop a natural feedback process and interaction with the audience and the art work.” [Lee, Kim, Lee.2014:3]



Castro, B. Velho, L. & Kosminsky, D. (2012) Integrate: Digital Art Using Body Interaction. Instituto Nacional de Matemática Pura e Aplicada. Brazil.

  • “In our relationship with the computer the sense of sight is usually emphasized and we often hear discussions about visual media, but these boundaries aren’t that clear anymore. All media are mixed and that perception is never through one sense only.” [Castro. Velho. Kosminsky.2012:2]

Making Gestures Project – Set Up [Part One]

Office space for the week. – Faithful Cat on hand.

With new technologies coming along every day, the ability for people to interact with the digital world, becomes easier and easier. Released by Microsoft in 2010, the Kinect has become an interesting tool for digital artists to experiment with, and thanks to its open source programming, programmers and artists are finding it easier to Implement the device within their projects.

With our ‘Making Gestures’ project, I decided to experiment with the device myself, using the software VVVV. Something I have briefly used before and wanted to get my hands on again to experiment with a different device. The Kinect, (happily on loan from my sister (Thank you Kel)) requires a large amount of space for its use, making the ‘lab’ for this week, my living room rather than my office.

Unreliable Experience Project [Part Three] – Moving backwards.

The idea of using text in place of full graphics is an interesting one, harkening back to text based adventures such as Colossal Cave Adventure [Crowther.1976], and Zork [Infocom.1977]. While this is an interesting idea, the fact of the matter remains in how do we implement it. Older text based games were programed to recognise certain phrases or commands, there is little time to implement such a feature here. Some early precursors (Though their exact names are escaping me right now), used a predetermined list that a player could choose from. While this would limit the players choices in a very obvious way, it would also be a change to impose a humorous tone, as originally intended.

However, this could be a good exercise in experimenting with fonts and colour per character, much in the vain of Five Nights at Freddy’s 4 [Cawthon.2015], and Undertale [Fox.2015]. This could be the better option, as the correct choice in font and colour could give a better impression for a villain than an image. Keeping them hidden, leaving the player to imagine their own personal villain. A look back through Why Fonts Matter [Hyndman.2015] will serve well here.

  • Cawthon, S. (2015) Five Nights at Freddy’s 4. [Programme] Microsoft Windows. Scott Cawthon Games.
  • Crowther, W. (1976) Colossal Cave Adventure. [Programme] Commodore 64. Washington: Microsoft Studios.
  • Fox, T. (2015) [Programme] Microsoft Windows. Toby Fox Games.
  • Hyndman, S. (2015) Why Fonts Matter. Random House: London.
  • (1977) Zork: The Great Underground Empire – Part I. [Programme] PDP-10. California: Activision.
  • Lammle, R. (2014) Eaten by a Grue: A Brief History of Zork. [Online] Mental Floss. June 16th Available from: [Last Accessed: 25/10/2016]

Unreliable Experience Project [Part Two] Creation Frustration


Originally I conceived the game to work as a side scroller, similar to some of my previous projects and inspired by games such as Super Mario Bros. and Sonic the Hedgehog. However, despite having a basic character created and a previous set of walk cycle templates on hand, I seem to have become unable to create any form of usable asset, leading to deep frustration.

However, this is maybe a setback, but it is also an opportunity to try something new. There are possible answers to this problem, that can take multiple form. photo-23-10-2016-12-38-22One, can we use something other than Unity? Something else at our disposal that is familiar. Two, does it have to be a side scroller? Point and click maybe? Three, can we draw on something else? Use text maybe? What about a point and click text based interface, where the game is lying to you about that the object actual is? What if the object is described but its label is completely different? Or, you are reading the text from someone else’s point of view, someone who wants to make you believe you are the hero while you are actually doing their bidding? Interesting…


Scott McCloud at TED

Scott talks a lot about vision, in both the physical sense and our visions for the future in regards to the arts, media, technology and comics. “Learn from Everyone, Follow no one, Watch for Patterns, Work like Hell.” Thinking about a scientific mind in an artistic field, attempting to make something and understand something at the same time. In regards to comics, despite them being a purely visual medium, they attempt to replicate all five senses. Something that innovation in the digital age could bridge the gap with in later years (e.g. Motion comics, interactive comics). The talk also goes through a history of visual narratives in print, from hieroglyphics, and tapestries, to modern day comics and digital comics. As well as talking about McLuhan-esc mistakes when transferring comics from print to screen, as they are attempting to recreate a previous medium in a new one, ignoring what the new medium can do. This talk borrows heavily from McCloud’s previous books Understanding Comics and Reinventing Comics.

I find this talk useful as it shows McCloud openly discussing and elaborating on his theories in a more open fashion, providing examples for each point he makes and going through the work with him to better understand his point. Since my focus is geared towards media theory in the realm of Comics, TV, Film, literature, and theatre, this elaboration of McCloud’s thoughts is interesting to behold.

Unreliable Experience Project [Part One]

After doing some research in 3 and 5 act structures, as well as Freytag’s pyramid, It became slightly easier than usual to attempt to construct a basic story, as I had a better understanding of what makes a story, including the ‘inciting incident’ and the ‘crisis point’.

As a comics fan, I wanted to build on my love of heroes, but I also wanted to mirror some aspect of our own world, So, I decided to structure the story as an allegory for our use of social media. How we all take centre stage as the heroes of our own story online, but in reality we are all just as important as each other, and can have just as much of an impact as anyone else. Good or bad.

I was thinking that Act 1 would work as an introduction, getting to know the world and building up the fact that your character is the hero. The inciting incident would be inhabitants of the world beginning to lie to you about how important you are, putting doubt into the players mind. Act 2 is a journey across the stage, building up a sense of importance and power, and the crisis point hammering in the fact that you have been lied to, that you are not the hero at all, that you are just like everyone else. Finally Act 3 is joining with other would-be heroes to actually make a difference, defeating the final boss.

Digital Semiotics – Oscar Bastiaens

Stemming from the field of linguistics, semiotics is the study of signs and the meanings we attach to those signs, an expression of man and animal behaviour. With birth of the internet age, and especially virtual reality, whole new breeds of signs are created that are unique to that environment. The existence of these signs are growing simply by the digital spaced being used and evolving. When it comes to virtual reality, semiotics takes on a whole new role. Recreating, or just creating, worlds that seem real to the user, will require a firm understanding of semiotics, an understanding of why a symbol or sign is the way it is and what it means to us. A miss communication of these signs, may be enough to break immersion for the user and create an incomplete experience due to this break in the imposed reality.

Semiotics is an important aspect in the realm of media theory, but I feel that it is important to understand that how we view signs in the physical world is quite different to how they are perceived in the digital. While this talk does not go very far in depth with this matter, it does provide a basis of understanding and some useful examples to keep in mind.

  • TEDx Talks (2015) Digital Semiotics: Making Sense of the World | Oscar Bastiaens | TEDxDordrecht. [Online Video] February 20th. Available from: [Last Accessed: 21/10/16]

John Yorke – Into The Woods: How Stories Work and Why We Tell Them. [Notes]

Yorke’s book provides a wonderful exploration of what makes a story, exploring how they are structured and giving some very helpful and interesting examples. While different media can tell stories in different ways, understanding the underlying principle is more than important regardless of the medium being discussed.

Three Act Structure:

“Everything must have a beginning, middle and end.” [Yorke. 2013:26]

Syd Field – First to articulate the 3-act paradigm, breaking the structure down to:

  • Set-Up,
  • Confrontation,
  • Resolution.

With Turning Points between the first and second (the Inciting Incident), and the second and third (the Crisis).


“It’s a model that lies behind all modern mainstream film and TV narratives. Contrary to the perception of many, though, it wasn’t invented by Field. One Only has to read Rider Haggard’s novel King Solomon’s Mines, written in 1885 and so clearly an antecedent of Indiana Jones, to see the structural prototype of the modern movie form.” [Yorke. 2013:26]

  • The Technique of the Photoplay. By Epes Winthrop Sargent. – First screenwriting manual.

Dramatic Structure:

  • Act One: Thesis
  • Act Two: Antithesis
  • Act Three: Synthesis

Hollywood Structure:

  • Act One: Establish a flawed character.
  • Act Two: Confront them with their Opposite.
  • Act Three: Synthesize the two to achieve balance

“From thesis to antithesis, from home to a world unknown.” [Yorke.2013:29]

“That’s what inciting incidents are too – they are ‘explosions of opposition’, structural tools freighted with all the characteristics the characters lack; embodiments, indeed, of everything they need.” [Yorke.2013:29]

“Cliff-hangers, inciting incidents and crisis points are essentially the same thing: a turning point at the end of an act; the unexpected entry point for the protagonists into a new world; bombs built from the very qualities they lack which explode their existing universe, hurtling them into an alien space of which they must make sense.” [Yorke.2013:29]

“Storytelling, then, can be seen as a codification of the method by which we lean – expressed in a three act shape.” [Yorke.2013:29]


Three Act Structure and Five Act Structure.

“It’s important to underline that a five act structure isn’t really different to a three act structure, merely a detailed refinement of it, and historically of course both forms can be traced back to the ancients.” [Yorke.2013:33]

Example – Comparing Shakespeare and Polanski’s Macbeth:


Freytag’s Pyramid.

Gustav Freytag

“In 1863, in his epic Technique of the Drama, he gave the world ‘Freytag’s pyramid.” [Yorke.2013:36]

photo-20-10-2016-17-28-381) Exposition – We meet the protagonist, and time and place are established.

2) Complications – Actions are complicated. “Events accelerated in a definite direction. Tension mounts, and momentum builds up.” [Yorke.2013:37]

3) The Climax of the action – Conflict reaches its high pint. Protagonist stands at a crossroads, victory or defeat.

4) Falling Action – Consequences of the climax. “Momentum slows, and tension is heightened by false hopes/fears. If it’s a tragedy, it looks like the hero can be saved. If [It’s not], then it looks like all may be lost.” [Yorke.2013:37]

5) Catastrophe – Conflict resolved. Either through catastrophe, downfall of the protagonist, or victory.

Freytag places emphasis on the midpoint of the story.

– Christopher Booker – The Seven Basic Plots.

  • Booker, C. (2005) The Seven Basic Plots  Continuum.
  • Haggard, R. (1885) King Solomon’s Mines. Cassell and Company. London
  • Shakespeare, W. (2016) Royal Shakespeare Company. London.
  • Yorke, J (2013) Into the Woods. How Stories Work and Why We Tell Them. Penguin Books. St. Ives.
  • Macbeth (1971) Film. Directed by Roman Polanski. [Blu-Ray] US: Caliban Films.
  • Raiders of the Lost Ark. (1981) Film. Directed by Steven Spielberg. [Blu-ray] US: Paramount Pictures.

Making it Real Project – Final Print [Part Four]

photo-19-10-2016-12-56-09Picking up the final product this morning, getting to see the 3D print for the first time, was a wholey facinating experience. While I had obvious seen the product in its digital form during its design, seeing something physically, which had only been rendered digitally before, was somewhat unique.

Studying the artifact, revieled small imprints from the printer, detailing its construction through physical memory. Imparting a map of how a machine persieves a physical object in layers.