Designing For Interactivity
Title: Future Cinema (Digital Media Production) and Designing for Interactivity (Interactive Media)
Unit start 04/10/2010
Unit Handin 08/11/2010
Update the cinematic experience for the 21st creating a narrative based around subject of ‘us versus them’.
(who is us who is them? This will be discussed further in the briefing)
Note: By ‘cinema’ we mean the event that is cinema, not the genre. We are expecting you to produce a shared social experience. Cinema in this case means the act of going to a building with other people to ‘watch’ a recorded and composed narrative.
(this will be discussed further in the briefing)
For those specialising in Future Cinema (Digital Media Production) you will focus on exploring and expanding the cinematic experience and planning writing, directing and producing content for this context and planning the narrative and content.
For those specialising in Designing for Interactivity (Interactive Media) focus on exploring the technologies of immersion and interactivity, and the observation and prediction of behaviour, psychology, game play etc through video tracking, gesture recognition etc.
What is cinema? Is it the experience of a narrative in a shared public space? Can a shared cinematic experience include game play, can it challenge and require a response? Is this character lead? Game play, narrative, human behaviour character and plot, story telling etc.
Approaches might include looking at methods with facebook and twitter to develop narrative, using traditional improvisation techniques, or generative rule based methods for computer-based work.
Realtime video processing using MAXMSP, 3D video mapping using Modulate, Computer Vision using eyesweb, TSPS, CCV, VVVV, etc. Flash and Flash media server for collaborative applications.
Name: Mathew Brewer
Course: Digital Media Production Level: 5
Unit: Designing for Interactivity
Reference number: IMD516
Credit points: 22.5
Study hours: 225 hours
Tutor(s): Liam Birtles, Phil Beards, Amanda Stephenson and Bob Cotton
Synopsis of study
For the Future Cinema/ Designing for Interactivity Unit we will be working on an ‘Interactive Mirror’. The idea behind this is that the user can put themselves in the scenario of a film. We are focusing on the conversational element of the cinematic experience; what you discuss with your friends and the people you are with when you leave the cinema. As they enter the room they will be given a choice of ‘Genre T-Shirts’, which have fiducial markers printed on them, each of the markers will trigger a different animation to turn the wearer into a character from that genre. For example, there will be a horror t-shirt that will then turn the wearer into a zombie, or a western t-shirt that would turn the wearer into a cowboy. As our main prototype we will be creating a sci-fi shirt, turning the user into an alien, we are going to make sure that the image is animated on to the person instead of it just appearing, this will make it a little more interesting for the user and make a little bit of suspense in what is happening.
We will go about this by creating our own fiducial markers to be transferred onto t-shirts. Using the FLARToolKit we will ensure that when the wearer comes in front of the camera an animation of the character will begin, turning them into, for example, an alien. We will transfer our markers on to different T-Shirts so that a different T-Shirt will trigger a different animation. The characters for the animation will be designed and created in Adobe Illustrator, they will then be animated in Adobe After Effects and exported within an Alpha Channel to ensure that when they are triggered the user can see the background that is actually around them at the time, to make it seem as if the ‘cinematic experience’ was being created within their own reality. Once this has been done they will be synced up with the coding of the markers in Adobe Flash, this means that when they are presented in front of the camera they will project our animations as opposed to some kind of stock footage.
We have chosen to do this because we decided that the most important part of the cinematic experience is when you are able to discuss it afterwards, comparing with friends and/or family what you liked, what you disliked, how you felt and what you inferred from the given scenarios. By allowing the audience to talk at the same time as receiving the experience it can, in theory, make it more enjoyable for the audience (as talking is generally frowned upon in the cinematic environment you cannot discuss at the same time the ‘action’ is happening). Our idea aims to increase the social aspect of the cinematic experience and with more time we would like to be able to create it so that more than one character can be in the same screen at any time and they can interact with one another.
For this Project I will personally take the role of Designing for Interactivity, and explore the best way of delivering our concept of the idea to the audience. For this I intend to find the most viable way of creating the interaction by testing MAXmsp in conjunction with Reactivision and to also look at the FLARtoolkit. I have chosen the role of Interactive Designer because I feel that I would learn the most from this role within the project, as I will have to develop an understanding of the programming language that is used by each of these applications, Also as part of this process I will also learn about the production of Fiducial Markers and how to design them for optimal functioning capacity.
I will evidence my work through screen-captured footage of the examples I create whilst they are running. Once I have concluded which application best suits the project I will then progress towards making some demo versions, in conjunction with the other members of my group who will be working on the content and visual design, in order to display what the final idea would be like and how it would work.
A1 To refine and develop an understanding of the appropriate use of interactive
technologies in the application of media solutions.
A2 To research and develop design solutions addressing the users’ needs.
A3 Evaluate the use and development of a prototype.
A4 To demonstrate an understanding of the role of multimedia in contemporary
LO1 Investigate, analyze and design appropriate applications of interactive media.
LO2 Analyze existing solutions to similar problems and demonstrate an understanding of their effectiveness. Changes in technology should be reflected in the analysis.
LO3 Design and assemble an interactive prototype.
LO4 Analyze and contextualize multimedia communication forms and solutions in contemporary media
- Evidence of understanding through research and analysis. (c) Report on effective uses of multimedia and its role in contemporary media (LO1, LO4)
- Evidence of knowledge of technical processes (b) - Through detailed report and practical assignments (LO2, LO3)
- Evidence of realization through communication of solutions (f) – Final multimedia presentations demonstrating effective use of media (LO2, LO3)
- Evidence of realization through application of technical knowledge and skills. (g) - Presentation of final production, web blog, written report (LO3)
All reference material can be found on the project’s blog:
Action plan/time plan
We have agreed with our tutors that for this project we will be making a prototype of our idea. For this we will print one or two t-shirts with a tracker, one of them will go with our proposed idea of a genre t-shirt, and for the purpose of the prototype we will make it according to one genre (we have chosen to do the horror genre and make a zombie character). As well as this we will make a different version using the semiotics of a genre and get them to hover and move around the person, with the idea that in the future they could move around and interact with the person on the screen. Likewise in the future we would like to be able to get the genre character to move about with the wearer so that it can interact with other users at the same time.
We have broken down everything that needs to be done and been assigned jobs for what needs to be done and we have set days by which things need to be achieved by so we will finish in time with a working prototype to present at the deadline.
Day one :
The 10 Usability Heuristics (Jakob Neilsen)
1: Visibility of System Status
Keeping users informed about what is going on, that they are doing the right thing and everything is happening.
2: Match between the System and the Real World
The system should speak the users language, no technical jargon that the user may not understand.
3: User Control and Freedom
The ability to undo and action should the user have done something wrong or made the wrong decision.
4: Consistency and Standards
Following the same rules, standards and ideas throughout the whole system.
5: Error Prevention
Helping to prevent errors to occur, and when they do, there is a need to alert the user.
6: Recognition Rather Than Recall
The system should be easy to use and remember.
7: Flexibility and Efficiency of Use
Accelerators speed up the use of the system for users.
8: Aesthetic and Minimalist Design
No superfluous information, any technical jargon should be explained.
9: Help Users Recognise, Diagnose and Recover from Errors
Error messages should be given in plain language, clearly indicate the problem and give clear advise how to solve it.
10: Help and Documentation
Help and advice should be available should the user require it.
Mind Map One
Mind Map 2
Papervision 3D (augmented reality blog ftw)
Desmond Morris (Human Behaviour)
- Predicting the future of cinema and producing something.
- User centred design.
- coversation is the most important thing (for me anyway)
For me one of the best parts of the “cinematic experience” is the dialogue or conversation which is generally speaking undertaken after the event of the aforementioned “cinematic experience”. As a result of this the Idea I would like to explore as part of this project very much relates to the conversational element of this experience.
What I would like to do is set up a way in which the users/cinematic experiencers were encouraged to engague in this conversation whilst the cinematic experince was taking place. I think that this would be best achieved by removing the narrative aspect from the cinema.
My Idea at its core is to introduce an experience where the users will enter a room which; as in a traditional cinema; has a screen at one end.
However the difference with this and the standard cinema is that this room will be an empty space with no seating and the screen itself will act in a similar fashion as a mirror. But this will be a mirror with a twist, the twist being that aswell as showing the users a mirror image of themsleves in real time, they will also see images/animations which appear to be moving in and around them. Hopefully this will instantly spark the conversation between the users during the cinematic experience and allow them to create there own narrative and a much deeper experience should be achieved than simply passively watching a movie.
Group Work Begins
After talking it through with Cat Abott and Alice Lewis, We have decided to move away from the bugs/fear theme as Fear would probably have ended up being cheesey and therefore not inherently Fear, which is theme breaking. (also bugs have been/are being done by other people). The Idea is now to have T-shirts with large Fiducial Tracker’s printed onto them, each representing a stereotypical genre of movie. The wearer will then see themselves in the Funky Interactive Mirror with movie genre based items animating over them. Paper Tweet 3D shows that this is theoretically possible.
I Tested : Papervision 3D’s FLARtoolikt experiment today and it is worth noting that you need very good daylight conditions for it to work. Mid afternoon (dimming light) isn’t substantial enough for the process to work. at least not through the iSight camera on my macbook.
Reactivision on Vimeo(examples)
Art is Open Source (tutorial)
Paper Tweet3D Augmented Reality T shirts (youtube)
PaperTweet3D (Squidder) Augmented Reality Twitter Feed T shirts
Real Time Face Detection In Flash ( does what it says on the tin)
Open Processing (great source of code and tutorials for processing)
QR code Generator (open processing)
3D Sound Clash. uses minim to apply sound to motion.
The Idea (explained)
We believe that one of the best aspects of “the cinematic experience” is the conversational aspect which generally takes place after the event. It is our intention to create an interactive cinematic experience which encourages the participants to engage in conversation during the “the cinematic experience”.
We have, as a group, come to the conclusion that the best way to achieve this goal is to produce an experience which puts the traditional idea of narrative within the cinematic experience to one side/in the trash. Our concept is that the narrative will be created by the viewer/user(s) who take part in our Interactive Mirror.
The Interactive Mirror + T-Shirts
The Interactive Mirror is a project where a set number of viewers will be invited to choose and wear a T-Shirt which will be emblazoned with a large Fiducial Marker, which represents that favourite stereotypical genre of film/movie from a pre determined list. Once wearing this t-shirt the user/viewer(s) will enter the Funky Interactive Mirror Room which will be a predominantly empty gallery space with a large cinema style screen at one end, to the user/viewer(s) this will appear to be a mirror image of the room.
However Upon approaching the “Mirror” the viewer/User(s) will be amazed by the augmented reality animations which have suddenly began to take place upon them. (these animations will be genre specific and correspond to there choice of t-shirt).
MaxMSP vs FLARtoolkit
Having Initially thought MaxMSP in conjunction with reactiVision would be easier to make an Augmented Reality piece with, purely because it’s a more visual UI, therefore less code would be involved I soon realized that this was an error in my personal judgement. I would still love to find a way to complete this project using Max but with the time constraints and the much vaster aray of tutorials available for the FLARtoolkit I have decided to use that instead. Using the FLARtoolkit we also gain the benefit of being able to track in 3D space and not just on a 2D plain which is what we would have been limited to using Max/reactiVision.
Narrative reason/bit that makes the idea work:
Having researched a bit about the idea of semiotics, I have come across Carl Jung and Sigmund Freud, who developed the Idea of Archetypes, which to my understanding is the idea of Public/Shared Concepts of specific Archetypes.
“The archetype is a tendency to form such representations of a motif” -Carl Jung
“representations that can vary a great deal in detail without losing their basic pattern … They are indeed an instinctive trend” -Carl Jung
“the archetype is essentially that motif which contains meaning
and it’s the content that those motifs evoke that allows them to be public and shared”-Mark Brewer
For example in terms of this project taking the theme of Space Invader aliens. Although as a whole we (humans) can completely appreciate that these are merely graphic symbols which only represent the idea of aliens, they are all widely recognised as Aliens, because as a collective we all play the same “Game”, wether that be a visual game, a language game or as previously mentioned the same Shared/Public Concepts.
Also because with our project the viewer is told to think of specific theme’s then they will things/themes, then they will be given the preconceived notion/idea of a theme when choosing a T-Shirt which corresponds to a specific filmic theme, i.e: Alien movies, Cowboy+Indian movies or Zombie Movies, they will already be subconsciously imbuing the images that they are viewing with congruent archetypal concepts. Basically seeing what they are told to see within the imagery.
Youtube: Tests with our demo t-shirt Custom Animation (demo marker)
Custom Marker and Custom Animation Demo Vimeo:
Overall I am happy with the outcome of this project. From the outset I was unsure how successful the results would be as I have never attempted to produce anything remotely like an Augmented Reality before, So I had quite a steep learning curve to over come and a little knowledge of how to set about starting the project. However this initial lack of knowledge did lead to some interesting experimentation with multiple software packages before I finally settled upon using the FLARToolkit.
After Initially looking at Processing, Reactivision in conjunction with MAXmsp and FLARToolkit, we decided as a group that we would first attempt to produce the project using MAX and Reactivision, mainly because MAX uses a very visual UI which appealed to my initial fear of FLARToolkit’s more objective C/heavy code based programming language. Upon further experimentation it became apparent that FLARToolkit would track imagery far more successfully within 3D space. It also meant that we would only require the use of one webcam/camera for the whole project to operate. The Augmented Reality aspect of the Archetypal Reflectomatic operates entirely within Adobe Flash CS4 calling upon the Papervision3D and ASCollada action-script libraries in order for the visual tracking to take place, the effect being that you can successfully track both 2D and 3D images in the real world in real time.
I have spoken with a number of people who have greater experience of Adobe’s Flash than myself, including Liam Birtles, Adam Jaggard and Josh Shard and all agree that the FLARToolkit can produce some very effective results.
When people use the Archetypal Reflectomatic they tend to move about quite a lot, waving there arms etc. Which although this can cause them to block the camera’s view of the marker and thus break the playback of the animation, it doesn’t seem to hinder there enjoyment of the experience, I put this down to the idea that people seem to quite enjoy jumping around infront of a live video stream of themselves.
To further improve this project I would like to spend a little more time on the visual aspect, in terms of additional animation and higher levels of interactivity. I would also like to get the project to a stage where multiple markers can be introduced to the stage at the same time, therefore allowing greater interactivity between both the animations and the users. Obviously this project could also be easily catered towards multiple user markets with a simple change of Marker and Animation design.
Post Presentation/Crit thoughts:
After Presenting the project for the crit, I have also realized that people would love this project to work with multiple markers. Also that something which I was aware of pre presentation that was confirmed during the presentation is that a standard webcam works fine in a well lit environment but for this piece to be succesful anywhere I would need to use an infra red camera so that lighting is no longer an issue.
Other aspects for consideration include: development of the project to work in combination with facial tracking so that it isn’t entirely reliant on the printed fiducial marker.