media art
"HandHeld": Pushing Interactivity in a Handheld Device
With the proliferation of portable handheld devices and mobile phones, there is a shift towards a more personalized—perhaps intimate—form of mediated communication and interaction. These devices are also at the forefront of the ever miniaturization of the computer—the personal computer becoming ever more personal. While games seem to fit naturally into the world of personal handheld devices, few have explored the possibilities for the medium of cinema and moving image.
In "HandHeld", simply picking up the device, then tilting and pitching left and right, forward and back, the viewer is able initiate a direct response in the imagery displayed on the screens. This response is both spatial and temporal, and allows the viewer to create, and participate, in their own audio/visual narrative experience, generated in real-time from both prerecorded material and live input.
Using a system of sensing and controlling software running on a separate computer, motion and orientation data are combined with digital video (and audio) clips, and computer-generated elements to create real-time affects and modifications on the playing narrative(s), all initiated by this simple tilting and pitching of the device in the viewer’s hands.
As a result, "HandHeld" challenges the normal assumptions and relationships between the screen (the cinematic space) and the viewer, and begins an artistic exploration into a range of possibilities for narrative moving image in a very tactile, visceral, and 3-dimensional work. By using two screens, the work explores some of the possibilities made available through the use of digital media (and digital technologies) in the areas of time/ space, simultaneity and parallelism in narrative forms, particularly cinematic narrative, and how new, 'expanded' audio-visual narratives might be created and experienced.
Interface, and Parameters of Play
"HandHeld" also explores the machine-user interface of handheld devices in relation to narrative by introducing a very direct, intuitive form of interaction—the simple tilting of the playback device in the viewer’s hands. Currently, most handheld device interaction is carried out through just one form of control: push-buttons. Of course, most commercially available devices have not been designed with narrative moving image in mind, but the conceptual and artistic direction of this project opens up the possibility of exploring other human-machine interface directions, and in turn, further exploring areas of the "physical grammar" of interactivity, with devices that are more 'responsive' in nature, and which may be capable of both conveying and sensing emotion (or creating the illusion of emotion).
Additionally, "HandHeld" challenges normal assumptions about games and game-play in relation to the experience of moving image. The cinematic experience is, by and large, a somewhat passive activity, where the viewer just watches narrative(s) on a screen, allowing a pre-determined narrative to unfold in front of them. "HandHeld", however, is a very personal narrative experience, one that introduces an element of participation and interaction into the cinematic experience, one that is directly controlled and influenced by the viewer him or herself.
In one sense, the viewer is able to "play" with the narrative, directly influencing their own narrative experience in real time by changing both spatial and temporal aspects of sequences that appear on the screens. In another sense, the addition of live video input allows the possibility of creating narratives (or certain aspects of narratives) in real time.
Summary
This element of live input, combined with pre-recorded digital material, enables the possibility of very unique narrative experiences: unique to the moment they are created (i.e. they will never be repeated) and unique only to that individual viewer's experience. This is unlike the normal experience of a video game where, no matter how complex the structure, most game narratives are pre-determined. Taken together, a strong element of discovery and surprise is introduced into narrative experience through this work.
download thesis docomentation (pdf: 7.6mb)