Jayne Vidheecharoen

Playing with the Here/Now/There/Then: Crafting Authentic Hybrid Experiences for a Networked Culture

1.0 Introduction

We live in a networked world with increasingly blurry lines between our physical and virtual lives. Our computation-centric society is more orderly, more systematic, and more governed by algorithms than ever before. Technologists often rush forward, in the name of progress and productivity, without looking back to reect upon the effects (or missed opportunities) in their wake. Focusing on the how instead of the why these new pioneers develop faster and better ways of systemizing, quantifying, streamlining, documenting and tagging our world objectively. As new algorithms get pumped out faster and faster, ubiquitous networked computing pervades everything from our built environment to our pockets.

In contrast to this rushed speed of progress, I believe in a more mindful yet exploratory approach to developing technology, in order to create idiosyncratic relationships between humans, computers, and the network. Instead of seeing a spacial or technological constraint as a potential problem to be cured away with technology, I see it as an opportunity to use play as a tool for challenging existing systems. I am specically interested in systems which enable uniquely subjective hybrid experiences which are made possible by leveraging the real world, the virtual world, and the human imagination. For instance, the physical world provides the benet of tangibility while

Jayne Vidheecharoen

the virtual world provides the collaborative power of our network culture. And the human imagination provides the cognitive mortar which holds everything together and allows a user to create meaning from the composited experience. Due to their hybrid nature, these experiences have their own unique sets of constraints and therefore their own unique opportunities for disruptive play. Using play, I aim to create a new hybrid experience while provoking questions about existing interaction norms, both online and in real life. In the end, I am hoping to create a new space of possibility for others to build upon and expand.

2.0 Work & Play: Magic Circles

Our future technology will not only enable us to work more productively, but in fact more playfully as well. Ubiquitous computing has created a fundamental shift in the way we separate work and play (Hilbrecht). The idea of work was once conned to the tangible real world, while play was allowed to exist in the intangible imaginary world. Now for many of us, much of our daily labor exists only in a virtual, intangible, and imaginary realm. As a result, this liberates our daily tasks from the rules of reality while subjecting them to the rules of play.

My early experiments, Zombie Mail & Play Slideshow, challenge the standard environments of banal digital productivity software to be more playful, while questioning established user interface norms. We often spend much of our work day in these dull virtual spaces, but being completely virtual they are especially viable

Jayne Vidheecharoen

candidates for becoming magic circles. Johan Huizinga, the classic expert of play in culture, denes a magic circle as a special temporary play space within the ordinary world where rules enable specic performances. In this case, having been traditionally designed for the rules of productivity and efciency, these environments enable a particularly objective and task oriented performance by the user. The modications in Zombie-Mail create custom metaphors for the e-mail interface, on both a visual and interaction level, as a way to reect a users subjective feelings about their experience with e-mail. It also favors a more discovery based navigation as opposed to a traditional list based navigation. Unlike a mere skin applied over an existing interface, this navigational shift fundamentally changed how the user interacts with the information and possibly their attitude towards the task at hand. Similarly, Play Slideshow makes the normally mundane task of creating slideshow presentations more challenging and playful by adding constraints. In this case, elements can only enter the slide by dropping in from above while the user is given limited controls (as in Tetris). By tweaking the interaction it begins to draw attention to the repetitive and homogenous act of creating slideshow presentations.

Not only does play, having its own rules of space and time, make something fully absorbing for a player it can also be used to expose tensions within a system. Since challenging a tension or established system is consciously outside of ordinary life (a formal characteristic of play), outlaws and revolutionaries have an inherent element of play in their activities (Huizinga). Thus, being playful is a potentially revolutionary and disruptive way to change established systems.

Jayne Vidheecharoen

3.1 Constraints & Systems: Activating Public Spaces

Creatively and playfully surmounting the constraints of a rational system is an opportunity for intervention and improvement. At the urban scale, people who reclaim public spaces to suit their own specic desires are defying the design of these urban systems, while actively challenging the way these spaces have been scripted by architects and urban planners. Margaret Crawford notes that mutability, contestation, and change is not what constitutes a failure of public space, but instead denes it. The emergent activities in Los Angeles public spaces are the very things that enable us to pose questions about urban citizenship. Therefore, we should not mourn the loss of our public spaces but instead see it as a space lled with possibilities (Crawford).

Although it exists in a completely virtual space, Google Street View is essentially a massive simulation of real world public spaces, frozen in time and lled with un-tapped potential. Through many of my experiments Im specically looking at ways to bring it to life and activate the space so it can be more like a real public space, subject to mutability and idiosyncratic uses. First Person Creator questions the use of Google Street View as a purely static and historically accurate tool for viewing an archive of our real world. While Google Earth allows people to contribute 3D models of buildings and monuments, Googles strict guidelines only accept models which represent real world structures textured with photographs. Therefore, it is impossible to contribute innovative or imaginative structures to the collective archive. In contrast, First Person

Jayne Vidheecharoen

Creator imagines a sandbox-game style interface which allows the user to augment and interact with Street View as they desire. The rst person perspective is also a shift from the section, plan, and birds-eye-view of traditional 3D modeling programs. This perspective empowers people to feel situated within the space, enabling a sort of spacial play testing. Other mash-up-mock-ups, such as Dancing in the Street View and Rolling in The Street View, continue this line of enquiry and begin to expose potential creative opportunities for adding life, through animation, to this static virtual public space.

3.2 Constraints & Systems: Improvisational Alternatives

In the physical world, Parkour is a clear example of both a form of play and form of resistance to the scripted use of urban systems and spaces. Nathaniel Bavinton explains that for Parkour enthusiasts, known as traceurs, constraints do not hinder fun but in fact add value and enhance it. Thus creative improvisation comes not from removing the constraints but by reinterpreting and nding new possibility within the constraints. This is particularly important to remember when considering the role of technology, which has historically been focused mainly on removing obstacles to make our lives easier. I am experimenting with what this approach of improvisational interactive play might look like when we substitute the traceurs buildings and railings for the technologies we are given as consumers.

Jayne Vidheecharoen

Users are not merely pedestrians adhering to a set clickable-path but instead digital traceurs who can be empowered to gracefully transcend technological obstacles through a wide range of embodied techniques. One of the key features of Portals is the ability to bypass the hurdle of the screen by being able to (seemingly) interact with the interface inside the screen. While touch screens and gesture based interactions already allow for a more embodied interaction than the traditional keyboard or mouse, in both cases the screen still acts like a glass wall between the user and the interface. Portals, on the other hand, is able to serve as a potential alternative to what Bret Victor calls Pictures Under Glass technology. Pictures Under Glass is a predominant vision for interfaces (in both the present and future) which forego the richness of tactile feedback in favor of high resolution visual representations. By looking for a fun and uid way to both bypass and take advantage of this constraint, I propose an alternative approach to digital interaction, which is capable of leveraging both the tactile and visual senses in order to create a more authentic simulation of interaction. The interaction in Portals may not be easier, as the system tends to cause the user some spacial confusion, but I believe this additional obstacle adds to the experience by making it more fun.

4.1 Truthiness: Believe

The idea of truth is another rational system open to exploitation & intervention as a way to create future possibilities. Questionably objective truths, created by algorithms, already shape our real life in many ways. In his TED talk, Kevin Slavin argues that the repetitive nature of algorithms enables them to gain the sensibility of truth (even if

Jayne Vidheecharoen

theyre not true) until eventually they solidify and become seemingly real. For instance, in the 2010 Flash Crash of 2:45 we saw algorithms taking control of the stock market, dropping the Dow Jones Industrial Average about 1000 points before recouping the losses minutes later. Despite the fact that no human had ordered it or had any control over what was happening, the effect of the crash was very real.

Code even controls the small decisions we make in our everyday lives. Netixs recommendation algorithms are responsible for 60% of the movies we end up watching (Slavin). Unlike the code crunching quantiable nancial data, the Netix code is essentially turning our subjective relationship with art & culture into something seemingly objective and true, which inuences the way we spend our time in real life. If algorithms have the power to make us believe in a truth that affects our lives, as individuals we too have the freedom to play with the truth in order to affect our own lives.

4.2 Truthiness: Fictional Facts

A subjective and loose denition of truth, when used in cahoots with the viewer, can become a valuable tool for creating a more real experience of the present. This can also allow for the viewer to create their own interpretations, beyond the creators original intentions. P.T. Barnum was infamous for playing with the expectation of truth in the context of museums. He often manipulated the press to stir up controversy about the authenticity of his own Dime Museum exhibits. This drove up ticket sales

Jayne Vidheecharoen

because he knew the audience loved trying to gure out if the exhibits were fact or ction. By inviting the viewers to question his exhibits, he was respecting the viewers intelligence. Many of the viewers enjoyed collaborating on the intellectual exercise regardless if it could be proven to be true (Dennett). The development of 19th century technologies also left people thinking that just about anything mechanically or organically probable was possible, making people very susceptible to hoaxing (Harris). This seems especially relevant in our modern times, given our dependence on the internet as a main source of knowledge in addition to the rapid pace of developing technologies which were once only fantasy, such as ubiquitous networked computing, and complex predictive algorithms.

The narrative I create in my Kickstarter campaign describes Portals as gateways to another place in the world, at a different point in time, even though theyre clearly not actual portals with teleportation capabilities. The description also states that it allows a user to go inside the screen, despite the fact that the user is technically behind the screen. I specically craft a description which is both true and potentially fantastical. I also include videos which are just rough sketches of how the Portals could potentially work. Some see these videos as examples of completely functional prototypes while some question their verity. Given the honor ecosystem of Kickstarter, it is especially interesting that this clearly ctional idea of going into a screen to access portals to a parallel world is an acceptably honest and authentic description of a technology project to backers. Like P.T. Barnum, I feel its important to respect the backers intelligence by inviting them to knowingly play along with me in crafting the projects

Jayne Vidheecharoen

story and advancing it further. As a result, various blogs continue the story with factualsounding headlines like Media Design Student Reaches Into Another Dimension (Art Center Dotted Line), Portal Becomes A Reality--Sort of--With This Kickstarter Project (PC World) and Thinking inside the box: The television you can 'reach inside' to grab things on screen (Daily Mail). Clearly, playing with peoples imagination is a potentially powerful point of reference for people to grasp a new idea.

4.3 Truthiness: Youre Doing It Wrong

It is also through challenging the true ofcial intended purpose of a system or object that we can create a new possibility space for things to enter the world. McKinzie Wark explains how hacking is essentially abstracting, and it is through this act of abstraction that a new space for possibility, beyond necessity, is created. By liberating objects from their designed purpose, it becomes possible to explore new uses. Within my Portals project I consciously explore the wrong ways to use technology (on both the software and hardware level) as a way to provoke questions and speculate about what another right way might soon be. While my wrong uses of technology may seem strange at the moment, in the future it may become second nature. Some blog commenters dont see any practical use for Portals while others are excited about the range of potential applications. Several Kickstarter members have contacted me excitedly with additional application ideas such as table top gaming, race training, presentation tools, and educational tools. Despite the fact that Im using everything wrong, to these people, my system feels like the right solution for their goals. Its

Jayne Vidheecharoen

important to give people enough context to understand the main ideas behind the project, while still leaving the space of possibility open enough to allow wrong personal trajectories of inspiration from the initial idea. Hopefully this ensures that the project is the start of a conversation as opposed to a denitive solution to a single problem.

5.1 Hybrid experiences: Alternative Reality

As more of our experiences exist in a hybrid reality we are increasingly open to enhancing our real experiences with an imaginary virtual layer. We are already comfortable calling upon the cloud to augment our experience of the real world. Whether its changing our behavior to earn a mayorship on Foursquare or letting Yelp determine our night out, we are perfectly comfortable with letting this virtual layer permeate our real lives. We can therefore freely borrow from techniques of traditionally virtual spaces as a way to enhance our real world. Harnessing the power of mixed reality enables us to challenge our expectations of what is possible in the real world. Jane McGonigal argues that we can x reality by employing tactics used in games, such as inviting the user to take on unnecessary challenges, in a personally meaningful way. She denes alternate reality games as games you play to get more out of your real life, as opposed to games you play to escape it. These alternate reality games are especially powerful because they give us explicit permission to play outside of social norms within our real life.

10

Jayne Vidheecharoen

In order to test McGonigals theory I created Media Design Bingo, an alternate reality game designed to increase interaction between all the students in the studio. Other design goals included designing a game that promoted specic values, such as gratitude and collaboration, to hopefully encourage a wider range of people to play. Using the studio oor plan as the bingo board, people were asked to keep track of the people who had helped them throughout the week, in attempt to complete patterns for badges. I was surprised to nd that people did indeed feel the game increased the level of interaction within the studio for the week, and people felt more comfortable approaching some peers they hadnt in the past. Clearly, this alternate reality game had created a sort of magic circle where the rules of the game enabled new types of interactions to occur and actually showed how play could be used to address and change real life tensions.

5.2 Hybrid Experiences: Movie Magic

While alternate reality games can be powerful, the gamication of real life is only one type of hybrid experience. If we think beyond gaming and include other forms of fake worlds, such as lm, we nd another rich set of tools for creating engaging immersive experiences. The imagined virtual space also has a rich tradition of leveraging technologically mediated illusions to create the suspension of disbelief, or the suspension of reality-testing all together. The world of animation, compositing, and special effects have advanced so far in the past decade that often times we can no longer separate between what was shot in-camera and what was added in post-

11

Jayne Vidheecharoen

production. As a result, the distinction between the real and the fake dissolves as we allow ourself to become fully immersed in the experience of the lm.

Illusions are powerful because they trigger a feeling of authenticity which can be manipulated and reshaped (Stark). With Portals I am consciously embracing movie magic techniques, such as chroma keying and layering footage, and applying them to the context of a real world interface. As a result, the user experiences a simulation of interacting with the interface directly, while taking advantage of the screens ability to augment and animate. Stop motion animation in particular, is an especially powerful tool because it has a realness in terms of materiality and motion, but it can still be stylized beyond what is possible in reality. As a medium, stop motion is inherently about illusions caused by playing with time. Due to its very nature, it also highlights the uncanny tension between the unexpectedly static or animated when in the context of the real world. By combining illusion and interaction I strive to create a more magical experience for the user, which takes advantage of both the real and virtual world, to create a new type of hybrid space.

5.3 Hybrid Experiences: The Real World

Of course, the hybrid experience goes both ways, in addition to drawing from the virtual world we should enhance our virtual spaces by drawing from the real world. With Portals, the ability to physically manipulate objects in the interface takes advantage of our hands sensorial feedback capabilities. Portals reality based visual

12

Jayne Vidheecharoen

interface also helps grounds the hybrid experience in the real world as opposed to only having a place in a virtual one. While virtual spaces like Second Life allow users to interact with each other as CG avatars in the same virtual space, they also only exist in a completely fantastical environment. Despite early hype around the potential use of Second Life as a legitimate alternative to in-person meetings, the idea failed to gain mainstream acceptance. Relying completely on CG, everything about the world completely disregards reality and becomes stuck as an escapist amusement, as opposed to one which could potentially enable the user to benet from the activity in their real life.

On the other hand, standard video chat allows for a completely reality-based conversational experience, but its limited in that each person is conned to their own little window of reality. It also completely ignores the potential for a more fantastical experience. While visually real, the experience of video conferencing distinctly lacks the ability to really co-exist and visually overlap in a shared third space and play together. Alternative solutions such as sharing a desktop screen or using a collaborative white boarding application still leaves the human element stuck on each side of the side of a screen. The only hint of a humans presence in these virtual workspaces is reduced to a wiggly mouse cursor, and maybe a chat box on the side. Tangibility is just one of many benets of the physical world, another is the ability to just co-exist in the same physical space with people while interacting with them. Portals strives to exist somewhere between virtual worlds, shared desktops, video

13

Jayne Vidheecharoen

conferencing, and the real world in order to challenge our standard manifestations of co-presence.

6.1 Network Culture: Strength in Numbers

While striving to create hybrid experiences that explore new modes of telepresence, network culture becomes a powerful tool for generative, collaborative, and distributed creation. Kazys Varnelis sees our current network culture as a distinctly new phenomenon, dened by the mix of real and virtual space and new forms of participatory media. While the effects this new culture seem minor to each of us as individuals, collectively its a radical shift. In the network, individuals become less important than the emergent outcome of the connections between people (and machines). Naturally, the process of remixing from many sources is the dominant form of creation in this highly interconnected culture, dissolving ideas of individual authorship (Varnelis). As such, experimenting with the affordances of network culture within my project is very timely and highly relevant to our current experience, particularly in relation to remixing real and virtual spaces and participatory media.

In many ways, Media Design Bingo is a tangible manifestation of our networked culture in the real world. The individual game boards log the help each student received from other students, essentially serving as a visualization of generosity within our real world social network. Additionally, customizing the individual boards is explicitly encouraged in the rules of the game, giving people freedom to freely express themselves. As a

14

Jayne Vidheecharoen

result, the ways in which people chose to customize their individual boards combined with the network activity generates a collective image of the studio culture that cant be predicted by any individual or an algorithm. The unique award badges created at the end of the game also serve as a visualization of the connections between students, demonstrating the emergent outcome of the network. Testing the capabilities of crowd sourced creativity more directly, I create a HIT (Human Intelligence Task) for Amazons Mechanical Turks and pay 20 people $0.25 each to modify a panorama. The space evolves as various people from all over the world playfully add their modications and alter the modications of others. While the test is relatively small and individual modications are minor, over time it becomes quite lively and demonstrates how a system of distributed creation could work. I imagine that if Portals were widely available (and fully featured) they would enable a similar kind of emergent and collaborative activity. Launching and running a Kickstarter campaign to raise funding for Portals also serves as another form of public research into our current networked culture and the power of the internet. The 30 day campaign has ended with the project 190% funded by 118 backers, with $2,853 pledged. About 35% of the funding comes from my personal network and their extended networks, with the other 65% of funding from random strangers on Kickstarter. These results clearly demonstrate our desire to participate and help each other through the network as a collective.

15

Jayne Vidheecharoen

6.2 Network Culture: Uncanny

While our network culture creates a whole new range of possibilities, it also creates an experience of tension within our minds as we reconcile the experience of transcending space and time. When we can get updates from the other side of the country at the speed of a tweet and travel the world with a drop of the Street View Pegman, our concept of the here and now blurs with the there and then. Through my experiments I intentionally aim to further muddle the distinction in order to create an uncanny experience for the user. Kio Stark explains that playing with the uncanny is a particularly powerful tool because of its ability to cause people to question a fundamental understanding, even if only momentarily. These uncanny experiences often cause us to suddenly reinterpret our experience using primitive beliefs such as magic or coincidence. But we are less likely to feel like something is uncanny when we are more familiar with our environment. As we become more and more accustomed to living in a network culture we run the risk of becoming immune to the strangeness of our experience. Thus, provoking an uncanny feeling can serve as a way to jolt us from our daily routines as a node in the network and cause us to question the things we currently just accept as normal.

In order to create an uncanny experience, I explore alternate ways to show different points in time within a single space in my panorama mashups. In Pan, looking around horizontally reveals a different point in time each time the viewer returns to the space. Scrub, on the other hand, stacks time vertically like a lm strip so the viewer can see

16

Jayne Vidheecharoen

one slice of a space over time while also being able to look around a space at a specic time. Both experiments explore the implications of being able to interact with and remix a time-based media (for instance animation or lm) within the context of a traditionally static environment. As these extra layers of reality co-exist with a static representation of reality, the user is able to weave their own non-linear understanding of the space. Similarly, in Portals the layering and compositing of static images from the past, footage of the present, and animated interactive stop motion creates a collage of multiple realities. The collage is by multiple authors and is neither real-time nor pre-recorded. By creating an uncanny experience of space and time I also strive to draw attention to the way in which our authentic representations of real space, in virtual environments, are actually stitched together from various moments in time, breaking the ties between space and time. In doing so I am suggesting alternative illusions of virtual co-presence that are equally open to a non-linear time structure.

7.0 Conclusion: Starting Principles

From the theory and experiments conducted thus far I have developed several principles to guide my work moving forward. First, by leveraging spacial and technical constraints, use play as a tool to disrupt and challenge existing magic circles in the real world. Through this act, instigate more improvisational interactions between both human to human and human to computer. Second, embrace alternative truths as a tool for sharing ideas and inviting participation from others. By provoking questions about future technological developments, hopefully this technique ensures the projects

17

Jayne Vidheecharoen

core ideas can continue exist beyond its original actual form. Third, combine the affordances of the virtual world, the physical world, and the human imagination in order to create an experience formerly impossible in the virtual or physical world separately, and therefore more authentic to the hybrid nature of the experience. Finally, using both the strength and strangeness of network culture, create an uncanny experience in order to disorient users and disrupt our habitual routines as active participants in the network.

With these core principles in mind, I feel well prepared to make progress on my thesis project through the Spring term. For the duration of the term, I intend to focus specically on developing Portals as a system. This includes the design of the objects in addition any supporting materials as part of the system. Hopefully the creation of this system will both address the issues outlined above and be the start of many other investigations to come.

18

Jayne Vidheecharoen

Works Cited

Bavinton, Nathaniel. From Obstacle to Opportunity: Parkour, Leisure, and the Reinterpretation of Constraints. Annals of Leisure Research. 10.3. (2007): 391. PDF File.

Crawford, Margaret. Contesting the Public Realm: Struggles Over Public Space in LA. Journal of Architectural Education. 49.1. (1995): 4-9. PDF File.

Dennett, Andrea. Weird and Wonderful: The Dime Museum in America. NYU Press, 1997. Print.

Harris, Neil. Humbug: The Art of P. T. Barnum. Phoenix ed. University Of Chicago Press, 1981. Print.

Hilbrecht, Margo. Changing Perspectives on the WorkLeisure Relationship. Annals of Leisure Research. 10.3. (2007): 368-384. PDF File.

Huizinga, Johan. Homo Ludens: The Play Element in Culture. Boston, Massachusetts: Roy Publishers, 1950. Print.

McGonigal, Jane. Reality is Broken: Why Games Make Us Better and How They Can Change the World. New York, NY: Penguin Press, 2011. Print.

19

Jayne Vidheecharoen

Slavin, Kevin. Kevin Slavin: How Algorithms Shape Our World. Ted Talks, Jul. 2011. Web. 4 Dec. 2011.

Stark, Kio. Notes on Authenticity and Uncannydiscussions. For Reals: Fall 2010. Oct. 2010. Web. 16 Jan. 2012.

Varnelis, Kazys. Conclusion: The Rise of Network Culture. Networked Publics. Web. 4 Dec. 2011.

Wark, McKinzie. A Hacker Manifesto. Cambridge, Massachusetts: Harvard University Press, 2004. Print.

20