Art 2.1 | New Frontier at Sundance 2013: 4D Art and Augmented Real

Yung Jake. “E.m-bed.de/d” (left) and “Augmented Real” (right), 2013. Photos by Nettrice Gaskins.

This is the second post in a series covering New Frontier art at Sundance 2013 (see the first post). Collectively, the artworks combine sensors, video, sound, streaming, web cams, projections, programming, the web platform, and other technologies to produce immersive, interactive and participatory performances and installations. Audiences are able to experience real and fictional worlds simultaneously. This series looks at artists from around the globe who are inventing new uses for existing technologies, or fashioning new technologies to fit their interests. These artists are innovators, creating new aesthetic forms and new ways to articulate their identities.

On the Internet, artists can assume any identity and are limited only by their imaginations—artist Cao Fei is China Tracy in RMB City. For the Art21 Blog, I covered artists who transitioned from the physical to the virtual in Second Life. These artists go by avatar names such as DanCoyote Antonelli and Bryn Oh. I first wrote about performative interventions and the ambiguity of identity three years ago. Today, artists’ avatars are virtually performing in physical (real) spaces. New Frontier artist Yung Jake has re-conceptualized the Happening as a mixed reality performance space for the Information Age marketplace (e.g. smartphones, tablet PCs, etc.). Yung Jake’s new media interventions eliminate the boundary between the artwork and its viewer. The audience, in a sense, becomes a part of the art or the art itself. Yung Jake does not normally interact directly with the viewer, except as an avatar.

Yung Jake is a character [but] he also lives by that character. I don’t even know who the other person is. I found out his legal name just for airport purposes, but I never talk to him other than calling him Yung Jake.
Shari Frilot, New Frontier curator

Yung Jake as Yung Jake. Photo by skylovestoeat.

Yung Jake a/k/a Yung Jake has been described as a “datamoshing, glitch-creating, meta-rapping artist” who employs “cross-genre collaborations, GIFS, and the sweet new app he and his buddies created for Sundance.” The new app is called Augmented Real and it is a mobile augmented reality (AR) rap music video that pops out of posters and magazines (see below). The app can be downloaded on mobile devices like smartphones and iPads. Yung Jake also stars in E.m-bed.de/d, another online performance that pops up on laptops and is triggered by sitting down in front of the device. I had a chance to speak with the recent CalArts grad and his designer Michael Ray-Von (also from CalArts) about their work at New Frontier.

Nettrice Gaskins: A few years ago, Lev Manovich coined the term augmented space to refer to physical space that is “overlaid with dynamically changing information, multimedia in form and localized for each user.” Manovich asks whether this form becomes irrelevant and ‘invisible’ or if people end up with a new experience in which the spatial and information layers are equally important. Can you tell me a little bit about how you, as the artist or performer, conceptualizes performance in augmented space?

Yung Jake: There are two performances: the embedded performance of the laptop and the new (augmented reality) application the device performs. For the live performance we used giant projectors on either side of the (physical) space, two laptops, a webcam and me performing with the computer. I was on a mic and people could hear and see me but I couldn’t see them.

NG: What inspired you to do this type of work?

YJ: It was easy to make something (online) and put it out in public. People want instant gratification. E.m-bed.de/d was meant for the internet. It’s a program that triggers another program. The (main) program opens up Google Chrome and makes decisions without the user’s input, using two really simple applications.

NG: How much of your work is planned and how much of it is experimentation?

YJ: Four people were involved. Vince McKelvie was the programmer and Temra Pavlovic was the cinematographer who also handled the face tracking for Augmented Real. Michael Ray-von was the (graphic) designer and of course there’s me. I worked with each person mostly one on one, based on my concepts.

You can experience Yung Jake’s work on YouTube or download the Yung Jake app from the App Store, then aim your camera-enabled mobile device at the Yung Jake rock, which can be found in the Sundance Insider magazine given to festival attendees or on certain people’s shirts and buttons.

Yung Jake’s “Augmented Real” postcard. Photo by Nettrice Gaskins.

So what happens when you download and launch the Yung Jake app? A red play button appears on the screen and when touched the virtual Yung Jake pops out of the rock and performs a rap song. During the song a constellation of pop culture artifacts (and female avatars) surround and pass through the virtual rapper as he performs. Face tracking enables the Yung Jake avatar’s mouth to move to the song lyrics.

Yung Jake. “Augmented Real” (AR performance). Photo by Nettrice Gaskins

Yung Jake’s telematic performances are also an example of techno-vernacular. Techno-vernacular creativity refers to cultural forms made and organized by (digital) indigenous communities for their own entertainment, using technology. These cultural forms generally evolve out of urban spaces and are social in nature, encouraging interaction and contact with spectators and the other artists. One way to understand how the mobile augmented reality (AR) art production works is to see it as a cognitive and computer-based system, so I created this chart:

Art, technology, and cognition: a system for creating mobile Augmented Reality and art. Image by Nettrice Gaskins.

Yung Jake is part of a community of digital native artists who produce, transform, appropriate, and consume technologies in their everyday lives. With access to several free online tools they can create cutting edge works of art that blur the lines between virtual/real (worlds) and high/low (cultures). Yung Jake’s performances reflect the evolving nature of contemporary art and technology in the Information Age. His work is part of the evolving art of participation that requires higher levels of technical knowledge and innovation to engage artists and their audiences…and it can be viewed anywhere, at anytime—just point your camera-enabled mobile device at the image.