Interactive art helps us extract impulses from our brains, thread by thread, and enact them in the world. Music takes this medium into mind-bending heights. What if we were able to transform any surface into a living, breathing musical instrument? Emerging designer and musician Felix Faire recently did just that with Contact, an acoustic Leap Motion experiment created for the Royal Academy’s “Sensing Spaces” exhibition.

As a first-year architecture student, Faire was struck by how listening to musical progressions as you walk through a space affects the way you move, so he designed an entire concert hall and gallery in the linear structure of a sonata. These initial musings grew into a much larger project on spatial music perception entitled “Music Aided Design.” It was then that coding became an integral part of Felix’s creative life, and he knew 3D motion control would become an essential exploratory engine for his thesis.

“The fidelity of the Leap Motion made me realize this kind of device could track even more subtle musical articulations, and perhaps even be used as a three-dimensional instrument in itself,” Faire told us. “Now I understand more of what is possible with Leap Motion. I have ambitions to try much more complex gestures and motions in future projects.”

For Contact, Felix used hand height, finger count, and a squeezing gesture to trigger various effects in the loop – the visual output influencing how the audience attempt to interact with the sound waves. “Abstract audiovisual synchronicity, while extremely elusive, can be a very exciting and compelling experience,” Felix concluded.

“Sensing Spaces” will continue to run at the Royal Academy in London through April 6th, 2014. It features architectural practices from six countries spanning four continents. If you plan to attend, be sure to tweet impressions, images, or video from your experience to @LeapMotion using the hashtag #SensingSpaces.

We’ve talked about the magic of WebGL before – how it unleashes the power of the web to do incredible things in 3D. With this latest experiment from Bartek Drozdz, you can reach into your browser and play with a variety of cool visuals to music. A liquid gem, cityscape, spherical lines, and more, all responding to Codex Machine’s S.P.Y. or even your own microphone.

As the creative director at Tool of North America, a production company based in Santa Monica, Bartek brings interactive digital experiences to life. Last month, Tool decided to throw a party for its employees and friends in a large building known locally as “the shed.” To bring some WebGL zest to the party, Bartek started working on sound-reactive visuals that would project on a wall – using the huge space to his advantage.

Next, Bartek took it a step further by introducing Leap Motion interaction to the setup. As you can see (and experience) for yourself, the results were spectacular. You can fly and steer between buildings, create strange wave reactions, or change your point of view. Each mode is different, and it can be challenging to discover what they all do. It’s all part of the fun.

“Part of the challenge was that we did not want to put any instructions anywhere, so we had to do something that was intuitive,” says Bartek. “Just let people know that they can hover the device with their hand and see what happens. It made people curious, but was also easy to use. Our guests could interact with the visuals using one hand, and holding a drink in another.”

The web is growing all the time. With these kinds of early experiments, we can catch a glimpse into what the future web might look like – with complex 3D sights and sounds created with little more than JavaScript and the magic of WebGL shaders.

Want to dive into Bartek’s trippy experience of sight and sound? You can check out his live demo on Tool’s website. To use Leap Motion on the web, be sure that you’ve checked the “Allow Web Apps” box in the Leap Motion Control Panel (General tab). As always, we recommend Google Chrome.

What do you imagine we’ll see in the web of the future? Let us know in the comments below, or give us a shout on Facebook and Twitter.

Ever since the first human stacked one brick onto another, architecture has been concerned with creating immovable things. Even with the rise of smart interconnected environments – where lights, heating, doors, and other systems within a building all work together – the physical structures of our buildings remain the same. As a result, the movements and interactions of people within these spaces are shaped by the buildings themselves, like water flowing through a canyon.

This is why architecture and urban design are about more than simply ensuring that our buildings are safe and efficient. Or that they are merely beautiful. Buildings can inspire or isolate, connect or divide, so that debates about everything from the nature of community to the fate of doorknobs have radical social implications. How we live and work every day cuts to the core of what makes us human. But what if buildings could respond to our movements and gestures? What would that change?

Inside the Aether Project

These were the questions asked by a team of students at the UCLA Department of Architecture and Urban Design. Refik Anadol, Raman Mustafa, Julietta Gil, and Farzad Mirshafiei wanted to explore how architecture itself could become smart and responsive. Technology, they say, is the story of how humans have embedded mind into matter – and architecture is no exception.

Using KUKA KR150-2 robots, projection mapping, and motion control, the Aether team created “an immersive interactive environment that gives a glimpse into the near future of artificial intelligence and its effects on human existence in an environment that bridges the physical and the digital.”

Aether begins with Leap Motion interaction. The movements of your hand have the power to transform a robot-controlled surface geometry. At the same time, another robot projects elaborate designs that are synchronized with the physical geometry, creating a choreographed robotic dance. The patterns that course along the surface are constantly shifting. Geometry, depth, shadow, and color – all rise and fall as the pattern changes.

Interaction as design medium

What does this have to do with architecture? With the growing use of robots and smart hardware, architects will have a lot more freedom in how they design the world around us. The Aether Project shows how a physical structure can be transformed through interaction, and takes it a step further by creating designs that can be layered on top of that structure. Interaction becomes a physical design medium, just like it already has for the digital world.

We’ve just scratched the surface of the ideas behind The Aether Project, which include thoughts on the nature of consciousness, artificial intelligence, and how human behavior might factor into the blueprints of the future. Check out the extended version of this post on our developer blog, including a Q&A with the creators. (For more interactive art with big ideas, you might want also to visit the digital forests of Growth or the stark beauty of Ascension.)

We’d love to hear what you think. Where will this desire to implant our consciousness within machines take us? And how will we interact with machines in the future?

When digital art and physical sculptures are melded together, the resulting creation can be spectacular and strange. Recently, visitors to an exhibition at Eyebeam, an NYC-based art and technology center, discovered what happens when you throw 3D interaction into the mix. You become an artist yourself – creating between the real and unreal. You become part of Ascension.

The exhibit was a collaboration between multimedia artist William Ismael and sculptor Carrie Mae Rose. At Hackerloop, an innovation lab and hardware playground, William has been developing interactive experiences to bring rooms to life. Carrie Mae is known for her evocative sculptures that shock and endanger – working in wire, scissors, and razor blades.

Ascension reflects their different backgrounds – bringing together a hand-built tetrahedral wing structure with digital animations and motion control to become something new. Recently, we caught up with William to ask about his creative process and the work that went into creating Ascension. Plus, a preview of his next installation, Visual Composer – which lets him generate live visuals with his fingers.

Ascension

Starting Point: Inspiration

My biggest visual influences are nature, the sky, the ocean, the cosmos, sacred geometry, architecture, mathematics, and human-built objects. What inspired me to use Leap Motion was the possibility of using our main manual tools as human beings – our hands – to generate animated art, in real physical spaces, in real time. It personalizes the spatial experience because people become co-creators of the space in a very intuitive way.

With the Leap Motion Controller, people become co-creators of the space in a very intuitive way.

Installations bring people together in a physical space, so using the Leap Motion Controller for the Ascension installation made it not just art to look at, but something reactive – in a unique co-creative experience. Leap Motion control was critical in giving people the power of triggering and controlling the animations of the projection mapping. Its precision when it came to subtle movements and the use of fingers made it the right device to use.

Creative Process

Incorporating 3D motion-controlled projection mapping onto the 3-dimensional angel-winged sculpture on the wall was a very technical process, involving trials and successes. My first spatial experience with Leap Motion was not for a space, but an interactive art app for the desktop as a way to test it right away.

For Ascension, first I programmed an interactive animation on my desktop, where Leap Motion was used to control it with my hands. I played with it on my MacBook Pro until I got to a place that felt right. Colors were vibrating and forms were interlocking – all by waving my hands in the air.

The next step was mapping the 3D structure through the projector. Using Processing with MadMapper, I ran a code-generated 3D animation, with the interactive animation triggered by Leap Motion interaction on top of it. Finally, I ran up and down the stairs for an entire week to problem-solve and adjust the details. Two projectors were used in the final installation.

Visual Composer

William’s experiment with Ascension led to his next project – a visual composer that brings together psychedelic colors with splattering paint and abstract geometry. While it’s still in development, William hopes to take Visual Composer to the next level as a live performance tool.

3D Motion Control & Performance

The ability to control motion graphics in a space with my body movements does not just change performance – it creates a new type of performance. The way coded animations precisely sync to my movements gives life to a new human experience through the way Leap Motion is used. I can now perform to a crowd live on a large stage with my movements creating real-time visuals. It’s extremely exciting.

An interactive room can be life-changing. It affects people emotionally…. It’s not happening on a screen, but in real life in a real space, where our senses are the most sensitive.

I think 3D motion control can radically turn a normal room into a living world – where we, as humans, affect the environment with a wave of our arms. An interactive room can be life-changing. It affects people emotionally. We’re affected by every detail of our environment. By creating such a space, people can be taken instantly to an incredible journey. It’s not happening on a screen, but in real life in a real space, where our senses are the most sensitive.

From high-concept art and storytelling to virtual objects and drone experiments – where would you like to take Leap Motion interaction? Let us know your favorite experiments with art, music, and design on Facebook and Twitter.

When a volcano erupts, a large, sunken crater called a Caldera is left in its wake. They are desolate and otherworldly voids, seemingly stolen from a post-apocalyptic future where – left with a blank canvas – we are suddenly faced with the challenge of reconstructing our environment completely from the ground up. But if we give ourselves the reins of Leap Motion technology, the power to reimagine our world is right in our grasp.

This concept of rebuilding after a great collapse inspired a team of digital media and motion graphics professionals enrolled at Stockholm’s Hyper Island school for digital enterprise in their recent project using Leap Motion. Equipped with our Controller, the students sought to create an experience that would challenge the current perception of what technology is capable of – then push it even further.

The result was Calderan.

To create the showpiece, the team projected WebGL images into a plexiglass pyramid to create a 3D hologram that can be manipulated in real time with the Leap Motion Controller. Attendees of the Distortional Evolution Exhibition approached Calderan with “wonder and excitement,” said digital media creator Ellinor Pettersson. “We had huge interest in from the visitors, and got great reactions. Calderan’s onlookers could repel the object in the pyramid or zoom it closer, rotate it this way and that, and then with a simple tap motion of the air, an entirely new object would appear behind the glass.

“People walked in with a sense of curiosity. We were really inspired by the reactions during the night. We heard so many great ideas from our visitors on how they would like this to be used in the future – everything from museum displays and interactive stores to apps for sign language. It was really fun to be able to inspire people through the installation, and to give people the chance to try new technology.”

In many ways, Leap Motion and Hyper Island are a perfect match. The programs and workshops the school provides help bring innovative professionals to the verge of a creative revolution – from the sidelines to the forefront of a rich new era of digital interaction.

It’s no coincidence that great storytellers often talk with their hands. They can become puppets, bring bedtime stories to life, or punctuate the anecdotes we tell our friends over drinks after a long day. Hands help us imitate and enrich and enlarge our everyday narratives when words alone fall short.

Interactive media artist Erik Loyer uses Leap Motion to bring the natural gestures of real life into the world of digital literature. Loyer is known for his pioneering use of fluid 3D navigation in narrative, founding Opertoon Studio to pursue the power and potential of “stories you can play” From sci-fi graphic novels and photorealist prose to iPhone love stories, his award-winning body of fiction and non-fiction brings the tactile use of devices to the forefront of the reading experience.

When Loyer began using Leap Motion as a vehicle for storytelling, he landed on the idea of breathing – using sweeping gestures to breathe into a space, as well as a guide for meditative contemplation. His latest piece, a surreal comic called Breathing Room, uses the Leap Motion Controller to push, pull, and wave through an audiovisual landscape of rustling trees.

Within Breathing Room, each movement above the Controller prompts the sound of a breath and a shift of the scenery, as if each inhale and exhale brings the world to life before sending it calmly into retreat. Fragments of the story unfold in each panel – sometimes one-off phrases, other times a multi-paneled vignette – but the story never repeats in the same order. It’s a brand new reading experience each time you play.

“I think it’s a huge thing to get people away from grasping a device.” Erik says, “There’s something really powerful about a reading experience with loose, broad gestures that help you move through a text.”

Each movement above the Controller prompts the sound of a breath and a shift of the scenery, as if each inhale and exhale brings the world to life before sending it calmly into retreat.

Recently, the Modern Language Association featured Breathing Room in its interactive tribute to the history of digital literature – Pathfinders: 25 Years of Experimental Literary Art. Students at Washington State University Vancouver will soon be using Breathing Room in an upcoming course on language, texts, and technology to explore N. Katherine Hayles’ classic How We Think in a whole new way.

To keep up with Erik’s explorations of digital storytelling with Leap Motion and beyond, you can follow Opertoon on Twitter or like on Facebook.

At a recent exhibition called Resortes, visitors to Mexico City’s Digital Cultural Center walked into a large room to be confronted by… nothing. A horizontal white line line projected against a silent surface, with a pair of Leap Motion Controllers mounted on either side.

“When the audience entered the installation, they didn’t have any clue how to interact with the piece – we wanted them to figure out how to control the piece without a guide,” says Thomas Sanchez Lengeling, one of Resortes’ creators. “Most people when they enter the installation didn’t know that they could activate the sensors – so when they did, some of them jumped!”

When the installation sprang to life, people quickly discovered that they could create huge waves of light and music with hand gestures. Elastic strings built on Newtonian physics and generated by the participants’ hand movements danced frantically between a set of particle nodes, shifting in color and tones to the sound of strange, otherworldly music.

With two people at the controls, the sounds and visuals generated by each person combined to form a massive composition. After the participants withdrew their hands, the giant strings continued to reverberate, throwing off light and sound for several minutes – like the dying vibrations of a digital guitar string:

Ironically, says Thomas, despite being a touchless device, the Leap Motion Controller made the installation possible by creating the illusion of touch. “We wanted the audience to feel that they could control a string with their bare hands and manipulate the physics of the environment. The Leap Motion device helps to create different forms of expression with art and technology, and I think if we had used another device we wouldn’t have had the same interaction.”

Want to create your own technicolor light displays? You can take control into your hands with particle apps like Midnight, Beautiful Chaos, and Gravilux. Let us know your favorite Leap Motion physics experiment – tweet us @LeapMotion or jump onto our Facebook page.

By letting people reach into thin air and create huge changes from tiny actions, interactive art exhibits can break down the barriers between art and audience. Last week, we looked at two recent art installations to use the Leap Motion Controller to challenge how we interact with nature and technology. Here are four more exhibits that made waves in 2013:

A Surreal World of Alien Shapes

One of the earliest art installations to use Leap Motion technology was Untitled°, created in June by NATURE graphique for Stereolux, a French digital arts and culture space. The exhibit allowed viewers to immerse themselves within a monochromatic 3D scene and explore the landscape with simple hand movements.

Human Actions and Natural Consequences

Vermont-based artist Craig Winslow’s Growth is a 3D-projection mapping experience that lets you manipulate digital vines, branches, and beams of light cropping up against a stark blue sky. Soothing movements create beautiful scenes, while aggressive swipes summon darkness – telling the story of humanity’s relationship with nature. Read more »

What if Social Media was Art?

Forge Collective’s Toronto installation Connexion Point allowed complete strangers to make connections and create a dynamic digital community through Leap Motion interaction. After contributing personal stories into a colorful world of geometric shapes, visitors were able to create, capture, and collect random collections of stories. Read more »

Welcome to Our Universe, Have a Seat

Created for production company B-Reel’s London office, Star Canvas is an interactive guestbook in the form of a giant projected star chart. It uses real star data and classical illustrations, some creative interpretations based on B-Reel’s current film projects, and a sound engine that turned the whole thing into a giant synthesizer. Visitors were able to draw and name their own constellations, then set it free as a shooting star.

While it takes a few extra steps to project it onto a building or within an art gallery, you can also create your own art with the Leap Motion Controller. The Airspace Store’s creative tools category includes a variety of artistic tools for everyone to enjoy – from finger painting to clay sculpting. To see the most popular creative apps, check out our Create collection.

What was your favorite moment of the past year, and what will you create with your Leap Motion Controller in 2014? Let us know on Facebook and Twitter, or join our 2013 retrospectacular forum thread.

What if social media was art? At Toronto’s Nuit Blanche in October, the designers at Forge Collective revealed their answer to this provocative question with interactive art installation Connexion Point. In the video below, you can see how complete strangers were able to make connections and create a dynamic digital community – exploring stories through Leap Motion interaction.

Once the sun went down, Connexion Point’s visitors used a mobile web app to share meaningful stories about themselves and choose a unique color. This generated a glowing avatar, which would ascend into a colorful world of geometric shapes, joining the avatars of others who visited the installation.

Standing before podiums embedded with Leap Motion Controllers and facing Connexion Point’s projected display, visitors then captured the free-floating avatars using circular gestures. As the world slowly populated over the course of Nuit Blanche, visitors were able to collect avatars and take them home – creating random collections of stories from strangers.

In this way, the creators of Connexion Point were able to bring people together with personal stories, rather than the torrent of noise that plagues conventional social media. Along with the event video, Forge Collective also documented the creative process behind Connexion Point, revealing a sneak peek into their process and highlighting the installation’s functionality. According to the creators:

As the Leap Motion technology is relatively unknown in the realm of public art, using the controller allowed us to create a bit of magic. It elevated the experience beyond that of a touchscreen, and by incorporating it into a podium, we were able to further that magical experience.

The ease of integration with our code also made these devices the perfect fit for our project. Also, most large-scale installations are touch-based – where participants have to stand too close to the screen, losing the impact of the piece as a whole.

With recent installations ranging from forests to graffiti, artists around the world are exploring the possibilities for touchless interaction. By breaking down the barriers between people and their devices, the Leap Motion Controller allows people to forge creative, emotional connections through technology – reaching into virtual worlds with no obstacles or distractions.

What sorts of virtual worlds would you like to explore with Leap Motion control? Let us hear your thoughts @LeapMotion and on our Facebook page.