Interactive art helps us extract impulses from our brains, thread by thread, and enact them in the world. Music takes this medium into mind-bending heights. What if we were able to transform any surface into a living, breathing musical instrument? Emerging designer and musician Felix Faire recently did just that with Contact, an acoustic Leap Motion experiment created for the Royal Academy’s “Sensing Spaces” exhibition.

As a first-year architecture student, Faire was struck by how listening to musical progressions as you walk through a space affects the way you move, so he designed an entire concert hall and gallery in the linear structure of a sonata. These initial musings grew into a much larger project on spatial music perception entitled “Music Aided Design.” It was then that coding became an integral part of Felix’s creative life, and he knew 3D motion control would become an essential exploratory engine for his thesis.

“The fidelity of the Leap Motion made me realize this kind of device could track even more subtle musical articulations, and perhaps even be used as a three-dimensional instrument in itself,” Faire told us. “Now I understand more of what is possible with Leap Motion. I have ambitions to try much more complex gestures and motions in future projects.”

For Contact, Felix used hand height, finger count, and a squeezing gesture to trigger various effects in the loop – the visual output influencing how the audience attempt to interact with the sound waves. “Abstract audiovisual synchronicity, while extremely elusive, can be a very exciting and compelling experience,” Felix concluded.

“Sensing Spaces” will continue to run at the Royal Academy in London through April 6th, 2014. It features architectural practices from six countries spanning four continents. If you plan to attend, be sure to tweet impressions, images, or video from your experience to @LeapMotion using the hashtag #SensingSpaces.

We’ve talked about the magic of WebGL before – how it unleashes the power of the web to do incredible things in 3D. With this latest experiment from Bartek Drozdz, you can reach into your browser and play with a variety of cool visuals to music. A liquid gem, cityscape, spherical lines, and more, all responding to Codex Machine’s S.P.Y. or even your own microphone.

As the creative director at Tool of North America, a production company based in Santa Monica, Bartek brings interactive digital experiences to life. Last month, Tool decided to throw a party for its employees and friends in a large building known locally as “the shed.” To bring some WebGL zest to the party, Bartek started working on sound-reactive visuals that would project on a wall – using the huge space to his advantage.

Next, Bartek took it a step further by introducing Leap Motion interaction to the setup. As you can see (and experience) for yourself, the results were spectacular. You can fly and steer between buildings, create strange wave reactions, or change your point of view. Each mode is different, and it can be challenging to discover what they all do. It’s all part of the fun.

“Part of the challenge was that we did not want to put any instructions anywhere, so we had to do something that was intuitive,” says Bartek. “Just let people know that they can hover the device with their hand and see what happens. It made people curious, but was also easy to use. Our guests could interact with the visuals using one hand, and holding a drink in another.”

The web is growing all the time. With these kinds of early experiments, we can catch a glimpse into what the future web might look like – with complex 3D sights and sounds created with little more than JavaScript and the magic of WebGL shaders.

Want to dive into Bartek’s trippy experience of sight and sound? You can check out his live demo on Tool’s website. To use Leap Motion on the web, be sure that you’ve checked the “Allow Web Apps” box in the Leap Motion Control Panel (General tab). As always, we recommend Google Chrome.

What do you imagine we’ll see in the web of the future? Let us know in the comments below, or give us a shout on Facebook and Twitter.

Ever since the first human stacked one brick onto another, architecture has been concerned with creating immovable things. Even with the rise of smart interconnected environments – where lights, heating, doors, and other systems within a building all work together – the physical structures of our buildings remain the same. As a result, the movements and interactions of people within these spaces are shaped by the buildings themselves, like water flowing through a canyon.

This is why architecture and urban design are about more than simply ensuring that our buildings are safe and efficient. Or that they are merely beautiful. Buildings can inspire or isolate, connect or divide, so that debates about everything from the nature of community to the fate of doorknobs have radical social implications. How we live and work every day cuts to the core of what makes us human. But what if buildings could respond to our movements and gestures? What would that change?

Inside the Aether Project

These were the questions asked by a team of students at the UCLA Department of Architecture and Urban Design. Refik Anadol, Raman Mustafa, Julietta Gil, and Farzad Mirshafiei wanted to explore how architecture itself could become smart and responsive. Technology, they say, is the story of how humans have embedded mind into matter – and architecture is no exception.

Using KUKA KR150-2 robots, projection mapping, and motion control, the Aether team created “an immersive interactive environment that gives a glimpse into the near future of artificial intelligence and its effects on human existence in an environment that bridges the physical and the digital.”

Aether begins with Leap Motion interaction. The movements of your hand have the power to transform a robot-controlled surface geometry. At the same time, another robot projects elaborate designs that are synchronized with the physical geometry, creating a choreographed robotic dance. The patterns that course along the surface are constantly shifting. Geometry, depth, shadow, and color – all rise and fall as the pattern changes.

Interaction as design medium

What does this have to do with architecture? With the growing use of robots and smart hardware, architects will have a lot more freedom in how they design the world around us. The Aether Project shows how a physical structure can be transformed through interaction, and takes it a step further by creating designs that can be layered on top of that structure. Interaction becomes a physical design medium, just like it already has for the digital world.

We’ve just scratched the surface of the ideas behind The Aether Project, which include thoughts on the nature of consciousness, artificial intelligence, and how human behavior might factor into the blueprints of the future. Check out the extended version of this post on our developer blog, including a Q&A with the creators. (For more interactive art with big ideas, you might want also to visit the digital forests of Growth or the stark beauty of Ascension.)

We’d love to hear what you think. Where will this desire to implant our consciousness within machines take us? And how will we interact with machines in the future?