For a while I was looking for an app that will let me dance to electronic music in VR. After all – every good dance club makes a big effort to augment the music and the beat with great visual stimulus.
Oddly enough – I couldn’t find one. The best recommendations from the community were AudioShield – a musical game, and Old Friend – a VR clip that was inspiring and very funny, but not what I was looking for. So I took the opportunity to learn more about user experience in VR and built one. I call it VRave.
This is a small glimpse of the experience (in “real” it’s much more vivid). It’s very basic, but you can imagine what it can evolve to:
Music: Pegboard Nerds – Hero (feat. Elizaveta)
Well, it’s not really part of VR, but a core part of this experiment. Analyzing a music track and extracting the beat is quite tricky, if not impossible (AI?). I mean – I can tweak the settings, like narrowing the range in the audio spectrum that I’m analyzing, converting linear scales to logarithmic to increase the contrast between treble and bass, etc, to intensify the peak moments of the track over the rest of the song. The problem is that each track requires tweaking to make it work well.
So my first idea for a workaround was to reverse engineer beat detection: Instead of relying on the music – I can analyze the body movement and use it to control the visuals. For example: Detect the height changes of the head for the main beat, or interpret the dancer raising his hands as a exciting moment in the track, and show more lights. That was an exciting theory, but it didn’t work out well so far. When the head movement wasn’t exactly on the beat, it ruined the dance. So for now it’s still just kind of an audio visualizer.
The second reason to use hands is that having some kind of control on the visual is magical. You can’t have that in a real dance club, but why not in VR?
At first I used the touch controllers, and it was OK, but holding controllers in tight fists while dancing is a burden. So I got a Leap Motion kit, and after a day of cursing I managed to fire sparks from my ‘real’, bony hands in the app.
The surroundings are based on randomness in multiple ways, and one of them is the position of these light balls. But the played with the range to find out how close can I bring them to the dancer. When they were close, it felt more immersive, and it made me want to reach out with my hands to the lights, but in many cases I felt it was too close, especially when it was a ‘boom’ moment.
So, to solve this, I added some particles around the dancer. That gave the effect that I was part of the surroundings, and I can reach out and touch something, and the lights could be distanced now.
Dancing alone is weird. Maybe it’s just a habit, but I feel like I need to see other people enjoying the same thing I am when I dance. I wonder if it will have an effect if I new these ‘people’ are not real…
Some would find it funny that I aiming for a solo dance experience and talking about social at the same time. I was not looking for a way to dance without the hassle of going out (although that’s not a bad value). My claim is that VR can provide a visual experience much more emotional and impressive than the main stage on the EDC event opening.
As I mentioned, this is a basic POC. I can imagine how artists can design these environments, and how you, as a dancer, could enjoy raving as the environments change. If anyone wants to try it or design an environment – shoot me a tweet (@eyalshahar).