What is it?
A small game inside a VR Cave to show kids how a ecosystem works.
It’s currently being displayed in “La Rodadora” Museum at Ciudad Juarez.
Here are some pics of the system in action:
The Cave had 5 proyections, the only thing left without proyection was the roof.
The interaction with the users was made using the “shadow” of the user in the wall taken from the kinect input.
What did I do?
I was given a theme and an overview of the application to propose and design the gameplay of the VR cave.
I also coded everything in the system from the kinect input processing and the networking to the game itself.
- Design of Gameplay and Interaction.
- Kinect Input Processing DLL using OpenGL and OpenNI.
- Networking between Client and Server PC’s in Unity.
- Proyection Distortion Shader for multiple screens.
- Mesh generation and collision from kinect Input.
- Game and Main Application Coding.
How did I do it?
To create the virtual cave it was necesary to use 2 computers with NVidia surround in each PC.
The server handled 2 Kinects and the 3 main proyections in the cave.
The client handled the other kinect and the entrance proyection as well as the floor.
I used OpenNI 2.0 for the Kinect input.
The kinect area was delimited using a box which could be setup inside the C++ proyect and was handleld using separate files for each kinect setup.
After that it was necessary using clustering to separate each blob to create a convex hull of every person. For this i used DBSCAN.
This module returned a 2D Array that contained each convex hull on a kinect at that frame. It was handled using a DLL to be able to use this information in Unity 3D.
What was the hardest challenge in this project?
The hardest problem was creating a individual convex hull that could collide with the elements of the wall.
Also as a extra note, after the release of the system it had a problem with a memory leak inside C#, which i was suprised to see because of the Garbage Collector, however it seems this was made because of the Mesh Generation every frame. It seems unity holds a reference to every mesh you create somehow so it had to be deleted manually before destryoing the object.
Looking back and after some research I strongly regret not using a ATI card and a stronger PC instead of 2 separate ones to avoid the network problems.
That would have helped a lot to improve development time and make it easier for the museum personnel to operate it.