polaris~
An Audiovisual Augmented Reality Experience Built on Open-Source Hardware and Software (2021)
← Software: CalibrationSoftware: Planning Musical AR Instruments →
Software: Running Demos
(May - July 2021)
Through-Combiner Recording
In July I added a camera to the headset, to be specific, I have a Raspberry Pi Zero, taking power over USB 2 ribbon cable from the CombineReality Integrator, connected to a ZeroCam (small camera), placed just to the right of the left screen on the headset. This results in the camera being in front of my left eye. Despite making it harder to see objects, for documentation and archival purposes, the camera has been a great addition. Following these instructions, the camera shows up as a standard USB camera through the headset’s USB 3 cable. This cable to the PC is now not only powering and transferring data to and from the sensors and displays, but now also the new camera, all through one cable!
In order to take these videos, I use OBS to composite my webcam, specific portions of my screen (during Unity Demos), as well as the through-combiner camera I have set up, and later on, binaural audio. Overall, whilst it looks and feels hacked together, and could probably do with a 3D printed enclosure, the system works perfectly well for documenting the experience of wearing the headset.
Paint & Cubes
Now that the headset is built, the SDKs are installed, and I have a 2D optical calibration, it was time to run a few demos. The first video above shows this original Unity demo from Leap Motion. As mentioned in the calibration page , if you want to run this yourself, you need to be on the multi-device support SDK.
Project Esky
Project Esky is a open-source software platform apable of high fidelity natural hand-interactions with virtual content, high field of view, and spatial mapping for environment interactions. This is the software framework by which I am creating my AR experiences in Unity. It is developed by Damien Rompapas, who has helped a lot in helping my project run smoother. Esky allows the North Star to be emulated as a Windows Mixed Reality Headset, meaning that you can use the Microsoft Mixed Reality Toolkit in Unity3D for desigining interactions (like the Microsoft Hololens 2 does).
There are a couple of important steps I had to go through before I could use Esky properly, importing calibration, setting up display settings, and aligning my hands. The third video covers importing my optical calbration into Project Esky, and the fourth video covers hand alignment, which makes sure that your virtual and real hands are aligned.
From here on, when working in Unity 3D, it is assumed that I am referring to the Project Esky Unity Implementation.
← Software: CalibrationSoftware: Planning Musical AR Instruments →