-
-
Alex driving his car using the Veyesor!
-
Side view of the car while Alex was driving
-
Veyesor logo
Inspiration
On the way to McHacks, we travelled by car and encountered quite a bit of traffic. Not to mention, the front and rear windshields were extremely snowy/icy at first, and our car was full of passengers, travelling supplies, luggage, and other miscellaneous items that obscured Alex’s (our driver’s) view of his surroundings. This made for a bit of an unsafe trip, and inspired us to create Veyesor—a VR-powered application that enables drivers to obtain a free, unobstructed view of their vehicle’s surroundings, as if they weren’t even inside the car at all.
What it does
You put on the VR headset, and you are shown a real-time, 180-degree view of your car’s surroundings—powered by 5 iPhone cameras on the outside of your vehicle.
How we built it
- Python + OpenCV for backend, stitching together our different camera views
- WebRTC for streaming the stitched video
- NextJS + Typescript for the web application
- Vercel for hosting the web app
- Oculus Quest and Unity for rendering in VR
Challenges we ran into
- Searching for an app to hook up a phone camera to the laptop to connect the camera feed
- Connecting the video feed of two phones together
- Finding a suitable way to stitch the videos together
- Deciphering the underlying linear algebra used to connect the camera feeds together
- Grueling over Unity development for 4+ hours
- Figuring out how to pay a parking ticket in Montreal
Accomplishments that we're proud of
- We were able to develop and configure an OpenCV project for live panoramic video stitching, which is generally regarded as a difficult project. Many available online resources are research papers on making this work, but there were very few code implementations of them.
- The project encompases a plethora of software development topics. Including networking, computer vision, pipelines to connect the front-end to back-end to utilize the computer vision model, and accessing it on the web through the VR headset.
- Winning the Super Smash Bros tournament!
What we learned
- Don’t visit Queens and McGill in the same weekend...
- Stitching together videos into a 3-D live stream is hard
- VR development is fun!
What's next for Veyesor
- Creating an fully immersive 360-degree camera view, and a more integrated camera build with the car we attach it to, creating a full build.
- Developing the VR app to have a dashboard with the local speed limit, traffic signs, current speed, time, temperature, etc.
- Map view: shows you the route that you should take to get to your destination, in AR, on the road in front of you!
