VR Portrait and Virtual Production

The VR Portrait and VR Window, and that obvious use we completely missed. 

Way back in 2000, Darrel Grien came to me with an idea. He wanted a virtual 'window' into places he'd been before. The idea was he'd set up a camera when he went on a trip, capture about an hour of video, and then play this video back on a screen on his wall. Pretty straightforward, and I started putting that together for him. We even got a patent on it. 

It was Glorious

https://patents.justia.com/patent/20010033344

But before we were at the patent stage, we'd gone off on a few tangents. What if it was portable? What if it was wireless? What if it could mirror your computer screen? What if you could take it with you to the bathroom so you could watch an important game play even if you had to tinkle? How could we make it feel more realistic, like a real window? What if you had these in a cubicle all around you?

Well, we needed to build a prototype first. Parts for it were crazy expensive. Even a small LCD screen was around $1,500. Most parts straight up didn't exist, at least in a form I could buy. Wireless transmission of video was tricky to get right. Lithium ion batteries weren't a mass market thing yet, so this thing needed a big honkin' brick of a battery. 

And we wanted parallax. I just didn't know what it was called. I think I called it projected motion. 

VR Portrait's Parallax System

(this turned out to be next to impossible at the time)

But the more we worked on it, the more it became obvious this had wider reach. I'd designed a wireless signal receiver and a secondary broadcaster that pumped video over it. And the screen had a battery pack, so you could hang it on a wall. It was bulky, but it worked. 

We were convinced that if we could just get it to an expo in Vegas, we'd get rich off this thing. I remember trying to solder the final components in place, in the lobby of the Embassy Suites in Houston, with Darrel waiting for his plane. 

Needless to say, we did NOT get rich off this thing. It barely worked, and was heavy as heck. 

And one of the big things I just couldn't figure out was parallax. I wanted to be able to track the person as they moved around the room, and fiddle with the video a little, to make it at least kinda look like the background was moving, like it would from a real window. 

The search for an answer there took me down a lot of paths I'm glad I went down (panoramic video, iStill, human shape detection, motion tracking), but I wasn't able to figure it all out. My best solution involved four cameras, and running their data through a DSP. It was not great. 

We did come up with a number of useful scenarios for this, such as showing a background behind a person for recording an advertisement, or using it as a temporary, portable screen for a wireless camera.  Our uses were much more tied to what we would use it for. 

So what we completely missed was the fact that a movie studio would eventually have the budget to build real walls of these things, and use this instead of building a real set.

And of course, someone would need to invent Unreal Engine 4, to drive the background video. The result is the incredible magic behind The Mandolorian. 

That last part is funny to me because I would often play Unreal late at night, to help stay up a little longer while I was working on the hardware. 

In the end, I built one prototype. I don't know if Darrel even has it anymore. But hey, we tried. Also, we got a patent for it. It turns out you have to *defend* a patent though, and we didn't do that. 

So.. moral of the story? I don't know. Maybe defend your patents, and show stuff to people who aren't in your field of experience. Maybe they have a better use for something than you do. Innovation lies at the intersection of disciplines. 

-Chilton