Oculus Crescent Bay interview: Nate Mitchell on prototype tech and VR presence

After testing out Oculus VR's new headset prototype Crescent Bay , I put the pieces of my brain back together enough to have a coherent chat with Nate Mitchell, VP of product at Oculus.

I last talked to Nate at E3, when he walked me through demos of SUPERHOT and Lucky's Tale. This time, at Oculus Connect, we talked about the new Crescent Bay prototype—what Oculus had to improve from DK2 to achieve "presence," what kind of hardware it takes to run games at 90Hz, and whether gamers who ordered a DK2 should be upset that there's already a new prototype on the way.

Wes Fenlon, PC Gamer: What the hell kind of hardware am I going to need to run this thing?

Nate Michell, Oculus VR VP of Product: The demos in there are running on high end machines. We try to be super transparent about that. They're running on the new Maxwell GPUs from Nvidia.

PCG: The 980?

Mitchell: I think it's the 980s. The demos are running at 90Hz. That really becomes the gating factor here. For that 90 frames per second, in UE4 where we're doing a lot of complex artwork and trying to render complex scenes, they're high end GPUs. They're high-end systems. i7s. You're going to need a high-end computer for that sort of experience.

But I think what's important to contrast is, looking at GearVR, there are these two clear categories. GearVR is not a high end GPU, but you're still able to have a really compelling experience. It's cool being able to show people that. We've been telling people that for a long time, and they're like, "you really have stuff running on mobile devices that's compelling?" And we're like "Yeah!"

But now you can see it with GearVR. Dropping into Oculus Cinema, it's compelling. You do feel like you're there. It's not the same level of presence or fidelity. It's not [Epic Games'] Showdown.

PCG: I would say it's nowhere near, but I was impressed with GearVR in that it felt like it was close to DK2 or Crystal Cove, the first time I experienced it.

Mitchell: Yeah, it's somewhere between DK1 and DK2. The lack of positional really hurts immersion. So you do this [tilts head] and nothing happens. That's killer.

PCG: In terms of the quality of display, I felt probably a little more motion sickness not having the same level of refresh and blank frames...

Mitchell: Definitely. And the optics aren't quite as good.

PCG: What can you talk about with the new Crescent Bay, hardware-wise, tech-wise, what you improved? The optics, the screen?

Mitchell: I'll run through the list and you can ask whatever you want and I'll refuse to answer if I have to. There's improved display technology. The displays are running at 90Hz. We've integrated audio. We have 360 degree head tracking. The LEDs on the back of the headset let you turn around and crouch. We have new optics.

And the optics are actually responsible for—one of the things about the whole system is that we talk about it as this total experience. We want people to focus on the experience. Some people come up and say "what is the resolution of the display? It's so high, I can't see the pixel structure!" Part of that is the optics working with the display, working with the mechanical engineering, working with the industrial design, to make it, perceptually—if you saw Abrash's talk, it's all about the human perceptual system. The perceived resolution seems really high, but it's not important what the real resolution is if the perceived resolution is that good.

And the ergonomics and the industrial design are obviously upgraded. It's significantly lighter, a lot more comfortable. The straps are a pretty cool system.

PCG: And just the way it fits on your face is different too. You get a little bit of vision—

Mitchell: Light leak...

PCG: Under the bottom of the headset. Was that actually intentional so you can see, basically?

Mitchell: I don't know if it was intentional or not with these prototypes. Sometimes we build prototypes that have that just so you can see. It's not super important when you're building prototypes to have it be fully mindblowingly immersive. On the flipside of that, it might be that we did that for nose clearance or something like that.

PCG: It felt like for the first time someone was experiencing something like that, you almost need that visibility there so that you don't fall on your ass or walk into a wall.

Mitchell: It definitely helps. It's something they actually talk about with the Valve room. Having that space out the bottom can be good for people so they're not walking into walls.

PCG: The crazy thing is, I guess if I look down deliberately and exaggerate that gap, it would break immersion, but in general, even when I could see that in my peripheral vision, it doesn't break presence the way I thought it would.

Mitchell: There's a threshold there. You're absolutely right. If you have light leak on the whole thing, it'll break presence. And light leak can come from other spaces too. In DK1, one of our early units came back, and there was so much light leak in the foam padding that the screen was bright and there was glare. And we were like, this is not usable. So there are different thresholds. Some space there can be good, but it depends what the user wants.

PCG: Let's go back to improved optics. What does that mean, exactly?

Mitchell: Ahh, I can't go into it too much.

PCG: Putting aside the improvement part you can't talk about—just in general, with optics, are we just referring to the glass that's between you and the screen? I feel like optics is a very encompassing topic.

Mitchell: Yes, there is more. So I'm a software engineer, and one of the things I often talk about when it comes to getting into hardware, was how surprised I was at the complexity of optics. We would sit in these rooms when we were going over the design of the Rift, and I'm not a hardware guy, but I'm in there as a product person. And they'd be like, we want to increase visual clarity, so we're going to increase that, and then everything else drops, and they're like, well, we can do this, turn all these dials...every single element changes everything else.

That's a super simplistic description of it, but the complexity of optics is just mindblowing. There's also all sorts of things like collimated light, right? Optics is affected by the screen to lens distance. So it is, you're right, not just the glass, but the actual mechanical engineering and industrial design as well.

What we've done here is we're using a different lens that DK2. A lens that we're generally more happy with. In terms of the lens, the screen and the lens together. The tradeoffs and the characteristics that you want. But when you look at the lenses that are in Crescent Bay, like I said before, the perceived resolution seems significantly higher.

PCG: But you can't tell me if it is or not?

Mitchell: There's a lot of optical magic happening there that really helps.

PCG: So, presence.

Mitchell: Presence. I wanted to say one thing—you asked me about all the Crescent Bay stuff. [Presence] is the final thing. The most important thing. Being able to deliver presence at a level that's impossible with any of our other VR headsets before.

PCG: I don't remember you guys even using the word presence before this point. Maybe you did and I don't remember—but at first I thought, okay that's a good buzzword, and I understand what it means in this context. I didn't really buy it until I used Crescent Bay. And now I buy it. What were the big steps that you took, when DK2 was done, what were the pillars you had to hit to get presence? When did that become an achievable goal?

Mitchell: Like Brendan said, a lot of the things we do are in the pursuit of presence. In Brendan's keynote, when he talked about the five different things—optics, latency, tracking, resolution, persistence—he actually talked about some of the rough specifications for some of those things. For persistence, he was saying it needs to be less than 3 ms, and for latency it needs to be sub-20 ms.

So we knew some of the thresholds, at least theoretically. In our experience, at least—this is a new field. It's not like "here is a line to draw for presence" and then you cross it. THere are all these different things, and you're like, probably true, probably true, probably true. But if you take any one piece out of this house of cards, it all comes tumbling down. We're pushing on every axis, in terms of better display technology, higher framerate, which meant lower persistence, better tracking. Some of these things are fundamental, others are technical.

But we knew where we wanted to go, and it was just a matter of putting it together and iterating on the hardware until it was showable. These things come online in individual parts. You might have a 90Hz panel running at your desk, but without the 360 tracking and the new optics and everything else, it's not a huge leap forward.

PCG: For people who bought a DK2, there are some people out there who don't even have it yet. Should they be kind of pissed right now that this awesome new thing has come out and now their old one feels outdated?

Mitchell: They're allowed to feel whatever they want—that'll be my PR answer. They shouldn't be pissed because we have no plans right now to release Crescent Bay. No plans. It's just a feature prototype like the HD prototype that we showed at E3 2013. But they might still feel pissed, and I don't have a good answer for that. And I'm sorry. They should be able to cancel their order, I think, if they want. But I would say, if you're a developer, get your hands on DK2. It allows you to build experiences for Crescent Bay. At the end of the day, this is a dev conference, it's not the consumer Rift. We still have a long way to go.

PCG: First DK1, then DK2. These both felt like very PC gaming oriented things, it's something the core guys are going to get and put it on and sit at their desk. It's going to enhance games in some way, make it more immersive or whatever.

Mitchell: It's [about] delivering on the dream of the Holodeck.

PCG: Now this is something you're going to use standing up...

CV1 is still targeting sitting down, but go on!

PCG: How do you bridge that gap between this being a product designed for the PC you have at your desk, but this being something that has to run at a high end machine, but you need to maybe have Kinect-type space around you.

Mitchell: For CV1 we're still targeting that sitting experience. We expect people to sit at their desk, set up the headset, or at the couch, whatever it is, and have an awesome VR experience. The reason we do this standing up here at Connect, is because standing up does amplify presence. So does audio. If we took the audio out of all those demos, you'd probably still achieve experience presence. And you put it in, and it's not great audio in [the demo], but it's just impactful. Your brain is more convinced. It does amplify it.

Standing is another thing that amplifies presence. We still want people sitting down for the consumer version of the Rift. We'll see if people build little spaces. Carmack talked about his VR lair. One of the beautiful things about mobile is that it's portable, so you don't have cables, but you don't want to be walking around with that either.

We've got a long way to go on that front, really getting it right, where you have a holodeck and aren't crashing it into walls all the time.

PCG: Maybe you should ship it with some foam blocks people can build a fort out of.

Mitchell: There you go. I like it. Product idea.

PCG: Thanks for your time, Nate!

Mitchell: My pleasure.