The concept of skyboxes has been around for years — a big dome around your virtual world which holds the backdrop; the horizon details beyond the important assets. Even first generation consoles often used a skybox to keep the world enclosed. With the advent of VR technology, skyboxes continue to be an important tool in any world designer’s kit.
The modern challenge for keeping this long-lived technique valid, however, is in the details. You could forgive a nineties arcade game for its 240p backdrop. Users entering the virtual world, however, expect a far higher quality experience, not to mention how headache-inducing blurry VR assets can be. When Corporation Pop was approached about a photo realistic VR experience then, it made sense to start with the boundaries and work in.
A Patchwork Sky
The biggest issue with a photo realistic skybox is getting a photograph for the backdrop of high enough resolution. The best 360 cameras on the market are pushing 16k, which is far more than the human eye can detect. It’s perfect for a standard photo, but actually not enough for a VR backdrop. This is because the 16k being stretched around a 360 sphere makes it far fewer pixels per degree than a standard photo, effectively reducing the resolution considerably. Instead, we opted for a 64k backdrop which comprised 8 8k images stitched together. The image below shows how we split the skybox into 900 subsections. Each subsection is a unique object that we can manipulate independently of the rest of the sphere. To keep storage down, we combined these images into a single material, using UVs of the sphere sections to assign the correct part of the image. The result is a skybox so high resolution that it feels genuinely real when wearing a VR headset. If you can ignore the Oculus Quests small field of view that is.
Smoke and mirrors
The technically savvy might wonder what the performance implications are for this level of detail in a skybox. There are 900 images being rendered, after all, in just the skybox alone. We haven’t added any actual models yet. The solution to this lends itself neatly from the patchwork approach used — occlusion culling. You see, because we already split the skybox into so many pieces, turning sections off that a user isn’t looking at is very simple. You can see an exaggerated example of this below, where fairly random sections of sky have been turned off. Essentially, even though our total skybox is an impressive 64k resolution, most major VR headsets can only boast a 110 degrees field of view, meaning we only need a third of our skybox rendered at once (a sixth if we consider the polar field of view being limited as well). The result is a smooth, high resolution backdrop to start building our virtual world in, which stands up to all but the sharpest head motion.
I wouldn’t recommend rapid turning in VR anyway — the bruise on my forehead will back that up.