The concept of skyboxes has been around for years — a big dome around your virtual world which holds the backdrop; the horizon details beyond the important assets. Even first generation consoles often used a skybox to keep the world enclosed. With the advent of VR technology, skyboxes continue to be an important tool in any world designer’s kit.
The modern challenge for keeping this long-lived technique valid, however, is in the details. You could forgive a nineties arcade game for its 240p backdrop. Users entering the virtual world, however, expect a far higher quality experience, not to mention how headache-inducing blurry VR assets can be. When Corporation Pop was approached about a photorealistic VR experience, it made sense to start with the boundaries and work in.
A Patchwork Sky
The biggest issue with a photorealistic skybox is getting a backdrop photo of a high enough resolution. The best 360 cameras on the market are pushing 16k, which is far more than the human eye can detect. It’s perfect for a standard photo, but not enough for a VR backdrop. The 16k being stretched around a 360 sphere makes it far fewer pixels per degree than a standard photo. This reduces the resolution considerably. Instead, we opted for a 64k backdrop which comprised eight 8k images stitched together. The image below shows how we split the skybox into 900 subsections. Each subsection is a unique object we can manipulate independently of the rest of the sphere. To keep storage down, we combined these images into a single material, using UVs of the sphere sections to assign the correct part of the image. The result is a skybox with such high resolution it feels real when wearing a VR headset. If you can ignore the Oculus Quests small field of view that is.
Smoke and mirrors
The technically savvy might wonder what the performance implications are for this level of detail in a skybox. After all, there are 900 images being rendered in just the skybox alone. We haven’t added any actual models yet. The solution to this lends itself neatly from the patchwork approach used — occlusion culling. Because we already split the skybox into so many pieces, turning sections off that a user isn’t looking at is very simple. You can see an exaggerated example of this below, where we have turned off fairly random sections of sky off. Essentially, even though our total skybox is an impressive 64k resolution, most major VR headsets can only boast a 110 degrees field of view, meaning we only need a third of our skybox rendered at once (a sixth if we consider the polar field of view being limited as well). The result is a smooth, high resolution backdrop to build our virtual world in, which stands up to all but the sharpest head motion.
I wouldn’t recommend rapid turning in VR anyway — the bruise on my forehead will back that up.
