Comparison of VR benchmarks with AMD and Nvidia

tl; dr Current VR benchmarks are rare. BitcoinMinersHashrate compares AMD and Nvidia, taking into account that FPS are not the measure of things in the virtual world. Instead, the supersampling factor is increased in the test until the GPU surrenders and frame drops occur. Nvidia clearly sees the results in the front.

For years, BitcoinMinersHashrate has provided the graphics card ranking list in which the best graphics cards are recommended from Full HD to Ultra HD. Depending on your own budget and the desired resolution, everything is here, from the entry-level model to the high end. Classic games form the basis.

GPU requirements in VR

But especially in view of the new generation of VR headsets in the form of, for example, Oculus Rift S (test), Valve Index or HTC Vive Cosmos, the question also arises for VR players which graphics card it is for immersion in the virtual world Insights from classic games cannot and should not be transferred 1: 1.

This is not only due to the APIs including their software tricks of the VR platforms, but also because V-Sync is inevitably active in VR. A classic benchmark is as long as all GPUs can meet the V-Sync limit, a car race through a Tempo 30 zone, in which all participants adhere to the speed limit.

The “currency” in which test candidates are measured is different. In VR you often want to turn up the internally calculated resolution, i.e. the supersampling factor, because this makes the picture much clearer and details that would otherwise be lost become recognizable. It is important to avoid the unsightly synthetic intermediate images or even frame drops, which in the worst case can lead to “motion sickness”.

GPU benchmarks in VR

Not only are the requirements for your own computer different in VR, but deviations from the usual procedure on the screen must also be considered in the benchmark. Classic FPS counters such as Fraps are out of the question here because, in addition to the really newly calculated images, there can only be slightly adjusted old images – and differentiating them is definitely relevant. As in previous VR benchmark tests, the editors used Nvidia's FCAT VR instead, which has proven itself as a benchmark for VR benchmarks since 2017 and, unlike the classic counterpart known to date, does not require a high-end computer with a RAM disk. The VR headsets used in the test are HTC's Vive Pro with 90 Hz and the Oculus Rift S with 80 Hz.

Real new versus synthetic (old) frames

Basically, there is no difference between the output on the screen and that in the VR headset. Put simply, the CPU delivers the data and the graphics card turns it into an image (frame). But that's not enough for VR. Depending on the headset and the situation, the image must be adjusted further before it can be displayed. Typical tasks are color adjustments and lens-specific corrections (warp). For both, the system has a total of a good 11 ms to deliver the 80 frames or 90 frames per second desired by the headset. With FCAT VR from Nvidia you can read out how long the graphics card took for each individual frame and calculate on this basis how many pictures would have been theoretically possible.

It becomes more interesting if the graphics card is not fast enough, i.e. does not complete a new image in 11.11 ms (Vive Pro) or 12.5 ms (Rift S). In a classic PC game for the screen, the following would then happen: the old image is displayed again and the user sees a short stutter. This is always annoying, but firmly anchored in reality does not make the user feel bad. In Virtual Reality, the unchanged image no longer fits the movements of the user and the brain realizes that something is wrong. The user feels uncomfortable to bad (“motion sickness”).

But both Oculus and Valve now offer software tricks in their SDKs that work around this problem. This is where the synthetic intermediate images come into play, which do not contain any new information from the game, but do contain new player position data. At Oculus this is called Asynchronous Timewarp (explanations in the developer blog), SteamVR offers a similar technology. FCAT VR makes it possible to measure the images actually delivered per second and shows exactly on the frame whether it was on time, was interpolated by a synthetic intermediate image, or there was a frame drop.

However, this happens rarely, especially with a fast graphics card and without supersampling, and in most games everything just runs constantly at the frame rate required by the respective headset. In these cases, a fast current GPU gets bored. The computing power that is left here can and should be converted into supersampling when playing in VR, as this massively improves the perceived sharpness and quality and often has a greater visual benefit than turning up the in-game settings.

Test system and graphics settings

The graphics card benchmarks were performed on an Intel Core i7-6700K overclocked to 4.6 GHz, which is equipped with 16 gigabytes of RAM according to DDR4-2666 in dual-channel mode. The drivers used were AMD's Adrenalin V 19.6.1 and Nvidia's GeForce V430.86.

Eight graphics cards from AMD and Nvidia

Graphics cards included an MSI Radeon RX 580 Gaming X 8G, an MSI GeForce GTX 1060 Gaming X 6G and an MSI GeForce GTX 1660 Ti Gaming X from MSI, the Inno3D GeForce GTX 1070 iChill X3, an EVGA GTX 1080 Ti SC Black Edition and the RTX 2080 Ti in the Founders Edition. The test field is rounded off by a PowerColor Radeon RX Vega 56 Red and the AMD Radeon VII. All graphics cards ran at the clock rates specified by the manufacturer.

The display was made on the headset specified in the benchmark. A room scale setup was used for the Oculus Rift S, two Lighthouse stations for HTC's Vive (Pro). For the test, the headsets were each attached in a fixed position, approximately at head height. BitcoinMinersHashrate determined all measured values ​​with the software Nvidia FCAT VR.

Graphics settings and test scene

The test sequence was chosen with the aim of being reproducible. On the one hand, it should have few or no elements that change significantly with each run, and on the other hand, it should place a requirement on the graphics cards that is in the range of what is required by the game in question.

For this, scenes were selected right at the start of the game: Trover Saves the Universe and Skyrim VR each had the intro, Robo Recall the start of the second mission, Moss the second story sequence after the first level. Project Cars 2 used the same test scene as the classic benchmark, only without an AI opponent.

The graphics settings were chosen so that at least for most graphics cards the 80 FPS or 90 FPS can be maintained without supersampling. This makes it especially necessary for Project Cars 2 to use the lowest settings, because otherwise it was impossible to achieve constant 80 FPS on all AMD GPUs, regardless of the supersampling used.

The measuring procedure in detail

As mentioned, just looking at the frame rate in VR does not provide a good comparison of the performance of the graphics cards. In order to do justice to this circumstance, a new evaluation method is used in this test for the first time, which should better correspond to what happens in real virtual use.

For this purpose, any kind of automatic resolution adjustment is first deactivated in every game, in SteamVR and in Oculus Home. With SteamVR this setting can be found in the sub-item "Video", while with Oculus Home using an Oculus Rift S there is the option "Prefer quality" which ensures that the full resolution is always used.

Turn it up until the graphics card breaks down

Now the individually calculated resolution for each graphics card is turned up via the Oculus debug tool or SteamVR settings until the native image refresh rate of the headset used can no longer be maintained. This is determined by looking at the “Delivered FPS” measured with FCAT VR. In the diagrams for the individual games, the FPS achieved is then plotted for three points over the supersampling used. First, the first value at which the native refresh rate is not reached, and the last two points before.

Supersampling is not the same as supersampling

It should be noted that SteamVR and Oculus specify supersampling in two different ways, both of which make sense in their own way. SteamVR shows the percentage change in resolution, while Oculus gives a factor by which each pixel is multiplied horizontally and vertically. An example:

To calculate the average values, the last value is now taken at which the frame rate still (almost) corresponds to the native of the respective headset. To calculate the average, the Oculus supersampling factors are then converted into the percentage of SteamVR and vice versa. Both representations are available in the diagram.

On the next page: test results and conclusion