Are you surprised that graphics test results sometimes differ significantly across websites? Let’s look at one of the many causes. This time we will take only one game and in a unique test we will thoroughly dissect it on a set with Ryzen 9 5900X and Core i7-13700K and Radeon RX 7900 XTX, GeForce RTX 4080 and RTX 4090. You will see how extreme results can sometimes be influenced only by a different test platform.
WARNING: All characters, circumstances and events in this article are fictional. Any resemblance to real persons, living or deceased, is purely intentional and intended by the author.
Once upon a time, there was an avid gamer and lover of powerful hardware, let’s call him Mr. Strelka, and one evening, while pondering over graphics card tests, he thought, noticed something strange, and became suspicious. And so he raised a question on his Facebook profile that was bothering him. But he was a bit of a mess, he sometimes confuses different models of cards (but who doesn’t, me too), sometimes he guessed the percentages rather than bothering to calculate them, and because he was hot on top of everything while writing the post, he messed up in him what he could. He swapped some cards, mixed together the results without ray tracing and with ray tracing, and he was not particularly excited that each of the sites measured on a different processor.
A few virtual friends saw the post and nodded their heads in agreement. There were also those who tried to oppose, and several of them even came up with hypotheses as to how such a thing is possible and why it does not surprise them and they knew it a long time ago. And hints of popular conspiracy theories began to appear again about what could be the cause of the fact that in some tests the results come out so differently from the tests of others… After a few days of noting together how bribed websites bend reality, Mr. Strelka somewhere he finally noticed that maybe he was a little off and there was something wrong with what he wrote. The post disappeared from the wall and another appeared in its place. Even in it, he replaced the absolute values with percentages so that it would not be so obvious that although Radeons are not that fast on PCTuning, the measured frame rates on all three sites differ slightly.
Then a wonderful and still relevant classic anecdote comes to mind:
Rúbíček meets Kohn:
“Greetings to them, Kohn. And as I see them like that, I have to tell them something. While they were visiting us with their mistress, we lost two pieces of silver cutlery.’
“Well, what do you think of me, Gag, that I, an honest merchant, will steal their silver cutlery!”
“They don’t get upset, Kohn, that’s why we found them sunk in the sideboard the other day. But, they understand, the shadow of suspicion remained!’
At first I laughed heartily at the thought that such a thing could actually happen. After I had enough fun with how funny I was, I realized that there was really no reason why something like this couldn’t happen. To a person researching hardware performance, some things seem obvious and do not need to be discussed at length. However, they do not realize that someone could interpret the results in the graphs differently, because they have less experience, or do not see so much in depth or do not perceive things in context (this is just an allusion to a popular saying). And that there may actually be enough people who shake their heads over why some tests on some website come out differently than on another and are looking for some kind of backstory behind it.
It is common for me to see users misinterpreting measurement results, including those that they have full control over because they measure them themselves. Typically, for example, in situations where owners are unhappy with the performance of their PC, they feel that the problem cannot be in the processor, because it is only 20 percent used, and they look for the error elsewhere.
It has also happened to me many times that I attributed unexpected measurement results to other causes than those that were actually behind them. It is doubly difficult to solve something like this when the person in question does not provide details about how and with what settings they worked to the given results. Here, for a change, I have to pour ashes on my head – the current description of the card testing methodology is anything but sufficient, so even in the case of my tests it is practically impossible to repeat the measurement independently. I still have more iron than I can handle testing, so I don’t have time to play around with text tuning any more.
Unfortunately, obfuscation around settings and testing methods is now a common practice. With how the general public has penetrated the Internet, it is now most convenient to stick a graph in the test, but not to write exactly how you arrived at the given results, because then you don’t have to explain what it means and why it turns out differently for you. And because if, for example, you screw up something when measuring or copying the results, or something comes out differently than it should, there is a minimal risk that someone would try to verify and contradict it. Arguing with someone about whether the given results are actually measured or made up from a finger is not pleasant. This is especially true for foreign tests, where instead of exotics from a small Czech pond, exotics from all over the world are piled under the article.
Why did Radeon win on Guru3D?
But that first “cited” graph from Guru3D is really interesting because the results measured by this website are quite extreme. Nevertheless, you can refer to it in discussions as an argument that if you take a modern game that is made properly and according to the most standard standards, the Radeon RX 7900 XTX can crush even the RTX 4090, and such games will only increase.
First, we will focus on why the results on Guru3D come out as they come out. At the outset, I will tell you that the quite fundamental difference is that while on Guru3D they are measured results without ray tracingon PCTuning I test the settings with ray tracing (and this is just one of the things you have to be careful about) and for the weakest cards I add a test in full HD without ray tracing.
We compare the performance of the RX 7900 XTX, RTX 4090 and RTX 4080 in the Far Cry 6 benchmark on a setup that is quite close to the Ryzen 9 5950X platform they measured on Guru3D. Let’s see how it turns out on the Ryzen R9 5900X, on which I’ve been testing graphics cards for Computer magazine for some Fridays. And then we compare the results with our current test platform built on an overclocked Intel Core i7-13700K.
Do not take this as a comparison of the performance of both processors! The Core i7-13700K in the PCTuning test set is overclocked and has all cores hard-locked at 5.4GHz. Ryzen, which I use in the test set for Computer, except for the XMP profile for memory, on the other hand, is on the board’s default settings (and it was probably the same on Guru3D, because there about overclocking in the description of the test platform they don’t say anything about overclocking).
