Benchmark tests are great, but all I care about is real world performance

Taylor Martin
 from  Concord, NC
| March 19, 2012

When it comes to tablets, there are two that have found themselves in the limelight as of late. Apple's new iPad, as most new iDevices do, drew the crowds out on Friday for its official launch. And the Internet has been chock full of new iPad content the entire weekend. For the Android crowd, though, there is one single tablet that takes the cake, the ASUS Transformer Prime, which launched back in December of 2011.

During the new iPad announcement, Phil Schiller took the stage and flipped to a slide describing the third-generation iPad's new A5X dual-core processor, touting that it has four times the graphics power as the NVIDIA Tegra 3, the very processor the Prime uses. Those were, as any Android fan would have it, fighting words. It even forced NVIDIA to speak up and ask Apple what benchmarks they had used to come to such an assessment.

We know it's not all about the cores; the Snapdragon S4 serves as a testament to that. But Apple wasn't calling out the Tegra 3 on processing power. They were claiming the A5X chip has better graphics performance, and comparing GPUs is even more tricky. But a lot of people were skeptical when Apple threw around such big claims. So, over the weekend, Laptop Mag took the Transformer Prime and the new iPad and put them to the test.

What exactly did they find? Using benchmarking software that can be found on both iOS and Android, Laptop Mag learned that the A5X used in the iPad outperformed the Tegra 3 in 3D animations. Laptop Mag's Avram Piltch explains:

"On the program's [GLBenchmark 2.1] Egypt Standard test, which shows a 3D animation of ancient Egypt, the new iPad and its A5X chip processed 6718 frames at a rate of 60 frames-per-second to the Transformer Primer’s 5,939 frames at 53 fps.

When GLBenchmark ran the Geometric test (vertex weighted), which measures low-level shader performance, the new iPad processed 7,530,524 frames at a rate of 57 fps while the Tegra 3-based Transformer Prime processed just 3,523,926 at a rate of 27 fps. We saw the same trend on the program’s Fill test, which measures texturing speed, as the iPad processed 1.98 billion textels per second to the Prime’s 404.61 million."

On Geekbench, though, the Tegra 3 blew the iPad away, says Pilch. The Tegra 3 scored a 1,571 compared to the A5X's 692. As far as integer, floating point and memory subtests go, the Tegra three edged out the A5X in every category. However, the A5X edged out the Tegra three (324 to 266) on the stream subtest.

But all of these numbers mean exactly zlich to someone who doesn't dwell on benchmarks.

I, for one, couldn't care less what the benchmarks read and really only care about how the devices fare in real world performance. I don't care if Device A bests Device B in some arbitrary test. There have been too many instances where high benchmark scores are not reflected in a device's actual performance. And in a test like this, where any other test might be a tie-breaker, the differences in benchmark scores are negligible aside from bragging rights and how the devices compare on paper.

What matters to me is how the devices perform when they're in your hands and you have a game loaded up and are pushing it to the limits – something I tend to do far too often. I often play games while streaming music on my tablet, and switch in and out of a game to respond to tweets, emails, etc. If a tablet or smartphone can handle all of that without keeling over, I'm happy.

Luckily, Laptop Mag also pitted the two tablets against each other in a subjective gaming test. They played Riptide and Shadowgun side by side on the two tablets. The iPad's Retina Display made for much sharper images, but the powerful Tegra 3 allowed for simulated graphics, like billowing flags in Shadowgun and water splashes in Riptide.

So what are the takeaways from Laptop Mag's tests? Apple's claims of the A5X having four times the graphics power of the Tegra 3 was pretty accurate and the Tegra 3, having two more CPU cores, bests the A5X in processing power. Both SoCs are great, leaps and bounds beyond the CPUs and GPUs of last year. Both the new iPad and Transformer Prime are going to play games, browse the Web, run applications and keep you plenty entertained. Neither of them will totally replace your laptop or desktop ... yet, and neither are going to bring you breakfast in bed.

Worrying about which chip is better and which one offers better graphics or better processing power is futile. Chances are, you are just going to replace it with the next best thing once it's available anyway, and software is really going to be the limiting factor in the end.

How do you feel about benchmark tests, readers? Obviously, you want any device you own to perform well. But do you worry about performance tests? Will you buy a device based solely on benchmarks? Or, like me, do you only care about getting a device in your hands and seeing how it performs in actual usage?

Products mentioned