Turns Out Pretty Much Everybody is Cheating on Android Benchmarking Tests
Further evidence that benchmark scores are a horrible way for the average consumer to choose an Android phone oozed to the surface this week, since it looks like most of the biggest vendors essentially cheat on the tests.
Fri, October 04, 2013
Network World — Further evidence that benchmark scores are a horrible way for the average consumer to choose an Android phone oozed to the surface this week, since it looks like most of the biggest vendors essentially cheat on the tests.
The idea is pretty simple devices designed to produce inflated benchmark scores have hidden subroutines that remove the usual limits on CPU or GPU power when known benchmarking apps are running, allowing them to achieve performance in excess of what would normally be possible. So the performance you get when actually using the device won't match up with the performance the benchmarks indicate.
[MORE GOOGLE:Google buys Flutter, a gesture recognition startup]
Samsung was the first Android manufacturer to get pilloried for this, a couple months ago, when it was caught cooking the hardware on the Galaxy S4, and the company seems not to have learned its lesson. Reviewers quickly noted that the Galaxy Note 3 uses extensive "benchmark optimization," and Ars Technica found that the same technique may partially mask potential performance issues in the latest Galaxy Note 10.1.
Sadly, this type of trickery isn't confined to Samsung's devices an extensive report from AnandTech shows that LG, HTC and Asus are all in on it as well. (Who had clean hands? Motorola and the people who make That Other Smartphone.)
Although I don't want to overstate the importance of this issue, it remains that getting involved with this kind of low-rent chicanery was a stupid decision by all involved. Deceiving customers and the media in the interest of minor increases on benchmark scores? Does that really seem like a decent risk-reward calculation?
Samsung, in particular, is having a rough week where its credibility is concerned. The company had promised that Galaxy Note 3s sold in Europe would not be region-locked which turns out to not be strictly true, either, according to a threadnaughtA on XDA Developers.
Users are finding the region lock still firmly in place, so I guess it just kind of sucks to be you if you were trying to use the Note 3 while traveling internationally. This news comes after Samsung had stated publicly that users would be able to unlock the devices for free. The company really, really needs to clarify what's going on here, and refund or replace customers who didn't receive the handset they thought they were buying.
(H/T: Android Authority)
On to slightly less aggravating news leaked information suggests that the rumored Nexus 5 from Google will be released before the end of the month, and AndroidWorld.it has what it says are the specs for the forthcoming new phone, thanks to Myce.com.