11.08.10

Commentary: StatCounter ‘Global’ Statistics

Posted in GNU/Linux at 8:10 pm by Dr. Roy Schestowitz

StatCounter bias

Summary: How StatCounter turns 4-5% of the world’s population into 25% and reduces the world’s largest Internet population (China) to just 2.46%, then claims to be measuring global market share (other surveys do the same thing)

AL submits: “Thank you for all your hard work in bringing us news through Techrights. I am reading it daily and find lots of interesting information.

“I read one of the comments from Mad Hatter in which he was talking about Wikipedia article on OS market share. I went to check it out and found that they use 1% for Linux (globally) based on the research by StatCounter Global. I was interested to see how this group is gathering their statistical data. If you go to their FAQ section they talk about sample size per country/region and there is a link to the full list of all countries. As they stated themselves their pool is 16,3 bln hits. Quite large I would say. But there is something interesting – the biggest group (region) is United States with 3,965,972,279 hits. That is almost 25% of the total pool. Now, my days of statistical studies are long gone but I still remember that in order to have accurate result you cannot over-represent one group. The result will be obviously skewed. We have one country that contributes almost 25% to the result compared to the rest of the world. As StatCounter states that they choose randomly that makes it very likely that lots of data on hits would be taken from USA. You know, for example, how much is the share of hits from China? 2,46%! In fact, looking at the whole list you can see that starting from Korea and further down the share is less than 1%! That includes countries like Poland, Greece, Japan, Russia, Switzerland etc.

“The result will be obviously skewed. We have one country that contributes almost 25% to the result compared to the rest of the world.”
      –Al
“I know some can say that there are many more computers sold in USA than in other countries (can’t be true). But market share is more complex. If we have 95% (example) Linux presence on desktops in China, they would hardly make any influence with representation of only 2,46% on the StatCounter data. Do you see what I mean? There are of course many more problems with that. What kind of websites StatCounter is using to get hits? If we put hit counter on the website with Silverlight I don’t think we will get many hits from Linux OS desktops, right? And even if the websites are getting hits from same amount of Linux OS and other OS desktops what will happen? StatCounter will randomly select hits from global pool and as data from USA will be more likely to get selected it will greatly skew the result and linux will always get under-represented. Lets say you have two crates: one with 10 pears and one with 250 tomatoes + 150 pears and you draw five times. However 3 times from first crate and 2 times from the second. You will have selected more pears than tomatoes. Even though there are 250 tomatoes and 150+10=160 pears. Is this reliable representation?”

Share in other sites/networks: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Reddit
  • email

This post is also available in Gemini over at:

gemini://gemini.techrights.org/2010/11/08/statcounter-bunk-data/

If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channels.

Pages that cross-reference this one

8 Comments

  1. StatCounterGlobalStats said,

    November 9, 2010 at 3:59 am

    Gravatar

    Hi there,

    I’ve just stumbled across your article and hope to clear up some confusion.

    Our StatCounter Global Stats measure various market share and other stats for all countries across the globe… hence the name.

    Our methodology is very simple and we’ve purposefully kept it that way. Specifically, our stats are based on more than 15 billion hits per month to our 3 million+ member sites. We’re not aware of any other publicly available service providing market share stats that has a bigger sample size on which they base their information.

    You’re absolutely correct about us NOT weighting our data. We do not impose artificial weightings on our stats and this is a conscious and deliberate decision. Weighting stats means that the stats are only as good as the weighting methodology used. If the weighting data is inaccurate or out of date, then it renders the data completely incorrect. For these reasons again, we choose NOT to weight our data in any way and instead we report it as we record – other commentators can, however, weight the data as they wish. All our work is shared under a Creative Commons Attribution-Share Alike License (http://creativecommons.org/licenses/by-sa/3.0/) for this specific purpose – so please feel free to download our data and apply whatever weights you see fit.

    StatCounter Global Stats came about because we decided to publicly share interesting trends that we were monitoring in-house. We aim to make our stats and methodology as clear as possible and appreciate all comments, queries and suggestions. If you have any questions for us, please don’t hesitate to contact us via our feedback form (http://gs.statcounter.com/feedback) or by direct email.

    Dr. Roy Schestowitz Reply:

    Hi,

    A few quick points:

    1. The size of the sample does not matter at all. Other companies like NetApps also brag about the number of UIPs, but this number is meaningless unless distributed correctly (nature of the sites sampled, geography, etc.)
    2. How does the data account for dynamic IPs, proxies/squid, and the imbalanced use of the Web browser depending on the user (e.g. # of page requests; this can be correlated to operating systems and browser, e.g. does it support tabs? What is the connection speed?)
    3. How are zombies PCs and other ‘junk’ traffic removed from the dataset?

    There are many other challenges/deficiencies, but it’s commendable when the data and methods (preferably code) are made publicly available for independent audits, provided of course they don’t violate privacy rule (which is a hard problem when browsers are specified very precisely with locational information too). That’s why such surveys cannot reach privacy-conscious sites, many of which appeal to civil rights-aware users (many would GNU/Linux), and that’s just one example. I wrote an article on the subject 3 years ago:

    http://itmanagement.earthweb.com/osrc/article.php/3687616/Can-Linux-Adoption-Ever-be-Accurately-Gauged.htm

  2. Matsi said,

    November 9, 2010 at 1:12 pm

    Gravatar

    Here are the statcounter results for Finland

    http://gs.statcounter.com/#os-FI-monthly-200910-201010
    (showing about 2,5-2,7% for Linux)
    …and here are the results of my homepage (non-geek, 95% non-computer, mostly politics, social discussion…)

    Windows 2785 83.18%
    Linux 302 9.02%
    Macintosh 185 5.53%
    Unknown 73 2.18%
    FreeBSD 2 0.06%
    CPM 1 0.03%

    (visiters: Finland 94,4%, Sweden 0,9, USA 1,1% others 3,4%)

    No big differences between clicks, unique visitors, unique sessions…

    There are several finns telling the same story: Linux has about 7-10% marketshare on their homepages, blogsites…

    So one thing is sure – Linux has much bigger marketshare in Finland than statcounter is telling. My guess is 8% +/- 1%. There is no doubt that situation is quite the same in other regions. Linux is some 3 times bigger than statscounter is claiming.

    Dr. Roy Schestowitz Reply:

    Statscounter needs to tell which actual sites in Finland it is sampling from. These are not randomly selected Finns.

    StatCounterGlobalStats Reply:

    There is still considerable confusion here!

    Our methodology is here:
    http://gs.statcounter.com/faq#methodology

    We are NOT sampling websites from Finland.

    Geo is determined via IP address based on location of visitor NOT location of website. Nothing is randomly selected either – we publish everything we track.

    In other words our stats for Finland are based on all hits we track (approx 60 million per month) from Finland (i.e. IP addresses in Finland) to all our 3 million plus member sites.

    If anyone has further questions, please do submit them to us directly – we’re more than happy to deal with any and all queries.

    Thanks!

    Dr. Roy Schestowitz Reply:

    That does not change my point of argument. What are those 3m+ sites? There are far more sites than that on the Web (IIRC, over 100m domains registered).

    What is the geographical distribution of these 3m+ sites? What proportion of them is Finnish for example? What proportion is Chinese or Brazilian?

    Danielh Reply:

    If i get this right you are sampling 3 million sites out of the 220 million sites available?

    I really hope those sites are very spread out in target groups etc because else there seems to be a great margin for error. Do they include any of the bigger sites like Google, Facebook, Slashdot, Youtube, QQ, Baidu, Blogger, Twitter etc or is it just smaller sites?

    Dr. Roy Schestowitz Reply:

    To be fair to StatCounter, it is an exceptionally hard problem to solve because of its massive scale. I just hope they make all of their experimental data and methods public. In academia we can hardly even publish a paper without this most fundamental requirement, not to mention rigourous scrutiny (no statistician would accept StatCounter’s charts without a challenge).

What Else is New


  1. Links 7/3/2021: Sparky 2021.03, SystemRescue 8.00, and FreeBSD 13.0 RC1

    Links for the day



  2. IRC Proceedings: Saturday, March 06, 2021

    IRC logs for Saturday, March 06, 2021



  3. How To Deal With Your Raspberry Spy -- Part V: All The Rest

    The final part of a series on liberating the Raspberry Spy from an untrustworthy OS that secretly adds Microsoft keys and proprietary software repositories of Microsoft



  4. How To Deal With Your Raspberry Spy -- Part IV: Doing The Task

    We now spell out the steps taken to actually replace the Raspberry Pi OS with something more trustworthy



  5. Corporations Do Not Represent Communities and Activists, They Just Exploit Them, Discredit Them, and Hijack Their Hard Work

    The AstroTurfing and the Googlebombing campaigns of large corporations would have us believe that genuine activists are toxic and malicious people, whereas corporations exist to save the world from evil people; don’t fall for those Public Relations tactics (a gross inversion of narrative)



  6. Why the 'Raspberry Spy' Blunder is a Lot More Serious and Profound Than the Corporate Media is Willing to Acknowledge

    As this video points out, the ongoing series by Gavin L. Rebeiro is justified by the fact that the 'Raspberry Spy' Foundation continues to work with and some might say for Microsoft; it sold out millions of customers



  7. Links 6/3/2021: “SLS” Mitigation and Exiv2/KDE Project

    Links for the day



  8. How To Deal With Your Raspberry Spy -- Part III: Fundamentals

    Following the introductory and preliminary parts we dive deeper into the steps taken to replace the Raspberry Pi's GNU- and Linux-based OS with something like NetBSD



  9. Links 6/3/2021: Linux 5.12 RC2 and OpenSUSE Tumbleweed Woes

    Links for the day



  10. IRC Proceedings: Friday, March 05, 2021

    IRC logs for Friday, March 05, 2021



  11. Links 5/3/2021: Qubes OS 4.0.4 Release and Wine's Project Leader is Open to Wayland

    Links for the day



  12. How To Deal With Your Raspberry Spy -- Part II: Introduction

    Following Part I, published a few hours ago, let's examine what happened from a technical perspective and what can be done about it technically



  13. How To Deal With Your Raspberry Spy -- Part I: Acknowledgements

    March 2, 2021 blog post series from a guest author; for some background, see blog posts from Microsoft in the official blog of Raspberry Pi and our response to these



  14. German Decision on Unitary Patent/UPC Will Take Years (and It Doesn't Matter Because the Whole Thing is Dead Already)

    Kluwer Patent Blog's Dr. Bausch explains why the UPC is pretty much doomed, as it cannot be ratified any time soon and probably will never be ratified either (for a multitude of reasons, including Brexit)



  15. Techrights in Australia (IPFS and Gemini)

    Allies in Australia will help Techrights serve material from another server; we're still bettering ourselves for an era of oppressive World Wide Web



  16. Professional Troll Matthew Garrett Spreads Libel, Defamation and Slander About the Free Software Community to Entertain Microsoft and Friends

    After months of parking in our IRC channels to provoke and troll people (and try to collect 'dirt' from responses) the professional troll Matthew Garrett has been for many years shows his true colours again



  17. Links 5/3/2021: Linux 5.12-rc2 Imminent, Linux Lite 5.4 RC1 in Review

    Links for the day



  18. IRC Proceedings: Thursday, March 04, 2021

    IRC logs for Thursday, March 04, 2021



  19. Links 4/3/2021: LibreOffice 7.1.1, Cockpit 239, Many Stable Kernel Releases

    Links for the day



  20. Links 4/3/2021: Pardus 19.5 is Out and Free Software Foundation Gets Consulting Grant

    Links for the day



  21. IRC Proceedings: Wednesday, March 03, 2021

    IRC logs for Wednesday, March 03, 2021



  22. The Free Software Foundation Should Re-add Richard Stallman to the Board

    Dr. Richard Stallman is missed by many who perceive him to have been wrongly treated; putting Stallman back in the Board (at the very least) would help the image of the Free Software Foundation more than the newly-announced work with Community Consulting Teams of Boston



  23. Free Software Calling

    Fewer people are willing to "put up with the shit" given by so-called 'Big Tech', seeing that it's mostly about social control rather than enablement or emancipation



  24. Meme: EPO Management Totally Gets 'Tehc'

    The bestest patent office in the whole wide world is besting the “hey hi” (AI) cutting edge; don't worry about exam and certification integrity



  25. The EPO's Software Blunders Are Inevitable Outcome of Technically Clueless Management Which Grants Illegal Patents on Software

    The "clusterfuck" which the EPO has become is negatively affecting not only EPO staff but also stakeholders, who sink into depression and sometimes anger, even fury, at great expense to their health; this is how institutions die (for a quick but short money grab, a culmination of corruption which piggybacks half a century of goodwill gestures)



  26. Links 3/3/2021: OpenSUSE Leap 15.3 Beta, GNU Denemo 2.5, and NomadBSD 1.4

    Links for the day



  27. What Free Software Organisations Can Learn From Australia's Rape Crisis

    Reprinted with permission from Daniel Pocock



  28. Microsoft Weaponises (and Further Spreads) Racism to Distract From Its Own Incompetence (and 'Five Eyes' Collusion for Back Door Access)

    Racist Microsoft is at it again; we're meant to think that China is evil for doing exactly what the United States has been doing but more importantly we're told not to blame Microsoft for shoddy code and back doors (classic blame-shifting tactics and overt distortion of facts, as we saw in the wake of SolarWinds backdoors)



  29. GNU/Linux News Sites Need to Promote Software Freedom, Not Binary and Proprietary Blobs Merely Compiled for GNU/Linux

    There has been lots of proprietary fluff in GNU/Linux 'news' sites so far this week; it merits an explanation or clarification, e.g. why we should generally reject proprietary stuff and instead promote Free/libre alternatives



  30. Links 3/3/2021: OpenSSH 8.5 and Absolute64 20210302 Released

    Links for the day


RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Home iconSite Home: Background about the site and some key features in the front page

Chat iconIRC Channel: Come and chat with us in real time

Recent Posts