Commentary: StatCounter 'Global' Statistics
- Dr. Roy Schestowitz
- 2010-11-09 01:10:38 UTC
- Modified: 2010-11-09 01:10:38 UTC
Summary: How StatCounter turns 4-5% of the world's population into 25% and reduces the world's largest Internet population (China) to just 2.46%, then claims to be measuring global market share (other surveys do the same thing)
AL submits: "Thank you for all your hard work in bringing us news through Techrights. I am reading it daily and find lots of interesting information.
"I read one of the comments from Mad Hatter in which he was talking about Wikipedia article on OS market share. I went to check it out and found that they use 1% for Linux (globally) based on the research by StatCounter Global. I was interested to see how this group is gathering their statistical data. If you go to their FAQ section they talk about sample size per country/region and there is a link to the full list of all countries. As they stated themselves their pool is 16,3 bln hits. Quite large I would say. But there is something interesting - the biggest group (region) is United States with 3,965,972,279 hits. That is almost 25% of the total pool. Now, my days of statistical studies are long gone but I still remember that in order to have accurate result you cannot over-represent one group. The result will be obviously skewed. We have one country that contributes almost 25% to the result compared to the rest of the world. As StatCounter states that they choose randomly that makes it very likely that lots of data on hits would be taken from USA. You know, for example, how much is the share of hits from China? 2,46%! In fact, looking at the whole list you can see that starting from Korea and further down the share is less than 1%! That includes countries like Poland, Greece, Japan, Russia, Switzerland etc.
“The result will be obviously skewed. We have one country that contributes almost 25% to the result compared to the rest of the world.”
--Al"I know some can say that there are many more computers sold in USA than in other countries (can't be true). But market share is more complex. If we have 95% (example) Linux presence on desktops in China, they would hardly make any influence with representation of only 2,46% on the StatCounter data. Do you see what I mean? There are of course many more problems with that. What kind of websites StatCounter is using to get hits? If we put hit counter on the website with Silverlight I don't think we will get many hits from Linux OS desktops, right? And even if the websites are getting hits from same amount of Linux OS and other OS desktops what will happen? StatCounter will randomly select hits from global pool and as data from USA will be more likely to get selected it will greatly skew the result and linux will always get under-represented. Lets say you have two crates: one with 10 pears and one with 250 tomatoes + 150 pears and you draw five times. However 3 times from first crate and 2 times from the second. You will have selected more pears than tomatoes. Even though there are 250 tomatoes and 150+10=160 pears. Is this reliable representation?" ⬆
Comments
Matsi
2010-11-09 18:12:21
http://gs.statcounter.com/#os-FI-monthly-200910-201010 (showing about 2,5-2,7% for Linux) ...and here are the results of my homepage (non-geek, 95% non-computer, mostly politics, social discussion...)
Windows 2785 83.18% Linux 302 9.02% Macintosh 185 5.53% Unknown 73 2.18% FreeBSD 2 0.06% CPM 1 0.03%
(visiters: Finland 94,4%, Sweden 0,9, USA 1,1% others 3,4%)
No big differences between clicks, unique visitors, unique sessions...
There are several finns telling the same story: Linux has about 7-10% marketshare on their homepages, blogsites...
So one thing is sure - Linux has much bigger marketshare in Finland than statcounter is telling. My guess is 8% +/- 1%. There is no doubt that situation is quite the same in other regions. Linux is some 3 times bigger than statscounter is claiming.
Dr. Roy Schestowitz
2010-11-09 18:23:16
StatCounterGlobalStats
2010-11-09 18:53:08
Our methodology is here: http://gs.statcounter.com/faq#methodology
We are NOT sampling websites from Finland.
Geo is determined via IP address based on location of visitor NOT location of website. Nothing is randomly selected either - we publish everything we track.
In other words our stats for Finland are based on all hits we track (approx 60 million per month) from Finland (i.e. IP addresses in Finland) to all our 3 million plus member sites.
If anyone has further questions, please do submit them to us directly - we're more than happy to deal with any and all queries.
Thanks!
Dr. Roy Schestowitz
2010-11-09 19:42:16
What is the geographical distribution of these 3m+ sites? What proportion of them is Finnish for example? What proportion is Chinese or Brazilian?
Danielh
2010-11-10 14:35:04
I really hope those sites are very spread out in target groups etc because else there seems to be a great margin for error. Do they include any of the bigger sites like Google, Facebook, Slashdot, Youtube, QQ, Baidu, Blogger, Twitter etc or is it just smaller sites?
Dr. Roy Schestowitz
2010-11-10 14:50:47
StatCounterGlobalStats
2010-11-09 08:59:57
I’ve just stumbled across your article and hope to clear up some confusion.
Our StatCounter Global Stats measure various market share and other stats for all countries across the globe... hence the name.
Our methodology is very simple and we’ve purposefully kept it that way. Specifically, our stats are based on more than 15 billion hits per month to our 3 million+ member sites. We’re not aware of any other publicly available service providing market share stats that has a bigger sample size on which they base their information.
You’re absolutely correct about us NOT weighting our data. We do not impose artificial weightings on our stats and this is a conscious and deliberate decision. Weighting stats means that the stats are only as good as the weighting methodology used. If the weighting data is inaccurate or out of date, then it renders the data completely incorrect. For these reasons again, we choose NOT to weight our data in any way and instead we report it as we record – other commentators can, however, weight the data as they wish. All our work is shared under a Creative Commons Attribution-Share Alike License (http://creativecommons.org/licenses/by-sa/3.0/) for this specific purpose – so please feel free to download our data and apply whatever weights you see fit.
StatCounter Global Stats came about because we decided to publicly share interesting trends that we were monitoring in-house. We aim to make our stats and methodology as clear as possible and appreciate all comments, queries and suggestions. If you have any questions for us, please don’t hesitate to contact us via our feedback form (http://gs.statcounter.com/feedback) or by direct email.
Dr. Roy Schestowitz
2010-11-09 15:05:57
A few quick points:
There are many other challenges/deficiencies, but it's commendable when the data and methods (preferably code) are made publicly available for independent audits, provided of course they don't violate privacy rule (which is a hard problem when browsers are specified very precisely with locational information too). That's why such surveys cannot reach privacy-conscious sites, many of which appeal to civil rights-aware users (many would GNU/Linux), and that's just one example. I wrote an article on the subject 3 years ago:
http://itmanagement.earthweb.com/osrc/article.php/3687616/Can-Linux-Adoption-Ever-be-Accurately-Gauged.htm