09.23.08
Gemini version available ♊︎“Independent” Studies… It’s Baaack
“Independent, yet sponsored”
Time after time we’ve warned that Microsoft’s involvement in ‘open source’ would prove detrimental to Free software [1, 2]. One company of a ex-Microsoft employee spreads FUD while another company that houses former Softies uses and advocates Mono. Here is another one, Ohloh, which was founded by Microsoft people and tends to attract like-minded folks. In the following post, which is probably shared lacking background of the parties involved, an impression is given that everything but C# is declining.
I came across my name in a site called Ohloh. I remember it coming out a few years ago. Now it has had time to really get going, I thought it was about time that I review the site here.
Watch the Web page. These are some very bad charts to show. They are population-biased.
So, former Microsoft employees entered the FOSS world and they now publish ‘studies’ (or at least charts) about C# beating everything. For insight incorporating more comprehensive studies, one must consider recent results like:
- Study: Java still top programming language
- Linux Development Thriving
- Vista’s big problem: 92 percent of developers ignoring it
- Coders Tell Why They’re Avoiding Vista
There’s some similar stuff in the news, which reader ‘twitter’ brought to our attention.
We previously explained why it’s hard to trust academic work covering Free software. It is sometimes funded by Microsoft (e.g. anti-GPLv3 ‘studies’), which also uses academic kickbacks. There are other equally-appalling things that Microsoft does on campus, so the following — although evidence is missing — raised a brow.
Stanford and Harvard teach businesses how to squash open source
[...]
It’s nice to see that $48,921 in Stanford MBA tuition going to a such a worthy cause.
More intriguingly, despite open source’s still-small market share relative to the Microsofts and Oracles of the world, it’s surely meaningful that professors from the world’s elite business institutions are turning their attention to figuring out how to beat open source. If it weren’t a threat, there would be no market for research like this.
Who is funding this thing? It’s probably innocent, but anything that harms Free(dom) software and serves an exploiter is conspicuous, particularly due to precedence. Remember money and tables. █
“Microsoft did sponsor the benchmark testing and the NT server was better tuned than the Linux one. Having said that, I must say that I still trust the Windows NT server would have outperformed the Linux one.”
–Windows platform manager, Microsoft South-Africa
Reference: Outrage at Microsoft’s independent, yet sponsored NT 4.0/Linux research
ᶃ Gemini Space
This post is also available in 
Content is available under CC-BY-SA
Needs Sunlight said,
September 23, 2008 at 12:35 pm
I thought MS licenses still banned benchmarking. Anyone with recent copies of the M$ licenses?
The originals (Java, perl, python, and everything else) still dust M$ imitations.
Roy Schestowitz said,
September 23, 2008 at 12:43 pm
Yes, it’s still forbidden [+], but the validity of this rule may not stand in some countries.
__
[+] Vista licensing also limits benchmarking
http://www.itworld.com/Comp/2218/061101vistalicense/index.html
Baby In The Bath Water said,
September 23, 2008 at 5:01 pm
I’m guessing that you are comparing Java to .NET, but if that’s the case I think that “dust Microsoft’s implementation” is a bit naive.
Microsoft’s Java implementation dusted Sun’s Java implementation.
As far as .NET, I haven’t seen any comparisons there, but considering you can’t even say that Java “dusts” Mono using the Debian Language Shootout tests (which are more likely to favor Java than Mono), even there it’s not quite so clear-cut.
Java is typically faster, but not by much and Mono typically uses a lot less memory than Java.
http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=javaclient&lang2=csharp
With the Linear IL work that Novell is putting into Mono, Mono 2.2 should be quite a bit faster than Mono 2.0 (which is already noticeably faster than 1.2)
http://www.mono-project.com/Linear_IL
Considering that it is very likely that Microsoft’s .NET implementation is faster still than Mono’s, I would not be surprised if it outperformed Java 6 in both memory usage and performance.
Note: this comment was posted from Novell’s headquarters.
Roy Schestowitz said,
September 23, 2008 at 5:03 pm
‘Baby’,
Who are you with? Your comments in this site fit a pattern.
AlexH said,
September 23, 2008 at 5:49 pm
That pattern being unwilling to unquestioningly consume?
Roy Schestowitz said,
September 23, 2008 at 5:52 pm
The pattern of coming here to just shoot the messenger.
Dan O'Brian said,
September 23, 2008 at 7:18 pm
Roy: stop trying to shoot the messenger
Roy Schestowitz said,
September 23, 2008 at 7:20 pm
You’re having fun trolling this site, aren’t you?
Dan O'Brian said,
September 23, 2008 at 7:30 pm
Ironic that you consider uncovering the truth to be “trolling”.
Says a lot about you, doesn’t it?
Roy Schestowitz said,
September 23, 2008 at 7:33 pm
No, I talk about heckling. Your ‘truth’ is subjective and biased.
Dan O'Brian said,
September 23, 2008 at 7:49 pm
No, my fact is provable. Yours is subjective and biased.
Don’t confuse my arguments with yours.
Dan O'Brian said,
September 23, 2008 at 7:50 pm
s/fact/truth/
Roy Schestowitz said,
September 23, 2008 at 8:15 pm
And yet, your refuse to disclose information. I asked you for disclosures before. All I know is that you want to hire a Mono fan from Novell.
Dan O'Brian said,
September 23, 2008 at 8:19 pm
If what I state is factually correct, what difference does it make who my employer is? Fact is, it doesn’t matter.
Roy Schestowitz said,
September 23, 2008 at 8:27 pm
Sites that I find factually unsuited for my perspective I hardly spend time leaving comments in. I just stay in my side of the fence. What compels you?
Dan O'Brian said,
September 23, 2008 at 9:54 pm
So what you are saying is that you avoid any discussions where your viewpoint is challenged?
Dan O'Brian said,
September 23, 2008 at 9:59 pm
FWIW, I don’t see where ‘Baby’ has attempted to shoot the messenger in this thread (haven’t searched any others). Instead, ‘Baby’ has provided an alternate viewpoint from the first poster and has provided supporting evidence (something the original poster did not).
AlexH said,
September 24, 2008 at 2:41 am
@Roy: Your claims about Ohloh are trivially checkable; I went to their site and used the Language tools and your claim about it showing C# in number one position is just clearly incorrect.
Roy Schestowitz said,
September 24, 2008 at 3:42 am
Dan O’Brian,
My point was that I do not endlessly argued with people whose site says stuff I disagree with.
Dan O'Brian said,
September 24, 2008 at 7:02 am
Roy: probably because you never have any evidence to disprove them
Roy Schestowitz said,
September 24, 2008 at 7:07 am
No, because I have my own sites to disprove them. I don’t need to s* in their own back yard.
Dan O'Brian said,
September 24, 2008 at 7:14 am
I’m guessing that Roy didn’t actually look at the charts that were posted on that site. C# is at about 1.5% usage. Java has also been increasing in usage and is up around ~6%. C usage has been coming down from around 10% to 7% and is starting to go back up again. C++ has been more steady, but is also starting to increase again. Ruby and Python are both steadily rising. Perl and Tcl are both on the decline (Tcl isn’t as surprising to me).
Where Roy gets his ideas about how “everything is declining but C#” is beyond me. I can only assume he didn’t read the article and that instead he wanted to badmouth Microsoft/C# and thought he could fool his readers into not clicking the links to actually read for themselves.
AlexH said,
September 24, 2008 at 7:18 am
@Dan:
I think he looked at the graphs; I don’t think he realised that the scales were not the same and therefore can’t be compared directly…
Roy Schestowitz said,
September 24, 2008 at 7:22 am
Graphs can be scaled and statistics cooked to give a particular impression. That’s just the point.
Microsoft did a lot of this to give a false impression of Silverlight adopted (and got smacked for it).
AlexH said,
September 24, 2008 at 7:28 am
@Roy: Except, that’s not what has happened here.
Someone created a graph on Ohloh to compare C# to Objective C. Not surprisingly, C# looks stronger on that one graph. But that article also says C# is “not heavily used in free/open source software” (true) and doesn’t seek to compare them to other languages.
The article doesn’t state that C# is the only language growing, and no-one who can read a graph is going to think that either.
But because Ohloh has some ex-Microsoft people on board, you’ve quickly misinterpreted the article and shot the messenger, again.
Dan O'Brian said,
September 24, 2008 at 7:45 am
Roy’s hypocrisy is one of the things that keeps me coming back for more laughs
Roy Schestowitz said,
September 24, 2008 at 7:47 am
AlexH: should we start learning about F/OSS also from CodePlex stats?
AlexH said,
September 24, 2008 at 7:51 am
@Roy:
CodePlex is a project hosting system, and stats on CodePlex are only useful for judging projects on CodePlex.
Ohloh do not host projects; indeed, all the projects I work on are tracked by Ohloh. So, there is an obvious difference there that I’m surprised you missed.
Can you give an example of the bias you think they have?
Roy Schestowitz said,
September 24, 2008 at 7:59 am
I don’t have time to check this now, but here comes something:
http://www.ohloh.net/licenses/Microsoft%20Shared%20Source%20Permissive%20License
This may not mean much, but I expect projects to earn inclusion based on the inclinations of Ohloh staff, who are former Microsoft chaps.
AlexH said,
September 24, 2008 at 8:12 am
So again you write something which you know is completely unsubstantiated…
The MS-PL is a free software license, and you can find out how inclusion works just by visiting Ohloh (hint: they automatically track any project hosted on known free software hosting systems, and you can self-include any other project for free).
AlexH said,
September 24, 2008 at 8:17 am
The fact that Ohloh tracks software licensed under a recognised free software licence is “evidence” that they’re biased?
Maybe you should go to Ohloh’s site and actually find out how inclusion works before speculating about, because you’re way off the mark.
Dan O'Brian said,
September 24, 2008 at 8:24 am
FWIW, I just searched all of the C# projects listed on Ohloh and not a single one of them was even licensed under the Microsoft Shared Source Permissive License, so it’s irrelevant anyway.
Roy Schestowitz said,
September 24, 2008 at 8:37 am
The mentioning of the licence of the saturation of C#?
Dan O'Brian said,
September 24, 2008 at 8:44 am
Since none of the projects even use the license, how is it relevant?
AlexH said,
September 24, 2008 at 8:45 am
Roy, they track ~200 active C# projects. and over 1000 in Java (as one example).
On what planet is that “saturation”?
Roy Schestowitz said,
September 24, 2008 at 8:49 am
Seems about right to me.
http://i.i.com.com/cnwk.1d/i/bto/20080807/Language_540x404.jpg
AlexH said,
September 24, 2008 at 8:54 am
A 1×1 gif is evidence of what, exactly?
Roy Schestowitz said,
September 24, 2008 at 8:58 am
It’s bizarre. Let me try that again.
http://i.i.com.com/cnwk.1d/i/bto/20080807/Language_540x404.jpg
It’s almost as though the site blocks references from other sites. See the figure here:
http://news.cnet.com/8301-13505_3-10009669-16.html
Dan O'Brian said,
September 24, 2008 at 8:58 am
That he’s mentally insane?
j/k
Roy Schestowitz said,
September 24, 2008 at 8:59 am
Very amusing, Dan. Try referencing that figure from CNET directly and see what happens. There’s some Web server trick there (referrer probably).
Dan O'Brian said,
September 24, 2008 at 9:02 am
Roy: that doesn’t disprove Ohloh.
Ohloh gathers statistics on F/LOSS software based on actual source repositories.
From the article:
So basically, the TIOBE stats are not based on code sampling, they are based on other criteria.
Dan O'Brian said,
September 24, 2008 at 9:05 am
In summary, neither source (Ohloh nor TIOBE) is necessarily representative of the actual market.
Ohloh represents only F/LOSS projects which may or may not be a representative sampling of the entire market.
TIOBE is based on guess-timation based on web searches, # of books on the topic, etc. This is hardly an accurate statistic.
AlexH said,
September 24, 2008 at 9:05 am
@Roy:
The graph you link to shows Java at ~22% right now, and C# at ~2.5%.
The Ohloh stat mentioned (which is commits per month, a different metric) says Java is ~15%, C# ~2%.
So they’re about the same. An indeed, if we look at the monthly lines of code metric on Ohloh we see Java regularly bounces over the 20% mark. So for the month of August, 2008 – when the Tiobe study was done – they’re actually in complete agreement.
And your point is, again?
Roy Schestowitz said,
September 24, 2008 at 9:09 am
Neither can be accurate and that’s the whole point. Mind the fact that the main criticism is directed at the second part of this post. A followup confirms that money came from foes of Free software to manufacture some ‘knowledge’.
Dan O'Brian said,
September 24, 2008 at 9:15 am
There’s no evidence that Ohloh manufactured anything.
AlexH said,
September 24, 2008 at 9:20 am
@Roy:
Your “main criticism” defence doesn’t stand up. We’re not looking at the second part of the article. We’re looking at claims like this one:
… which have been thoroughly and factually debunked. Ohloh didn’t “publish” any “study”, let alone those charts (users create them), and the charts do not show C# beating everything – they just show you didn’t read them properly. We’ve also seen that Ohloh’s numbers are in line with other studies that you trust.
Roy Schestowitz said,
September 24, 2008 at 9:22 am
Dan O’Brian,
Read what I wrote a moment ago. It has nothing to do with Ohloh. it’s the study Asay links to.
Baby In The Bath Water said,
September 24, 2008 at 10:37 am
I see that instead of arguing with my undisputable statements, Roy has instead elected to “shoot the messenger”.
I guess he has concluded that if he can’t fight the facts, he’ll try his best to attack the messenger. Luckily, the Free Software community is made up of intelligent people like myself who do not fall for such garbage.
If I back up my claims with facts, what difference does it make who I work for?
(FWIW, I’m an undergrad student)
Note: this comment was posted from Novell’s headquarters.