09.11.10
Mono Outperformed by .NET Sometimes, Ubuntu Netbook Edition 10.10 Dumps Novell’s Banshee
Summary: The technical deficiency of Mono is shown once again and Mono dependency in Ubuntu has just been reduced
Mono is a copy of a copy of “the real thing”, which is Java. Mono is a way to advance .NET at Java’s expense, totally under Microsoft's control and at its mercy. According to this new benchmark, Mono continues to be more like “.NET express”, i.e. an introductory version to Microsoft’s stack, which the “real experts” will invest in and eventually use. Mono is essentially like a recruitment tool for .NET and its advantage is that it manages to scoop up Free/libre open source people who foolishly believe that they advance freedom by advancing C#. Here are some numbers from the new benchmark (Mono works well only in small examples, at least in this experiment):
As you can see on both platforms the sones GraphDB is able to work through more than 2.000 queries per second on average. For the longest running benchmark (1800 seconds) with all the data imported .NET allows us to answer 2.339 queries per second while Mono allows us to answer 1.980 queries per second.
Why would anyone believe the spin from Mono proponents, who claim that Mono does more than .NET (even though the MCP forbid them, using software patents as ‘teeth’). The OMG!Mono! folks continue promoting Mono-based software like Sparkleshare [1, 2, 3], as usual paying no attention to Mono by at least naming it as a dependency in the post (it’s usually left for comments to do).
Sparkleshare – the open-source cloud storage (think Dropbox) alternative – has hit its first beta milestone.
Although this site does not pay attention to the Mono problem, it does currently state that Ubuntu Netbook Edition 10.10 won’t have Banshee by default [1, 2, 3], despite the original plan. This is excellent news and it buys users more time to explain why Mono should not be included by default in Ubuntu. The reason which is named for dropping Banshee from this release is the number of bugs:
Deemed to contain too many bugs to make it an acceptable default this cycle, the popular player’s default pinning has thus been held over to Ubuntu 11.04.
It does seem like Ubuntu reduces its dependency on Mono then, at least to an extent. █
hexx said,
September 13, 2010 at 12:24 am
The benchmark actually shows decent performance. Lets compare this to Java. If you take any free non-Sun JVM you will see similar figures vs Sun Java. In many cases you will see free Java perform very poorly.
When it comes to VM languages it’s a well known fact that the mature, highly invested and highly optimize VMs perform much better that free implementations. Even Harmony or Dalvik cannot keep up with the Sun VM in terms of JIT optimization.
Nobody with a good knowledge of Java expects free implementations to come close to Sun JVM, because that is simply not realistic. When choosing an alternative VM the only real requirements are stability and “reasonable” performance. Which often means 1.5 to 2 times slower.
So the numbers you see in the benchmark are actually very good, especially for a free implementation.
Of course the big difference between .NET and Java is that Sun JVM was open sourced in the form of OpenJDK. So you can have a VM that is both free under GPL and performs very well.
Mono should be critisized where it deserves to be critisized (patent issues, closed platform, etc), but misreporting facts like this seriously hurts your credibility. Very disappointing.
Dr. Roy Schestowitz Reply:
September 13th, 2010 at 1:48 am
Hi, hexx,
Mono is not different from Java’s free implementations/clones in that regard. The difference is that Sun/Oracle Java is/was free software which runs on GNU/Linux too. The same cannot be said about .NET.
hexx said,
September 13, 2010 at 11:09 am
Never thought I will find myself defending mono, but here it goes…
The numbers do not support your headline nor your analysis. The benchmark shows mono performing on par with .net most of the time. In the numbers you quote mono is only 18% slower. This is nothing. The conclusion should be that mono performs reasonable well and is comparable to .net in terms of performance.
Nobody claimed that mono outperforms .net in the first place (strawman argument?). In general I had a strong feeling of being misled when reading this article. Then I looked at the numbers and realized just how much it was true.
Again, I think mono is a patent timebomb waiting to go off. I also believe that the mono team uses questionable tactics in order to promote their platform and supress the opposition. However this does not give you an excuse to twist (spin?) the facts.
Dr. Roy Schestowitz Reply:
September 13th, 2010 at 11:29 am
hexx,
I looked at the numbers carefully before I posted this item and I chose my words carefully. I wrote “sometimes” in the title. In the body I called Mono “.NET express” (someone suggested “starter edition”) because it worked better on smaller-scale data. I wrote reasonably clearly that “Mono works well only in small examples, at least in this experiment” and then quoted what happens in larger ones.
The last thing I wrote was the claim that Mono does more (e.g. OpenGL), but the MCP prevents Mono from fulfilling its full potential — that is, without being at risk of lawsuits (like Banshee). Discussions about it go back to 2008, maybe 2007.
Can you please explain what you found deceiving so that improvements can be made?
twitter Reply:
September 13th, 2010 at 10:13 pm
18% faster is huge for something like a web server. The slow version would need 5 machines to the fast one’s 4. In power consumption, this is also huge. Of course, none of this matter because both are pigs that no one with any sense uses.
hexx said,
September 13, 2010 at 12:16 pm
From the tone of the article and “.NET express” title i expected to see mono perform significantly worse. However the numbers tell an opposite story.
If it has to boil down to one sentense, then I would pick this quote from the article:
“According to this new benchmark, Mono continues to be more like “.NET express”…”
However there is nothing in the benchmark suggesting that mono is like “.NET express”. The quoted numbers show that .net is 18% faster for large datasets. This difference is simply not enough to call mono an “express” version.
You could call it a strong contender with a second place, you could call it almost as good as .net, you could even say “close, but no sigar”. However hinting that benchmarks are disappointing or saying that it performs like an “express” version is misleading.
Dr. Roy Schestowitz Reply:
September 13th, 2010 at 12:37 pm
It was intended to be somewhat humourous and self-evidently so. We sometimes like to joke that MonoDevelop is the other Visual Studio “lesser version” (getting people ‘hooked’ and then buying the ‘real thing’). MonoDevelop may also work better under Windows, which helps in weaning developers off GNU/Linux.
twitter said,
September 13, 2010 at 12:53 pm
.NET is nothing like a, “mature, highly invested and highly optimize VM” nor is it clear that a free implementation of a VM will not perform better than a non free one. I know from personal experience as a user and from the London Stock Exchange melt down that .NET is more of the same from Microsoft. It is sad to hear that mono is not better in everyway than .NET, but the effort is a Trojan horse and there can’t be much of a real community working on it. People who know what they are doing use perl, php, Java, or any other language that’s GPL’d and these free software implementations all eclipse .NET and mono performance. I expect that java has been improved by freedom and that it will continue to outpace both .NET and mono until both of those are placed into the Microsoft software graveyard.
hexx said,
September 13, 2010 at 3:39 pm
You often blame pro-microsoft press in creating a reality distoring bubble. However in this instance you are doing exactly the same thing.
“Mono outperformed by .NET Sometimes”. Ok, and Java outperforms C, Sometimes. Many things outperform many other things, SOMETIMES. Doesnt say much. Doesnt say anything at all. The headline itself is a spin.
You are trying to be factually correct while creating the feel that mono underperforms, unsing several deceptive tactics. Using words like “sometimes” and “express version” you are creating ambiguity which could be interpreted in many ways.
Then you are trying to weasel your way out by saying that calling mono “.net express” is akin to an inside joke. Well, you are reporting to the public, and most people arent going to be aware of your inside jokes.
The end result is that you are creating a perception that mono is slow when in fact it isnt.
You may call it what you want, but this is not an objective reporting. At the very best this is a cheap shot, at worst it just looks silly when you quote numbers that do no back your point. Either way, you are giving ammunition to the people claiming that BN is not an objective source of information.
There are plenty of technical and licensing problems with mono, there is no need to be tacking cheap shots like this and risking your credibility.
Dr. Roy Schestowitz Reply:
September 13th, 2010 at 3:51 pm
There was a similar discussion about Mono performance about 2 years ago. Benchmarks need to be broad and diverse to be meaningful. Got a golden test for Mono?
hexx Reply:
September 13th, 2010 at 4:31 pm
I dont have a benchmark. I wouldn’t touch mono or C# with a six foot pole.
What I’m saying is that benchmark you are quoting is not consistent with your conclusion and the tone of your aticle. Also, as you have just pointed out one benchmark is hardly enough to draw such broad conclusions as to oveall platform performance.
Maybe I am very bad at eplaining myself. However I am starting to get the feeling you are simply dodging the issue.
I have no interst in continuing this discussion, as it requires me to take the side of mono. Just wanted to let you know that this article does not pass as objective reporting. Hopefully the message will get through even if you are not willing to publicly aknowledge it.
BN has been very helpful at highlighting deceptive tactics and point out spin in all shapes and sizes (usually coming from microsoft sponsored press). Also you have had many articles about analysts gaming or spinning the benchmark data to get the desired results.
You have tought me (and hopefully many others) how to spot these things. However when the very same analysis is applied to this article, it does not pass.
Dr. Roy Schestowitz Reply:
September 13th, 2010 at 4:45 pm
hexx,
I’ve read maybe thousands of items about Mono over the years and many left me with the impression that Mono was still catching up with .NET, performance-wise. If Mono was consistently faster, some companies would transition workloads from .NET to Mono (that’s rare).
See
http://bytes.com/topic/net/answers/629771-net-vs-mono-performance
http://www.phpvs.net/2008/02/08/benchmarking-mono-aspnet-vs-php-a-slight-problem/
If Mono was faster, maybe Microsoft would learn how to improve. The same does not work the other way (.NET is secret code).