Let’s hope someone tests Microsoft by forking mono and then diverging from dotnet.

To properly fragment and harm .NET, it would take quite some userbase. Either way, from my point of view, Mono developers willingly put themselves at Microsoft’s mercy, enabling the tools to be restricted, pulled or sabotaged (remember what Microsoft did to Java and DR-DOS?) at any time. They need to keep “The Beast” satisfied.

For the Free Desktop to depend on Mono is a big gamble.

I haven’t used it, but I don’t see why it wouldn’t be possible at least at some point.

I have also said that I personally would not tax my brain with MS protocol specifics. mono follows Microsoft and so helps grow the dotnet ecosystem where advantage goes to Microsoft in many ways. This is the main reason I would not use mono but would instead opt for a different set of specs if I ever wanted to use something like mono.

Let’s hope someone tests Microsoft by forking mono and then diverging from dotnet. I would welcome such success since eventually that would mean I would have yet another FOSS tool at my disposal.

Consider a company, Monowallow, that controls servers of a certain type, builds corresponding client apps that compete with third party client apps, and publishes the protocol specs to the server-client interactions.

A particular call/message accepts various parameters including a “quality” parameter that has a few values, eg, OPTIMIZE_FOR_SPEED, OPTIMIZE_FOR_FAIRNESS, OPTIMIZE_FOR_PAYLOAD_SIZE. There are also two private/undisclosed values used internally and a few repetitions of these but using older slightly buggy implementations within the server (for “backwards compat”).

The OPTIMIZE_FOR_FAIRNESS is defined in the spec, but it’s a bit ambiguous. Examples of use are given, but this doesn’t solve the confusion. Most third party devs avoid this value.

Monowallow internal devs use the 2 undisclosed calls. For these, the semantics and range of values for the other parameters also change, perhaps subtly. Maybe a “size” value is actually a shifted and scaled value and maybe a few such values map even more strangely. Undisclosed values give performance boosts, new functionality, etc.

The internal devs also use a set of C macros that map to the integral values; however, there is a “typo” in the macro files used internally relative to the published spec. The internal developers always use the macro names so never know the actual numbers except in passing when they debug (or the tools they use hide the numbers). The various levels of macro indirection led to the bug/typo so that the OPTIMIZE_FOR_FAIRNESS is implemented as OPTIMIZE_FOR_SPEED and vice-versa. This means the calls will not do what the public api say, yet virtually no one, even internal devs, will realize this. Since these are optimization hints, it’s not a big deal except for perhaps servers under pressure. Here using the wrong optimization will have some noticeable penalties. Those doing reverse-engineering along with profiling may catch this bug (and maybe keep it as a company secret for market advantage). Others will likely try to use OPTIMIZE_FOR_SPEED frequently but instead get the performance characteristics of the confusing OPTIMIZE_FOR_FAIRNESS and never realize it.

Also, you can use your imagination as to why there will be good and buggy versions of the same code in there. The buggy was simply bit rotted in some cases. Maybe a small boundary error occurs in the other (so some packets might be dropped unnecessarily). Maybe some of the shipped developer tools (Monowallow also sells devel tools) map to one set of quality implementations or another based on who is receiving the tools (eg, a tier 2 or tier 3 customer or the community vs the deluxe tools).

Maybe some of the values in the actual published spec map to buggy implementations, so that by default everyone uses a less than efficient/correct call for some values. Maybe the error is only noticeable in conditions that occur infrequently. Also, we have old API vs. the new and improved API. The old have problems to discourage use (bit rot might mean that only 16 bit sizes are used where nowadays you are likely to find a larger size). OTOH, the new API’s might do DRM.

Also, the undocumented values used internally lead to superior performance but only for internal client/server interactions since these server calls make assumptions that only client apps made by the internal developers would also make (eg, info known already from the OS or from out of band from initialization). Others that try and use these calls (eg, they heard from a friend who.. or they were reverse-engineering and noticed better performance at some time) likely will get corrupted data at some point.

There is a ton that can be done, but this should give a flavor of how easily it is to naturally find ways to give handicaps to outsiders and benefits to insiders while partly hiding your tracks and even having plausible deniability should some of this ever get discovered.

Really, this are small items. Anything can be done when you have closed source and put decoys and errors in place.

“I dunno. We have sloppy devs.”

To be fair, I should ask, “once we find a mistake, can we fix it?”

With FOSS we also will find mistakes (in many cases anyway), but once the bug is diagnosed, we can fix it and usually very quickly. We have the source. This is important not just to find little bugs but to verify that very strange things or violations are not being attempted or lie in wait (eg, the closed source “time-bomb” that is not disclosed and maybe does something even more nasty).

That reverse-engineering sometimes does a good enough job (within a limited context), and many times this is not even true, is because developers don’t usually spend time crafting tricks in their code or into the toolchain. Microsoft has a multi-billion dollar per month revenue set of monopolies to protect. They have also shown to have gotten into that position because of their willingness to stretch the law and the boundaries of ethics. Use your imagination to see what they might do with some of that money. Consider too that it only takes a few developers working at a high clearance level to adjust the code the others do (or perhaps automatically through the in-house tools).. “to foil reverse engineering of our IP.”

On boycottnovell, I’ve seen a 1988 quote of Gates apparently wanting the OS to isolate third party apps for misbehavior. Back then computers, the Internet/networking, Microsoft’s software, their revenues, their size, etc, were a small or even tiny fraction in magnitude of what it is now. I think this was still the large-floppies-in-a-bag world with sparse online connectivity and low bandwidth. Microsoft was very small.

Fast forward to today many years and experiences later. Much has changed, including what is at stake for Microsoft and the many times they have managed to have their way with the law. Even simply coding in bugs is enough to foil reverse-engineering if the bugs are well placed (not to mention that bugs, ambiguous specs, tired employees, etc, offer plausible deniability).

It’s very easy to foil apps if you control the OS. I might eventually get to doing a distro that does just this (but with source code).. it tracks you, it collects data, it gives variable quality hooks to the apps isolating each app individually.. Then maybe have people use the distro and through online updates change the behavior based on some secret formula (ie, “which project forums treated me well”). …

The problem:
(a) Do we know what the software is supposed to be doing? Yes for FOSS or honest closed apps. No for Microsoft sw.
(b) Is there an active effort to foil spec’d behavior and reverse-engineering? No for FOSS or honest closed apps. Yes for Microsoft sw.

This is why testing and reverse engineering is likely to miss many problem areas. And Microsoft has lots of money to help build their “protections of IP.”

Also, in practice simply by keeping secrets and using other levers, Microsoft can come out ahead. They likely reserve their greatest tricks when they target specific apps or groups. I can’t stand competition where you are only safe so long as you don’t pose a threat or aren’t doing a very good app.

And my answer was that you can only feel “good enough” comfortable about what you specifically tested. You can take your chances with everything else.

In practice, ie, among friends, you can get great coverage if you know what the behavior is supposed to be and you test boundary points and various key data in general.

With Microsoft software, the spec is not what their software is supposed to do. What is their software supposed to do if not the spec? What their software is supposed to do is something only Bill Gates and I know, but I’d have to shoot you if I told you

Let’s talk a second about continuous well-defined (etc etc) math functions vs. a computer program. If you know you have a linear function in mathematics, you need only know two test points and then you know everything. That is because math defines a line perfectly that way. A discrete computer program OTOH can produce any arbitrary map from its inputs to its outputs. It’s the math equiv of an arbitrary mapping from an infinite domain to the corresponding infinite range. This means that if you don’t measure every output for every input, you could leave holes behind that don’t conform to the model you are reverse-guessgeneering. In other words, a “line” function in the computer can plot all X’s to the proper Y’s except for 34.32. Why? Because that happened to have been the value the developers of the program decided to use for trap/secret bypassing backdoors or to execute a set of secret extensions… or to create rarely occurring misbehavior for the interfacing third party app… or because of a bug.. because they want voting tallies to reverse and start counting backwards when some point is reached.. why? I don’t know, but the point is that anything like this can happen.

If Microsoft could be assumed to want to follow the spec at all costs, we would not be here. They would not be in court every day of the year, and they also would not have a monopoly. Bill Gates would have much less money. ETC.

Show me a single third party product you think works identically to any specific MS product in every single respect at all times. .. oh, time! I was just looking above at space mappings at a single point in time. You mean that the “tricks” and traps can change location and behavior over time, too? [BTW, list such an app for the record, but I won't make an attempt to use it to disprove you. Surely, you can find 10 or 20 such clones, right? Surely, reverse engineering is an art.. I mean, a science many practice, right? I don't think I can find or create such apps, but I think you have that magic reverse-engineering touch. I look forward to your submission. Surely, people would not keep trying unless several had succeeded before among the thousands that had tried.. unless of course there is money involved for continuing to try.]

The people wanting a good standard would strive for a quality product capable of allowing perfect interop merely through the study of the standard, at least this would be true in the ideal case (eg, no implementation bugs, no spec ambiguities, etc).

I don’t believe English can be made that precise in general. OASIS is working on testsuite specs for ODF (tests are useful if on top of a solid standard). OO.o, KOffice, … source code are available to enable interop with those particular products.

Does MS reveal source code? Well, they should reveal some for $$, but even here there is no way to verify what they showed you is what they ship. You also don’t have access to the source code of their toolchain. This means deviations in their compilers from the standards will lead the code they showed you not to work as you would expect. Finally, I am assuming good faith on their part since they can play many sorts of games even through revealing source.

Everything about OO.o is open source (all functions calls all the way straight into the heart of the kernel).

Everything about MSO carries a red flag, from the lack of source code all the way to the expected motivations of the monopolist vendor.

2. Jose: I’m guessing what BITBW was referring to was that you write a bunch of tests that check the behavior of something in a bunch of different situations. From that, you can deduce how something works. You then go and implement something else with the same API, and you can very effectively mimick the behavior.

This is called reverse engineering.

Don’t be silly. Why does having 10 unit tests matter? Isn’t 1 enough? Maybe 1000000 is better? Why?

A unit test simply shows that that particular test works. It says nothing about what wasn’t tested.

More importantly, however, is that you can’t measure the behavior on Windows because it is not transparent (unless you are running it so that every single instruction is trapped, recorded, analyzed, compared, while every single memory byte and byte on the drives are similarly recorded and tracked for each such instruction).

I know you did not do this. Possibly a “Hello World” program would take at least millions of such (eg) x86 instructions. Not to mention that hardware on the PC bus would also have to be tracked (including opaque graphics cards and motherboard chips etc).

Look. I am saying something that is not supposed to be an issue among friends, but it is when Microsoft gets together with hardware vendors and chooses to mess things up. And they don’t have to get together with any hardware makers. Windows is plenty. Windows is opaque unless you literally do what I just described above to in effect reverse engineer Windows. This is not a task humans or their computer programs are likely to do anytime soon for such a large blob that is an OS like Windows. [How well we succeed depends on how much or not Microsoft wants to hide their tracks and succeeds at doing so, though, in any case Windows is a monster of a program in size and complexity.]

If I have lost you, let me try again with an example.

How do you define “something works”? You can look and study the code and believe that X should happen. Then you might observe X on both platforms and conclude that the two implementations work the same. And perhaps they mostly do.

But what if when you study the code, you think X yet on Windows Y happens which is behavior that is subtly different, eg, the registry is modified so that on next log in you app fails to work. Maybe you observe Y as if it were X because you weren’t measuring too carefully or tracking instructions, memory, nor the registry (btw, you can’t track memory the OS occupies on x386 and above unless the OS allows you and no OS does this.. or if you run within the OS).

I can re-explain if you want. Just ask.

This crazy things I am talking about matter because it’s ridiculously easy for Microsoft to pull off (just by being sloppy, in fact), and means that your app might end up doing or lead to anything in reality (eg, phoning home to Microsoft behind your back). Another example, your next OS call to allocate a memory buffer simultaneously corrupts memory you were already using.

We can achieve some level of interop (a high percent), especially, if there is no intelligent/concerted effort to sabotage. Otherwise, you can’t really know/measure.

If you trust Microsoft for all your applications for all time, go ahead. I do not. I don’t even trust their (superhuman) ability to code bug free or make specs that are completely unambiguous, consistent, fully specified, etc. I certainly don’t trust their intentions or business plan.

You are talking about interop at a very high level for toy programs. The reality is that MS lock-in will never be solved except in pieces and even then we can’t be sure. We play the rat race and our software is never quite just right with their stuff. This is especially true for complex programs.

So in practice things might work most of the time “good enough”, but that can change the moment Microsoft feels threatened (and they always feel threatened). It’s especially easy to thwart your app in predictable ways if you reveal your source. Also, Microsoft likely always gives you something subpar so that you can’t compete on an even field with their integration-apps.

As for performance, that is not really an issue for toy apps with today’s computers, but you might be using a lot more CPU than you should.

Was your app a toy? Did you even test everything you can see? Do you know if the exact input string “http://www.google.com/X/Y/Z” won’t translate to “http://www.microsoft.com/A/B/Z”? Have you tested every possible input string? .. I am just getting started.

If you insist you can have interop, I did not make myself clear above, but I don’t know what your background is so don’t know how much more to explain without you asking me first for more examples.

>> From what I’ve seen, Mono has a LOT of unit tests, more than any other Free Software project I have seen.

I have no idea, but surely you haven’t tested everything under the sun. For arbitrary inputs (eg, mouse clicks on screen or typed strings), this would constitute an infinite set).

Mono might work if you guys separate from Microsoft’s lead. This way, you can at least have as much confidence as you want/need whenever you run the app over a FOSS platform. You would also find out quickly if Microsoft will sue.

Try it and see if Microsoft comes after anyone. Ditch Windows, for security, if for nothing of what I have said in this reply or if you don’t care about privacy.

BTW, I don’t pretend that FOSS has behavior (especially on a desktop) that anyone is likely to completely predict, but being able to check up and verify or being able to find the problems is priceless.

As you can see, for practical “good enough” results, Microsoft platforms are usually fine.. at least until it’s time for an upgrade.

[I *may* not post again until earliest tomorrow.]

You don’t actually need source code to get compatibility, you simply need to write unit tests and make sure they work on both systems.

From what I’ve seen, Mono has a LOT of unit tests, more than any other Free Software project I have seen.

Note: this comment was posted from Novell’s headquarters.

Microsoft is not an ordinary player. Re-inventing their wheels, especially when they wave patent threats around and work from a closed source monopoly position, is a very good idea. We re-invent wheels in real society when necessary. You just don’t want to do it when the existing wheels will do.

>> The amount of research and development that went into designing .NET was huge and Microsoft had a lot of the best minds in the field working on it. Does the Mono team have those kinds of things going for it? Probably not.

This excuse is weak unless you don’t mind using that wheel. People reinvent wheels all the time. Microsoft leveraged much existing material. Don’t overestimate their minds. FOSS is not supposed to work, remember — a ragtag bunch of hippies scratching themselves — but it does.

>> Besides having a a design to start with by implementing the .NET specifications, they also get compatibility with Microsof’s .NET (which is a bonus)

Not really possible.

Not to mention this is inconsistent with Novell’s (inaccurate) position that they are achieving interop because they are working closely with Microsoft, unless the mono people never expected to achieve (essentially impossible) interop except through special deals and contacts with Microsoft. If the mono people believed this, then this is as wonderful a reason as any to reinvent the wheel unless you want to run a Microsoft rat race (and risk much time and money much as IBM did with OS/2). If the mono people don’t believe this (ie, if they believe that you *can* achieve interop straight from following the spec), then they (would be fools and) would be contradicting Novell’s position that they have an advantage over all other implementations because of the deals they have with MS.

>> It seems to me that if it is made trivial for them to write applications that work both on Windows and Linux, then Linux users win because they get a wider selection of software to choose from.

Again, interop is not achieved from the spec (ask Novell). You need to have access to the entire source code (and then only with the assumption there are no bugs).

The reality is that full interop is not really possible but some % of it is possible. Of course, if you think Microsoft will help that percentage be very high and cover everything of significance or else offer money to the customers when mis-matching occurs, I have some land in Florida….

When FOSS apps fail to interoperate, have bugs, etc, you have the source code right there. There is good faith among most FOSS developers (not that faith is needed when the source is right there). These things are not only not true with Microsoft, but Microsoft has every incentive to be sloppy and break with the “best” interpretation of any given standard since that would help secure their monopoly position.

Working on mono seems to me as practice for trying to get a job with Microsoft. The whole premise to trying to interop with Microsoft is broken.

“What about OO.o .doc formats,” AlexH will say? That is most certainly not a perfect or complete conversion by any means. Fortunately, for adequate subsets of MSO, it can do a good enough job.

But there is a real difference. If text is a little off on the page, you can still read that and fix it perhaps. You may not even notice in some cases.

If a bit is off within a long instruction stream, the whole system might come “crashing down” (ie, interop broken perhaps horribly badly leading to nothing useful).

Now, it’s very easy to be off by a bit somewhere, especially when one side of the conversation has real interests in having those bits be incorrect here and there.

Also, the bit thing also can apply to documents to the extent off-by-a-bit can lead to complete garbage (eg, change a single random bit on a .tgz file and likely you will get garbage). To this end, OOXML might be a format more “properly” designed to make it easier for such garbage to result from such a small error. OO.o tries to find all the things it needs so that .doc work. In some cases, they fail a little, in others (probably more so with macros and instructions), they fail much more.

[For those not following, "bit" is the term used to describe a single on/off entity within a computer.]

>> Why didn’t they use the Parrot VM?

I think parrot was new as well, but the question isn’t why didn’t they use parrot, it’s why didn’t they join a group to build parrot instead of starting mono? Of course, you already gave a partial response to that (>> too much trouble to re-implement that wheel).

Don’t mean to be feisty in this post, btw, or to appear to be attacking you. It’s a bit of frustration on my part. The interop thing is exactly why people spend lots of dollars and time playing a never-ending game of catch with Microsoft (a Rat Race).

Deal with a closed source monopolist? No no.

[Mono can be useful without interop with MSdotnet. I don't deny that. In earlier posts, I explained why I wouldn't take the mono path, but different strokes for different folks. ..And maybe in the future some day I will be using mono! .. ah-ah- ah ahh aaaachew... sorry.]

When you say something is not correct, you should point out how it is wrong.

I think what you are saying is that Microsoft is adding as their extensions CSS 3.0 trial properties. This doesn’t contradict what I said. It simply adds new information.

For those following along, AlexH is saying that Microsoft’s extensions are innocent features-in-testing from the yet to be fully specified/ratified CSS 3.0.

So to repeat, Microsoft is doing what they have to do to change CSS 2.1 while not violating the spec.

AlexH claims that the (only) extensions they are adding are items being considered for CSS 3.0 and (presumably) nothing else.

If all of this is true, there is at this point not much to get upset over unless you want to practices ;-0.

FWIW…

I have no clue what IKVM is (I’ll have to look it up).

You gave absolutely no reason why parrot can’t work (as you mentioned). It is free software so could be changed, but I won’t buy it until I have a reason. I’ll take a look at the article you linked to see if it gives the clue you are withholding.

Point out otherwise, but I believe Java has had introspection since before dotnet existed or at least it has since version 1.2 if not earlier (not sure if dotnet existed then). If you are correct wrt to dotnet’s timeline (I don’t much know the history of dotnet), certainly Java (the standard and Sun’s version) has at least had introspection for something close to a decade and almost certainly before mono got it implemented.

>> Now, these are effectively “interfaces” but they don’t make any implementation decisions. You have the JVM as Sun ship it. … And obviously, you have Mono, which can run stuff via IKVM.

You are confusing me. You can’t mix VM from different unrelated standards. Maybe what you mean is that you can implement one VM to work over another, but I doubt dotnet’s VM follows the same definitions/interface as Java’s one (I don’t know dotnet).

Anyway, I was expecting more from you here given how you said we didn’t understand dotnet/mono. If you don’t want to share, just don’t criticize. As far as anyone here can tell, you don’t have too much of a clue yourself beyond vague generalizations. [I'm just saying based on what you have posted -- isn't that how you judged us?]

So Microsoft is doing what they have to do, according to the spec, in order to add their own “keywords” or “property names”

Sort of, but still not right. They’re adding CSS 3 properties for people to test. They cannot add those with the CSS 3 names because that would conflict with CSS 2.1.

So, they prefix them with a vendor code: this allows them to develop CSS 3 features without conflicting with existing CSS.

…

http://www.w3.org/TR/CSS21/syndata.html
“Section 4.1.2.1 Vendor-specific extensions”

“Keywords and property names beginning with -’ or ‘_’ are reserved for vendor-specific extensions.”

This section itself doesn’t describe everything (eg, conformance for an implementation is not addressed here).

Also, note the following:

“Authors should avoid vendor-specific extensions”

So Microsoft is doing what they have to do, according to the spec, in order to add their own “keywords” or “property names” (assuming I’m not missing anything).

The amount of research and development that went into designing .NET was huge and Microsoft had a lot of the best minds in the field working on it. Does the Mono team have those kinds of things going for it? Probably not.

Besides having a a design to start with by implementing the .NET specifications, they also get compatibility with Microsoft’s .NET (which is a bonus). There are a lot more Windows developers out there than there are Linux developers. It seems to me that if it is made trivial for them to write applications that work both on Windows and Linux, then Linux users win because they get a wider selection of software to choose from.

The biggest reason a lot of people I know stick with Windows is because of the apps. They won’t even go Mac even though they love the interface (ugh) because the software they need doesn’t exist for Mac.

As far as “did they fall for the microsoft hype?” – I can’t speak for why the Mono team started developing Mono for certain, but from what I’ve read – they chose to implement it because they got tired of writing large applications in C and were in search of a better platform. At the time, Java was not free (and as I discovered a while back, some of the Mono developers were the creators of the GNU Classpath project), so it seems likely that Java had been considered.

Why didn’t they use the Parrot VM? That’s a question you’d have to ask the Mono team, but quite possibly they weren’t aware of the project; maybe the parrot vm wasn’t a good fit (at the time?), etc.

Note: this comment was posted from Novell’s headquarters.

Mono is a good enough VM, Java is a good enough VM, and the thing in Android is probably good enough.

I’m going to try to address the main point of what you’ve written. Let’s talk in terms of Java, since that seems to be more readily understandable to everyone.

You have Java the language, Java the bytecode, and a standard library.

Now, these are effectively “interfaces” but they don’t make any implementation decisions. You have the JVM as Sun ship it. You have gcj. You have the Android system, which is a fundamentally different design being stack-based rather than register-based. And obviously, you have Mono, which can run stuff via IKVM.

Parrot could well be sufficient to run .net; aside from introspection features (which Java only got recently), you could easily have an IKVM-style setup which allowed it.

At the end of the day, the actual VM engine doesn’t really matter one jot. Mono has two VM engines of completely different designs alone! And the main one compiles code into native code as Hotspot does.

Virtually none of the interesting stuff is the platform code. If you haven’t already, I highly recommend cgwalter’s essay on a single free software VM.

CSS 3 is in development, and these are the features that they are developing. CSS 3 is not a finished specification, though.

The vendor prefixes are part of the CSS spec, and ensure that they can add these new features without conflicting with the CSS specification. So with the vendor prefixes, they are valid CSS 2.1. But they’re developing CSS 3 features.

They’re “extending CSS 2.1″ to the extent that they are developing CSS 3 features.