OSDev.org

Hi guys,

I'm currently reading the Singularity publications and this seems to be an really cool approach for an operating system. So I'm wondering if someone of you tried/is trying to implement something similar.

The idea to code the entire (almost) OS in C# (or Sing# as they call their extended language) looks interesting. I'm reading through the Singularity and CLI docs now with the idea in mind, to turn my hobby OS in such a direction the singularity guys done it.

So to my question, is there someone out here who has experience with an custom implementation of the CLI and know which pitfalls there are to watch out for? Any hints are appraciated.

What is your opinion on the Singularity stuff?

regards

In my opinion C# is a nasty language and Microsoft is a nasty company..
I think that OS development is much more efficiently done in C/C++ or Assembly.

Thats my two cents worth anyway..

Brynet-Inc, while it may be true that writing an OS in asm or C would likely result in more efficient code...that's kinda not the point.

The goal of singularity is not performance (though I am sure that they keep it in mind). The goal is to attempt a new approach to how operating systems work.

The concept of using strongly typed languages as a protection mechanism is interesting, and to be honest I would like to see where it goes. I do see some things that would need to be addressed.

See the whole point is, that if the language doesn't allow dangerous constructs such as pointers (and you can therefore not trample on memory which is not yours), then you don't need true hardware isolation. Well what happens if I write an application which claims to have been built with C#, but wasn't? Can they accurately detect and prevent this? If not, can they detect "dangerous constructs" at runtime? If not do I not now have instant full access to all of memory and therefore the system?

Of course the easiest answer is to only allow interpreted code which would make it much more difficult to write code which doesn't play nice (though I would argue still possible). But I imagine they would allow using applications written in C# which are compiled to x86 code for efficiency sake.

proxy

You do not have to use c#. The application must be compiled by some compiler to IL. The app therefore could be written in every language an CLR-Compiler is available for.
At install-time the applications IL-code is verified and compiled to native code by the operating system. So it seems to be a lot harder to "pretend to be trustfully".
I'm not really sure how strong the validation of intermediate code can be performed, but it seems there are a lot of possibilites to validate the code before compiling it natively.

The concept of sealed processes pushes this a bit further, so no process can ever directly access memory of another process. This is allows an amazingly stable system if implemented well. The worst case would be that an malicious program would crash it's own process, but never harms any other running application, driver or service.
The entire interprocess communication in singularity is working with channels (as far as I understand this, it's similar to a win32 pipe) with some restrictions which data is allowed to be passed and which not.

And as you mentioned the hardware protection mechanics aren't needed in such an environment, which safes a lot of execution time that can be used by the CLR for task like scheduling, garbage collecting, ...

I really like this whole approach, because it seems to be a huge gain in stability of the entire system.


Whether MS is nasty or not is not really the point, I think ...
I'm not sure if efficiency should be reduced to performance. Of course well written C/C++ code and optimized assembly will beet any CLR code in case of performance. The main problem is to get the code of the entire OS and especially the 3rd party software to this level.
In the last years I saw much more mediocre C/C++ code, than highly optimized. And the maintance of optimized code is almost worse than cancer. On the other side, it's much more "cost-efficient" and "time-efficient" to write high-quality C# code, which will in 80% perform better then mediocre C/C++ code. This is the advantage of a higher level language.

Oh **** ... this looks like "War and Peace" ... sorry guys, I stop here

as stated...kinda missing the point. That aside, C# isn't that bad of a language either. Obviously its not good for everything. But then again, neither is C/C++/Assembly. I sense a bit of bias there just because it is a spawn of microsoft.

I personally am pretty intrigued by singularity. Its nice to see new ideas being implemented just to see how they work. Even if for nothing more than the pure curiosity of doing it just because you can.

Interestingly enough, some of their benchmarks look pretty good, despite the fact that good performance wasn't even a goal of the project.

The programming language you use is becoming less and less relevant to performance. I would argue that what really matters is your language's run-time environment. As a really obvious example, the performance of interpreted code is always going to suffer when compared with compiled code. C# compiles to native code (with an intermediate compilation step in between), so you have to look a little closer to find differences from C/C++ that matter to performance. These differences include things like array bounds checking, boxing/unboxing, and garbage collection. Most bounds checks can be eliminated in Singularity because the IL-to-native compiler has total knowledge of all the code that is going to run in the process (i.e. -- no dynamic linking is allowed, which would otherwise make many optimizations impossible). Excessive boxing and unboxing is becoming less of an issue now that C# has generics. GC is the really controversial item here. I don't want to turn this into a GC flame-fest... suffice it to say that I don't think performance comparison of automatic and manual memory management is as clear-cut as most people think.

The real reason why Singularity performance looks promising is that the cost of IPC is really cheap in a system that doesn't use the MMU. All message passes between software-isolated processes are zero-copy (basically sending pointers around). There are no TLB misses, and no need to deal with page tables when allocating memory. They dissect the costs of these operations in detail in that paper I linked to before.

Brynet-Inc, next time I think you should read up on a subject before delivering your opinion. You may find that you lose credibility otherwise.

They are taking an interesting aproach and I am currently triing to write an similar OS(yet based on my own language).
There are some interesting interwievs availible on the net with singularity developers. But the thing which I had not figured out is how they do the message passing... They say that they do it without copiing and I can't imagine any way doiing it without breaking heap integrity. OK to add some points which I would like to change about Singularity(and strugle to implement it in my own OS):
-GC would be only fall back for nasty applications. Applications would clean memory by combination of reference counting and explicit deallocation. Reference counting cannot handle circular references (eg A points to B, B points to A), but theese circles could be explicitly broken by aplication
-No closed processes - you can use final when you need and it has nearly the same effect and is more flexible
-Legacy domains(maybe) allowing custom domains running eg C or C++ code that would be protected by HW protection
-Not processes but domains. The processes are classic microkernel approach, that means there is some client and server each of them running it's own thread. But domains can share the threads (you are aible to call A domain's function foo from domain B in single thread)
-Domains could be completely erased from this concept and leaving all to work with simple function calls. Domains are here only for additional protection, but even without domains nasty app can not break the system. But it can created for example circularly referenced object to make GC run over the whole system an thus slowing it down.[/list]

I haven't read all the replies yet, but I just want to answer the first post before I go to bed.

Actually I have an idea about creating something similar to Singularis. I think it is a nice concept. It also shows the power of invariants, which I love a lot. However I'm still stuck with my test operating systems for learning different OS concepts and issues. But when I create my future OS I won't choose CIL as the language. I find it too restrictive. It is about programming paradigms and CIL is tightly bound to the single-inheritance object-oriented paradigm. What about multi-inheritance or mix-ins used in a lot of languages? If for instance you would create a C++ to CIL compiler you would have to translate the multi-inheritance into single-inheritance loosing some information and linking ability. And what about LISP? A friend of mine pointed out that it is not that easily executed in .NET because it is another paradigm, even when LISP is interpreted like CIL. This might not be one of the pitfalls you request, but maybe a disadventage. Can you live with it? If so, then go for it! The CIL standard will definitely help you. Maybe the source code of Mono will.

My own idea is to create a even lower level language. A language which is not bound to any specific paradigms. It would be an assembly like language without the notion of objects and similar. But it is strongly typed, has ability for annotated data structures (I call them variable invariants or data invariants) and you can't manipulate pointers in a bad way. It will be optimized for translation as to use JIT (just-in-time) interpreter. I have some ideas about that as well to get the code start faster. The language is still on the drafting board. I have been working on it for a year now. Designing, learning. I have several text documents containing design issues and ideas. I'm a bit of a theoretician. Maybe when I have had the "hands on" compiler course in Computer Science I will create it. Even when it's done I have to compile a high level language to that. I don't want to be stuck with assembly. At that time I will begin programming my dream OS. Just wait a few years

Hmm, what Walling describes sounds rather like HLA...

HLA?

Would it be High Level Assembly?

I found these on Wikipedia: http://en.wikipedia.org/wiki/High_Level_Architecture and http://en.wikipedia.org/wiki/High_Level_Assembly

I have found a few languages incorporating some of the ideas I have, but I haven't found one, which is just similar to the one I'm designing. I don't know if HLA is close to it or not. There are C-- and UNCOL. The first being specified and implemented, the other not. However they are both designed for compiler writing, not as a general purpose virtual machine.

In some sense the language will be highlevel, using annotations and types. But the instructions itself will be low level. There will only be a few instructions, 20 maybe, or less. There will be two branching instructions (conditional, unconditional). But you don't need more. If you need operators you just create annotated functions and tell what it should do, ie. add two variables. It will compile because in the annotation you use a namespace, an URL, which tells the compiler where to download a specification for the specific platform. Your compiler will cache them, or you can download the whole database in advance. No need to be online all the time.

In my opinion it is needed to have common object system, because programms need to interoperate. It is needed that all developers of the system accept this language. Similar situation is today when there are main languages for developement like C,C++ and Delphi and most of devels is not using Lisp. Theese are based on similar prinipies and could be easily based on common object system(but unsafe beacause of pointers). And multiple inheritance can be emulated....I am currently in my system planning to have generics implementation by erasure, but additional system can be added(might be using class loaders) to the environment and this would allow mixins(in my opinion they are not very usefull anyway). They could even mixin existing classes. And that is not what you can achieve with HL assembly because mixins would probably bee incompatible. HL asm is very good and flexible solution but because of flexibility is it is destroing the interoperabiliti.

And i do not knownearly anything about Lisp but if it can not be implemented atop of existing runtime, it can be placed in separated domain. Of course communication between different domains is slower because it has to be converted to scalar data in common format that can be passed throught inter-domain channel.

I have an idea to write Singularity-like OS too. It's concept seems to be interesting innovation.

Writting such an OS means I have to implement most of CLI. But I'm not experienced in writting JIT compiler or even ordinary compiler. Although I can imagine how to implement GC and class library, I can't write the most important part of CLI!

I have an idea to translate CLR byte-code to c or c++ code and compile it to native using gcc or any other open-source compiler. But it needs huge crt to be implemented before such a compiler or one of its parts can be executed!

Is Bartok accessible for our use? And can we ask Microsoft to get it?

They have to agree on some high level language, yes. This will be implemented in my language as a "paradigm model". You can create the object-oriented model, the procedure model, functional programming model, etc. The developers have to agree on the same model to communicate, but all the models can co-exist on the same platform and in the same operating system. Surely models like procedure calls can be used by most languages if no higher level model is available. My language is not something you write in (unless for fun or if you're a compiler writer). If you have read Structured Computer Organisation by Andrew Tanenbaum: My language should be seen as an architecture upon the hardware ISA level rather than just a high-level assembly. Just like the hardware ISA level (fx. I386 assembly) doesn't feature any specific high-level languages, neither does my language.


If the developers agree on a model featuring mixins, there is no problem at all.

Like many other I want to create an OS because I think it should be done different than existing operating systems and similar reasons. I have the same feeling about CIL and Java bytecode, which I think is not very different from a theoretical standpoint. CIL is very interesting, but still... I just have to enhance it, create it the way I want it to be.