OSDev.org

I was looking around for (free) C(++) compilers and came across the Digital Mars compiler:

http://www.digitalmars.com/download/freecompiler.html

It seems to be in active development, it is free and no install or anything fancy needed (compares to gcc/cygwin, especially on Windows). Anyone ever tried it for OS development?

They also have a "new" language called D, seems to be a bit like Java/C# but compiled directly to native code and still allows native asm. So you have things like garbage collection and all other kind of "higher-level" stuff, without loosing the low-level stuff:

http://www.digitalmars.com/d/

They also have a comparison to Java/C#/C(++):

http://www.digitalmars.com/d/comparison.html

Thoughts?

As for D... that "comparison table" pretty much disqualifies the language in my eyes. Why do people keep comparing their respective language to C++ without taking the standard library into account?. C++ does have resizable arrays (<vector>), it has associative arrays (<map>). It does have a for_each. It does have "typeof".

They haven't really done their homework, and if I hate something it's biased advertising disguised as "comparison".

That being said, I am deeply distrusting languages with garbage collectors. Especially in OS development, where I consider it a show-stopper outright because of the non-deterministic run-time impact of a GC starting up. I've seen trivial bugs in Java that result in memory leaks. And if you can't handle pointers correctly, stay away from kernel space - IMHO.

- being developing under Linux, their free compiler for MS-* is of no use to me.
- D language is around for about 5 years now (i think) but still i see no "programs written with D" section ...
- i'm not sure i'm willing so much a garbage-collected language for writing a kernel ...
- do we have access to the D runtime ? that will be mandatory to make it part of your kernel runtime, no ?

Fair points by all. Thanks.

Solar, you say: "...because of the non-deterministic run-time impact of a GC starting up..."

I've always had a hard time wrapping my head around the "non-deterministic" 'thing', mainly since English is not my primary language. Could you please elaborate on it and in combination with the rest of that sentence (ie, run-time impact, GC starting up)?

When you're writing C++, you know exactly when your destructors will fire: when the object is deleted or goes out of scope.

With a garbage-collected language, you can't tell when or even if your destructors will be executed. (At least that's the case with Java.) That makes destructors nonsense; if there is some cleanup that has to be done before an object goes out of scope (like, closing connections etc.), you have to call the cleanup function yourself. (Tell me again where that is an improvement over having your destructor called automatically.)

As for the run-time impact, when your GC (garbage collector) decides it is time to collect the garbage, your execution sequence is halted while the GC does its job. Imagine this happening while you are in the midst of some kernel-level operation: Your whole system will stop operating for the duration of the garbage collection. It is easy to picture how this wreaks havoc in, say, an interrupt handler, or anything timing-related...

As for primary language: Me neither.

well, the garbage collector needs to run from time to time, but you cannot tell precisely *when* it will run, nor *how long* it will take ... that's precisely "non-deterministic" (more a math concept than english one, actually).

That's not quite a problem for most programs ... if handled properly, it can even remain unnoticed in user-interaction programs but it's not something i'd like to take place.

In kernel environment, however, it's unclear what to do if GC starts when you're routing a packet or handling a disk request.

About GC in kernel land:

AFAIK, typically the GC is implemented in a seperate thread, driven by an event such as a timer. So you simply put it into a kernel thread of _appropriate_ priority - lower than interrupt handlers, higher than most userspace programs. It might be useful however to offer manual control over memory management when appropriate, the 'garbage collected heap' should be a seperate component. So when you have time (the system is idle), you let the GC do its work, when there are better things to do, surpress it.



If you are going to use such a level at kernel language, you should implement its runtime yourself anyway, so you can trust it.

To clear this beforehand, I wouldn't use such a language at the low levels of the kernel either - but I still think it could bring many benefits at the 'higher' levels, such as higher level device drivers and stuff, just things which are not that timing sensetive.

cheers Joe

Deterministic stuff has a pre-determined path (it will always do that). When you make a program that does 20 nops after another, on a 386 that has a deterministic execution path, you can calculate how long it'll take. On a P4, it's nondeterministic, you can't say how long it'll take due to execution units being taken, pipeline stalls etc.

When I go to my work, I drive along a road. If there's no other car and I drive a certain speed, I can calculate when I'll arrive. I'll have a deterministic arrival time.
The truth is that there are other cars and there are stuff like accidents, traffic jams and traffic lights that are all nondeterministic compared to my car movement (they won't go green or dissolve when I drive toward them). That makes my arrival time non-deterministic, that means, I can't guarantee that I'll be in time. I can leave in time and account for some delay, but if a plane crashes in front of me on the motorway (highway) I can't turn around and I'm going to be late.

Same thing for automatons (state machines). There's a subclass of them called Finite Automaton (FA), which have an ending (which may take very long to reach, but for each given input it has 0 or more paths to take). There are two variations of this, non-deterministic (NFA) and deterministic (DFA).

The NFA has paths that you can take based on nothing, you can take any number of paths on a certain input from a given position and you can't actually use it unless you keep track of all paths until there's only one left you could've taken.

The DFA has only states that have nonempty output paths, it has 0 or 1 output path for each possible input and it will not be accepting for more than one outcome.

The point of difference, in a DFA there's nothing left to choice or accidence. There's ONE path that you can follow on any given input and it either ends in an accepting state (you accept then) or it ends, where you give an error. On an NFA, you COULD take this path, but you could just as well take any other.

PS: Je kunt ook in het woordenboek kijken, determinisme is ook een Nederlands woord

I distrust the concept. I've seen people provoke memory leaks in Java with about 20 lines of code. I've seen C++ apps running uninterupted for months without leaking a byte.

Garbage Collection is for those areas where you throw a hundred half-trained code monkeys at a problem, IMHO. Unfortunately that's the rule rather than the exception in today's software industry. For those things where you spend most of your time in design and crafting of algorithms, you need highly qualified people to do the job, and the advantage of GC languages - allowing code monkeys to avoid a couple of pitfalls - is void anyway.

All heavily IMHO, of course. (And yes, I know GC languages and have used them in production.)

Candy: haha, bedankt

Thanks everyone, interesting subject and discussion. I guess a kernel level GC would indeed be best to happen in idle time and/or when needed (like when a memory allocation is happening and a certain threshold is reached or you are running low on memory)

I think I once read a paper somewhere that it might not be such a bad idea to (almost) fully utilize your memory and only release / replace it when you need some more. This seems to be what some (enterprise) database engines use as well.

Food for thought!

@Solar:
I think you'll have to lose some of your distrust of GC's, because IMO they're here to stay. They're good for more than just "enabling code-monkeys" -- GC also makes many lock-free algorithms a lot easier to implement, and it makes memory management much more scalable on multiprocessor systems (allocation is much faster and therefore holds locks for less time, and deallocation can happen asynchronously).

Singularity has its own GC in the kernel. It's for kernel-use only, so it's not a systemwide GC -- each process has its own GC so that applications can use the type of GC that is most suitable. AFAIK the Singularity kernel's GC uses Concurrent Mark-Sweep, although to be honest I'm not entirely sure how it works.



Take a step back and ask yourself: Why do we need destructors? 10 or 15 years ago, the answer most people would give is "to free memory", because they'd happily forget about things like file handles, network connections, etc. The truth is, we will always need deterministic destruction, but IMO for non-memory resources only. This is where Java falls flat, and finalizers really make no sense -- I think here we agree.

The best approach I've seen is in the upcoming C++/CLI extensions. You can still write destructors that get called deterministically for so-called "unmanaged resources", but you also have the GC to free memory. I think this is a pretty good balance.

I know next to nothing about D, but MS Research wrote a kernel in a variant of C#, so IMO anything is possible...

We seem to share a common interest in Singularity, Colonel Kernel. Interesting system!

Just as a matter of interest, or you working on an OS of your own?

In some respects D is an improvement over c++. It does not "feature" a preprocessor, which normally screws up your source. Instead you have version & debug statements. Another thing are these modules instead of including thousands of header files, which just increase compile time. And you don't have to foreward-declare your functions/classes. There is also stronger type checking, that means e.g. that typedef really creates a new type, which was a problem in C++ when it comes to overloading functions. Another thing that comes to my mind are functions declared as "synchronized" (or was it "multithreaded"... not sure), which means that the compiler automatically puts a mutex around these functions. Imho also contract programming _seems_ to be a powerfull concept, but I'm not yet sure of that (I'm not sure if these contracts are also included in the "release version" of your programm, which imho would make no sense). Perhaps someone (with more programming experience) could elaborate on that. And perhaps the "unittesting" is also somehow usefull, but I didn't read about it yet. I also think that single inheritance + interfaces are better than this complex multiple inheritance in C++.
What I'm not sure about are mixins. Didn't get what that is for...

What makes D not very good for osdev is, that the most/interessting parts of the D ABI are to be defined (but that's also an improvement over C++ - an ABI that is specified is better than one that is not. For me it was very annoying to see that there is not much info about the different implementations of C++ through the compilers/different compiler version). Well and the thing about garbage collection... but you mustn't (wait: is it really mustn't here or is it needn't ??? ) use it in D. You "just" have to overload new/delete if you don't want to use it.

well, iirc, having a garbage collector in your language requires that you cannot do pointer arithmethic, right ? but in the case of a kernel we rather _need_ pointer arithmetics, be it only for the memory allocation functions ...

i'm not quite sure of how page tables and the like could be manipulated from a GC'd language. Yet i should admit i haven't tried it, so it might be possible anyway ...

D does not prevent you from using pointer, they are still there. D just wants to minimize the need for pointers. D is still a low level higher language with inline assembler and the like...

[edit: but I'm neither a D expert nor did I programm in D already... I just read through the specs from time to time to get inspired]