OSDev.org

Often, beginners in OSDEV imagine their OS being compatible with all the major operating systems. However... wine is the best (not an emulator!) compatibility layer we have in this field so far, and it is still notorious in 2024 for being unreliable. Sure, it's made substantial progress, but many Windows programs that many users find very important (Adobe Creative Cloud, Visual Studio, ROBLOX) still do not work reliably.

There have been many notable attempts at supporting another operating system's software, to varying degrees of success. The (esoteric) Longene project, the entirety of the 20-year-old ReactOS project, Wine, and even Managarm who aims for source compatibility rather than binary compatibility.

On the surface, it seems relatively simple. Support another system's syscall interface (of which, many do not conflict, as Linux uses interrupt 0x80, and Windows uses interrupt 0x2a IIRC, making multi-compat possible), and the rest of the cards should fall in line, right?

Obviously not, but why?

"Just support the syscall layer". You say that as though it was simple.

Supporting compatibility with Linux for example would require you to support all of Linux' system calls to some degree at least. That includes SYS_fork and SYS_clone. With those alone, you have already made the choice that processes/threads are created in the fork style, as copies of their parents, with only select things changed. It also requires you to support POSIX file descriptors, and rdos has previously dumped entire buckets of bile onto that concept, so it is not always well-received.

If you want to also support Windows system calls, you also have to support Windows concepts. Such as drives. I just made a topic on BIOS updates for an HP laptop. The HP program I need to run for this has a function to write its files onto a USB stick. It somehow queries all drives in the system to find a suitable candidate to do the dumping on. Now, this does not work at all on a UNIX-like system, which handles devices completely differently.

Next problem you will have is that the actual Windows system call layer is completely undocumented. It is hidden beneath at least two layers of DLLs, which are allowed to change without notice and frequently do. Of those, only the interface of the highest layer is defined, but a lot of applications actually use the lower layers.

The info you have on the interrupt numbers is outdated; all remotely modern operating systems use syscall or sysenter. So now the system calls may intersect! Although that issue can be taken care of with a "personality" mechanism like Linux has (basically you set a flag on the process, and that flag is preserved across execve() and inherited into children by default),

The next issue you have is that many applications use undocumented "features", or just observed behavior, that you also have to implement in order to not break. For example, on Linux, there is a select() function. That function is defined in POSIX to take a struct timeval as timeout. All other functions take a timeout as a timespec structure, and all of those error if the structure given is invalid. Not select(). It will normalize the timeout argument. Or at least, it did historically. Now POSIX doesn't say that that will happen, but if you implement Linux syscall compatibility, you have to implement stuff like this as well.

And finally, sometimes emulation is just not feasible within your framework. It is no coincidence that many of the applications you mention are graphically intensive. Windows has DirectX, a proprietary graphics library. And on Linux, you have basically no good way to implement it. Emulating DirectX with OpenGL is like emulating a Gamecube with a PS2. They are just two very different approaches.

I think it's worth noting that you've overlooked the whole POSIX ecosystem, of which Managarm here is just the most recent of possibly hundreds of OSs. POSIX and the preceeding Standard Unix standardised the many Unix-like and Unix-compatible OSs which appeared in the 80s if not the late 70s. Compatibility has always been source-level only and has always required patches, but at least you can patch. It has its difficulties but I find it less painful than any other form of compatibility. Well, no, there are frameworks, sometimes attached to languages, which try to work the same way on every OS they support. Those are nice, though there are gotchas such as Windows file semantics and performance characteristics being substantially different from POSIX's.

Nitpick: Proton, which is effectively a translation from DirectX to Vulkan, works pretty fine.