OSDev.org

(note, this is all over the place and rambling on and on and on).

i dont remember putting anything down in this thread. rough ideas here.

basically something like os400...

physical memory + disk are the same. you have a singular memory pool. using a DB as the filestore. everything is a DB. you have libraries and files.

processes run either as batch or interactive.

geared more toward a storage system than user desktop system. nothing to say you couldnt run it as a user desktop system, i'm just not desinging in that direction first and foremost.

i want generalised schemas for everything. youd have an email DB, that all email is stored into.
Using this kind of universal storage, you take a lot of complexity out of an application. An email app no longer has to fart around with .pst/mbox files. (ever deleted an email from the middle of your mbox file in pine?)... apps essential query their file store and the os takes care of create/delete/read/write/update/index/search etc

this is whisfull thinking as your not going to get kmail/mozilla mail/outlook/eudora all sharing the one filestore, but I like the concept. If you can get developers to make use of what you give them, who knows what boundaries might be pushed.

think of implementing an itunes type media library, without the complexity of manually scanning mp3 files, parsing meta data, futzing around with sorting and searching, maintaing lists. etc.

someone copies a file onto the system. bam, instant update into the library.

i'm not talking some kind of meta-db like BeFS with its queries,but a full on DB system.

things not in a schema just get stored as BLOB-esque items. IFS would sit inside the DB. say, a record becomes a sector. thus if you installed UFS/XFS/Reiser/EXT2 on top of the DB, you could grow it without hassle since it sits on top of another FS (the db), you attach disk and voila, system just grows. With a special implementation of say EXT2, you could have records for superblock, inodes, direntries, data, using a physical<>logical sector number mapping, thus you only store used sectors, rather than entire swaths of unused disk space. like sparse sector allocations. at any rate, filesystems as disk images can be very felxibly implemented.

in unix, everything is a file (pipes, devices, etc), in my os, everything is a DB.

everything you look at is in its basic form a DB. mp3 for example, you have meta data (artist, title etc), then you have the mp3 audio frames themselevs.

the biggest drawback here obviously, is moving files externally to alien systems (any _other_ os or fs). you get the mac data fork problem all over again.


thinknig kernel wise, i'm interested in a monolithic style kernel. the kernel itself would be stored as objectfiles and the master loader would link them at load time. Am using the PE format, so I can just relocate on load the executables etc. There is so much CRAP in the pe format, but at the same time, I find it nicer than ELF. Im more using the ECOFF than the PE side of things.

There is no concept of admin/user, since it will be all rights based, and rights will be assigned on a user basis. Not much predefined at startup. A user may or may not have blah permission on the system library. or permission to change the run queue priority, or can only be allowed to use 10% of the processor, etc.

Security will also be fine grained, something like windows nt uses with permissions per user, than unix rwx+groups.

I'm still tossing ideas around as, when you get finegrained, things start to become heavy and you need a good and easy way to manage a large and unweildy set of ACL's across an entire filesystem.

(one thing I'd like in windows is to edit some user and see what odd permissions that they have across the entire FS in one single tree like structure, so I can go hmm how did sgeorge get 'full control' to \system32\blah.dll??)

everything on the system is also tied to a security ID which would generally map to a user. Think a global email file, you have to know who's message belongs to whom on the system, etc.

on the resources side of things, part of the security control, subsystems get tied to resource usage. So you can say, 'my web server' cant use any more than 30% of the available processor at any given time, and the entire instance of your OS can only use 2.5 processors on a 4 cpu box. (think partitioning of multiple instances of OS on 1 machine).

Or that the entire OS cant use more than 95% of its allocation, or that XYZ automatically gets 10% when its ran (think emergeancy console access on a DOS'd machine, you get smooth usage, rather than fighting a 100% busy tcpip stack or 100% IO bound process). There is nothing worse than trying to do emergeancy work on an unresponsive box.

(im rambling on now, so I'll stop here before you all go mad).

I like that idea. Sounds good for the style of system I'm designing. It gives you the simplicity of linking drivers in at compile time with the advantages of easily loadable drivers. How do you plan to do it. It sounds quick hard to do. The simplest way I can think of is:-
- Have the different modules of the kernel (core, driver1, mm etc.) linked to archives with ar.
- Then write a stage2 boot loader with a simple linker built in which takes the object files and outputs the binary directly to a certain memory location and then jumps to it.

Interesting. But would require almost a full OS in the stage2 loader.

Pete

What do you guys think of the idea of extensive module use? As in all apps are split up into modules. Each module represents a specific task and they can all interoperate together. You can essentially "assemble" your own custom app from modules of different developers.

It would be something like the shell commands we have today


(Sorry I don't have a Windows/DOS example)

I've heard variations of this around here and read a few articles on the idea...

You could equate this functionality to that of DLLs and ActiveX's in Windows. Instead of utilizing multiple application modules, each application can utilize DLLs or ActiveXs.
One of the drawbacks of Windows's implementation is that the provided functionality of most DLLs and ActiveXs found on a user's system isn't well documented or known to developers. Hence you end up with a couple hundred DLLs that all do the same things but in their own way, and developers are left without any standard implementations for many functions. Perhaps this was intentional on Microsoft's part, to force developers to rely on Microsoft's Windows SDK.

As far as I've understand, KDE has fairly good object model too. Gnome people are also trying to build something like that I think. I don't know about OS X but I'd assume there's something similar there too. So basicly, we have all this so called "component object models" or whatever that are trying to do this.

The KDE one is quite interesting (although I've not tried KDE personally since something like 1.0) with all it's KIOslaves and what not..

Anyway, the most successful "component model" is probably still the unix piping model, Windows with Visual Basic coming next IMHO. I've understood that Python is doing wonders on the OSS and inhouse fronts, although I have no personal experience with that either.

One could say that "component model" is pretty much one of the current buzzwords. Similar trend are different desklet system and such, which share the idea that you take a few basic components, chain them together, draw a nice GUI with photoshop, describe it with an XML file or something, and GO!

Anyway, the more sophisticated and general purpose you make your object model/module system/whatever, the more the users need to know to benefit from it. It's the tradeoff between learnability (initial ease-of-use) and flexibility. Think carefully who will be the "user" and you might build an useful system. I think the "user" here would be the developer building apps for the system, since after all, most people have trouble understanding that if you pipe something from one program to another, you can actually pipe from the second program to a third too..

Thanks for you replys. mystran, as for the OS X stuff, I posted this a while ago explaining the "services" menu. It's piping in a way...

I don't know how many of you watch the alt.os.development newsgroup but Microsoft seem to be deserting COM and DCOM and are going to use messages.

http://zdnet.com.com/2100-1104_2-5148148.html

Pete

Pete:


Sounds like the idea I had, but slightly different. A stage2 boot loader needs almost all the code a kernel does too when it is running, administering paging, segmentation, possibly multithreading (init one module while the other is loading, speed up your boot process, or let the user use the system in its basic form while the rest is still loading...). If stage2 is almost like a kernel, why not make it a kernel? Say, a kernel that is statically linked to a certain place in memory (like F0000000) and is thus loaded there, information about symbols going to a separate place in memory, for later linking, and all the modules being loadable (except for a certain core, which is the "stage2" loader). A single archive of those files is loaded by the real after-BS loader (that doesn't know linking nor other address spaces) to a specific place, and those objects are extracted and thereafter linked in the kernel, releasing their own objects etc. All other drivers are only loaded on request (by a driver most likely, in PnP fashion).

Those core drivers should suffice for the main tasks, and the core archive of driver files is constantly administered for new updates to those drivers, additions, and possible searching of new drivers for unrecognised devices. Some sort of sysadmin thread should do that work.

The end result of this system is that the kernel as itself is only the core, therefore having a "perfection-point", a point at which there is nothing to fix anymore. It should be a near-trivial end (trivial compared to a 20M monolithic kernel that is) so it can be fully tested to work. All other work should rely on that basis.

The second side of the "kernel" (speaking in old terms again) should be the interface between system modules that are loaded and the programs that want to use them. As always, drivers are present that release modules to the module space, and some types of modules (predefined by the end layer) are always taken by the end layer to be connected directly to the end point. So, during bootup, all physical drivers are loaded, then all logical drivers, and then the objects the logical drivers release are picked up by the upper layer.

The upper layer should be replaceable at run time, and should be constantly kept up to date. It stores all its data in a predefined format, which shall not be allowed to change. Only stable versions should be released.

After loading all this the one and only "startup program", like init, is loaded and is activated. It starts all its subprocesses, and possibly configures some devices that need configuration. All device drivers must place their devices in usable, always-working conditions (such as using the slowest form of DMA for harddisks etc). The configuration must always allow choice between this form and all faster forms. At the end of configuration, the system is up and running, and the login connections are made. Only at the point where the configurator is at a certain point in its function will there be any progress display.

*kicks forum* I want longer replies!

Chris:


Seems to be the most common idea on Mega-Tokyo. I have my own take of course:

All file formats and conversions are module-like. Each file format consists of a module for saving and loading. All conversions consist of a conversion between an in-memory type and another in-memory type (possibly, and even usually, the same). The system has functions that allow a program to specify a starting format and an ending format and ask the system to find and make a string of filters that do that conversion.

An application as it is defined by current terms is outdated. People use PSP or Photoshop or the Gimp. They would continue to use those variants of a theme even if they could all do exactly the same. They'd use those because they like that interface best. Taste is a matter per person, so using a file format or something similar should not limit a person to use his/her own environment. If you like Wordpad, why not let it save to .sdw or .xml files?

An application in my new definition is therefore an empty shell. It is only a frontend that allows the user to apply module functions to files in the way the user wants it to happen. there are a lot of trivial-ish modules, that can be stringed together like it's nothing (say, edit-text or save-txt). Others can be as complicated as anyone would like them to be. The interface of a module is defined in an XML-like file including special parsers that open this file and give the applications quite plain entrypoints as to what is required (a save-txt would require an in-memory text buffer, a mix-movies would require two movie buffers and an in-memory mix pattern).

The function of application development is thus reduced to either new frontends or new backends, and no more mixing of those two.

Special care though, some applications are more than they might be. Applications can also use system modules (if given permission to do so). Most if not all applications can use the system time module for reading the time & date, and most can use the file modules to open or save files, but there are a few applications that can load modules in the TCP stack, or modify the settings of a number of random modules.

All system modules again must have an XML-based description of their settings, which is easily parseable and can be stuffed in a configurator. The configurator only needs to know the XML files and the system module types it needs to address (which is in the xml file too) and it can give the user all the power he/she needs to do 100% pixel-perfect administration.

I can only hope this stuff ever comes true. I'm off to start the system module loader (finally got the motivation...).

Thanks for your response Candy .

If you used the module idea, in what areas would you have to approach the design of your OS differently compared to designing something similar to what there is today?

The response was only to be criticised. I am still hoping for some .



Well, since this is still the first I did I'm not quite aware of what the others do. As far as I can judge, the main differences I have with for example linux, are that I load all modules from a completely nonlinked format (still relocatable object, not a shared object) and that they end up being an integral part of the kernel that does not occupy the name space, but is in fact separated by a function linking mechanism.

Didn't figure out the function linking mechanism though, but it should work out something fast enough.

Differing with Clicker or something (hope I get the details right, correct me if I'm wrong) I would load most modules in kernel space for speed, make the links permanent (if possible) speeding it up some more, not use marshalling for parameters, things like that. In other words, sacrifice some independance for speed. You can tell me anything, but computers are fast and shouldn't be slowed down by protecting me from drivers I know are correct.

For the final conclusion about the design, the entire OS design is around it being the faster responding one. If the system is slow, you don't want to work with it. If it takes a minute to load a program, that's bad. If you can set off loading the language checking, auto-correction, macro functionality, picture inclusion and special anti-aliased font rendering, you can get it to load within a second. Gives a lot better impression than MS word for instance...

Candy's post reminds me about one thing ... Most moderns GUIs designers have decided that windows should be moving painted under what-lies-beneath. That is slow (for you have to call apps to repaint themselves, in addition to moving a large picture on the screen :-/). Older systems had a "outlined box" when moving windows -- which was quite fast if not eyecandy. Enlightenment's translucent moving window on top of the current screen snapshot (frozen) is my favourite one ...

What's my point, here ?

Here it comes. I know how to set up my favourite one because i know the option should exist ... but what if in version x' new options come that i dunno ?

I'm dreaming of a system where i could ask quickly for "options about task X", regardless of what system component actually manage that option ... And if i dunno what the task is (imagine i ignore i should change "look&feel - window management - move&resize - outline box", i could ask the Watcher to listen to me, then i move the window and the Watcher reports "Window Highligt (hover) - title highlight - window move"

i can then select "window move" and be sent to a "Customize Window movement" menu

-- pop -- time to wake up :p

With a clean implementation you can do that without freezing the below windows. Keep that one window set to non-visible for the normal window rendering software, add an extra layer that takes the "display", adds a possible transparent window you're moving and then dumps it to the screen.

Also, this does away with freezing anything on the screen. Since you're not going to move more than one window at a time (seems hard to do with only one mouse cursor ) you can make the mixing layer very fast, it's a static setup. Mix image of window-size at offset X,Y on image of screen-size while writing screen-size to the video memory. Options for optimization enough

I've been thinking about adding a overlay APIs to my windowing system. You could create an overlay layer and designate either one colour as transparent, or use an alpha channel. When any app tried to draw underneath that overlay, the app's output and the overlay would be blended together, and the product written to the screen.

This came out of thinking of ways of handling the mouse pointer without flickering (i.e. "what do you do if an app tries to draw over the mouse pointer?"). The pointer could be just another overlay, as could the rectangles you get when you drag windows, as could selection boxes in graphics programs, etc...

@tim: ;D overlayed GUI was one of the things i wished to implement, but which i never really had time to check in depth ...