OSDev.org

Ok, I have a new OS design principal idea now. I will be using an object sytem like the one above (tree made up of linked lists, where each node can link to another list), however, I'm making it much simpler. The idea of using object names as a parameter has been irradicated (it makes everything more complicated, and slow for that matter), and also (I'm not sure I described this before), the idea of stacking methods in a tree has gone. The original idea was that you could go to object net.all_interfaces.IP."192.168.0.1".tcp.80.sendpacket.[pointer to packet contents] and that would send the packet data to the specified ip. However, there are some propblems with this idea: First of all, how the tcp method and ip method would communicate, and secondly, how the parameters (or virtual objects) are transfered to their related method.

This bred my current idea. There is now no more 'method stacking' or 'virtual objects'. The lack of these makes the system a little less beatiful (e.g. not being able to refer to a remote computer as an object), although it gains loads in simplicity. On an execute_object system call, the handler automatically exectues the code pointed to by the last object refered to in the request. The other change is to accomodate for the loss of method stacking, by introducing the new system of streams. Streams are quite like unix pipes -> they are simply areas of memory mapped into two address spaces, by which the relevent proccesses can communicate. The protocol by which this happens can be managed by libraries linked into the applications.

Thus, applications can use the services provided by drivers, and connect them together in the way they choose, much similar to an audio system (i.e. cd player linked to he amp, which links to the speaker), a program could link the physical internet layer dirver to the ip packet assembler, which could be told to recieve input from the tcp packet assembler, which could recieve its raw data from the application. An event manager could be used to wake handler threads up to manage these thing when, say, a packet arrives.

Thankyou for reading my waffling, I hope you could make some sense out of it! BTW is it possible I could start a seperate thread to write all of my os's desing into (I promise I'll try not to waffle as much), as not as many people I think look at this then other threads, and I selfishly wish for lots of people to comment without other peoples designs being in the way, and after all, this is mainly used for showing off your OS desing and talking about all aspects of it in general as opposed to getting lots of feedback, and saying your design is finished is a pretty big desicion and you don't want to have to go and change it later because of something you didn't realise. Perhaps we should have a whole OS-design section (if that's what it's called),

OScoder

Bit of Filesystem ranting:

Virtual Folders: Map folders to other locations on the harddrive or to URLs, mounted devices, or even to perform shell commands.

Folder Permissions: Folder permissions will be handled in a rather simplistic way. Anything in your folder (for example /usrs/nelson/) will be yours unless you allow another user to access it.

Administrative action: I know this is starting to slip into user management, but I'll get back on track. There will be no "root", whenever you need to perform an administrative action (edit devices, change network configurations, etc) , you would be prompted for an Administrative password and allowed to perform that task.

Meta data: Meta data for each file will be stored in an (sqlish)database server for speedy access to make searching a breeze (finally my dream of spotlight on my OS can come true ;o)

Devices wouldn't be stored in the traditional "/dev/" like Unix and others do, rather they would be stored in the same meta database which would act sort of like .INF files work with Windows to provide information about the driver.

Of course, the filesystem will be ontop of a virtual filesystem layer which would allow other filesystems to be mapped to virtual folders (read above)

Uh, out of ideas for now.
Comments?

I've been thinking about making OS handle everything as objects yielded with one kind of interface to access the data inside object. So you'd have HD objects, input device objects, file objects and a database where to access that. The whole system would work on simple FORTH compiler. User would make simple forth scripts to push objects into programs ( they'd auto lock until program is finished ), so programs would never access random devices or subsystems without user interaction.

I gues this may be done already somewhere, but not the way I'm thinking about doing it.

This were a great idea to test my account I made since I'm propably going to write-access this forum later, but done/configured the account already because it's fun.

Has anyone else thought of using a very-high-level language somewhere near to the core of the OS?

My idea is to expose special, privileged system calls that work on the kernel's internals (because I sure as hell am not reimplementing the kernel in another language), run an interpreter for such a language (I'm thinking Lisp) and hand it Foreign Function Interface access to the innards of the kernel. This could allow scripting or reprogramming of the kernel by any sufficiently privileged user.

Hello! I'm new here as well as to OS development in general, but I still have some ambitious plans for the future. Of course, I will probably first have to go through several smaller projects just to get a good grip on good practices and knowledge for OS development. All this on top of classes, too. :-\

There are two main areas where my ideas are the most unusual. The first is security. It comes in a few layers, but all layers have the same dimensions of access.

* The first layer is at the hardware devices themselves (or, at least, the software abstractions of those devices). Pretty much every major hardware device is protected (hard drive, sound, display, network card, keyboard, printer, maybe USB in general?).
* All other layers are device specific. For example, the hard drive's next layer would be in the file structure, specifically directories (Do we have permission to access this directory?). After that would be specific files. After that would be the exact permissions on the file in question (basically just read, write, and execute like in Unix systems). In the case of the network card, the next layer might be the IP and port.
* Each layer is governed by Access Control Lists. These ACLs keep track of users, programs, and user-program combinations that explicitly do or do not have the permissions to breach each layer.
* Ignoring the possibility of setting up users with some predefined sets of permissions, even a user that has permissions to do everything will, by default, have to type his password to do almost anything. This could be compared to a sudoer in Unix.
* Whenever a user is prompted to breach a layer in security, they may set options so as not to be bothered by that particular layer under identical or similar circumstances again later. For example, the user could decide that he doesn't care what program accesses his sound card and explicitly allow any program he runs to play audio. This would not affect other users. As another example, a user could allow a certain application to access any IP, but only on a specific port or set of ports. This would be ideal for programs like web browsers or file sharing applications. It would only apply to that user, but also only to that program.
* Some users could be created to only have restricted permissions. Basically, instead of a password prompt, these users would receive only a notification that they do not have the permission to perform that specific task.

This approach to security, if used correctly, will only silently allow things that the user doesn't care about, and will ask for approval from the user for things that the user does care about.

The other main idea is one that is not explored often in general purpose operating systems: energy efficiency. Energy is becoming more and more of an issue with computers, and while hardware designers have been working pretty hard on the issue, software developers have not.

* Manage energy just like any other system resource. Just like tasks are allocated CPU time, memory, drive space, and network bandwidth, so must they be explicitly allocated energy.
* This requires a pretty good profile of the hardware of the machine, so this scheme would only work on hardware devices that have been profiled for energy use in all their different states.
* Units of energy (probably in terms of Watt-hours or some related unit) are allocated to each task at the beginning of each timeslice. Tasks are aware of their own energy usage at all times.
* This energy is used up by CPU usage and system calls (because system calls are usually used for hardware access).
* The entire system can be limited to a certain rate of energy usage, or individual tasks or groups of tasks can be limited to certain rates of energy usage.
* Hardware access can be queued up by the kernel to further save energy. For example, hard drive accesses can be queued up to minimize spinning up and spinning down.
* Even use of fans in the computer can be limited by slowing down use or clock rate of the hardware (maintaining a low temperature), for example. This leads to the possibility of a "quiet mode," which could be especially useful for laptop users or people who are recording audio.
* Programs can give hints to the kernel about its future plans, such as files it will be accessing shortly or bursts of extra energy they might need to assist the kernel in queuing up its resources and allocating appropriately.
* Misbehaving programs, of course, can be limited in their energy usage appropriately.

So basically I am wanting to put control over the energy usage of a machine in the hands of the users. This would be beneficial to nearly all types of computers whether they be servers, desktops, or laptops.

I have some vague ideas for other areas of the OS, but nothing quite so new or unique. I, of course, want the whole thing to be clean, simple, extendable, and so on, but these are not special goals at all. Before I do any extremely serious research into these areas, I need to better learn how to get the lower-level parts of OS development started.

To my disappointment, this thread does not seem to be as active as I was hoping. Oh well.

As of late, I have realized that my security ideas are nothing new and special at all, or, at least, they have not been expressed very well in this thread or defined very precisely in my head, but the security stuff has taken kind of a back seat anyway so far.

As for my ideas on energy efficiency, I extrapolated the idea a bit. I realized that while energy really would be used more efficiently when it is treated as a limited resource in the system, but it occurred to me that most operating systems do not treat any of the system resources as limited. To the processes on most operating systems, all resources they use are free and unlimited, when this is, in fact, never the case.

My solution is to treat system resources as things that must be considered with a tradeoff. For example, in order to take more memory (from another process), it would be more fair and efficient to supply that process with an equally-valued amount of CPU time, network bandwidth, power, etc. These values are to be determined by how much time is saved or lost due to the tradeoff. Each resource would have different values to different tasks, so there will generally be many opportunities to trade resources. Not only that, but the amount of each to trade can be determined based on the principle of marginal utility; the more of a resource a task has, the less one additional unit of that resource would help. Each process would keep a small table of data to assist in estimating and storing this data to a limited scope. Because these marginal utilities would change over time, there will need to be some experimenting to keep these tables roughly in sync with the changes in resource needs.

I am thinking of using time as my virtual unit of currency. This would have the effect of being able to more directly compare, for example, CPU time with memory. For example, the decision to buy some more memory could be made based on whether it would cut back enough time spent on paging to justify the CPU time given up to buy it.

Just a rough description of my idea. I actually have a very specific algorithm worked out, but it would be a bit pointless trying to describe it here right now, especially this early.

@geezusfreek: pretty long posts, too ... plus this thread is not exactly the kind of thread where people reply directly to each other ...

btw, you might discover that telling what files or what resources are used in the future is not necessarily easier for the program than it is for the OS. I mean think of a large database server ... before a request arrive, it cannot tell what tables will be needed to serve that request. same for web servers ... and certainly similar things for plenty other apps. All you can do is give "generic" hints such as "those data behave better if least-recently used cache policy is applied" or "this is a (video) stream. which means i don't care re-reading bits i've read, but i _do_ care about getting _next_ bytes on schedule".

i'm a bit puzzled by your "trade memory for energy", i should admit ...

Did you know there have been lisp machines since 1975 which are designed for Lisp? http://www.cliki.net/Lisp%20Machines

If you use Lisp: When done well. The entire operating system would be extensible all along. And not much slower than C implementation. ( if slow at all )

I've heard of Lisp machines, which are part of what inspired me.

So far I've got a reasonably working interpreter that compiles down to about 270K, requiring only enough OS/lib support to read and write lines of text to terminals.

Features to add before it's done:
1.Block/return-from and tagbody/go special forms. Once I figure these out they'll be pretty easy to implement.
2.Progn special form. Really easy to implement.
3.CLOS, or something like it.
4.UFFI support, or something very like it.
5.Some way of streaming code to it so the programmer doesn't have to type out all the definitions of some OS-involved Lisp functions involving UFFI bindings. Currently this could only be accomplished by redirecting the standard input stream, but where does that leave you when the preparation work is done and you want an REPL?

Fun.

I'm developing an OS (provisional name: Hermes) and it's intended to be used to run robots. My first robot will be a simple one, just drive around and turn around when in front of something;D

But anyway, has anyone done this before? I heard that QNX could be somehow used for this purpose (it's used on the Hubble), but it's closed-source... :-\

Deving an OS that will only on robots frees me from the tentation of osdeving like James T. Klik ...

Here is my design:

- Modular hybrid kernel
- Full virtual memory
- Multitasking/Multithreading
- Later release: Multiprocessor (SMP & NUMA)

This isn't anything special, I guess most of you here have a similar design ;D

However, I want the robots running my OS to communicate in order to realize tasks in teams (like pushing a block too havy for only one of them). Apart from that, I want the robots to donate their idle time to other robots that have some difficult calculations to do.

I also want an AI server in my microkernel at later releases. This server would implement certain AI techniques and provide them to the robot's controller.

The robot's controller is the key part of my OS. I'm not sure whether to run it inside the kernel or as a server. But anyway, this controller would be similar to an extended device manager. There will be two device managers. One at a low-level, which all devices are registered with and that just starts them as a server/module for the microkernel. Then, the controller is started and loads an XML (can also be something else, I haven't come to this part of my OS) file in order to program it's behaviour. (What is it expected to do on which events, etc.). Thus, the controller is re-usable. Then, the controller asks the low-level device manager for the needed device drivers and puts them at different priority levels: For example, a robot has the task to get to a given point (a given GPS coordinate or something similar), but the task of walking and avoiding to walk into objects will always be at a higher priority, as it is fundamental for the robot. Like this, a robot can also be given different tasks to be executed simultaneously, but with a different priority each. This is going to be implemented like this: An event coming from a B-level device or program will be ignored if the driver is processing an event/interrupt coming from an A-level device. BTW, I'm talking of real devices, but I'm also envisioning some kind of virtual devices needed to program the robot's behaviour. These virtual devices could also be wrappers around two or three other devices needed to do the same task. A physical device can also be mapped in different virtual devices, thus also obeying first the high-level drivers.

Right now, I've got my basic kernel, but in order to actually make a /robot/, I need devices, minimally an engine to move it and a contact sensor... Has anyone done this before?

Candamir/Robin

Ok, I've got an idea that I'm interested to get some people's opinions on:

I was thinking of building a microkernel which had NO system calls whatsoever, just IPC through shared memory.

Would there be any advantages or disadvatages in working this way?

Thanks,
OScoder

How would you implement IPC without a system call? How would the kernel know when to take a peek in the shared memory and see what requests are there? How would the client know when to take a peek in the shared memory and see when the kernel's response has been delivered?

What's so bad about system calls anyway?

My approach to system calls is going to be similar - IPC over pipes to a kernel process which is in ring 0, and has access to the system functions direct. Thus, context switches occur when the kernel process becomes unblocked, but multiple system calls can be handled in one context switch (as multiple calls could be queued on the incoming pipes).

Andrew

You're describing the implementation of abstractions in terms of other abstractions. How do you implement "pipes", a "kernel process", and "unblocking" without system calls?