OSDev.org

The Place to Start for Operating System Developers
It is currently Fri May 17, 2024 12:35 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 237 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16  Next
Author Message
 Post subject:
PostPosted: Sun Mar 04, 2007 10:48 pm 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 6:06 pm
Posts: 1437
Location: Vancouver, BC, Canada
bubach wrote:
even if i would try to reuse code with diffrent types of calling conventions, it's not really much of a problem in asm, just use the method any comments for that function tells you to.


It's great if someone took the time to write those comments, but it takes a human to write them and a human to read, understand, and apply them. You don't get any sort of automated help from your development tools. Like I said, it takes self-discipline. This is the point I was trying to make.

_________________
Top three reasons why my OS project died:
  1. Too much overtime at work
  2. Got married
  3. My brain got stuck in an infinite loop while trying to design the memory manager
Don't let this happen to you!


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 04, 2007 11:32 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

Colonel Kernel wrote:
I guess what I'm trying to say is that writing reusable asm requires far more self-discipline. Any development process requiring that much self-discipline is inherently not scalable.


IMHO you can deliberately write reuseable, or you can deliberately write highly specialised code. Sometimes these are mutually exclusive and sometimes they aren't.

I always try to write highly specialised code. Sometimes I can cut & paste older code, but in general I can't. Usually (for me) there's larger changes, and it's these larger changes that cause the rewriting.

For an example, let me compare my last code to the current code - they're both mostly similar (modular) micro-kernel designs and the only main difference is that the newest code is designed to support headless systems and diskless systems. I'll focus on all the boot code because neither of them got much further than that (so far).

First, there's the boot loaders. The 1440 KB floppy boot loader started as the old code, but I added code to make the PC speaker beep when there's an error (for headless systems) and some code to detect if a VGA card is present (to automatically force a "headless boot" if there's no VGA). The Bochs ROM boot loader and the PXE/network boot loader are both entirely new code (nothing similar in the previous version).

Next is the "Boot Manager". About 80% of it is re-used code while the other 20% is was rewritten due to different physical memory usage and different keyboard/video methods (it used to directly access keyboard and video, but this was split into a seperate module).

There's also a "fake" boot manager that decompresses boot images, and a utility to compress the boot images (written in portable C code). The algorithm I was using to compress the boot images was too slow (especially for large boot images), so I rewrote this utility to use a different algorithm (now it uses more RAM, but compresses much faster). When I rewrote the compression utility I didn't care about making the output compatible with the old version, so when in the end I rewrote the OSs decompression module too.

Then there's the boot user interface code. There's 2 versions of this - one for VT100 over the serial port (for headless systems) and the other for keyboard/VGA. The VT100 code is entirely new. The keyboard/VGA code was mostly rewritten because I changed the kernel log format (stripped out the colour control codes and replaced them with bold and highlight codes, because VT100 can't reliably handle colour) and improved the way it works for better performance (instead of scrolling each time the cursor reaches the bottom of the screen, it pre-checks the new text and does one big scroll, then draws the new text without caring about hitting the bottom of the screen - end result is a lot less time spent scrolling when several lines are added at once).

The physical memory manager boot module is used to detect the system memory map and dynamically allocate pages before the kernel is started. The last version took into account page colour and NUMA domain, so that any code that allocates a page got the best possible physical page for it's purposes. Because of the complexity of it I had an "allocate a page" function in the physical memory manager that was added to the boot API, and the physical memory manager had to remain in memory (like a TSR, if anyone remembers DOS programming). I decided this wasn't such a good idea - the boot code was "32-bit only" and couldn't handle pages above 4 GB, it tended to impose it's data structures on the kernel's own physical memory manager (which was meant to be easily changeable), and because it was the only boot module that needed to stay in RAM after it had done it's thing it made the boot manager messier.

The new version uses a much simpler approach - one bit per page, without caring about page colours or NUMA domains. This means the physical memory manager used during boot is radically different (different function parameters, different data structures, etc), and meant changing all the code that uses the physical memory manager during boot (mainly code that sets up paging for the kernel) so it allocates it's own pages directly from the "free page bitmap" instead of using the boot API. It also means that the kernel will need to be able to optimise the physical pages it's using (something like a "kernel memory optimiser" running in the background that exchanges the physical pages the kernel is using for better physical pages).

The "CPU detection" module was mostly cut & paste, but I added code to detect "CPU forgery" and code to handle the errata for a lot more CPUs.

Then there's a module that handles NUMA domain information. This was mostly cut & paste, but the old code detected areas of the physical address space that are reserved for hot plug RAM. This got removed as it has nothing to do with NUMA domains and belongs in the physical memory manager boot module (it's unfortunate that the ACPI SRAT table contains a lot of NUMA information *and* the hot plug RAM information - the old code parsed this table once, while for the new code the same table is parsed twice).

Overall, I'd estimate about 20% of the code was reused, 40% was rewritten due to change, 10% was rewritten for no good reason, and the remaining 30% is entirely new. If I was working in C (or any other HLL) and if I had deliberately written the original code to be reusable (which I didn't), it really wouldn't have made much difference (I'd still rewrite a little bit for no good reason, that's just the way I do things). ;)


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 12, 2007 5:46 pm 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2294
Location: USA (and Australia)
What do people think about something like http://fabrice.bellard.free.fr/tcc/tccboot.html?

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 12, 2007 8:40 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 27, 2006 12:00 am
Posts: 1444
Cool, people use tcc to code C program for DexOS, which seems to work fine.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 2:42 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2294
Location: USA (and Australia)
No, I mean compiling the kernel at runtime.

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 3:54 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 18, 2006 3:45 am
Posts: 9301
Location: On the balcony, where I can actually keep 1½m distance
IMHO its a waste of time. What tcc must do every boot can be done by gcc just once.

_________________
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 5:26 am 
Offline
Member
Member

Joined: Sun Jul 30, 2006 8:16 am
Posts: 149
Location: The Netherlands
Combuster wrote:
IMHO its a waste of time. What tcc must do every boot can be done by gcc just once.

I agree, though this might be useful on a livecd, if tcc performs CPU-specific optimizations (eg it uses new CPU features that can't be counted on to be present in all current CPUs, but happen to be present in the current one). A single gcc run can't do that, since it won't know which CPUs will be booting from the CD.

It could also be useful for Linux (kernel and/or driver) developers, at least for their test systems.

But for a normal install it doesn't make much sense, since the harddisk (which is presumably where the kernel is stored ;) ) is mutable, so if you're going to compile your kernel specifically for that CPU you might as well do so once and store it on disk.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 20, 2007 12:24 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2294
Location: USA (and Australia)
How do people layout their OS directory structure?

Mine is (my OS uses \ instead of the Unix /):
\System\
kernel.bin
Configure\
system
users.cfg
...
\Users\
Superuser\
Applications\
Source\
Include\
Kernel\
Drivers\
Servers\
Boot\
Libraries\
\Applications\
ProgramName\
\Devices\

When requesting a file from \Applications\ProgramName\ (such as a config file), my API says a program should first search in \Users\--\Applications\ProgramName\ for user-specific files.

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 21, 2007 9:34 am 
Offline
Member
Member

Joined: Thu Mar 15, 2007 8:48 am
Posts: 214
The FS design I've got right now is pretty basic. It's POSIX-style, but incompatible.

Code:
bin/
    -- Programs. Duh.
    local/
        -- Sysadmin-compiled programs. Put here for easy removal.
lib/
    -- Shared libraries.
    local/
        -- Same as /bin/local.
var/
    -- Data. System-wide configuration, log files, etc.
    log/
        -- Logs.
    cfg/
        -- Program configuration, except OS-wide. User-specific settings override these.
    src/
        -- Source of any kind! Sysadmins should dump them here before compiling.
    local/
        -- Guess. Of course, contains log,cfg,src.
usr/
    -- User directories.
    <usrname>/
        -- User's  home directory. Whatever they want. Optionally:
        var,lib,bin/...
            -- User-installed {anything}. For when they're not sysadmin.
boot/
    -- Grub, kernel, etc.
sys/
    -- System related stuff.
    var,lib,bin/...
        -- Core system binaries, libraries, and data (/sys/var/src contains system sources).
dev/
    -- Devices. :)


I plan stuff too far in advance :-)


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 21, 2007 3:49 pm 
Offline
Member
Member

Joined: Sun Jan 14, 2007 9:15 pm
Posts: 2566
Location: Sydney, Australia (I come from a land down under!)
All my FS is so far is:

Code:
/boot
     -- grub stages, kernel and config file for grub
/
    -- the ( < 512 byte) file that stores user login info, storage for anything else


As you can see, it's really simple :( . Mainly because ATM my file i/o drivers only read consecutive sectors (so a fragmented file means serious problems). I'm still trying to figure out how on earth to do the whole FAT12 'fat table' thing (properly).

Once I do get it:
Code:
/
    -- root dir
/boot
    -- grub etc...
/sam
    -- user login data, unavailable for access from OS
/usr
    -- user folders
/bin
    -- shell programs
/help
    -- help system files
/tmp
    -- system temporary files
/dev
    -- wrappers for devices


My plan is to emulate a typical ext2 filesystem on a FAT disk. Once I get file i/o working (Somehow) I'll implement this.

_________________
Pedigree | GitHub | Twitter | LinkedIn


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 21, 2007 7:42 pm 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 9:29 pm
Posts: 2426
Location: Canada
Good luck with permissions ;)

_________________
Image
Twitter: @canadianbryan. Award by smcerm, I stole it. Original was larger.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 21, 2007 8:26 pm 
Offline
Member
Member

Joined: Sun Jan 14, 2007 9:15 pm
Posts: 2566
Location: Sydney, Australia (I come from a land down under!)
As I said, I'm a long way from getting it to work :(

Only thing between me and trying now is getting FAT16 onto a floppy...

Edit: And as they say, better to aim for the sky and hit a tree then aim for a tree and hit a rock.

_________________
Pedigree | GitHub | Twitter | LinkedIn


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 23, 2007 11:35 am 
Offline
Member
Member

Joined: Thu Mar 15, 2007 8:48 am
Posts: 214
Just don't take the __.__linux file route, for the love of all things holy. (*has visions of religion topic*)


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 24, 2007 9:22 pm 
Offline
Member
Member

Joined: Sun Jan 14, 2007 9:15 pm
Posts: 2566
Location: Sydney, Australia (I come from a land down under!)
Hmmm.. Just got it working :D

Now, to figure out permissions :?

_________________
Pedigree | GitHub | Twitter | LinkedIn


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 02, 2007 11:15 am 
Offline
Member
Member
User avatar

Joined: Tue Mar 06, 2007 11:17 am
Posts: 1225
I'm currently just taking projects from others to try to learn something, leave it well explained and extract the interesting bits from there. Hopefuly it will reach a point of didactic value for the ones that document and organize it (currently only me) and the ones that will consume that information.

So, I'm currently "designing" better methods and techniques to more easily crack down the complexities of understanding and so...

Maybe somebody will donate me so I can keep up with that work :lol: :P :)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 237 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: 8infy, Bing [Bot], Majestic-12 [Bot] and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group