chaos-devel Mailing List for The chaos Operating System

On Mon, 2006-04-24 at 18:16 +0200, Johan Hermansson wrote:

Better late than never, I hope... :) I have been having trouble catching
up with the mailing list (even though I was spending time at work
reading it!) and right now we're in the middle of a move so... I guess
that's a reasonable excuse. :)

> All these chaos mails suddenly appearing are really confusing. Some
> arrive in the wrong order, and lots of them aren't showing up at all,
> probably because some of you aren't replying correctly.

Yep... Reply to all is important, as Anders Öhrt mentioned. Appearing in
the wrong order must be because they are being forwarded later to the
list, or that they are cross-sent to both the list and the person whom
you're replying to... and that person might be able to reply to it
before the list get it - or something? Do you think this problem still
exists?

> May I ask what triggered this, as far as I can tell, spontaneous burst
> of activity, Per?

I did a nasty thing and just mass-subscribed a bunch of people that used
to write to the chaos-misc mailing list, and posted a message. However,
I didn't add them to the list but just subscribed them. (so a lot of you
got a confirmation email) That didn't trigger very much but some of the
people. The next day, I took this a step further and did a mass join to
the list of all email addresses (by scanning my email with some
scripting) and sent a new email, and Joel Jose replied to this... and
his email became the little spark that triggered a flood of email. 139
messages archived on SourceForge so far. Not too bad a daily average
since april 17th!

> > > The idea of layer above the kernel - either kernel independent or tied to
> > > a chosen one sounds good. However, I have a fundamental question
> > > - what value does this add?
> To whoever wrote the above: I'm not even sure if I should find it
> amusing or disturbing that you think an idea is attractive, when you
> at the same time are having troubles seeing its value.

:-) It was Shanky, who volunteered to take over the project a couple of
years ago (and did so, he started an effort but unfortunately, it never
bore any fruit). As far as the statement itself, I'll let him explain it
if he hasn't already (I still have 38 emails to read from this list).
-- 
Best regards,
Per Lundberg

On Fri, 2006-04-28 at 10:12 +0200, Anders Öhrt wrote:
> > Or making a QNX-ish clone?
> We don't want to make a clone of anything. Why could someone use chaos
> instead of the cloned original?

Very good point indeed... In the case of QNX, the idea could be to have
the clone be free software/open source and have that be the reason for
cloning. (Please don't get the erroneous idea that I'm now suggesting
that we *should* do a free QNX clone, it should be taken figuratively)

As far as micro-kernel versus macro-kernel, I think I was more speaking
about how storm G3 worked and the ideas behind it rather than how we
should now do it. I said you were too systematic; that is, I think you
were more systematic and thought-through than I was when designing storm
G3. But I think we could have these 3 variants:

	A: A pure micro-kernel. All services provided from within user-space.
Very clean, and very slow. Doing anything basically involves at least
two task switches; one task switch to the service provider and one task
switch back. Yes, can probably be made more efficient now with dualcore
(you could have the second core running the service tasks?) but still...
probably some years in the future before something of this can really
win.

	B: A semi micro/macro-kernel approach (what you were asking whether we
should do). Speed-critical services (such as hardware drivers and VFS)
are written as kernel modules where non-speed critical services (such as
the log and console servers) are written as regular user processes. This
is not how we did with stormG3 though, I think I more or less shifted
over to the C paradigm which is:

	C: A pure macro-kernel approach. Everything is done as kernel modules.
The rationale behind doing so is simple; doing things in userspace makes
things more complicated and doesn't add so much more security anyway (at
least for PCI drivers as I've mentioned before). Or, at least so we're
trying to pretend. :-)

I think the A paradigm can be discarded from the start; it is simply too
much work and impossible to combine with busmastering PCI anyway.
(because you tell the PCI device itself where to put its memory buffer,
so a PCI device driver will in effect always run in ring 0 whether we
allow it to or not. You could probably use this to read any memory in
the system if you are a bit clever)

The C paradigm can seem a bit stupid so, perhaps going with road B is
better. Or, we wait with this kind of decision until we've done the
userspace stuff to the extent where we are ready for writing a kernel of
our own. Ideally, we would write the high level stuff in such a way that
it can run on both a micro-kernel (L4a/pistachio) as well as on a
macro-kernel (Linux), so we can compare up/down-sides of them both.

I'll write more at a later time but now, I'll go back to putting up the
last wall covering ("tapet") in our new computer room... :)
-- 
Best regards,
Per Lundberg

On Fri, 2006-04-28 at 00:28 +0200, Anders Öhrt wrote: 
> You hardly ever change "/etc", and when you do you don't need
> atomicity. And once you've made a change, do you really want that
> change to be saved and reversible for ever? I can see you want the
> last step, in case you do something wrong, but having a full history
> is just unnecessary.

Perhaps, but sometimes changes are made where atomicity would be a plus
-- such as software installations or operating system upgrades. Imagine
being able to reverse the installation of an application. The upgrade of
an application -- or even an upgrade of the whole operating system, in
case of incompatibility because of (for example) API changes. Of course,
someone can always implement this as an add-on but wouldn't it be great
and a step in the right direction to have this implemented in the system
by design?
-- 
Best regards,
Per Lundberg

On 4/28/06, Henrik Hallin <hen...@st...> wrote:
> This also means that the OS can have greater control of what is going
> on. It can map seamingly local objects to remote objects transparently
> for applications and do some serialization automatically. It's at least
> interesting to think about.

I think that with such an approach, it would be much easier to
implement an "objectified VFS".

> >> Yes, something like that. And other safe languages that doesn't have
> >> direct pointers could also be modified to compile to this virtual
> >> machine...
> >
> > But there is still a security issue, we would have to supply and
> > verify the VM or someone could hack other applications, right?
>
> Of course the VM would be part of the OS, or BE the OS in fact.

That would mean a huge monolithic kernel, right? If even the VM is
part of it. Sounds strange, I don't get it.


> Basically all the standard microkernel components like MM,
> process/thread handling etc would be there with an added translator
> module that translates and binds code objects on demand/on load.

But, what protects this module, is must be a (compiled in) part of the
kernel...?


> But this means writing a kernel, which wasn't what we had planned right n=
ow?

I didn't think we had any plans yet, and were keeping the discusions
in the meta physical plane... *sigh*

// Anders

Anders =D6hrt skrev:
>> > How can that be enforced? Should Java (or C#) be the only language y=
ou
>> > can write applications in?
>>
>> Yes, something like that. And other safe languages that doesn't have
>> direct pointers could also be modified to compile to this virtual=20
>> machine...
>
> But there is still a security issue, we would have to supply and
> verify the VM or someone could hack other applications, right?
Of course the VM would be part of the OS, or BE the OS in fact.=20
Basically all the standard microkernel components like MM,=20
process/thread handling etc would be there with an added translator=20
module that translates and binds code objects on demand/on load.

>> It's just a thought for discussion...
>
> Well, there really isn't much point to writing Yet Another OS,
> something have to be unique about it, so I'm not discarding anything
> right now... =3D)
Good. :)

But this means writing a kernel, which wasn't what we had planned right n=
ow?

Henrik

> > How can that be enforced? Should Java (or C#) be the only language you
> > can write applications in?
>
> Yes, something like that. And other safe languages that doesn't have
> direct pointers could also be modified to compile to this virtual machine=
...

But there is still a security issue, we would have to supply and
verify the VM or someone could hack other applications, right?


> It's just a thought for discussion...

Well, there really isn't much point to writing Yet Another OS,
something have to be unique about it, so I'm not discarding anything
right now... =3D)

// Anders

Anders =D6hrt skrev:
>> > o Perhaps microkernels and message passing are outdated? A type-safe
>> > language/env could run things isolated with different costs that=20
>> might make it
>> > a better option than everything coded in C and super-optimized=20
>> message passing.
>> >
>> I have been thinking a bit about this lately and I have grown to like
>> this idea very much. Using a type safe language to write all chaos
>> applications,
>
> How can that be enforced? Should Java (or C#) be the only language you
> can write applications in?
Yes, something like that. And other safe languages that doesn't have=20
direct pointers could also be modified to compile to this virtual machine=
...

It's just a thought for discussion...

Henrik

> > o Perhaps microkernels and message passing are outdated? A type-safe
> > language/env could run things isolated with different costs that might =
make it
> > a better option than everything coded in C and super-optimized message =
passing.
> >
> I have been thinking a bit about this lately and I have grown to like
> this idea very much. Using a type safe language to write all chaos
> applications,

How can that be enforced? Should Java (or C#) be the only language you
can write applications in?

// Anders

> > There will be an API difference, so the same module can't be
> > configured to run either in-kernel or in user space without adding
> > some API wrappers.
>
> Why not? For performance reasons? There are certainly no technical reason=
s it
> couldn't be a configurable option while having the same API for both
> situations.

In-kernel would use direct function calls (that's where the
performance comes from), which user space will use some totally
different, like a filesystem (/servicefs).

You could add both to a module, but then you would need API wrappers
for each function.


> That depends on how good it is and what we feel it could contribute with.=
 The
> L4 team have worked for years trying to make things like context switches=
 as
> fast as possible.

We could use L4 as the kernel and build from there. If we used Linux,
that would be a temporary solution, but I guess L4 would work as a
permanent one.


> o Linux is like 50+MB of source all compiled to run in the same address s=
pace,
> how can they expect it to be stable?

That's the problem with monolithic kernels, you sacrifice stability
and need to make sure noone messes up and stomps all over the memory.


> o Package-systems in Linux dists suffer from the old file system layouts,=
 and
> too fine-grained division of libraries. Here on my Mac I can drag a game =
from
> its CD to the desktop to install it. Even MS Office can be installed by
> drag-and-drop.

Microsoft's MSI package system is a good inovation, something similar
should be used to simplify application installation. Package-systems
solve dependencies, but none work "perfect".


> o Desktop tech like gtk+ would probably be much more robust and faster if=
 it
> was written in C#.

Can't comment here. What I've seen of C# made my stomach turn, it's
even more crappy that Java, and I don't have any experience with GTK.


> o GUIs should be liberated from pixels

Why, and what do you want instead?

// Anders

Henrik Hallin skrev:
> I have been thinking a bit about this lately and I have grown to like 
> this idea very much. Using a type safe language to write all chaos 
> applications, there isn't even a need for separate address spaces 
> which simplifies things a lot and makes task switches very cheap. You 
> can still have all your applications and threads as before. There 
> might be some technical issues with stacks and things, but I'm sure 
> they can be worked around.
>
> Also, using a type safe language that compiles into a virtual machine 
> language has several advantages and could be made exceptionally fast 
> with dynamic translation techniques. It also makes all chaos apps 
> platform independant without a noticable speed loss.
>
> Imagine implementing all of chaos, including tornado, device drivers 
> and applications in Java. Just play with the thought.
This also means that the OS can have greater control of what is going 
on. It can map seamingly local objects to remote objects transparently 
for applications and do some serialization automatically. It's at least 
interesting to think about.

Henrik

Tommy Hallgren skrev:
> o Perhaps microkernels and message passing are outdated? A type-safe
> language/env could run things isolated with different costs that might make it
> a better option than everything coded in C and super-optimized message passing.
>   
I have been thinking a bit about this lately and I have grown to like 
this idea very much. Using a type safe language to write all chaos 
applications, there isn't even a need for separate address spaces which 
simplifies things a lot and makes task switches very cheap. You can 
still have all your applications and threads as before. There might be 
some technical issues with stacks and things, but I'm sure they can be 
worked around.

Also, using a type safe language that compiles into a virtual machine 
language has several advantages and could be made exceptionally fast 
with dynamic translation techniques. It also makes all chaos apps 
platform independant without a noticable speed loss.

Imagine implementing all of chaos, including tornado, device drivers and 
applications in Java. Just play with the thought.

Henrik

> > First I was about to say it could be configurable, but then I realized
> > things like VFS also would gain a lot by being configured to be running
> > "in-kernel".
> There will be an API difference, so the same module can't be
> configured to run either in-kernel or in user space without adding
> some API wrappers.

Why not? For performance reasons? There are certainly no technical reasons it
couldn't be a configurable option while having the same API for both
situations.

> > How about looking into the L4 kernels?
> For what? Design solutions, a temporary base while developing, or a
> permanent usage?

That depends on how good it is and what we feel it could contribute with. The
L4 team have worked for years trying to make things like context switches as
fast as possible. 

> > Or making a QNX-ish clone?
> 
> We don't want to make a clone of anything. Why could someone use chaos
> instead of the cloned original?

So what do you want to do then?

I feel that there are many good ideas and code out there, but there is always
some parts of the existing operating systems that suck.

For example:
o Linux is like 50+MB of source all compiled to run in the same address space,
how can they expect it to be stable?
o Package-systems in Linux dists suffer from the old file system layouts, and
too fine-grained division of libraries. Here on my Mac I can drag a game from
its CD to the desktop to install it. Even MS Office can be installed by
drag-and-drop.
o Desktop tech like gtk+ would probably be much more robust and faster if it
was written in C#.
o Squeak is slow, lack of native threading and generally lack polish. Even
then, it has so many advantages that it might be a good choice for many things.
o GUIs should be liberated from pixels
o Perhaps microkernels and message passing are outdated? A type-safe
language/env could run things isolated with different costs that might make it
a better option than everything coded in C and super-optimized message passing.

> > I think this is a pretty important point (much more important that the
> > other things discussed!). Either we do microkernel or we don't. If we
> > do, all services, with _very_ carefull exceptions, are put in user
> > space.
>
> First I was about to say it could be configurable, but then I realized th=
ings
> like VFS also would gain a lot by being configured to be running "in-kern=
el".

There will be an API difference, so the same module can't be
configured to run either in-kernel or in user space without adding
some API wrappers.


> How about looking into the L4 kernels?

For what? Design solutions, a temporary base while developing, or a
permanent usage?


> Or making a QNX-ish clone?

We don't want to make a clone of anything. Why could someone use chaos
instead of the cloned original?

// Anders

--- Anders Öhrt <and...@gm...> skrev:
> I think this is a pretty important point (much more important that the
> other things discussed!). Either we do microkernel or we don't. If we
> do, all services, with _very_ carefull exceptions, are put in user
> space.

First I was about to say it could be configurable, but then I realized things
like VFS also would gain a lot by being configured to be running "in-kernel".
How about looking into the L4 kernels? They have very fast process switching
and other nice features. Or making a QNX-ish clone?

> >I believe we will never get the high speed that other
> >
> ...? I think you missed the rest here. :)

Yes, my fingers went in one direction, and my mind in another. I was
going to say something about never reaching the throughput of
Linux/*BSD, and having that in mind as we design...


> >This means we will have "user space" services that are low
> >performance, and kernel modules that are high performance, right?
>
> We can do it like that, yes. With stormG3 we never did that distinction;
> all was put into kernel space. It makes things so much easier. But of
> course, being able to provide services from user space is a good thing.

I think this is a pretty important point (much more important that the
other things discussed!). Either we do microkernel or we don't. If we
do, all services, with _very_ carefull exceptions, are put in user
space.


> >The services will be self contained, and only communicate via
> >messages, keeping them isolated and secure, while the kernel modules
> >will use integrated and "dangerous"? Is the goal to keep a low count
> >of kernel modules (only for direct HW, or performance critical stuff)
> >and more the rest into services?
>
> You are too systematic my friend... :-) The goal was to get rid of the
> complexity/speed bottlenecks that the storm G1 implementation had.
> Easier to implement as well as faster.

Well, I think we need some firm decisions here. First we said we were
going to go the micro kernel way, then you totally broke that because
of speed issues...? I mean, sure, if it was undoable, then we should
change. Better to go with what works than continue down the wrong
path...

// Anders

> What makes you think SVN would be overkill? Do you think it would mean
> too much overhead? I think storing all of the /etc equivalent in a
> Subversion repository is an excellent idea. You can do it with Linux
> already but having it implemented in chaos by default would be a very
> good idea.

You hardly ever change "/etc", and when you do you don't need
atomicity. And once you've made a change, do you really want that
change to be saved and reversible for ever? I can see you want the
last step, in case you do something wrong, but having a full history
is just unnecessary.

// Anders

Anders =D6hrt wrote:

>(Linux has an EXPORT_SYMBOL macro)
> =20
>
Okay. I still think they export much more than is needed though, because=20
the system has never been built with modularity as one of the key design=20
goals.

>I believe we will never get the high speed that other
> =20
>
...? I think you missed the rest here. :)

>This means we will have "user space" services that are low
>performance, and kernel modules that are high performance, right?
> =20
>
We can do it like that, yes. With stormG3 we never did that distinction;=20
all was put into kernel space. It makes things so much easier. But of=20
course, being able to provide services from user space is a good thing.

>The services will be self contained, and only communicate via
>messages, keeping them isolated and secure, while the kernel modules
>will use integrated and "dangerous"? Is the goal to keep a low count
>of kernel modules (only for direct HW, or performance critical stuff)
>and more the rest into services?
> =20
>
You are too systematic my friend... :-) The goal was to get rid of the=20
complexity/speed bottlenecks that the storm G1 implementation had.=20
Easier to implement as well as faster.

This is how the boot program looked btw. I don't know if this code ever=20
worked because the code was commented out. It might have been how I=20
thought it would work rather than that it ever came to this point:

    size_t services =3D 1;
    service_t service_list;
    service_lookup_t service_lookup =3D
        {
            "kernel", NULL, NULL, NULL, NULL,
            KERNEL_PROTOCOL_MAJOR_VERSION,
            KERNEL_PROTOCOL_MINOR_VERSION
        };
    service_connection_id_t connection_id;

    if (system_call_service_lookup (&service_lookup, &services,=20
&service_list) !=3D STORM_RETURN_SUCCESS)
    {
        return -1;
    }

    if (system_call_service_connect (service_list.id, &connection_id) !=3D=
=20
STORM_RETURN_SUCCESS)
    {
        return -1;
    }

    if (system_call_service_invoke (connection_id, KERNEL_FUNCTION_LOG,
                                    test) !=3D STORM_RETURN_SUCCESS)
    {
        return -1;
    }

    if (system_call_service_close (connection_id))
    {
        return -1;
    }

All of these system calls seemed to exist though so it might have=20
actually worked. :) This is actually a bit similar to XPCOM and perhaps=20
CORBA etc, except that they don't have to do a=20
system_call_service_invoke but rather just call the method on the=20
object. If it involves shifting to/from kernel space, that approach is a=20
bit harder to implement of course.

--=20
Best regards,
Per Lundberg

Just to clarify, I haven't thought of all the details of how to
implement an "objectified VFS",  it's not even a finished concept. But
seeing such a thing in reality is really tempting. :-)


> > As for what sets of primitives to have, I like how Lua is doing things
>
> Which is? :) More specifically in this context.
> One can probably go a long way with just strings, numbers and lists. Keep=
ing
> the core set of datatypes simple simplifies the userland implementation.

I'll try explain how it works in Lua (see www.lua.org for more information)=
.

First we have the containter type, which is an associative array with
key=3D>value that's called table in Lua. Each value can be either
another table, or any of the follow primitives:

string, 8-bit clean of any size and which can contain embedded zeros.
Thus, this could be used for all regular files in the objectified VFS.
number, Equals a double in the standard Lua distribution.
function, Functions (and methods) are first class values in Lua.
bool, true or false
nil, Like null. Trying to access an unexisting key in a table also
evalutes to nil.

What makes it beautiful is that tables are used for everything, both
objects and data structures. Specifically, table["key"] and table.key
is the same.

On 4/25/06, Peter Schuller <pet...@in...> wrote:
> > Lets try to be forward-compatible, so to speak. :)
>
> On the other hand it is difficult to ensure the rules of ABI compatibilit=
y are
> truly 100% followed; not to mention newly introduced bugs.
>
> It would almost certainly be best to allow multiple versions of the same
> library to be installed, and be able to choose which version to use on a
> per-application basis (without having to link to a specific version).

Yes, this is probably a good idea.

> This kinda overlaps a bit with what's being talked about for pkgsrc and
> DragonFly BSD. One envisions a system where a given application sees
> *exactly* that which it should see; its own files and those of its
> dependencies. (For *nix this would mean that /lib, /usr/lib and /usr/pkg/=
lib
> are populated differently depending on which binary is run. Whether this =
be
> done through special operating system support ("views" at the VFS layer o=
r
> similar) or though some more conventional mechanism (symlink farms etc) i=
s
> not that important.)

I like this idea very much, and I don't think it should be limited to
just hiding/showing libraries for an application. :) Maybe the same
could be done to hide inaccessible files for users too? One nice thing
with having a single namespace for both VFS/services and message
passing / function calls is that you could have the same ACL system
for all of it. You could probably add network connections and process
management into this single namespace too.


Regards,
  Johannes Lundberg

Anders =D6hrt wrote:

>>Shouldn't be too hard either? Store all the database in a Subversion
>>back-end and commit everytime the changes are put in place.
>>   =20
>>
>Guys, really. You should not kill flies with cannons, that's what fly
>swatters are for. Using a SVN for this is just way to much... That
>would be the overkill of overkills.
> =20
>
What makes you think SVN would be overkill? Do you think it would mean=20
too much overhead? I think storing all of the /etc equivalent in a=20
Subversion repository is an excellent idea. You can do it with Linux=20
already but having it implemented in chaos by default would be a very=20
good idea.

--=20
Best regards,
Per Lundberg

Nice to know. I've thought more about this and maybe using something like
COM/XPCOM is a better idea if the toolset around it are there.

Some features in COM are very nice. It's network transparent from the
start(HRESULT, anything can fail), it has possibilities for both high
performance(making something in-process) or distributed/safer mode (transparent
proxying of a component).

Downsides with COM are the associated feeling of Microsoft bloatedness and the
lack of a nice language to develop components in. C++ with ATL may work, but
it's hardly nice and clean. Oh, event handling and all those threading modes in
COM is also something one should rethink.

Something that I feel is very interesting is this project:
http://research.microsoft.com/os/singularity/

.Net with things like virtual machines, application domains and sandboxes could
make a fast and safe microkernel-ish system without all the overhead of a
traditional processes/mailboxes system.

Regards, Tommy
 
--- Per Lundberg <pe...@gm...> skrev:

> Anders Öhrt wrote:
> 
> Tommy, I just want to tell you that I have seen your email but I will 
> read it more carefully and see what I think about it at anoher time. 
> Just so you know. :) Great to see that you're taking time to help out 
> the project!
> 
> >I seem to remember Per talking about kernel modules due to performance
> >issues...? Are we still determined to have a micro kernel?
> >  
> >
> Well... What I started doing with stormG3 was to change from a 
> service-based system to a more traditionalistic dynamic linking 
> approach. However, it was more smart than Linux in that it did not 
> export *all* symbols (I don't know if Linux does that either but let's 
> pretend it does) but only a set of well-defined API functions defined in 
> storm/interface.h.
> 
> See http://chaos.lundcom.se/stormG3.pdf for more info. I quote myself 
> (as usual :-) from this file. The latest paragraph is the most 
> interesting one:
> 
> Design Goals
> 
> Each and every successful project needs a vision, a clear understanding 
> of what is to be achieved. Here
> are some of the goals that we aim for in storm G3.
> 
> * Not Yet Another Kernel. There are very many kernels available on the 
> "market". Now, being different
> is not a goal in itself, even though many people live as if it were so. 
> However, if storm G3 will be
> exactly like an already existing kernel, writing it would not fit a 
> purpose (other than the general
> education of its developers). Therefore, storm G3 ought to be different 
> than i.e. Linux or the FreeBSD
> kernel.
> 
> * Modules Are Everything. We have come to realize that the success of a 
> kernel will be a result of how
> well-modularized it is. How easy will it be to modify one part of the 
> system, without having to restart
> the computer every time? With the old chaos design, we had all hardware 
> device drivers running as
> separate processes. This forced us to be very modular in our thinking, 
> and this thinking should persist
> in storm G3.
> 
> * Clean And Fast. In previous generations of the kernel, we made it 
> clean. However, a system that is
> only clean will not succeed. It needs to have the performance neccessary 
> to run high-capacity
> applications, like modern games, 3D rendering programs, space ship 
> simulations, et cetera. With our
> old design, sending a packet to the network, for example, was a very 
> heavy operation that included at
> least two task switches: one to the IPv4 server and one to the ethernet 
> card server. Task switches,
> especially between tasks with different address spaces, will always be a 
> fairly time-consuming
> operation and will therefore be avoided for system tasks. This is one of 
> the reasons we have decided to
> move away from the server-based design to a modular macro-kernel design 
> instead. But the biggest
> reason is that it makes everything so easy - the kernel design will be 
> much cleaner. We believe this is a
> Good Thing.
> 
> -- 
> Best regards,
> Per Lundberg
> 
> 

Anders =D6hrt wrote:

Tommy, I just want to tell you that I have seen your email but I will=20
read it more carefully and see what I think about it at anoher time.=20
Just so you know. :) Great to see that you're taking time to help out=20
the project!

>I seem to remember Per talking about kernel modules due to performance
>issues...? Are we still determined to have a micro kernel?
> =20
>
Well... What I started doing with stormG3 was to change from a=20
service-based system to a more traditionalistic dynamic linking=20
approach. However, it was more smart than Linux in that it did not=20
export *all* symbols (I don't know if Linux does that either but let's=20
pretend it does) but only a set of well-defined API functions defined in=20
storm/interface.h.

See http://chaos.lundcom.se/stormG3.pdf for more info. I quote myself=20
(as usual :-) from this file. The latest paragraph is the most=20
interesting one:

Design Goals

Each and every successful project needs a vision, a clear understanding=20
of what is to be achieved. Here
are some of the goals that we aim for in storm G3.

* Not Yet Another Kernel. There are very many kernels available on the=20
"market". Now, being different
is not a goal in itself, even though many people live as if it were so.=20
However, if storm G3 will be
exactly like an already existing kernel, writing it would not fit a=20
purpose (other than the general
education of its developers). Therefore, storm G3 ought to be different=20
than i.e. Linux or the FreeBSD
kernel.

* Modules Are Everything. We have come to realize that the success of a=20
kernel will be a result of how
well-modularized it is. How easy will it be to modify one part of the=20
system, without having to restart
the computer every time? With the old chaos design, we had all hardware=20
device drivers running as
separate processes. This forced us to be very modular in our thinking,=20
and this thinking should persist
in storm G3.

* Clean And Fast. In previous generations of the kernel, we made it=20
clean. However, a system that is
only clean will not succeed. It needs to have the performance neccessary=20
to run high-capacity
applications, like modern games, 3D rendering programs, space ship=20
simulations, et cetera. With our
old design, sending a packet to the network, for example, was a very=20
heavy operation that included at
least two task switches: one to the IPv4 server and one to the ethernet=20
card server. Task switches,
especially between tasks with different address spaces, will always be a=20
fairly time-consuming
operation and will therefore be avoided for system tasks. This is one of=20
the reasons we have decided to
move away from the server-based design to a modular macro-kernel design=20
instead. But the biggest
reason is that it makes everything so easy - the kernel design will be=20
much cleaner. We believe this is a
Good Thing.

--=20
Best regards,
Per Lundberg

Henrik Hallin wrote:

> Anyone has any experience with CORBA, IDL or related topics?

I have, as I mentioned. I worked with creating a custom Mozilla 
component (for handling a user-defined URI protocol). It worked pretty 
well actually. The idea was to be able to run PHP in the client (that 
is, have PHP pages on the hardrive and being able to see the output in 
the Mozilla).

Mozilla uses IDL for the interface descriptions and a custom component 
framework called XPCOM for the component architecture. This is of course 
operating system generic. All of it is written in C++. Is uses reference 
counting for the object references, which are simple double pointers to 
an XPCOM base class which is then being (don't scream now :-) typecasted 
to the expected type. Of course, there is a way to check whether the 
reference is of the proper type before this is being done.

I think definitely going with IDL as our service definitions is a good 
way to go. It doesn't limit us to a specific implementation language 
either but it will probably be hard if not impossible to do a C 
implementation; it is much more C++:ish. But that doesn't have to be a 
problem. XPCOM components can be accessed from Javascript for example. 
:) (just showing the point that even though the components themselves 
are C++, they can be accessed from other languages)

-- 
Best regards,
Per Lundberg

Henrik Hallin wrote:

> For a block device, we mostly want to read and write data randomly. 
> For this to work, the block device interface needs to at least 
> implement open/read/write/seek/close (perhaps all service interfaces 
> should implement these).

I think a distinction between the BlockInterface and the 
CharacterInterface is reasonable. They should both derive from 
IOInterface though (which has the basic read/write function defined). 
And perhaps this interface should derive from ServiceInterface which has 
the getParam and setParam methods?

I have worked with Mozilla a bit during the years away from chaos and it 
uses IDL interfaces heavily. We should probably be smart enough to do 
the same. :)

> Specific servers can then implement additional functionality and have 
> their own IDL description.

Perhaps they are better off by subclassing the BlockInterface etc. In 
other words, a busmastering BlockInterface implementor will likely need 
more methods (for setting up the busmaster IO buffer and so forth) and 
it should create a BusmasterBlockInterface or something like that. But 
that's a bit too generic example but you can probably see through it 
anyway. :-)

> This doesn't at all address the datatype question though. What data 
> types are needed? Do they need to be complex if read/write handles 
> reading and writing raw data? Aren't fundamental data types enough 
> (like boolean, int, float and string).

They will probably need to be as complex as the functions are. But as 
you're saying, if the functions are basic, trivial such as read/write, 
not very complex things will be needed. Can we avoid structures 
altogether and only use fundamental types?

>> Even cooler would be if there was a transactionally safe method of 
>> modifying multiple settings at once, where all changes are either 
>> committed or rolled back [...]
>> Think of the ease with which one could perform an atomic, safe and 
>> remote reconfiguration of a firewall! Combined with an automatic 
>> time-delayed rollback it would even allow for an easy method of 
>> recovering from misconfigurations.
>>   
>
> Yes, I believe we talked about this earlier today, but it might just 
> have been jibberish. You are right, it would be cool.

Shouldn't be too hard either? Store all the database in a Subversion 
back-end and commit everytime the changes are put in place. It gives all 
of this with not much work at all as far as I can see it. The only 
problme would be if the Subversion backend would crash or have database 
inconsistencies, it could cause horrendeous problems but they can 
probably be worked around with weekly backups or something and 
Subversion should be quite mature anyway. If we want to be really safe, 
we can use CVS but it is much more limited in many ways (cannot handle 
renames and similar things).

-- 
Best regards,
Per Lundberg