chaos-devel Mailing List for The chaos Operating System

Per Lundberg wrote:

> You can find it here:
> 
> http://chaos.lundcom.se/download/chaos-20001010.tar.bz2
> 
> Of course, ideally we would put up a floppy image/ISO image of this 
> stuff in precompiled form... Someday, sometime. :)

Today, now: http://chaos.lundcom.se/download/chaos-20001010-floppy.img

These things are missing/broken:
* tetris. It is on the floppy, but I failed to get it running.
* Network support. ipv4 is included as well as bunch of drivers, but 
they are not in the grub menu file. Can easily be loaded if you like, 
order is important: if I remember correctly, ipv4 first and then the 
network driver of choice. (you can easily find it by tab completing in 
GRUB) At least 3C905 and Realtek 8139 should work.

I will see if I can get the 2002 code running now. :) I think this is 
interesting, if not for anything else but for archival purposes... I 
mean, it can be nice to see what the system could do almost 6 years (!) 
ago. Time flies, really.
-- 
Best regards,
Per Lundberg

>>  > [... Good example btw. ...]
> Thanks! I was sure I would be flamed for it. =)

Not this time. :-)

>> I agree... Configuration changes are a bit complicated to apply
>> atomically. I think that's a bit unavoidable. But storing the changes as
>> a change-set (that is, a revision with a set of changes) in a CVS/SVN
>> repository should be a piece of cake. This does not give atomicity
>> though, only from the versioning system's point of view. :)
> Sort of like System Restore Points in Windows...? =)

Yes, exactly the same thing. I mean, all operating systems (I wrote first
"modern operating systems" but they are not modern :) in use today,
including Windows, Linux etc has some good and bad points. The key is just
to eliminate the biggest downsides in the existing systems and design new
killer ideas that will make the system attractive for perhaps 90% of the
computer users. The lowest 5% wouldn't know how to switch operating
systems anyway (nor how to burn an ISO with a demo CD or similar). Perhaps
this group is much bigger than 5%? :-) As well, the top 5% wouldn't care,
they have already handicrafted their init procedure in an old version of
OpenBSD which they upgrade by hand with CVS diffs of security updates...
because everything is so extremely customized to fit their specific ideas
of how they want the computer environment to work. They haven't written
their own operating system but they have surely patched the kernel to give
it a more personal bent. Those people would be hard to win over to any new
operating system, regardless of the pros/cons/new features.

But, the "middle ground" is what is most interesting to us I would say,
just like Gustav Sinder was implying in another email.
-- 
Best regards,
Per Lundberg

>  > [... Good example btw. ...]

Thanks! I was sure I would be flamed for it. =3D)


> I agree... Configuration changes are a bit complicated to apply
> atomically. I think that's a bit unavoidable. But storing the changes as
> a change-set (that is, a revision with a set of changes) in a CVS/SVN
> repository should be a piece of cake. This does not give atomicity
> though, only from the versioning system's point of view. :)

Sort of like System Restore Points in Windows...? =3D)

// Anders

Tommy Hallgren wrote:
> Thanks. I'm aware that it's a bit confusing and misses things like events and
> object references. I think this subject will be a hot subject one way or
> another. Kernel-neutral or not, services must be able to communicate. Also,
> Anders and Per may not agree with the concepts I tried to express. You had a
> very similar system in the old Chaos?

I think (I have still not looked more into detail into what you write, 
anyway...) that the big question is whether we go the A, B or C route 
that I wrote to Anders Öhrt about. That is, whether we choose a 
micro-kernel (message-based) approach a la what you like or how chaos G1 
worked or a more monolithic approach like storm G3 (even though it uses 
shared libraries so it is not just one big bloated binary). Perhaps this 
gets irrelevant if we decide to go with CLI/.Net though. Henrik, what do 
you say?

Everybody: see this URL for some nostalgic feelings. :) 
http://chaos.lundcom.se/download/chaos-20001010-screenshot.jpg

It is the chaos 20001010 snapshot that I posted recently running in 
bochs right now. cluido + console switching seems to work (had to hack 
it to use use control-tab instead of alt-tab though because alt-tab is 
reserved by Windows).

I will put up a hdimage that you can use with bochs sometime when I have 
the time... I booted this one over tftp to my Linux server. :) with this 
menu.lst:

title = chaos
bootp
root (nd)
kernel /chaos/system/kernel/storm.gz
module /chaos/system/servers/console.gz
module /chaos/system/servers/vga.gz
module /chaos/system/servers/log.gz
module /chaos/system/servers/keyboard.gz
module /chaos/system/servers/pci.gz
module /chaos/system/servers/boot.gz
module /chaos/programs/cluido/cluido.gz
-- 
Best regards,
Per Lundberg

Anders Öhrt wrote:
>> Could you elaborate on this a bit? It shouldn't be too hard. I mean, as
>> long as we keep the configuration text file-based or similar, it should
>> be doable (if we think in terms of a subversion repository). What
>> problems do you see, can you explain a scenario where this would be hard?
> The problem is not with the text files, they can be changed atomically
> without that much problem. But with a configuration utility, you might
> have hundreds of options and if any one failes you should roll back
> the changes.
 >
 > [... Good example btw. ...]

I agree... Configuration changes are a bit complicated to apply 
atomically. I think that's a bit unavoidable. But storing the changes as 
a change-set (that is, a revision with a set of changes) in a CVS/SVN 
repository should be a piece of cake. This does not give atomicity 
though, only from the versioning system's point of view. :)

So, point taken in other words. :)
-- 
Best regards,
Per Lundberg

--- Per Lundberg <per...@ha...> skrev:
> This seems really, really interesting. If we do this right, we should be 
> able to run Windows (.NET) applications with no modifications 
> whatsoever. Of course, this requires an implementation of Windows.Forms 
> but still... I think this would be great! As long as we can implement 
> the chaos goals (which are yet to be formulated, I have some thoughts 
> here which I will share ASAP, in a couple of weeks or so in other words 
> :-) within this platform, I think it might be the wisest choice to make.

There's quite a lot of Win32 things left in .Net. You can find some pieces here
and there. I thought the interesting thing was the VM, IL and some part of CLR
like application domain, attributes on classes/methods/etc and reflection. I've
worked quite a bit with .Net and I try to avoid everything "non-pure" so i
can't show you examples. But the overall impression is that much of the basic
framework does the job pretty well. But it also has that "version 0.9" feeling,
I can't figure out what I don't like.

Gustav Sinder wrote:
> I'm glad to see that these, most important aspects are being discussed 
> again.
> I started editing a wiki page some days ago. It's just quick thoughts; 
> feel free to rewrite, update and extend the current text available at
> http://chaos.lundcom.se/index.pl?Vision

Great initiative! I think we should use this as a place where we can 
list different things that we think chaos should be or not be, much like 
what you've done but more specific. I suggest that changes to the page 
also be e-mailed to this list so we can all see what is being done. 
Please keep the subject "The vision" or something like that so it is 
easily filtered out.
-- 
Best regards,
Per Lundberg

Tommy Hallgren wrote:
>> Ok. So I could download a technical description from somewhere and write 
>> my own interpreter? If so, this would be REALLY interesting, since there 
>> are already lots of compilers available.
> http://www.ecma-international.org/publications/standards/Ecma-335.htm

This seems really, really interesting. If we do this right, we should be 
able to run Windows (.NET) applications with no modifications 
whatsoever. Of course, this requires an implementation of Windows.Forms 
but still... I think this would be great! As long as we can implement 
the chaos goals (which are yet to be formulated, I have some thoughts 
here which I will share ASAP, in a couple of weeks or so in other words 
:-) within this platform, I think it might be the wisest choice to make.

I'm glad to see that these, most important aspects are being discussed 
again.
I started editing a wiki page some days ago. It's just quick thoughts; 
feel free to rewrite, update and extend the current text available at
http://chaos.lundcom.se/index.pl?Vision

/Gustav

Henrik Hallin wrote:
> Per Lundberg skrev:
>>> We need a vision, and preferably written down and easy to understand.
>>> * Which fundamental flaws do we see across most or all of the modern
>>> operating systems?
>>> * Are these flaw enough to justify building a completely new system,
>>> instead of patching one of those?
>>> * What will the major benefits of chaos be to the end users, 
>>> compared to these?
> We need a system that is easier to manage. This systems needs a 
> unified configuration framework, a drag'n'drop installer/uninstaller 
> and an intuitively structured filesystem.
>
> Also, we need a GUI which is really easy to use for computer newbies 
> (we should ask our parents and people like that how they would want a 
> computer to work to get some ideas here). This GUI still needs to 
> satisfy the needs of an experienced user.
>
> We need a system that is well integrated from the start. If you know 
> one application and you know how to configure it, you should feel at 
> home using any other chaos application. Systems like Linux and Windows 
> have lots of applications but they sometimes look and behave 
> completely different from each other.
>
>>> * Who's our end users? Geeks? Computer novices? Everyone? If so, on
>>> what will our initial focus be on?
> Everyone. Easy to use but still powerful. Initial focus on getting a 
> basic framework with some working chaos libraries so we can start 
> experimenting with things. After that, probably filesystem layout, 
> application packaging and configuration framework. Alongside of that, 
> tornade can be discussed.
>
>>> * Will chaos be more than just an operating system, will it be more
>>> like a application environment? If so, how?
> See above.
>
>>> * How will chaos change the way we use computers?
> See above.
>
> We could put these items on the wiki and start jotting down the 
> details as we go along. I'm tired now but might do it during the week.
>
> Good night, y'all
>
> Henrik
>
>
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache 
> Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> chaos-devel mailing list
> cha...@li...
> https://lists.sourceforge.net/lists/listinfo/chaos-devel

> Ok. So I could download a technical description from somewhere and write 
> my own interpreter? If so, this would be REALLY interesting, since there 
> are already lots of compilers available.

http://www.ecma-international.org/publications/standards/Ecma-335.htm

Tommy Hallgren skrev:
> --- Henrik Hallin <hen...@ma...> skrev:
>   
>> Isn't MSIL completely closed and impossible to use license-wise or 
>> something?
>>     
>
> Nope. That's part of the CLR and it's an ECMA standard. However, there might be
> patents covering the CRL, both MS patents and patents by others. Google for ECM CLR
>   
Ok. So I could download a technical description from somewhere and write 
my own interpreter? If so, this would be REALLY interesting, since there 
are already lots of compilers available.

Henrik

--- Henrik Hallin <hen...@ma...> skrev:
> Isn't MSIL completely closed and impossible to use license-wise or 
> something?

Nope. That's part of the CLR and it's an ECMA standard. However, there might be
patents covering the CRL, both MS patents and patents by others. Google for ECM CLR

Per Lundberg skrev:
>> We need a vision, and preferably written down and easy to understand.
>> * Which fundamental flaws do we see across most or all of the modern
>> operating systems?
>> * Are these flaw enough to justify building a completely new system,
>> instead of patching one of those?
>> * What will the major benefits of chaos be to the end users, compared 
>> to these?
We need a system that is easier to manage. This systems needs a unified 
configuration framework, a drag'n'drop installer/uninstaller and an 
intuitively structured filesystem.

Also, we need a GUI which is really easy to use for computer newbies (we 
should ask our parents and people like that how they would want a 
computer to work to get some ideas here). This GUI still needs to 
satisfy the needs of an experienced user.

We need a system that is well integrated from the start. If you know one 
application and you know how to configure it, you should feel at home 
using any other chaos application. Systems like Linux and Windows have 
lots of applications but they sometimes look and behave completely 
different from each other.

>> * Who's our end users? Geeks? Computer novices? Everyone? If so, on
>> what will our initial focus be on?
Everyone. Easy to use but still powerful. Initial focus on getting a 
basic framework with some working chaos libraries so we can start 
experimenting with things. After that, probably filesystem layout, 
application packaging and configuration framework. Alongside of that, 
tornade can be discussed.

>> * Will chaos be more than just an operating system, will it be more
>> like a application environment? If so, how?
See above.

>> * How will chaos change the way we use computers?
See above.

We could put these items on the wiki and start jotting down the details 
as we go along. I'm tired now but might do it during the week.

Good night, y'all

Henrik

Per Lundberg skrev:
> Henrik Hallin wrote:
>> I wouldn't recommend designing a new programming language. There are 
>> already enough of them, some better than others for an OS of 
>> course... I don't know much about Java bytecode, but I don't see why 
>> other languages couldn't be compiled into Java bytecode instead of 
>> x86 code.
>
> Yes, that can surely be done. But if we are to go that route, I 
> suggest we look at the .NET architecture instead because it already 
> has exactly that -- compilation to bytecode from different programming 
> languages.
>
Isn't MSIL completely closed and impossible to use license-wise or 
something?

Henrik

Johannes Lundberg wrote:
>> I still feel we're starting at the wrong end here. Most of the things talked
>> about here are interesting ideas, but we still need to start with the
>> basics.
> I guess it's because we all enjoy thinking and playing with the
> technical details.

Yep... This is a pretty old mail but I think this is very right on the 
spot. If anyone has any good ideas about answers to these questions, 
please step up and give us your thoughts! They are more important than 
the technical details actually:

> We need a vision, and preferably written down and easy to understand.
> * Which fundamental flaws do we see across most or all of the modern
> operating systems?
> * Are these flaw enough to justify building a completely new system,
> instead of patching one of those?
> * What will the major benefits of chaos be to the end users, compared to these?
> * Who's our end users? Geeks? Computer novices? Everyone? If so, on
> what will our initial focus be on?
> * Will chaos be more than just an operating system, will it be more
> like a application environment? If so, how?
> * How will chaos change the way we use computers?
-- 
Best regards,
Per Lundberg

Henrik Hallin wrote:
> I wouldn't recommend designing a new programming language. There are 
> already enough of them, some better than others for an OS of course... I 
> don't know much about Java bytecode, but I don't see why other languages 
> couldn't be compiled into Java bytecode instead of x86 code.

Yes, that can surely be done. But if we are to go that route, I suggest 
we look at the .NET architecture instead because it already has exactly 
that -- compilation to bytecode from different programming languages.

Of course, writing a kernel and hardware drivers in a purely managed 
language such as Java or C# is not entirely possible. You need to have 
at least tiny bits being written as unmanaged code (that is, code with 
pointers to direct memory access). But these bits can be fairly small.
-- 
Best regards,
Per Lundberg

> > But the proof need to be constructed somehow and this is highly
> > non-trivial. I'm not saying the compiler is buggy, I'm saying you
> > can't construct the proof because it's too hard.
>
> It's clearly not too hard, because it's already been done:

Okay, my bad.

// Anders

G'day all.

Quoting Anders =D6hrt <and...@gm...>:

> But the proof need to be constructed somehow and this is highly
> non-trivial. I'm not saying the compiler is buggy, I'm saying you
> can't construct the proof because it's too hard.

It's clearly not too hard, because it's already been done:

    http://www.cs.berkeley.edu/~necula/Papers/touchstone_pldi00.ps

This paper is especially interesting, because it describes a system for
calling provably safe native code from a type-safe language:

    http://www.cs.berkeley.edu/~necula/Papers/tr96-165.ps.gz

Cheers,
Andrew Bromage

> Sorry to jump in late in the discussion guys, but I hope this be OK
> anyway...

That's OK, I'l jump out to make room... =3D)

// Anders

> I for one would *love* atomicity. One of my wet dreams is a fully ACID
> compliant filesystem for *nix.

The main problem with ACID is it's slow. You could not feasibly have
an ACID FS, it would kill performance. You could construct an ACID FS
on top of some SQL engine, which runs on top of a regular FS but most
applications/users don't want the performance sacrifice.


> I realize most of the key stuff like /etc/passwd are in pratice
> already 'solved'; but in general I think it is pretty clear that collback
> capability and atomicity could go a long way to robustify things at the
> general level.

Actually, I think it would go in the opposite direction. It adds
complexity and complexity is always error prone. The only application
I can think of at the top of my head where atomicity is a benifit is
firewall configuration, most other configurations doesn't benefit at
all.

// Anders

> > Depends on what we are talking about. Atomicity in a package manager
> > should be easy, in a configuration utility might be harder...
>
> Could you elaborate on this a bit? It shouldn't be too hard. I mean, as
> long as we keep the configuration text file-based or similar, it should
> be doable (if we think in terms of a subversion repository). What
> problems do you see, can you explain a scenario where this would be hard?

The problem is not with the text files, they can be changed atomically
without that much problem. But with a configuration utility, you might
have hundreds of options and if any one failes you should roll back
the changes.

Assume it's a firewall configuration utility, containing hundreds of
rules. In order to change one rule you need to stop networking,
"check-in" the new rule, try to apply all rules, roll back if it
didn't work, and then start networking again. You never want the apply
part to fail, so the configuration utility should know if something
was wrong and tell you while you're making the changes. This means
having the same error checking semantics in the utility as in the
networking module. This means either have an API in the module for
checking, which seems unnecessary, or sharing code, which makes them
inter-dependend.

If the utility just took each rule you made and tried to apply it when
you made it (no atomicity) the networking module would tell you when
something was wrong without the utility having to know why it was
wrong. It would just report that the rule was wrong and tell you to
change it.

// Anders

Per Lundberg skrev:
>>> Of course the VM would be part of the OS, or BE the OS in fact.
>> That would mean a huge monolithic kernel, right? If even the VM is
>> part of it. Sounds strange, I don't get it.
>
> Well, I think his idea (if I understand it correctly) is that the 
> standard kernel layer with system calls et al be replaced with a 
> Virtual Machine instead. That is, the kernel will in effect be a 
> bytecode interpreter/JIT compiler, that compiles the bytecode out to 
> machine code on demand. How the machine code then interfaces with the 
> kernel (system calls) is then being kept kernel-internal because the 
> conversion is being done within the kernel.
>
Exactly. The kernel would export an interface that device drivers and 
applications could use which would call into the kernel code. See my 
other mail for more details.

Henrik

Per Lundberg skrev:
> Henrik Hallin wrote:
>> Imagine implementing all of chaos, including tornado, device drivers 
>> and applications in Java. Just play with the thought.
>
> We actually discussed this (me, Peter Schuller, Magnus, Johannes etc) 
> a couple of years ago in Uppsala. :-) The idea then was kind of to run 
> it with gcj, which compiles it out to native binary code which is then 
> being run. But of course, running everything within the chaos Virtual 
> Machine might be an even better approach... But if we end up writing 
> our programming language, we are getting dangerously close to the 
> Tunes project... (if you remember?)

I wouldn't recommend designing a new programming language. There are 
already enough of them, some better than others for an OS of course... I 
don't know much about Java bytecode, but I don't see why other languages 
couldn't be compiled into Java bytecode instead of x86 code.

Henrik

Anders =D6hrt skrev:
> No, that reasoning gives that nothing totally safe is ever going to be
> written because you can't prove soundness except for trivial parts of
> code.
>
Which means that to be "the most safe" so to speak, one needs to keep=20
the trusted unsafe parts of a system as small as possible. If they can=20
get small and trivial one can be pretty sure that they are safe and will=20
behave well. The rest can then be written in a language that produces=20
safe bytecode. That bytecode language is then proven to be safe and=20
everyone will be happy and live a long prosperous life!


>> If you
>> can prove that and base your OS on that language, it still seems more
>> safe to me than running native code.
>
> But you do lost _a_lot_ by this, so it needs to be weighed against the=20
> gains.
>
> Lets take Java as an example. If you rip out JNI and assume the rest
> is provably sound, you can start writing an OS in it and it will be
> good. But, without JNI you can't bootstrap. You can't write low-level
> (like PCI) is it, you can't write a VM in it, etc.
>
Right. Some parts will obviously have to be written in C/asm. But these=20
parts can be wrapped in interfaces to the safe language. These=20
interfaces could do access and bounds checking to ensure that a device=20
driver written in a safe language only accesses the I/O ports or the=20
memory it is allowed to access. Who is to decide this I leave as an=20
exercise to the reader... ;)


>> If the VM/translator/whatever part
>> of the core OS is buggy you fix that and it makes all applications
>> behave better. That seems better to me than fixing tons of buggy
>> exploitable unix daemons.
>
> You're really comparing apples and oranges here... Chaos doesn't have
> anything buggy to begin with, so if we start writing everything in
> Java, C, perl, or erlang doesn't really matter. And language will not
> save us from trouble anyway.
>
I'm not sure I'm getting your point here. Anyway, this whole way of=20
thinking is the result of reading about MS's Singularity which got me=20
inspired. It just seems to me that it simplifies the OS design, makes it=20
safer, opens up lots of possibilities since the OS can have more control=20
over running processes than with native executables and also reduces the=20
need for complex hardware mechanisms in the CPU which makes the OS more=20
portable. Also, applications and most of the OS doesn't even need to be=20
ported but will be written in a generic processor bytecode. A=20
translator/bytecode to native compiler plus a layer of drivers need to=20
be implemented for a new platform.

I'm not saying this is for chaos, I just think it's neat!

Henrik

Anders Öhrt wrote:

Sorry to jump in late in the discussion guys, but I hope this be OK 
anyway...

>> Of course the VM would be part of the OS, or BE the OS in fact.
> That would mean a huge monolithic kernel, right? If even the VM is
> part of it. Sounds strange, I don't get it.

Well, I think his idea (if I understand it correctly) is that the 
standard kernel layer with system calls et al be replaced with a Virtual 
Machine instead. That is, the kernel will in effect be a bytecode 
interpreter/JIT compiler, that compiles the bytecode out to machine code 
on demand. How the machine code then interfaces with the kernel (system 
calls) is then being kept kernel-internal because the conversion is 
being done within the kernel.

> But, what protects this module, is must be a (compiled in) part of the
> kernel...?

Yes, it would in effect be the kernel, or at least the kernel from the 
application POV.

>> But this means writing a kernel, which wasn't what we had planned 
>> right now?
> I didn't think we had any plans yet, and were keeping the discusions
> in the meta physical plane... *sigh*

There was kind of a consensus that writing a kernel was too much work 
right now and that we would be better off starting in userspace. (In 
userspace, no-one hears you scream?)
-- 
Best regards,
Per Lundberg