OLS: A proposal for a new networking API

Posted Jul 23, 2006 11:09 UTC (Sun) by nix (subscriber, #2304)
In reply to: OLS: A proposal for a new networking API by flewellyn
Parent article: OLS: A proposal for a new networking API

One of the possible underlying media over which these events may flow *is* a netlink socket.

But fundamentally the BSD sockets API is ugly, quite non-Unixlike, and hard to use and even harder to use correctly, and the thought of using it for *everything* including files is horrible. So no matter what the underlying layer is composed of, we need something better for people to actually *use*. (Ulrich is focused on that, not the implementation details alone: he *is* the libc maintainer.)

The replacement for select() had me cheering. At last, One Inner Loop! :)

to post comments

OLS: A proposal for a new networking API

Posted Jul 23, 2006 23:58 UTC (Sun) by flewellyn (subscriber, #5047) [Link] (7 responses)

I'm curious what a new, socket-replacing API would look like. Keeping in mind that sockets are more than just "networking", they're "generic IPC", I imagine one improvement would be to eliminate the distinction between "Unix domain" and "network" sockets. Since currently you have to specify one or the other, either limiting your application to local connections only, or using a relatively heavyweight IP or other networking protocol when its overhead wasn't necessary, OR coding in unpleasant and inelegant conditionals to detect which situation you're in, for each connection or message.

Yick. It would be really really nice if, instead of having to say "I want local namespace" or "I want internet namespace", you could just tell the kernel "I want a socket", and the kernel would determine based on the destination of your message what type of internal semantics was needed. That way, you could code for both local performance and network transparency.

Along with that, I could see real value in adding permissions to sockets themselves. Right now, you can get permissions on Unix domain sockets through the filesystem, but internet sockets don't have that protection. If you could, instead, assign permissions to a socket itself, you could have a daemon that would only accept connections from certain users, or with certain credentials such as RSA keys or Kerberos tickets.

Alternatively, imagine making ALL sockets files, including internet domain sockets. Now, a socket would have a file name, no matter what namespace or protocol you use, and you would still get the benefit of permissions on the sockets. Plus, instead of using special system calls to connect with them, all you'd need is calls to open(), read(), write(), and close(). A server program would just call "makesocket()", which would take parameters for address, protocol, port, and so on, to create a socket, and then start reading on it (assuming blocking I/O); a client would call "makesocket()" and then open it to start writing data. The kernel would handle connection details (if TCP or something else connection-oriented), or else just start sending datagrams (UDP or something else connectionless) or byte streams (address was for a local socket), and neither the client nor server programs themselves would need to know or care where the connection was coming from.

Hmm...now that I think about it, this has potential.

OLS: A proposal for a new networking API

Posted Jul 24, 2006 5:47 UTC (Mon) by BrucePerens (guest, #2510) [Link] (2 responses)

I'm curious what a new, socket-replacing API would look like.

open("/net/localhost/http", O_RDWR, 0)

Maybe this is not quite what you were asking for :-) . Of course, it's inspired by Plan 9. Using this, or using the socket calls, you get back the same object: a file descriptor. I think socket calls come from BBN's ARPA-sponsored Unix TCP/IP implementation, filtered through Berkeley BSD. They are indeed non-Unix-like, as are the net devices which live in their own unique name space.

Once you get the FD, you can call magic DMA functions on it as Ulrich proposes, which should be valid for plain files, not just network devices. Sometimes, however, what you need to do will fit the simpler sendfile().

Bruce

OLS: A proposal for a new networking API

Posted Jul 24, 2006 6:59 UTC (Mon) by neilbrown (subscriber, #359) [Link] (1 responses)

The (a) problem with
   open("/net/localhost/http", O_RDWR, 0)
type approaches is that they combine "socket" and "connect" into one
call, and so there is no room for doing anything in between like bind
or setsockopt.  Those things aren't always needed, but sometimes they are.

You could try
  open("/net/local/http:bind=1023;sourceaddr=xx.xx.xx.xx;tcp_syncnt=5")
but I don't think you would get very far.

Unfortunately IPC is simply very different from for file IO and trying
to use the same API will be a problem.  Having an open_url library call
is about the best you can do.

OLS: A proposal for a new networking API

Posted Jul 24, 2006 15:14 UTC (Mon) by mheily (subscriber, #27123) [Link]

Or better yet, extend the path heirarchy to include the IP address and port, then allow setsockopt to operate on file descriptors:

fd = open("/net/localhost/ip/192.168.0.1/1023", O_RDWR, 0);
setsockopt(fd, O_TCPSYNCNT, 5);

You could bind to all interfaces by binding to 0.0.0.0 like so:

fd = open("/net/localhost/ip/0.0.0.0/1023", O_RDWR, 0);

OLS: A proposal for a new networking API

Posted Jul 24, 2006 6:15 UTC (Mon) by drag (guest, #31333) [Link] (3 responses)

The way things go everything should be network capable.

In fact there was a group that did a presentation this year at the Usenix conference that did something very odd with computers buses and IP.

Basicly they made the entire computer and all the communication run IP.
See http://www.usenix.org/events/usenix06/tech/ben-yehuda.html

Of course it's closed to non-members until next year.. (Is anybody here a member?)

The way things are going pretty soon ethernet networks will be faster then local disk I/O. If latencies decrease along with that, or something like that, then shared memory scemes may start to work.

Distributed operating systems maybe? Or something like treating a small cluster of machines as a single Numa machine with single kernel?

If most IPC stuff uses something that could just as easily be local as network'd then that would make something like that more easily workable. Like how X is mostly network transparent.

er.. or something like that.

OLS: A proposal for a new networking API

Posted Jul 24, 2006 9:39 UTC (Mon) by dlang (guest, #313) [Link] (2 responses)

I am a member and went and read the paper.

what they did was to create a pci card that claimed to be a VGA/keyboard/mouse card and remoted that out to another machine via IP (the purpose being to investigate the potential for more 'legacy free' motherboards, including being free of any graphics, keybard, usb, IDE, etc ports)

they found a 40-60% performance hit for doing so, but point out that normal sysadmin tasks don't use these channels so it was deemed acceptable.

while this is a very interesting idea for server manufacturers (and performance can be improved with a custom ASIC instead of the slow FPGA)it's of much less interest for home use (where you don't normally have another machine to serve as the head for your systems, and gamers aren't willing to sacrafice any performance)

while an interesting paper it doesn't seem to be very relavent to the issues being dicussed here.

OLS: A proposal for a new networking API

Posted Jul 24, 2006 10:03 UTC (Mon) by drag (guest, #31333) [Link]

ah, I see.

Thank you. I was curious about what exactly they were talking about.

Old news?

Posted Aug 4, 2006 5:36 UTC (Fri) by ringerc (subscriber, #3071) [Link]

I'm fairly sure I've seen cards on the market that do exactly that - and have for a while. The IBM RSA (Remote Supervisor Adapter) comes to mind, and I think Dell do something similar for their servers.

(By the way: I WANT ONE VERY BADLY. I hate the way x86 servers - even Intel server boards etc - can't handle serial console access for POST and BIOS. It's completely retarded, and one area where Sun has been trouncing `PC' servers for years.).

There are also older approaches, such as the `PC Weasel' VGA+keyboard -> serial console cards that've been around for yonks.

That said, you just wrote a summary, so I'm sure there's more in the details.