Android gets a toybox

Posted Jan 16, 2015 18:42 UTC (Fri) by dlang (guest, #313)
In reply to: Android gets a toybox by rsidd
Parent article: Android gets a toybox

Yes, Rpb's intent is to have compilers etc on Android as well. However, it's not a case of having to port the compilers. If they can already run in a chroot and can compile native android code there, then it seems to me that the problem is one of organization, not porting.

Getting Android to have a fully capable shell rather than a crippled one is more work. That's where toybox replacing toolbox helps.

to post comments

Android gets a toybox

Posted Jan 16, 2015 22:01 UTC (Fri) by rsidd (subscriber, #2582) [Link] (11 responses)

Android uses mksh, hardly a crippled shell. And not sure why busybox would help in porting bash or some more heavyweight shell.. It's probably portable already if anyone wants it.

Android gets a toybox

Posted Jan 16, 2015 22:19 UTC (Fri) by dlang (guest, #313) [Link] (10 responses)

toolbox/toybox/busybox aren't the shell, they replace full-blown versions of apps to save space and ram (minor things like ls, sed, etc)

Android won't use busybox by default, they were using toolbox, but it didn't implement a lot of commands that people want, so the move to using toybox, which does implement a lot more (and is aimed at supporting a full self-hosting environment) is a move forwards.

Android gets a toybox

Posted Jan 17, 2015 1:02 UTC (Sat) by rsidd (subscriber, #2582) [Link] (9 responses)

I know all that but you said something about a 'crippled shell' above.

Busybox/toolbox do make life easier but there's much more to a development environment, unless one goes the chroot way.

Android gets a toybox

Posted Jan 18, 2015 8:29 UTC (Sun) by landley (guest, #6789) [Link] (8 responses)

> Busybox/toolbox do make life easier but there's much more
> to a development environment, unless one goes the chroot
> way.

Actually no, there isn't. Years ago I got a self-bootstrapping environment down to 7 packages (busybox, uClibc, linux, make, gcc, binutils, bash), and I still maintain the project here:

http://landley.net/aboriginal/about.html

That project is not just my current main toybox test harness, it's how I got into busybox development in the first place (eventually becoming maintainer because I was doing more work than Erik, so he handed over the reigns). I started with Linux From Scratch (circa 3.0 or so) and tried replacing all the gnu packages, one by one, with busybox. They broke spectacularly (or were missing), so I fixed them, or completely rewrote things like "sed" and "sort" so the new implementation would actually work in a development environment.

I actually used to keep a list of which gnu packages I'd replaced with busybox. Search this page for "coreutils" to see the old list:

http://landley.net/aboriginal/old/

(Yeah, I renamed my project years ago because "firmware" wasn't quite a correct description and is impossible to google for anyway. The above link is from before the restart where I switched from User Mode Linux to QEMU as my native build environment. Started over from scratch, the current project's repository only goes back to the start of the qemu-targeted stuff.)

I got Aboriginal Linux to 1.0 years ago with an automated Linux From Scratch build that runs under it (http://landley.net/aboriginal/control-images) which I run under each new toybox release. (Currently LFS 6.8, upgrading it is on my todo list but not a high priority.) I got it to work with busybox years ago, and now I'm replacing each busybox command with toybox, one by one. This config file lists all the commands needed by a development environment to rebuild itself:

https://speakerdeck.com/landley/developing-for-non-x86-ta...

(Experimentally determined using the record-commands stuff, see "command logging wrapper" in http://landley.net/aboriginal/FAQ.html#debug_logging)

The config symbols before the comment containing the word "toybox" are still used from busybox. The ones afterwards are now provided by toybox. It's in "miniconfig" format, ala:

http://landley.net/aboriginal/FAQ.html#dev_miniconfig
https://lwn.net/Articles/161086/

This is why I keep pushing patches to remove perl from the kernel build, I do _not_ want to add perl as an 8th package required to get a minimal development environment capable of rebuilding itself under itself from source code:

https://lkml.org/lkml/2008/2/15/548

I gave a long presentation on Aboriginal Linux at three different conferences about 5 years back. My 260-slide deck is at:

http://landley.net/aboriginal/downloads/presentation.pdf

(In theory it's also at https://speakerdeck.com/landley/developing-for-non-x86-ta... but speakerdeck appears to be down at the moment...)

Rob

Android gets a toybox

Posted Jan 18, 2015 10:47 UTC (Sun) by rsidd (subscriber, #2582) [Link] (7 responses)

Sure it's possible, but is it useful or is it mainly a technology demonstration? Android takes about 8GB RAM and huge amounts of disk space to compile. Yes, phones will get there soon enough, but desktops aren't going away or getting slower...

As noted above, I do use my phone for work when I'm waiting somewhere (and that foldable bluetooth keyboard is very comfortable). But if it's serious programming stuff, I open a few terms, ssh into my work machine, and get into the GNU screen sessions that I have running there.

Android gets a toybox

Posted Jan 18, 2015 12:28 UTC (Sun) by magnus (subscriber, #34778) [Link]

Desktops are not getting slower, but since the price depends heavily on economy of scale it's possible they get more expensive over time. It sounds to me like a good idea to make sure there is a path to general-purpose computing with local data storage on whatever devices are available for the average person (smartphones, SBCs or whatever). At least it's nice to know the possibility is there.

Android gets a toybox

Posted Jan 19, 2015 20:44 UTC (Mon) by landley (guest, #6789) [Link] (3 responses)

> Sure it's possible, but is it useful or is it mainly a technology
> demonstration?

My use case is an 8 year old girl in rural india who inherited an old phone from her parents and should be able to learn to program with just that.

Having a phone should be sufficient to program for the phone, and not just ROM BASIC du jour but actually digging into the thing. Linus started Linux because he had a minimum level of knowledge and equipment, and way more time than money. He's no longer that guy, and it's starting to show. But the _next_ generation is gonna do it on phones, not PCs.

Android is a long way from that, but responds to gentle nudges. (Meanwhile iOS is actively locking people out. No user serviceable parts inside, shut up and subscribe to your feeds.)

> Android takes about 8GB RAM and huge amounts of disk space to compile.

AOSP can build in less memory when you do a non-parallel build, but doing so takes about 5 hours.

Fixing this is a "meet in the middle" sort of thing, Moore's Law marches on and it should be possible to pare AOSP down to a less monstrous version. (Right now AOSP only functions as a complete distro build in a giant integrated monolith with hundreds of packages. It should be possible to extract a base system build and make the rest more configurable. Remember we had a bunch of releases of Red Hat before we got Linux From Scratch...)

> As noted above, I do use my phone for work when I'm waiting somewhere
> (and that foldable bluetooth keyboard is very comfortable).

The keyboard linked elsewhere in the comments is very nice, but almost $200. I started learning to program when I was 11 on shared family resources, I didn't have $200 (even inflation adjusted) to spend on anything (even counting indirect purchasing power from birthdays and christmas) for like 6 years after that.

Ideally having the device should be sufficient for programming the device. Ergonomically bigger keyboard and display are really cool, but to matter they need to be cheap and ubiquitous. (I'm tracking how long until USB 3 hubs are available used. Old keyboard, old mouse, easy to come by already. People throw _out_ ones that still work. Old USB->HDMI adapter allows you to plug into an old television, but that requires USB 3. USB2 only gives you usb->vga adapter, and old monitors aren't nearly as ubiquitous as old TVs. Where "old" is "today's technology in 5-10 years".)

Still, a teenager's _already_ going to be a lot more comfortable with an on-screen phone keyboard than I'm ever likely to get. "How can you live with that cramped horrible thing, my fingers hurt just looking at you" is what an old person (like me) says to a 12 year old who has no problem with it. And there are tablets. (Did you ever notice that successful tablets are big phones, not small PCs? Except a phone you carry with you 24/7 and a tablet you generally don't, so that's not a solution unless you transiently associate your phone to the tablet just like you would to any other larger display/input devices you happened to be near...)

(Pity Google's glasshole spycams poisoned the well for glasses as an I/O device. Oh well, maybe that's temporary or doesn't apply to Korea or something...)

> But if it's serious programming stuff, I open a few terms,
> ssh into my work machine, and get into the GNU screen sessions
> that I have running there.

Microsoft's Altair BASIC was cross compiled (well, cross-assembled) from a PDP-10. Ken and Dennis cross-assembled the first unix code on a GE 645 mainframe. Looking at the previous generation as a "real computer" and the new one as "just a toy" is a common stage, and old programmers usually stick with what they know until they retire still writing COBOL. Meanwhile the new kids come up to speed on the hardware in front of them, and consider it a thing in itself rather than an extension of something else.

I'm trying to allow serious programming to be done natively on a phone. Yes, an android phone is about 7 years behind the PC in hardware specs. But I was building Linux systems 7 years ago on the hardware of the day, the fact I could get a much more powerful mainframe and just use that is not actually a new argument, it was made about minis and micros before it was made about phones.

Honestly, we've been through all this before:

mainframe -> minicomputer -> microcomputer (I.E. PC) -> smartphone.

This is the fourth iteration of the same old story. Too many people ask "is this worth doing today" rather than "would this be nice to have in 5-10 years".

I can't fix all the problems, but I can chip away at obvious bottlenecks to try to steer the narrative through the paths I want it to take rather than yet more DRM lockdown nonsense (replaying the lost decade of shrinkwrap software turning into abandonware the PC went through).

Android gets a toybox

Posted Jan 20, 2015 6:03 UTC (Tue) by rsidd (subscriber, #2582) [Link] (2 responses)

>The keyboard linked elsewhere in the comments is very nice, but almost $200.

Lazy google link on my part. Newegg, where I got mine, lists it for $23 (plus shipping which was significant in my case, but still a total of perhaps $40).

If you can indeed get kids to program on their phones, that would be great. I get what you're saying about power, but with phones it's not just the power, but the interface. For programmers, the mainframe->mini->desktop transition came with improved interfaces (punch card -> keyboard -> keyboard+mouse, teletype -> vt200->cga->vga->modern monitors). Phones are a huge step back, and if you are going to hook them up to external keyboards and displays, you lose most of the benefits of a phone. But I'm happy to be proved wrong.

Android gets a toybox

Posted Jan 21, 2015 4:25 UTC (Wed) by landley (guest, #6789) [Link] (1 responses)

> I get what you're saying about power, but with phones
> it's not just the power, but the interface.

I'm sad I can't find the article I wrote for linuxandmain.com back around 2002 on exactly this issue. (I was arguing that the next logical step beyond laptops was hooking a palmtop (newton/ipaq/palm) up to cell tower modems, the reply was that the ergonomics made it impossible for that to ever grow beyond a tiny power user niche. Alas the site went down years ago and archive.org's copy is swiss cheese. The iPhone was announced about 4 years later; presumably it was already in development when I wrote that article)

Here's a project that put a linux dev environment chroot on an android phone a year or two back:

https://play.google.com/store/apps/details?id=champion.gn...

That's the current baseline of sad ui that we have to improve upon. (And of course the above can't come preinstalled because it's full of GPL stuff Android has an official policy about not shipping. And I'd prefer a version that didn't cause my phone to reboot if I left it running more than an hour.)

> For programmers, the mainframe->mini->desktop transition
> came with improved interfaces (punch card -> keyboard ->
> keyboard+mouse, teletype -> vt200->cga->vga->modern monitors).

Minicomputers didn't start with improved UI, the first one to take off was the Altair which had toggle switches and LEDs as its input and output. The old PBS computer history documentary "Triumph of the Nerds" starts describing the altair at 18 minutes 30 seconds into here:

https://www.youtube.com/watch?v=2tsBAS91lTI

It goes on to describe the first thing an Altair ever actually _did_, which was use a series of timed loops to generate radio interference, so you could put an AM radio on top of it and it would play a song.

Intel's 8008 processor was designd to go in a "glass tty", I.E. a video display terminal hooked up to a minicomputer via serial port. The 8080 in the Altair was a sequel to the 8008. (The Intel section of http://landley.net/history/mirror/ starts with interviews with all four of the principals in the 4004->8008->8080 (and z80).)

From the Altair, we went to various ROM Basic systems hooked up to _televisions_ (max possible horizontal resolution about 340 pixels due to the NTSC signal, see http://landley.net/history/scans/icpug-1986-jul/page290-2... so with 8 pixel characters you have 42 columns max), and only years later managed DOS and CP/M (single-tasking, booted from a floppy, not a step up from a PDP-11 with a vt100). The PC _grew_ better UI as it went along, but it started out with monochrome video (then EGA graphics, four colors! One of which was Cyan!)

Meanwhile, Ken Thompson wrote unix on a PDP-7 that he _originally_ started using because it had a vector graphics terminal hooked up to it, so he ported an early version of lunar lander to it. (I don't believe unix used the graphics terminal, but the vector graphics terminal is why he started poking at that mothballed machine.) The reason it was mothballed and abandoned it was a cheap low-end system, nothing special.

The first 3D card (SGI's first product) was an add-on board for a VAX that powered a display terminal. (The Iris 1000: http://www.nekochan.net/wiki/Silicon_Graphics#First_gener...)

So by the time the PC grew 3D cards (ala Voodoo and such) circa 1995, minicomputers already had that for a decade (and tron was rendered on minicomptuers in 1982).

The convenience of the PC was you didn't need to sign up for a timeslot on the terminal down the hall when you could have your own dedicated system on your desk. Now you don't need the computer on your desk when you have a comptuter in your pocket.

Oh, the founder of DEC (Ken Olsen) was proud that the PDP-1 had a built-in video display terminal in the operator's console. That's the machine spacewar" was written for. Here's an interview with the Ken Olsen where he talks about that: http://landley.net/history/mirror/interviews/olsen.html

And the wikipedia article about spacewar (in 1961, 14 years before the Altair shipped):

http://en.wikipedia.org/wiki/Spacewar_(video_game)

> Phones are a huge step back,

The advantage the phone has is that there are billions of them deployed today and people carry them around 24/7. We don't have to make it a better programming environment than big iron, we just have to make it _possible_ to program on them. Having a device should be sufficient to program the device, and not in a second class "you can write BASIC programs but to make real programs like the one it comes with you need a real computer which this isn't".

Hobbyists developers learn to program on the machines they have. I'm not trying to put phones in people's hands, that's _today_. I'm trying to make those phones fully programmable without being second-class citizens dependent on external big iron. (And these days, a PC is big iron. Already happened, past tense. That was what the http://youtu.be/SGmtP5Lg_t0 talk was about.)

> and if you are going to hook them up to external keyboards
> and displays, you lose most of the benefits of a phone.

The USB keyboard/mouse/monitor allows old farts like you and me to switch over from our big iron VAXes to the new micro-er computers. I doubt your average 12 year old will find an onscreen keyboard "too small and cramped" if that's what they grew up with, and the UI should naturally evolve in response to what people are doing with it.

We are the unix greybeards singing "all the world's a vax" and squinting at those strange kids with their 16 bit PCs going "what a silly toy". Except computer history's a hobby of mine, and this is the fourth generation of this particular transition. I don't have to know how the problems will be fixed to know they will, I'm just trying to steer the future towards android and away from iOS because one is a multivendor commodity system with a theoretically modular software architecture, and the other is a hermetically sealed and locked down system produced by a monopoly that sells the hardware too.

> I'm happy to be proved wrong.

I'm operating on the assumption that there are already 10x as many smartphones in the world as PCs, that people already carry them around 24/7, and that historically economies of scale have always strongly favored the system with the higher unit volume.

Given that smartphones exist, they should be fully natively programmable. If you're going to write code for a phone, you should be able to do it on the phone.

Rob

Android gets a toybox

Posted Jan 21, 2015 5:46 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

> I doubt your average 12 year old will find an onscreen keyboard "too small and cramped"
They certainly do - children are not some kind of alien species.

Android gets a toybox

Posted Jan 22, 2015 23:27 UTC (Thu) by xslogic (guest, #97478) [Link] (1 responses)

That's 8GB of RAM to compile the whole OS.

Most people don't compile their whole OS that much - unless they're updating it.

Mostly, people will (and do) compile applications.

Android gets a toybox

Posted Jan 23, 2015 3:45 UTC (Fri) by rsidd (subscriber, #2582) [Link]

Rob mentioned a "self-bootstrapping" environment, which has to mean a working OS. Or it could be a chrooted environment, but that's already possible on android.

Also, most people developing on android devices would want to develop android apps. The development environment for that (sdk+ide etc) is pretty heavyweight, and I'm not sure it even exists on ARM. But it would definitely be brilliant if you could compile, debug and run an app right on your mobile device without needing a computer. Even better if you could recompile the kernel and install it while keeping the earlier one as a backup -- we are all used to doing that on desktops, but doing it on a mobile device is much trickier.