Fish - The friendly interactive shell
A User-Friendly Shell
Introduction
A great deal of effort has been spent in the last decade trying to make computers more user friendly. While much progress has been made on making graphical user interfaces more user friendly, much less has happened with non-graphical programs such as shells. This is unfortunate, since there are still many things that are inherently easier to do using a shell. The concept of commands, pipelines and environment variables are somewhat complex, but I believe modern shells are harder to use than they have to be, both for the beginner and for the seasoned shell hacker. I have written a new shell called fish, or the friendly interactive shell, that tries to solve several issues that I have found with other shells.fish features syntax highlighting, advanced tab completion features, discoverable help, a revised shell syntax and many other features. In this article, I will describe some of the features found in fish, and explain why I think they are useful.
Syntax Highlighting
Beginners may like syntax highlighting because the pretty colors make them feel comfortable. But syntax highlighting also makes it easier for humans to parse commands and find errors. fish features an advanced error detector that highlights a large number of errors in red. Errors that are flagged include misspelled commands, misspelled options, reading from non-existing files, mismatched parenthesis and quotes and many other common errors. fish also features highlighting of matching quotes and parenthesis.
Powerful History Mechanism
Modern shells save previous commands in a command history. You can view earlier commands by using the up and down arrows. Fish extends this concept by integrating the history search functionality. To search the history, simply type in the search string, and press the up arrow. By using the up and down arrow, you can search for older and newer matches. The fish history automatically removes duplicate matches and the matching substring is highlighted. These features make searching and reusing previous commands much faster.Advanced Tab Completion
Tab completion is a feature that saves a lot of time for beginner and seasoned shell professional alike. The fish tab completion engine is powerful, but easy to use.- fish has a large number of command-specific tab completions, including tab completions for manual pages when using the man command and completion of host names using the known hosts file for commands such as ssh. Both bash and zsh support command specific completions, but neither enables them by default.
- Completions feature a description. When tab-completing a command or a man page, the description is the whatis info. When completing a variable, it is the value of the variable. When completing a file, it is the file type description from the mime database, etc. zsh has limited support for this feature.
- It is possible to use tab completion on strings with wild cards like '*' or '?' as well as in strings with brace expansion like 'input{a,b,c}.txt'. zsh can be configured to expand wild cards, but it does this by replacing the wild cards with one instance of the text they match, making it impossible to tab complete strings that are meant to match more than one file.
- fish tries to squeeze in all completions on one page by truncating long completions and descriptions, but if this fails, a built-in pager is used that supports scrolling up and down using the arrow keys, page up/page down and the space bar. If the user presses any other key, the pager exits and the corresponding character is inserted into the command buffer.
Default Settings
zsh provides command-specific tab completions, a history file, tab completion of strings with wild cards and many, many other advanced functions. But none of them are turned on by default. In fact, a user who starts zsh for the first time would think it was a small improvement over the original Bourne shell. bash does better here, but features like command specific tab completions are turned off by default in bash as well, and the default history settings are not very useful.It is quite possible that a few of my complaints against bash and zsh could be configured away, this is not on purpose from my side, but even though I have been an avid shell user for nearly a decade, I keep discovering useful new features that are poorly documented, turned off by default and implemented in a less useful way.
Fish does not hide it's functionality. The design philosophy of fish is to focus more on making things work right and less on making things configurable. As a result of this, there are very few settings in fish.
Context Sensitive, User Friendly Help Pages
While the man pages give you a decent amount of information on how to use specific commands, the documentation for the shell and it's built-in commands is often hard to use. The bash man page is nearly legendary for how hard it is to get the information you want. fish tries to provide context sensitive documentation in an easy to use form.To access the fish help, one should use the 'help' command. Simply writing 'help' and pressing return will start the user's preferred web browser with the local copy of the fish manual. There are a large number of topics that can be specified with the help command, like 'help syntax', 'help editor', etc. These open up a chapter of the documentation on the specified topic. In order to make the help system easy to find, a message describing how to access it is printed every time fish starts up. Finding a specific help section is easy since the section names can be tab completed.
Built-in commands in fish support the -h and --help options, which result in a detailed explanation of how that command works. The only exception to this are the commands the start a new block of code, such as 'for', 'if', 'while', 'function', etc. To get help on one of these commands, type the command without any options.
Error reporting is an often-overlooked form of help. On syntax errors, fish tries to give a detailed report on what went wrong, and if possible, also prints a help message.
Desktop Integration
Since most users access the shell from inside a virtual terminal in a graphical desktop, the shell should attempt to integrate with the desktop. fish uses the X clipboard for copy and paste, so you can use Control-Y to paste from the clipboard to fish, and Control-K to move the rest of the line to the clipboard.Opening Files
In a graphical file manager it is usually easy to open a document or an image. You simply double-click it and it is launched with a default application. From the shell, this is much more difficult. You need to know which program can handle a file of the given type, and also how to launch it. Launching an HTML file from the command line is no easy task, since most browsers expect a URL, possibly with an absolute path, not a filename. fish features a program called open that uses the same mime-type database and .desktop files used by Gnome and KDE to find the default application for a file, and opens it using the syntax specified by the .desktop file.
A Better Shell Syntax
While shells have gained some features since the seventies, the shell syntax of moderns POSIX shells like bash and zsh is very similar to the original Bourne shell, which is about 30 years old. There are a large number of problems with this syntax which I feel should be changed. Unfortunately, this means that the fish syntax is incompatible with other shells. While it should not be difficult to adapt to the new syntax, old scripts would have to be converted to run in fish.Blocks
There are many cases in shell programming where you specify a list of multiple commands. This includes conditional blocks, loop blocks and function definitions. In regular shells there is very little logic in how these different types of blocks are ended. Conditional statements end with the reverse command, like: 'if true; echo yes; fi', but loops end with the 'done' command like: 'while true; do echo hello; done', individual case conditions end with ';;' and functions end with '}'. Arbitrary reserved words like 'then' and 'do' are also sprinkled over the code. fish uses a single, consistent method of closing blocks: the 'end' command. For a few examples of block syntax in POSIX shell and in fish, see the table below.
| POSIX command | fish command |
|---|---|
| if true; then echo hello; fi | if true; echo hello; end |
| for i in a b c; do echo $i; done | for i in a b c; echo $i; end |
| case $you in *) echo hi;; esac | switch $you; case '*'; echo hi; end |
| hi () { echo hello; } | function hi; echo hello; end |
Quoting
The original Bourne shell was a macro language. It performed variable substitution, tokenization and other operations on one line at a time without understanding the underlying syntax. This results in many unexpected side effects: Consider the following block of code:smurf=blue; smurf=evil; echo Smurfs are $smurfOn the Bourne shell, it will result in the output 'Smurfs are blue'. Macro languages like M4 and Bourne are not intuitive, but once you understand how they function, they are at least predictable and somewhat logical. bash is implemented as a standard language using a bison grammar, but still chooses to emulate some of the quirks from the original Bourne shell.
The above example would result in bash printing 'Smurfs are evil'. On the other hand variable values are still tokenized on spaces, meaning you can't write 'rm $file', since if the variable file contains spaces, rm will try to remove the wrong files. To fix this, the user has to make sure every use of a variable in enclosed in quotes, like 'rm "$file"'. This is a very common source of bugs in shell scripts since it is simply a case of the default behavior being unexpected and very rarely what is wanted.
In summary, by making bash a non-macro language that sometimes behaves like one, it becomes unpredictable and very hard to learn.
fish is not a macro language and does not pretend to be one. Variables with spaces are still just one token. Because of this, there is no need for the double quotes to mean something different from single quotes, so both types of quotes mean the same thing, and quotes can be nested.
Variable Assignment
Variable assignments in Bourne shell are whitespace sensitive. 'foo=bar' is an assignment, but 'foo = bar' is not. This is just a bad idea. fish does something somewhat unexpected while fixing this. It borrows syntax from csh, and uses a command called 'set' to assign variable values. The reason for doing this is that in fish everything is a command. Loops, conditionals and every other kind of higher level language construct is implemented as yet another built-in command, following the same syntax rules. This makes the language easier to learn and understand, as well as easier to implement.To set the variable smurf to the value blue, use the command:
set smurf blueBy default, variables are local to the current block and disappear when the block goes out of scope. To make a variable global, you need to use the -g switch.
Two Methods of Creating Functions, and Both are Bad
bash, zsh and other regular shells allow you to create stored functions in two ways, either as aliases or as functions.Aliases are defined using commands like 'alias ll="ls -l"'. Aliases are simply string substitutions in the command-line. Because of this, aliases have the following limitations:
- You can only redirect input/output to the last command in the alias.
- You can only specify arguments to the last command in the alias.
- Alias definitions are a single text string, this means complex functions are nearly impossible to create.
ll() { ls $*;}
While this solves the issues with aliases, I think this is just as bad
a syntax. It looks like C code, but anyone expecting it to work
anything like C will discover it is really not. You can not specify
argument names in the parenthesis, they are just there to make it look
like C code. The curly brackets are some sort of pseudo-commands, so
skipping the semicolon in the example above results in a syntax
error. And perhaps the most strange quirk of all is that removing the
whitespace between the opening bracket and 'ls' will also result in a
syntax error. Clearly this is not a very well though out syntax. fish
uses a single syntax for defining functions, and the definition is
just another regular command:
function ll; ls $argv; endThis is slightly wordier than the above examples, but it solves all the issues with either of the above syntaxes in a way that is consistent with the rest of the fish syntax.
Impossible to Validate the Syntax
Since the use of variables as commands in regular shells is allowed, it is impossible to reliably check the syntax of a script.For example, this snippet of bash/zsh code may or may not be legal, depending on your luck:
if true; then if [ $RANDOM -lt 1024 ]; then END=fi; else END=true; fi; $ENDBoth bash and zsh try to determine if the command in the current buffer is finished when the user presses the return key, but because of issues like this, they will sometimes fail.
fish solves this by disallowing variables as commands. Anything you can do with variables as commands can be done in a much cleaner way using either the eval command or by using functions.
Minor Problems
The strings '$foo', "$foo" and `$foo` all look rather similar, while doing three completely different things. fish solves this by making '$foo' and "$foo" mean the same thing, as described above, but also by making the syntax for sub-shells use parenthesis instead of back-ticks.A large number of standard UNIX commands, like printf, echo, kill and test are implemented as shell built-ins in bash and zsh. As near as I can tell, there is only one advantage to this, a minimal performance increase. But the drawbacks are many:
- Bugs in one of these built-ins risk crashing or corrupting the entire shell.
- Commands will change their behavior based on what shell you are using.
- Users of other shells will not benefit from the commands you have written.
- It breaks the UNIX philosophy of doing only one thing but doing it well.
- Memory usage is increased.
Summing Up
None of the problems with the regular shell syntax are big enough to make the shells unusable. For thirty years, the shell has been a valuable tool for many computer users. But even well designed, powerful programs sometimes need to be updated to remove some of the old mistakes. The changes introduced in fish makes the shell language easier to discover and remember for beginners, and once learned, it should introduce fewer bugs for experienced users. Regular computer languages have evolved and been replaced multiple times in the last thirty years, why shouldn't that be the case for shell languages?Future Plans
Mostly, the fish codebase is in good shape. It is well documented and reasonably small. But since the code is new and untested, it contains more bugs then one would like. It definitely needs an audit for security and stability problems, and a much more comprehensive test suite. There are also a number of new features that I wish to add, but none of these features require any major rewrites.There is one important piece of syntax that is not yet implemented, namely full IO redirection for blocks and functions. It is currently possible to use functions in pipelines and to redirect the output of functions, but functions cannot output binary data, and input redirection is not supported. It is not possible to pipe or redirect the input/output of blocks of code.
A future version of fish will allow code like this:
for i in (find . -name "*.c"); echo $i; grep "mbstowcs" $i; end | lessto view the output of multiple commands in less, or:
cat foo.txt | while test -z quit
read type
switch $type
case quit
set quit 1
...
end
end
to read multiple lines from a file. I am working on implementing these
features and hope to release a new version with these features in the
coming weeks.
Fish currently has command specific completions for about 60 commands. There are a great number of additional commands that would benefit from completions. I hope that it will be possible to use Doclifter to automatically convert manual pages into tab completion specifications.
Another area that still needs more attention is the documentation. While the project already has a great deal of documentation, fish would benefit from shell scripting tutorials, an introduction to UNIX concepts, and various other forms of documentation.
The syntax validation and error reporting of fish also needs more work. Fish already does some basic syntax checks, but a future implementation will be able to detect most syntax errors before running a file.
There are also a large number of minor features that I plan to try out:
- Undo/redo support.
- Interactive directory history, use Alt-up and Alt-down to insert next previous directory into the command buffer.
- Multi-line editing.
- Mouse support, including moving the cursor by clicking in the window, as well as selecting a completion by clicking on it.
- Suggest completions by writing them out in an understated color.
- Store the output of previous commands. Calculators often use the variable 'ans' to refer to the result of the previous calculation. It would be nice if $ans could be the output of the previous shell command.
| Index entries for this article | |
|---|---|
| GuestArticles | Liljencrantz, Axel |