.HTML "Rc — The Plan 9 Shell
. \" /*% refer -k -e -n -l3,2 -s < % | tbl | troff -ms | lp -dfn
.Tm shell programming language g
.de TP \" An indented paragraph describing some command, tagged with the command name
.IP "\\f(CW\\$1\\fR" 5
.if \\w'\\f(CW\\$1\\fR'-4n .br
..
.de CI
.nr Sf \\n(.f
\%\&\\$3\f(CW\\$1\fI\&\\$2\f\\n(Sf
..
.TL
Rc \(em The Plan 9 Shell
.AU
Tom Duff
td@plan9.bell-labs.com
.AB
.I Rc
is a command interpreter for Plan 9 that
provides similar facilities to UNIX's
Bourne shell,
with some small additions and less idiosyncratic syntax.
This paper uses numerous examples to describe
.I rc 's
features, and contrasts
.I rc
with the Bourne shell, a model that many readers will be familiar with.
.AE
.NH
Introduction
.PP
.I Rc
is similar in spirit but different in detail from UNIX's
Bourne shell. This paper describes
.I rc 's
principal features with many small examples and a few larger ones.
It assumes familiarity with the Bourne shell.
.NH
Simple commands
.PP
For the simplest uses
.I rc
has syntax familiar to Bourne-shell users.
All of the following behave as expected:
.P1
date
cat /lib/news/build
who >user.names
who >>user.names
wc
followed by a command in braces causes the command to be run with
its standard output or input attached to a pipe. The parent command
.CW cmp "" (
in the example)
is started with the other end of the pipe attached to some file descriptor
or other, and with an argument that will connect to the pipe when opened
(e.g.,
.CW /dev/fd/6 ).
Some commands are unprepared to deal with input files that turn out not to be seekable.
For example
.CW diff
needs to read its input twice.
.NH
Exit status
.PP
When a command exits it returns status to the program that executed it.
On Plan 9 status is a character string describing an error condition.
On normal termination it is empty.
.PP
.I Rc
captures command exit status in the variable
.CW $status .
For a simple command the value of
.CW $status
is just as described above. For a pipeline
.CW $status
is set to the concatenation of the statuses of the pipeline components with
.CW |
characters for separators.
.PP
.I Rc
has a several kinds of control flow,
many of them conditioned by the status returned from previously
executed commands. Any
.CW $status
containing only
.CW 0 's
and
.CW | 's
has boolean value
.I true .
Any other status is
.I false .
.NH
Command grouping
.PP
A sequence of commands enclosed in
.CW {}
may be used anywhere a command is required.
For example:
.P1
{sleep 3600;echo 'Time''s up!'}&
.P2
will wait an hour in the background, then print a message.
Without the braces,
.P1
sleep 3600;echo 'Time''s up!'&
.P2
would lock up the terminal for an hour,
then print the message in the background.
.NH
Control flow \(em \f(CWfor\fP
.PP
A command may be executed once for each member of a list
by typing, for example:
.P1
for(i in printf scanf putchar) look $i /usr/td/lib/dw.dat
.P2
This looks for each of the words
.CW printf ,
.CW scanf
and
.CW putchar
in the given file.
The general form is
.P1
for(\fIname\fP in \fIlist\fP) \fIcommand\fP
.P2
or
.P1
for(\fIname\fP) \fIcommand\fP
.P2
In the first case
.I command
is executed once for each member of
.I list
with that member assigned to variable
.I name .
If the clause
.CW in "" ``
.I list ''
is missing,
.CW in "" ``
.CW $* ''
is assumed.
.NH
Conditional execution \(em \f(CWif\fP
.PP
.I Rc
also provides a general if-statement. For example:
.P1
for(i in *.c) if(cpp $i >/tmp/$i) vc /tmp/$i
.P2
runs the C compiler on each C source program that
cpp processes without error.
An `if not' statement provides a two-tailed conditional.
For example:
.P1
for(i){
if(test -f /tmp/$i) echo $i already in /tmp
if not cp $i /tmp
}
.P2
This loops over each file in
.CW $* ,
copying to
.CW /tmp
those that do not already appear there, and
printing a message for those that do.
.NH
Control flow \(em \f(CWwhile\fP
.PP
.I Rc 's
while statement looks like this:
.P1
while(newer subr.v subr.c) sleep 5
.P2
This waits until
.CW subr.v
is newer than
.CW subr.c ,
presumably because the C compiler finished with it.
.PP
If the controlling command is empty, the loop will not terminate.
Thus,
.P1
while() echo y
.P2
emulates the
.I yes
command.
.NH
Control flow \(em \f(CWswitch\fP
.PP
.I Rc
provides a switch statement to do pattern-matching on
arbitrary strings. Its general form is
.P1
switch(\fIword\fP){
case \fIpattern ...\fP
\fIcommands\fP
case \fIpattern ...\fP
\fIcommands\fP
\&...
}
.P2
.I Rc
attempts to match the word against the patterns in each case statement in turn.
Patterns are the same as for filename matching, except that
.CW /
and
.CW .
and
.CW ..
need not be matched explicitly.
.PP
If any pattern matches, the
commands following that case up to
the next case (or the end of the switch)
are executed, and execution of the switch
is complete. For example,
.P1
switch($#*){
case 1
cat >>$1
case 2
cat >>$2 <$1
case *
echo 'Usage: append [from] to'
}
.P2
is an append command. Called with one file argument,
it appends its standard input to the named file. With two, the
first is appended to the second. Any other number
elicits an error message.
.PP
The built-in
.CW ~
command also matches patterns, and is often more concise than a switch.
Its arguments are a string and a list of patterns. It sets
.CW $status
to true if and only if any of the patterns matches the string.
The following example processes option arguments for the
.I man (1)
command:
.P1
opt=()
while(~ $1 -* [1-9] 10){
switch($1){
case [1-9] 10
sec=$1 secn=$1
case -f
c=f s=f
case -[qwnt]
cmd=$1
case -T*
T=$1
case -*
opt=($opt $1)
}
shift
}
.P2
.NH
Functions
.PP
Functions may be defined by typing
.P1
fn \fIname\fP { \fIcommands\fP }
.P2
Subsequently, whenever a command named
.I name
is encountered, the remainder of the command's
argument list will assigned to
.CW $*
and
.I rc
will execute the
.I commands .
The value of
.CW $*
will be restored on completion.
For example:
.P1
fn g {
grep $1 *.[hcyl]
}
.P2
defines
.CI g " pattern
to look for occurrences of
.I pattern
in all program source files in the current directory.
.PP
Function definitions are deleted by writing
.P1
fn \fIname\fP
.P2
with no function body.
.NH
Command execution
.PP
.I Rc
does one of several things to execute a simple command.
If the command name is the name of a function defined using
.CW fn ,
the function is executed.
Otherwise, if it is the name of a built-in command, the
built-in is executed directly by
.I rc .
Otherwise, directories mentioned in the variable
.CW $path
are searched until an executable file is found.
Extensive use of the
.CW $path
variable is discouraged in Plan 9. Instead, use the default
.CW (.
.CW /bin)
and bind what you need into
.CW /bin .
.NH
Built-in commands
.PP
Several commands are executed internally by
.I rc
because they are difficult to implement otherwise.
.TP ". [-i] \fIfile ...\f(CW
Execute commands from
.I file .
.CW $*
is set for the duration to the reminder of the argument list following
.I file .
.CW $path
is used to search for
.I file .
Option
.CW -i
indicates interactive input \(em a prompt
(found in
.CW $prompt )
is printed before each command is read.
.TP "builtin \fIcommand ...\f(CW
Execute
.I command
as usual except that any function named
.I command
is ignored.
For example,
.P1
fn cd{
builtin cd $* && pwd
}
.P2
defines a replacement for the
.CW cd
built-in (see below) that announces the full name of the new directory.
.TP "cd [\fIdir\f(CW]
Change the current directory to
.I dir .
The default argument is
.CW $home .
.CW $cdpath
is a list of places in which to search for
.I dir .
.TP "eval [\fIarg ...\f(CW]
The arguments are concatenated (separated by spaces) into a string, read as input to
.I rc ,
and executed. For example,
.P1
x='$y'
y=Doody
eval echo Howdy, $x
.P2
would echo
.P1
Howdy, Doody
.P2
since the arguments of
.CW eval
would be
.P1
echo Howdy, $y
.P2
after substituting for
.CW $x .
.TP "exec [\fIcommand ...\f(CW]
.I Rc
replaces itself with the given
.I command .
This is like a
.I goto
\(em
.I rc
does not wait for the command to exit, and does not return to read any more commands.
.TP "exit [\fIstatus\f(CW]
.I Rc
exits immediately with the given status. If none is given, the current value of
.CW $status
is used.
.TP "flag \fIf\f(CW [+-]
This command manipulates and tests the command line flags (described below).
.P1
flag \fIf\f(CW +
.P2
sets flag
.I f .
.P1
flag \fIf\f(CW -
.P2
clears flag
.I f .
.P1
flag \fIf\f(CW
.P2
tests flag
.I f ,
setting
.CW $status
appropriately.
Thus
.P1
if(flag x) flag v +
.P2
sets the
.CW -v
flag if the
.CW -x
flag is already set.
.TP "rfork [nNeEsfF]
This uses the Plan 9
.I rfork
system entry to put
.I rc
into a new process group with the following attributes:
.TS
box;
l l l
lfCW l l.
Flag Name Function
_
n RFNAMEG Make a copy of the parent's name space
N RFCNAMEG Start with a new, empty name space
e RFENVG Make a copy of the parent's environment
E RFCENVG Start with a new, empty environment
s RFNOTEG Make a new note group
f RFFDG Make a copy of the parent's file descriptor space
F RFCFDG Make a new, empty file descriptor space
.TE
Section
.I fork (2)
of the Programmer's Manual describes these attributes in more detail.
.TP "shift [\fIn\f(CW]
Delete the first
.I n
(default 1) elements of
.CW $* .
.TP "wait [\fIpid\fP]
Wait for the process with the given
.I pid
to exit. If no
.I pid
is given, all outstanding processes are waited for.
.TP "whatis \fIname ...\f(CW
Print the value of each
.I name
in a form suitable for input to
.I rc .
The output is an assignment to a variable, the definition of a function,
a call to
.CW builtin
for a built-in command, or the path name of a binary program.
For example,
.P1
whatis path g cd who
.P2
might print
.P1
path=(. /bin)
fn g {gre -e $1 *.[hycl]}
builtin cd
/bin/who
.P2
.TP "~ \fIsubject pattern ...\f(CW
The
.I subject
is matched against each
.I pattern
in turn. On a match,
.CW $status
is set to true.
Otherwise, it is set to
.CW "'no match'" .
Patterns are the same as for filename matching.
The
.I patterns
are not subjected to filename replacement before the
.CW ~
command is executed, so they need not be enclosed in
quotation marks, unless of course, a literal match for
.CW *
.CW [
or
.CW ?
is required.
For example
.P1
~ $1 ?
.P2
matches any single character, whereas
.P1
~ $1 '?'
.P2
only matches a literal question mark.
.NH
Advanced I/O Redirection
.PP
.I Rc
allows redirection of file descriptors other than 0 and 1
(standard input and output) by specifying the file descriptor
in square brackets
.CW "[ ]
after the
.CW <
or
.CW > .
For example,
.P1
vc junk.c >[2]junk.diag
.P2
saves the compiler's diagnostics from standard error in
.CW junk.diag .
.PP
File descriptors may be replaced by a copy, in the sense of
.I dup (2),
of an already-open file by typing, for example
.P1
vc junk.c >[2=1]
.P2
This replaces file descriptor 2 with a copy of file descriptor 1.
It is more useful in conjunction with other redirections, like this
.P1
vc junk.c >junk.out >[2=1]
.P2
Redirections are evaluated from left to right, so this redirects
file descriptor 1 to
.CW junk.out ,
then points file descriptor 2 at the same file.
By contrast,
.P1
vc junk.c >[2=1] >junk.out
.P2
redirects file descriptor 2 to a copy of file descriptor 1
(presumably the terminal), and then directs file descriptor 1
to a file. In the first case, standard and diagnostic output
will be intermixed in
.CW junk.out .
In the second, diagnostic output will appear on the terminal,
and standard output will be sent to the file.
.PP
File descriptors may be closed by using the duplication notation
with an empty right-hand side.
For example,
.P1
vc junk.c >[2=]
.P2
will discard diagnostics from the compilation.
.PP
Arbitrary file descriptors may be sent through
a pipe by typing, for example,
.P1
vc junk.c |[2] grep -v '^$'
.P2
This deletes blank lines
from the C compiler's error output. Note that the output
of
.CW grep
still appears on file descriptor 1.
.PP
Occasionally you may wish to connect the input side of
a pipe to some file descriptor other than zero.
The notation
.P1
cmd1 |[5=19] cmd2
.P2
creates a pipeline with
.CW cmd1 's
file descriptor 5 connected through a pipe to
.CW cmd2 's
file descriptor 19.
.NH
Here documents
.PP
.I Rc
procedures may include data, called ``here documents'',
to be provided as input to commands, as in this version of the
.I tel
command
.P1
for(i) grep $i <[1=2]
exit 1
}
NT=n # default nroff
s='*' # section, default try all
for(i) switch($i){
case -t
NT=t
case -n
NT=n
case -*
echo Usage: $0 '[-nt] [section] page ...' >[1=2]
exit 1
case [1-9] 10
s=$i
case *
eval 'pages='$s/$i
for(page in $pages){
if(test -f $page)
$NT^roff -man $page
if not
echo $0: $i not found >[1=2]
}
}
.P2
Note the use of
.CW eval
to make a list of candidate manual pages.
Without
.CW eval ,
the
.CW *
stored in
.CW $s
would not trigger filename matching
\(em it's enclosed in quotation marks,
and even if it weren't, it would be expanded
when assigned to
.CW $s .
Eval causes its arguments
to be re-processed by
.I rc 's
parser and interpreter, effectively delaying
evaluation of the
.CW *
until the assignment to
.CW $pages .
.NH
Examples \(em \fIholmdel\fP
.PP
The following
.I rc
script plays the deceptively simple game
.I holmdel ,
in which the players alternately name Bell Labs locations,
the winner being the first to mention Holmdel.
.KF
.P1
t=/tmp/holmdel$pid
fn read{
$1=`{awk '{print;exit}'}
}
ifs='
\&' # just a newline
fn sigexit sigint sigquit sighup{
rm -f $t
exit
}
cat <<'!' >$t
Allentown
Atlanta
Cedar Crest
Chester
Columbus
Elmhurst
Fullerton
Holmdel
Indian Hill
Merrimack Valley
Morristown
Neptune
Piscataway
Reading
Short Hills
South Plainfield
Summit
Whippany
West Long Branch
!
while(){
lab=`{fortune $t}
echo $lab
if(~ $lab Holmdel){
echo You lose.
exit
}
while(read lab; ! grep -i -s $lab $t) echo No such location.
if(~ $lab [hH]olmdel){
echo You win.
exit
}
}
.P2
.KE
.PP
This script is worth describing in detail
(rather, it would be if it weren't so silly.)
.PP
Variable
.CW $t
is an abbreviation for the name of a temporary file.
Including
.CW $pid ,
initialized by
.I rc
to its process-id,
in the names of temporary files insures that their
names won't collide, in case more than one instance
of the script is running at a time.
.PP
Function
.CW read 's
argument is the name of a variable into which a
line gathered from standard input is read.
.CW $ifs
is set to just a newline. Thus
.CW read 's
input is not split apart at spaces, but the terminating
newline is deleted.
.PP
A handler is set to catch
.CW sigint ,
.CW sigquit ,
and
.CW sighup,
and the artificial
.CW sigexit
signal. It just removes the temporary file and exits.
.PP
The temporary file is initialized from a here
document containing a list of Bell Labs locations, and
the main loop starts.
.PP
First, the program guesses a location (in
.CW $lab )
using the
.CW fortune
program to pick a random line from the location list.
It prints the location, and if it guessed Holmdel, prints
a message and exits.
.PP
Then it uses the
.CW read
function to get lines from standard input and validity-check
them until it gets a legal name.
Note that the condition part of a
.CW while
can be a compound command. Only the exit status of the
last command in the sequence is checked.
.PP
Again, if the result
is Holmdel, it prints a message and exits.
Otherwise it goes back to the top of the loop.
.NH
Design Principles
.PP
.I Rc
draws heavily from Steve Bourne's
.CW /bin/sh .
Any successor of the Bourne shell is bound to
suffer in comparison. I have tried to fix its
best-acknowledged shortcomings and to simplify things
wherever possible, usually by omitting inessential features.
Only when irresistibly tempted have I introduced novel ideas.
Obviously I have tinkered extensively with Bourne's syntax.
.PP
The most important principle in
.I rc 's
design is that it's not a macro processor. Input is never
scanned more than once by the lexical and syntactic analysis
code (except, of course, by the
.CW eval
command, whose
.I "raison d'être
is to break the rule).
.PP
Bourne shell scripts can often be made
to run wild by passing them arguments containing spaces.
These will be split into multiple arguments using
.CW IFS ,
often at inopportune times.
In
.I rc ,
values of variables, including command line arguments, are not re-read
when substituted into a command.
Arguments have presumably been scanned in the parent process, and ought
not to be re-read.
.PP
Why does Bourne re-scan commands after variable substitution?
He needs to be able to store lists of arguments in variables whose values are
character strings.
If we eliminate re-scanning, we must change the type of variables, so that
they can explicitly carry lists of strings.
.PP
This introduces some
conceptual complications. We need a notation for lists of words.
There are two different kinds of concatenation, for strings \(em
.CW $a^$b ,
and lists \(em
.CW "($a $b)" .
The difference between
.CW ()
and
.CW ''
is confusing to novices,
although the distinction is arguably sensible \(em
a null argument is not the same as no argument.
.PP
Bourne also rescans input when doing command substitution.
This is because the text enclosed in back-quotes is not
a string, but a command. Properly, it ought to
be parsed when the enclosing command is, but this makes
it difficult to
handle nested command substitutions, like this:
.P1
size=`wc -l \e`ls -t|sed 1q\e``
.P2
The inner back-quotes must be escaped
to avoid terminating the outer command.
This can get much worse than the above example;
the number of
.CW \e 's
required is exponential in the nesting depth.
.I Rc
fixes this by making the backquote a unary operator
whose argument is a command, like this:
.P1
size=`{wc -l `{ls -t|sed 1q}}
.P2
No escapes are ever required, and the whole thing
is parsed in one pass.
.PP
For similar reasons
.I rc
defines signal handlers as though they were functions,
instead of associating a string with each signal, as Bourne does,
with the attendant possibility of getting a syntax error message
in response to typing the interrupt character. Since
.I rc
parses input when typed, it reports errors when you make them.
.PP
For all this trouble, we gain substantial semantic simplifications.
There is no need for the distinction between
.CW $*
and
.CW $@ .
There is no need for four types of quotation, nor the
extremely complicated rules that govern them. In
.I rc
you use quotation marks when you want a syntax character
to appear in an argument, or an argument that is the empty string,
and at no other time.
.CW IFS
is no longer used, except in the one case where it was indispensable:
converting command output into argument lists during command substitution.
.PP
This also avoids an important UNIX security hole.
In UNIX, the
.I system
and
.I popen
functions call
.CW /bin/sh
to execute a command. It is impossible to use either
of these routines with any assurance that the specified command will
be executed, even if the caller of
.I system
or
.I popen
specifies a full path name for the command. This can be devastating
if it occurs in a set-userid program.
The problem is that
.CW IFS
is used to split the command into words, so an attacker can just
set
.CW IFS=/
in his environment and leave a Trojan horse
named
.CW usr
or
.CW bin
in the current working directory before running the privileged program.
.I Rc
fixes this by never rescanning input for any reason.
.PP
Most of the other differences between
.I rc
and the Bourne shell are not so serious. I eliminated Bourne's
peculiar forms of variable substitution, like
.P1
echo ${a=b} ${c-d} ${e?error}
.P2
because they are little used, redundant and easily
expressed in less abstruse terms.
I deleted the builtins
.CW export ,
.CW readonly ,
.CW break ,
.CW continue ,
.CW read ,
.CW return ,
.CW set ,
.CW times
and
.CW unset
because they seem redundant or
only marginally useful.
.PP
Where Bourne's syntax draws from Algol 68,
.I rc 's
is based on C or Awk. This is harder to defend.
I believe that, for example
.P1
if(test -f junk) rm junk
.P2
is better syntax than
.P1
if test -f junk; then rm junk; fi
.P2
because it is less cluttered with keywords,
it avoids the semicolons that Bourne requires
in odd places,
and the syntax characters better set off the
active parts of the command.
.PP
The one bit of large-scale syntax that Bourne
unquestionably does better than
.I rc
is the
.CW if
statement with
.CW "else
clause.
.I Rc 's
.CW if
has no terminating
.CW fi -like
bracket. As a result, the parser cannot
tell whether or not to expect an
.CW "else
clause without looking ahead in its input.
The problem is that after reading, for example
.P1
if(test -f junk) echo junk found
.P2
in interactive mode,
.I rc
cannot decide whether to execute it immediately and print
.CW $prompt(1) ,
or to print
.CW $prompt(2)
and wait for the
.CW "else
to be typed.
In the Bourne shell, this is not a problem, because the
.CW if
command must end with
.CW fi ,
regardless of whether it contains an
.CW else
or not.
.PP
.I Rc 's
admittedly feeble solution is to declare that the
.CW else
clause is a separate statement, with the semantic
proviso that it must immediately follow an
.CW if ,
and to call it
.CW "if not
rather than
.CW else ,
as a reminder that something odd is going on.
The only noticeable consequence of this is that
the braces are required in the construction
.P1
for(i){
if(test -f $i) echo $i found
if not echo $i not found
}
.P2
and that
.I rc
resolves the ``dangling else'' ambiguity in opposition
to most people's expectations.
.PP
It is remarkable that in the four most recent editions of the UNIX system
programmer's manual the Bourne shell grammar described in the manual page
does not admit the command
.CW who|wc .
This is surely an oversight, but it suggests something darker:
nobody really knows what the Bourne shell's grammar is. Even examination
of the source code is little help. The parser is implemented by recursive
descent, but the routines corresponding to the syntactic categories all
have a flag argument that subtly changes their operation depending on the
context.
.I Rc 's
parser is implemented using
.I yacc ,
so I can say precisely what the grammar is.
.NH
Acknowledgements
.PP
Rob Pike, Howard Trickey and other Plan 9 users have been insistent, incessant
sources of good ideas and criticism. Some examples in this document are plagiarized
from [Bourne],
as are most of
.I rc 's
good features.
.NH
Reference
.LP
S. R. Bourne,
UNIX Time-Sharing System: The UNIX Shell,
Bell System Technical Journal, Volume 57 number 6, July-August 1978