When writing to a large variable in /env, we
spam 'Write error', in spite of the env var
working. This is new behavior, as of commit
b90036a062.
It produces a lot of scary, noisy warnings,
which are probably bogus.
Pattern matching with lists no longer works:
; ls /tmp/*.c
/tmp/npage.c
/tmp/pagedebug.c
/tmp/pageold.c
/tmp/scheduler.c
/tmp/writeimagetest.c
; ls /tmp/^(*.c)
ls: /tmp/*.c: '/tmp/*.c' directory entry not found
; 9fs dump
; bind /n/dump/2021/1002/amd64/bin/rc /bin/rc
; rc
; ls /tmp/^(*.c)
/tmp/npage.c
/tmp/pagedebug.c
/tmp/pageold.c
/tmp/scheduler.c
/tmp/writeimagetest.c
the fix:
we have to propagate the glob attribute thru lists
as well. before it was only handled for single words
and propagated thru concatenations...
the Xglob instruction now works on list, and we
propagate the glob attribute thru PAREN and WORDS
and ARGLIST nodes.
also, avoid using negative numbers for the Tree.glob
field as char might be unsigned on some targets.
Eckard's test case:
cat <<! | cat
asdf
!
The issue is that we have to continue parsing until we see
the '\n' before consuming the here document.
So we revert to the old approach of having two functions:
heredoc() which remembers if we'v seen a heredoc redirection
and a second readhere() function that reads the doc from
the lexers input and sets Tree.str on thee REDIR node.
Fixup remaining Plan9 dependencies (chartorune()).
Add Makefile for UNIX-like systems (tested with Linux and APE).
Make error printing consistent, use Errstr() explicitely.
Get rid of NSTATUS buffer limit, just malloc it.
Untangle the lexer and interpreter thread state.
Fix the file and line number error reporting, getting rid of
Xsrcfile instruction, as the whole code block can only come
from a single file, stuff the source file in slot[1] of the
code block instead.
Remove limitations for globber (path element limits)
and be more intelligent about handling globbing by
inserting Xglob instruction only when needed and not
run it over every Xsimple argument list.
Remove fragile ndot magic and make it explicit by adding
the -q flag to . builtin command.
Add -b flag for full compilation.
Make exitnext() smart, so we can speculate thru rcmain and
avoid the fork().
Get rid of all print(2) format functions and use io
instead.
Improve the io library, adding rstr() to handle tokenization,
which allows us to look ahead in the already read buffer
for the terminators, avoiding alot of string copies.
Auto indent pcmd(), to make line number reporting more usefull.
Implement here documents properly, so they can work everywhere.
To reproduce run the following on a terminal:
<snip>
cpu% leak -s `{pstree | grep termrc | sed 1q | awk '{print $1}'}
src(0x00209a82); // 12
src(0x0020b2a6); // 1
cpu% acid `{pstree | grep termrc | sed 1q | awk '{print $1}'}
/proc/358/text:amd64 plan 9 executable
/sys/lib/acid/port
/sys/lib/acid/amd64
acid: src(0x0020b2a6)
/sys/src/cmd/rc/plan9.c:169
164 if(runq->argv->words == 0)
165 poplist();
166 else {
167 free(runq->cmdfile);
168 int f = open(runq->argv->words->word, 0);
>169 runq->cmdfile = strdup(runq->argv->words->word);
170 runq->lexline = 1;
171 runq->pc--;
172 popword();
173 if(f>=0) execcmds(openfd(f));
174 }
acid:
</snap>
Another `runq->cmdfile` leak is present here (captured on a cpu server):
<snip>
277 ├listen [tcp * /rc/bin/service <nil>]
321 │├listen [/net/tcp/2 tcp!*!80]
322 │├listen [/net/tcp/3 tcp!*!17019]
324 ││└rc [/net/tcp/5 tcp!185.64.155.70!3516]
334 ││ ├rc -li
382 ││ │└pstree
336 ││ └rc
338 ││ └cat
323 │└listen [/net/tcp/4 tcp!*!17020]
278 ├listen [tcp * /rc/bin/service.auth <nil>]
320 │└listen [/net/tcp/1 tcp!*!567]
381 └closeproc
cpu% leak -s 336
src(0x00209a82); // 2
src(0x002051d2); // 1
cpu% acid 336
/proc/336/text:amd64 plan 9 executable
/sys/lib/acid/port
/sys/lib/acid/amd64
acid: src(0x002051d2)
/sys/src/cmd/rc/exec.c:1056
1051
1052 void
1053 Xsrcfile(void)
1054 {
1055 free(runq->cmdfile);
>1056 runq->cmdfile = strdup(runq->code[runq->pc++].s);
1057 }
acid:
</snap>
These leaks happen because we do not free cmdfile on all execution paths
where `Xreturn()` is invoked. In `/sys/src/cmd/rc/exec.c:/^Xreturn`
<snip>
void
Xreturn(void)
{
struct thread *p = runq;
turfredir();
while(p->argv) poplist();
codefree(p->code);
runq = p->ret;
free(p);
if(runq==0)
Exit(getstatus());
}
</snip>
Note how the function `Xreturn()` frees a heap allocated instance of type
`thread` with its members *except* the `cmdfile` member.
On some code paths where `Xreturn()` is called there is an attempt to free
`cmdfile`, however, there are some code paths where `Xreturn()` is called
where `cmdfile` is not freed, leading to a leak.
The attached patch calls `free(p->cmdfile)` in `Xreturn()` to avoid leaking
memory and handling the free in one place.
After applying the patch this particular leak is removed. There are still
other leaks in rc:
<snip>
277 ├listen [tcp * /rc/bin/service <nil>]
321 │├listen [/net/tcp/2 tcp!*!80]
322 │├listen [/net/tcp/3 tcp!*!17019]
324 ││└rc [/net/tcp/5 tcp!185.64.155.70!3516]
334 ││ ├rc -li
382 ││ │└pstree
336 ││ └rc
338 ││ └cat
323 │└listen [/net/tcp/4 tcp!*!17020]
278 ├listen [tcp * /rc/bin/service.auth <nil>]
320 │└listen [/net/tcp/1 tcp!*!567]
381 └closeproc
cpu% leak -s 336
src(0x00209a82); // 2
src(0x002051d2); // 1
cpu% acid 336
/proc/336/text:amd64 plan 9 executable
/sys/lib/acid/port
/sys/lib/acid/amd64
acid: src(0x00209a82)
/sys/src/cmd/rc/subr.c:9
4 #include "fns.h"
5
6 void *
7 emalloc(long n)
8 {
>9 void *p = malloc(n);
10 if(p==0)
11 panic("Can't malloc %d bytes", n);
12 return p;
13 }
14
</snap>
To help fixing those leaks emalloc(…) and erealloc(…) have been amended to use
setmalloctag(…) and setrealloctag(…). The actual fixes for other reported leaks
are *not* part of this merge and will follow.
fn foo @{bar} is now equivalent to
fn foo {@{bar}}. As a side effect,
this disallows creating functions
named after keywords without first
quoting them.
We weren't correctly skipping the location operators
in codefree. This would mostly be work, but sometimes
you'd get unlucky and have one of the argmuents mismatch,
and that would lead to an invalid free.
This correctly skips the args in codefree.
When loading a file using ".", we could
end up with our line numbers thrown off
due to the mutation of lexline. Putting
lexline into the runq beside the file
that we're reading from causes it to get
pushed and popped correctly, so that we
no longer lose track of our location.
basically, we want the following commands to print
the same pid twice:
rc -c 'cat /dev/pid &;echo $apid'
vs:
rc -c 'a=1 cat /dev/pid &;echo $apid'
basically, Xsimple() calls exitnext() to determine if
a simple command should be promoted to exec, by peeking
ahead into the code and searching for Xexit instruction.
Xexit might not follow immediately after the Xsimple
instruction because of redirections, which exitnext()
would skip.
but it would not skip the Xunlocal instructions that
where added by the variable assignment.
BurnZeZ reported this the other day. It seems like if we have
a pipeline that looks like:
fn foo{cat < <{echo hi}}
then the '<' will get merged in /env/'fn#foo'. This change
fixes pcmd to add a space. It looks to me like this is the
only token that can get merged this way by pcmd.
we emitted an error on heredoc tags, but we
continued on, and added a heredoc entry to
the list, with a tag that we couldn't handle.
when processing this heredoc, rc would segfault.
fix: don't add a heredoc to the list on error.
rfork F closes all file descriptors, so we have to
invalidate the redirections as they are now refering
to closed files. not doing so causes the wrong file
descriptors being closed later on as the fd numbers
get reused.
to get $"1 right, remove Xqdol() and instead implement it in
terms of Xdol() instruction and use the new Xqw() instruction
to quote the resulting list.
Xpipefd wants the pipe descriptor to be closed in turfredir(), so
it pushes the redirection, but this breaks Xpopredir after normal
redirection. so we shuffle the Xpipefd redir to the bottom of the
stack.
add glob information to the word structure so we wont accidently
deglob quoted strings containing the GLOB. we store Globsize(word)
in in word->glob which avoids recalculating that values and the
check if a word should be globbed quick.
globlist() now substitutes the word inplace avoiding the copying
when all words are literals and avoids recursion.
minor cleanups: use list2str() in execeval(), move octal() to
unix.c, remove the (char*) casts to efree().
The execexec() function should never return, as it irreversably changes
the filedescriptor table for the new program. This means rc's internal
filedesciptors for reading the script get implicitely closed and we cannot
continue the rc interpreter when Execute() fails. So Execute() now sets the
error status, and execexec() runs Xexit() in case Execute() returns.
Xqdol() used to take quadratic time because of strcat(),
the code isnt really needed as list2str() aready does the
same thing in linear time without the strcat().
add estrdup() which uses emalloc() so allocation error are
catched.
move strdups() of name from callers into newvar().
avoid recursion of conclist(), and avoid copying of word
strings by providing Newword() function which doesnt copy
the word string.
on races... normal forks will all share the /env environment but
not the in memory variables of rc. so when we would normally fork
whoever does an exec (flush) first will override what the values of the
/env variables are, *independent* of the variables that where
actually modified *in* the process.
when we flush *before* fork, then at least both processes start out
with marked clean in memory variables and the processes will flush
only the things they actually change.
