Page 1 of 1
shell-like scripting
Posted: Tue Jul 11, 2017 7:02 am
by xificurC
Looking at
http://www.newlisp.org/downloads/newlis ... #processes I see there are several ways to start an external process, namely !, exec, process and spawn. So I thought let's write my typical toy tool - parallel git pull. It's a simple script that gets a set of root folders, looks for subfolders that are git projects and runs a git pull inside them. This should run in parallel since most of the time is spent on waiting for the network. The output to the console should be updated live and each line should be a simple "{project-folder} finished pulling with exit code {exit-code}".
So I need the bash equivalent of something like "git pull &>/dev/null; exitCode=$?", i.e. suppress all output and fetch the exit code. I see a combination of process, exec+which and wait-pid can do this job.
Having this explained my question is - can I retrieve this output live from subprocesses with the Cilk API without resorting to send/receive? I see that the result of a subprocess is stored in the symbol one gives to spawn but I don't see a reasonable way to fetch that information using sync. Am I missing something?
Re: shell-like scripting
Posted: Tue Jul 11, 2017 1:19 pm
by ralph.ronnquist
As you know, one way to pass command return code is via
! and
wait-pid, the former captures the return code of the invoked shell command, and the latter of a child (though the function return values need downshifting 8 bits to be the actual return code). I made the following toy example to experiment with this
Code: Select all
(constant 'commands
'( "sleep 3; exit 1" "sleep 2; exit 2" sleep 2; exit 3" ))
(define (psub x)
(fork (exit (>> (! (format "%s >/dev/null 2>&1" x)) 8))))
; Set up a table of forked pids and their commands
(setf subs (map list (map psub commands) commands))
; Process child return values as they come about
(dolist (p subs)
(let ((x (wait-pid -1)))
(println (lookup (x 0) subs) " ended with " (>> (x 1) 8))))
Perhaps you envisioned something like that?
Re: shell-like scripting
Posted: Tue Jul 11, 2017 8:12 pm
by xificurC
Yes, that pretty much does it, thanks! As a nitpick - is there no way to wait-pid for something that is forked only once? Looking at the sources ! uses system which is a fork+exec sh -c and you need to wrap that in another fork to get the pid. And why is the exit code shifted?
Re: shell-like scripting
Posted: Tue Jul 11, 2017 9:46 pm
by ralph.ronnquist
The
! function waits for the command to complete, so without the outer
fork, only one sub process at a time would be performed. You could redesign it to use
process, which includes its own
fork+exec, to avoid the explicit outer
fork (and the first explicit right-shifting); maybe that'd be more succinct, really. Though in practice you may well need an explicit
fork anyhow so as to deal with that all sub processes share stdin/out/err with the parent process.
The right shifting is needed because the return value is that of the
wait system call, which in short, combines the process return code with a control byte. See
for the gory details. (i.e. look up the "wait(2)" man page). Thus, the right shifting discards the control byte and becomes the actual exit code.
Re: shell-like scripting
Posted: Wed Jul 12, 2017 9:47 am
by xificurC
Yeah, I just felt a bit of redundancy, ! is synchronous which means it calls fork, exec and wait (or something similar, not sure how system is coded exactly). Then we call another fork and wait. But as you said process could solve this, although that requires a command-string which means I either have to go {sh -c "..."} or find my executable (e.g. git) with an exec which first. I would probably prefer having access to execlp directly in some cases :)
Anyway, thank you for your help, my script works flawlessly now ;)