System supervisor
- Shell 100%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
|
||
| definitions | ||
| examples | ||
| lib/services | ||
| scripts | ||
| tools | ||
| .gitignore | ||
| README | ||
| supervisor.talc | ||
services: the supervisor of the machine's services, and the libraries it
is made of. It reads a definition per service, starts them in the order
they need each other, knows when each is ready, keeps them running, hands
every program it starts the environment that says where the others are,
stops them in reverse when the machine goes down, and answers every other
program over a local socket: what is running, and how.
supervisor.talc the program: one isolate changes state, every
wait is an isolate that sends it a message
lib/services/definitions.talc (services definitions): the *.service files
and services.config, every clause checked
lib/services/graph.talc (services graph): needs, uses and provides
resolved; cycles and missing needs refused;
the start order, the closure of a name, the
dependents, what blocks a start
lib/services/supervise.talc (services supervise): one service as a state
machine of data and time -- events in,
actions out -- with the backoff, the give-up
and the grace in it
lib/services/ready.talc (services ready): the readiness checks -- a
socket that answers, or the service's own
word at a notify socket -- each an isolate
that reports once
lib/services/forms.talc (services forms): forms one a line, both
ways (../network's, copied)
lib/services/serve.talc (services serve): the socket, its clients
(../network's, copied)
lib/services/client.talc (services client): what a program imports to
ask -- services-list, services-start!, ...
tools/services.talc `services`: what is running, asked of the
supervisor; start, await, stop, restart,
reload, log, follow, watch, shutdown, reboot
definitions/ what the vm's image carries at /etc/services:
network, console, screen, and services.config
examples/ the libraries' tests: definitions, graph,
supervise, forms; examples/regress/ the
throwaway services the live regress runs and
examples/programs/ the programs they run
scripts/build.sh builds bin/supervisor, bin/services and the
throwaway programs into build/programs
scripts/regress.sh the tests, then the supervisor on the
throwaway services, asked over its socket
scripts/vm-services.sh ../vm's machine booted through the
supervisor on a serial line, `services`
asked at its prompt, (exit 0) taking it down
Build and check:
scripts/build.sh
scripts/regress.sh
scripts/vm-services.sh
which need the installed toolchain (compiler on PATH) and, for the last,
../vm built after this (scripts/build.sh there ships the supervisor) and
qemu.
RUNNING IT. On the vm, init (../vm/init.talc) does nothing but mount,
start the supervisor and wait for it: when the supervisor ends with 0
the machine is switched off, with 1 it restarts. The supervisor is
started with the definitions under /etc/services and listens at
/run/services/socket; everything it starts has SERVICES in its
environment naming that socket, which is how `services` and (services
client) find it; nothing assumes a path.
supervisor [-definitions <directory>] [-socket <path>] [-logs <directory>]
[-only <name> ...] [-seconds n] [-trace]
-logs is where a service's output goes when its definition does not say
(/run/log, one file per service): the output comes through a pipe to the
supervisor, which appends it and, past the definition's (log (most
bytes)) -- a megabyte unless said -- turns the file over to <name>.old
and begins again; a service on a terminal writes there instead. -only
starts only those at boot. -seconds takes
everything down after that long, for tests; -trace logs every message.
On the host, with no privilege:
bin/supervisor -definitions examples/regress -socket build/sock -logs build/log -seconds 8 &
SERVICES=build/sock bin/services
THE MODEL. A DEFINITION says how a service is run; a SERVICE is one as
the supervisor runs it. The supervisor is the parent of every service,
which runs in the foreground and never puts itself in the background: no
pidfiles, and the kernel says when one ends. A definition, <name>.service:
(service "network"
(run "/bin/network" "-profiles" "/etc/network" "-socket" "/run/network/socket")
(needs) ; names it waits for (symbols)
(uses screen) ; waited for only if asked for too
(provides net) ; names beyond its own
(ready (socket "/run/network/socket") (within 10)) ; or (ready started),
; or (ready (notify)): it says so itself
(environment ("NETWORK" "/run/network/socket")) ; for what comes after
(log "/run/log/network" (most 1048576)) ; default: <logs>/<name>, a megabyte
(restart always (backoff 1 30) (give-up 5)) ; or on-failure, never
(stop terminate 5) ; the signal, then seconds of grace
(start boot) ; or manual
(terminal console) ; the kernel's console= is its own
(directory "/root")
(root "/jail") ; all it can name; the program is a path inside
(on-end shutdown) ; the machine goes down when it ends
(user 1000) (group 1000))
Only (run ...) is required. services.config in the same directory holds
(services (environment ("PATH" "/bin") ...)), the base every environment
starts from; the supervisor adds SERVICES, then the (environment ...)
pairs of every running service in start order.
A service's states, as `services` shows them:
stopped defined, not running, not asked for
waiting asked for; a need is not ready, or a backoff is running
starting the process exists; the readiness check has not passed
running ready: what needs it may start
stopping told to stop; the grace period is running
failed ended, and the policy says not to try again, or the
give-up count is spent; `services start` resets it
done ended 0 and (restart never): a thing meant to run once
A service whose definition says (ready (notify)) is started with NOTIFY
in its environment naming a socket beside the supervisor's, and is ready
when it has connected there and said (ready) -- (services-notify-ready!)
from (services client), one call, the only line such a program needs.
A NEED is satisfied when a provider of it is running; a USE blocks only
while a provider is waiting or starting, so a desktop that uses the
network starts without it when the network has failed. Stopping a service
stops those that need it first (not those that merely use it), and the
service itself once they have gone; restarting it does the same and then
asks for them all again; shutting down stops everything the same way,
wave by wave from the services nothing running needs. A service
that ends while wanted is started again after the backoff (the first
wait, doubling, never more than the second), unless (restart never), or
on-failure and it ended 0, or it has failed give-up times in a row; a
run as long as the backoff's cap starts the count over. A service that
is not ready within its seconds is stopped and counts as a failure.
THE PROTOCOL, forms one a line over the socket; a request gets one answer:
(services) (services (service "network" running (seconds 12)
(pid 42) (restarts 0) (provides "network")
(ending #f) (start boot)) ...)
(start "x") (stop "x") (restart "x") (reload)
(ok), or (refused "start" "why")
(await "x") asks for x; answered (ready "x") once it is running,
however long that takes, or (refused "await" "why")
when it has failed -- a client that wants a service
on demand
(log "x" 20) (log (line "...") ...); (log "x" 20 40) the 20 lines
before the last 40, for paging back
(follow "x") (following), then (output "x" "text") as x writes it
(shutdown) (reboot) (ok); then everything stops in reverse order and
the supervisor ends with 0 or 1
(watch) (watching), then (changed "x" running "ready") as
they happen
A client that is not the supervisor's user may ask (services), (log ...),
(follow ...) and (watch) only. (services client) wraps it: (services-list),
(services-start! name), (services-await! name), (services-stop! name),
(services-shutdown!) ... The shell binds it, so (services-list) works at
the prompt; system-properties has a Services tab; and the desktop's
`shutdown` window (../desktop) asks (shutdown) or (reboot).