pygos/init
1
0
Fork 0
mirror of https://github.com/pygos/init.git synced 2024-11-24 12:00:42 +01:00
Code Issues Releases Wiki Activity

Cleanup and update documentation

Browse source 
Signed-off-by: David Oberhollenzer <david.oberhollenzer@tele2.at>
This commit is contained in:
David Oberhollenzer 2018-08-22 00:43:11 +02:00
parent 066efaa33e
commit 0ed964c8a5
8 changed files with 568 additions and 167 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

71
README.md
View file

@ -1,43 +1,66 @@
# About
This directory contains the source code for a tiny init devised for
the Pygos system.
This directory contains the source code for a tiny service supervision
framework devised for the Pygos system, consisting of an init daemon,
a small syslog daemon and various command line utilities.
The main goal of this project is to create a simple framework for:
- system boot up and initialization
- service supervision
With the additional aims of having something that:
- simply works
- is easy to understand
- is easy to configure and maintain
The individual parts of the framework are designed to be independent of
each other (for instance, the tiny syslogd is intended to work with *any*
init system and other components of the framework don't depend on the presence
of this *specific* syslog implementation) and everything that is not strictly
part of the init system (such as the syslog daemon) can be disabled through
the configure script.
The init process is intended to run on top of Linux and makes use of some
Linux specific features (e.g. signalfd), but if sufficient interest exists,
it should still be possible to make it run on some BSDs or whatever else.
The programs of this package are developed first and foremost for GNU/Linux
systems, so there are some GNU and some Linux extensions used and some of the
code may unintentionally rely on Linux specific behavior.
Nevertheless, if sufficient interest exists, it should be possible to make it
run on BSDs or other Unix-like systems, but some effort may be required.
The init system tries to mimic the concept of unit files from systemd as those
were considered to be a good design choice.
Those parameterizeable service description files are stored in `/usr/share/init`
by default. Services are enabled by creating a symlink in `/etc/init.d`. This
can be done using the `service` command line tool.
In a typical setup, the parameterizeable service description files are stored
in `/usr/share/init` by default. Services are enabled by creating a symlink
in `/etc/init.d`. This can be done more conveniently using the `service`
command line tool.
See [docs/services.md](docs/services.md) for more information on service
description files.
See [docs/bootup.md](docs/bootup.md) for more information on what the init
daemon does during system boot.
A default setup is provided, as needed for the Pygos system, including helper
scripts for setting up mount points and for network configuration. If you want
to use the init daemon for another system, you may have to toss out or adapt
some of the default configuration and make your own.
Right now, the system is in a "basically works" proof of concept stage and
needs some more work to become usable.
There are plans for *maybe* *eventually* adding support for Linux name
spaces, seccomp filters and cgroups as needed in the medium future.
There are plans for *maybe* *eventually* adding more fancy features like
support for Linux name spaces, seccomp filters and cgroups or network
back ends for the syslog daemon, but right now, features are added only
when the need arises.
See [docs/init.md](docs/init.md) for more information on the design,
implementation and caveats of the init daemon.
See [docs/cmdline.md](docs/cmdline.md) for an explanation on the available
command line tools.
See [docs/services.md](docs/services.md) for more information on service
description files.
See [docs/network.md](docs/network.md) for information on how the network
configuration works.
See [docs/defconfig.md](docs/defconfig.md) for an explanation on the default
services and configuration provided with this package.
See [docs/usyslogd.md](docs/usyslogd.md) for details on the tiny syslog
implementation.
## Why

74
docs/bootup.md
View file

@ -1,74 +0,0 @@
# System Bootup Process
## Initial Ram Disk to Rootfs transition
After mounting the root filesystem, either the kernel or the initial ram disk
startup process is expected to exec the init program from the root filesystem.
At the current time, there is no support for re-scanning the service files
*yet*, so when init is started, the final configuration in `/etc/init.d` has to
be present. As a result, we currently cannot perform mounting of `/etc/` or
packing init into the initial ram disk and doing the rootfs transition.
Also, as a result of this, changing the service configuration requires a system
reboot to be effective.
This _will_ change in the future.
## Processing Service Descriptions
The init process reads service description files from `/etc/init.d` which are
usually symlinks to actual files in `/usr/share/init`.
The exact locations may be changed through configure flags when compiling init.
Service files specify a *target* which is basically like a SystemV runlevel and
can be one of the following:
* boot
* reboot
* shutdown
* ctrlaltdel
After parsing the configuration files, the init process starts running the
services for the `boot` target in a topological order as determined by their
*before* and *after* dependencies.
Services can be of one of the following *types*:
* wait
* once
* respawn
Services of type `wait` are started exactly once and the init process waits
until they terminate before continuing with other services.
The type `once` also only runs services once, but immediately continues
starting other services in the mean time without waiting.
Services of type `respawn` also don't stall the init process and are re-started
whenever they terminate.
## Service Process Setup
If a service description contains only a single `exec` line, the init process
forks and then execs the command directly in the child process.
If the service description contains a `tty` field, the specified device file
is opened in the child process and standard I/O is redirected to it before
calling exec. Also, a new session is created.
If a service description contains multiple `exec` lines, the init process forks
off to a single child process that does the same setup as above, and then runs
the command lines sequentially by forking a second time for each one, followed
by an exec in the grand child and a wait in the original child.
If a single command line returns something other than `EXIT_SUCCESS`,
processing of multiple command lines is immediately stopped and the offending
exit status is returned to init.
The init process reads environment variables from `/etc/initd.env`.

43
docs/cmdline.md Normal file
View file

@ -0,0 +1,43 @@
# Available Command Line Tools
## The service command
The `service` utility can be used for control and administration of services.
It is composed of several sub commands run by issuing
`service <command> [arguments..]`.
Currently available service commands are:
* enable - enable a service with the given list of arguments.
* disable - disable a service. If the service is parameterized, requires the
same arguments used for enabling, to disable the specific instance of the
service.
* dumpscript - generate an equivalent shell script from the `exec` lines of
a service after applying all parameter substitutions.
* list - list all enabled service. A target can be specified to only list
services for the specified target.
* help - display a short help text and a list of available commands.
## shutdown and reboot
The programs `shudown` and `reboot` can be used to signal to the `init` program
that it should transition to the `shutdown` or `reboot` target respectively.
The option `-f` or `--force` can be used to by pass the init system entirely
and force a hard reset or power off by directly signalling the kernel.
Running any one of those programs requires superuser privileges.
## syslog
If the `usyslogd` service is built as part of this package, a program called
`syslog` is built that can be used from the command line to send syslog
messages.
This can for instance be used to produce log messages from shell scripts.
The log level, facility and identity string can be specified.
See `syslog --help` for more information.

106
docs/defconfig.md Normal file
View file

@ -0,0 +1,106 @@
# Default Service Configuration
## Pseudo Services
The default configuration contains a number of "pseudo services" in the boot
target that don't actually do anything but are merely used as anchors in
service dependencies, i.e. they indicate that some sort of milestone in the
boot sequence has been reached. Everything that is part of that milestone
specifies that it should be run *before* that pseudo service and everything
that requires that this milestone has been reached, specifies that it wants
to run afterwards.
The pseudo targets are (in the order that they are executed):
* vfs
All services that do mount point setup go before this, all service that
depend on the fully mounted rootfs go after this.
* sysinit
The system has reached a sane state, i.e. the hostname is set, the system
clock has a sane value, modules and kernel parameters are loaded, some
very basic, fundamental services are running (e.g. syslog).
Everything that is part of that setup process goes between `vfs` and
`sysinit`, everything that requires a sane setup goes *after* `sysinit`.
* network
Network configuration is done. All services that do network configuration
should position themselves between `sysinit` and `network`. Everything that
requires a fully configured networking setup should go *after* `network`.
## Default Bootup Services
This section outlines the services for the boot target that are enabled by
default.
The following services are enabled by default and run *before* the `vfs` target
for filesystem setup:
* procfs - mount `procfs` to `/proc` and try to mount additional pseudo
filesystems in `/proc` such as `binfmt_misc`
* tmpfs - mount a `tmpfs` to `/tmp`
* sysfs - mount `sysfs` to `/sys` and try to mount additional pseudo
filesystems in `/sys` (e.g. `securityfs`, `configfs`, ...)
* devfs - mount `devtmpfs` to `/dev`, try to mount additional pseudo
filesystems in `/dev` (e.g. `devpts`, `mqueue`, ...) and try to create
some additional device nodes and symlinks.
The following services are enabled by default and configured to run *after*
the `vfs` target and *before* the `sysinit` target:
* hostname - reload hostname `/etc/hostname`
* loopback - bring the loopback device up
* usyslogd - if the `usyslogd` services is compiled with this package, this
service is enabled by default and starts `usyslogd`.
* modules - iterate over the file `/etc/modules` and try to load each module
using modprobe.
* sysctl - restore kernel parameters using `sysctl --system`. See `sysctl(8)`
for a list of possible locations that the parameters are read from.
The following services are enabled by default and configured to run *after*
the `sysinit` target and *before* the `network` target:
* ifcfg - static network configuration
Does the static network configuration outlined in [network.md](network.md)
## Default Shutdown and Reboot Services
For the shutdown and reboot targets, the following services are executed:
* sigterm - send the SIGTERM signal to all processes and wait for 5 seconds
* sigkill - send the SIGKILL signal to all remaining processes
* ifdown - bring all network interfaces down
* sync - run the sync command
## Additional Services not Enabled by Default
* agetty - A parameterizeable, respawn type `agetty` service. The first
parameter is the terminal device that the getty should run on.
* dhcpcdmaster - If one or more network interfaces should be configured using
dhcpcd, this service starts a central `dhcpcd` master instance.
* dhcpcd - A parameterizeable single shot service that signals the `dhcpcd`
master that it should configure a specific interface. The first parameter
is the interface that should be configured by `dhcpcd`.
* dnsmasq - A respawn type service for the `dnsmasq` DNS and DHCP server.
* hostapd - If the system should operate a WIFI access point, this respawn
type service can be enabled to manage an instace of the `hostapd` program.
* unbound - A respawn type service that manages an instance of the `unbound`
name resolver.
* usyslogd - A respawn type service that manages an instance of the `usyslogd`
syslogd implementation that is part of this package.
* hwclock - If the system has a hardware clock, this service can restore the
kernels clock from the hardware at bootup, between the `vfs` and `sysinit`
targets.
* nft - If enabled, restores net filter table rules during boot.
* swclock - For systems that don't have a hardware clock, this service
restores a somewhat usable time from a file during boot.
* swclocksave - For systems that don't have a hardware clock, this service
saves the current time to a file during shutdown or reboot.

71
docs/init.md Normal file
View file

@ -0,0 +1,71 @@
# The Init Daemon
## Initial Ram Disk to Rootfs transition
After mounting the root filesystem, either the kernel or the initial ram disk
startup process is expected to exec the init program from the root filesystem.
At the current time, there is no support for re-scanning the service files
*yet*, so when init is started, the final configuration in `/etc/init.d` has to
be present. As a result, we currently cannot perform mounting of `/etc/` or
packing init into the initial ram disk and doing the rootfs transition.
Also, as a result of this, changing the service configuration requires a system
reboot to be effective.
This _will_ change in the future.
## Service Types and Targets
A service file has a *type*, specifying whether the service should be run once
or restarted when it terminates and a *target*, specifying when the service
should be run.
The *target* is similar to a runlevel in System V init. The init daemon
currently knows about the following targets:
* boot
* reboot
* shutdown
When `init` is run, it starts all the services for the `boot` target. From the
`boot` target it can transition to any other target and execute the services
for the specified target.
The `reboot` and `shutdown` targets cannot transition to any other target and
when invoked, cause initd to drop everything else it intended to do.
For the `reboot` and `shutdown` targets, respawn type processes are no longer
restarted when they terminate and once all services have been executed, the
`init` program performs a hard system reboot or power off.
The init program tries to capture the `CTRL+ALT+DELETE` key sequence (or its
local equivalent) and transitions to the reboot target if pressed.
## Service Configuration Rescan
TBD
## Control Socket and Signals
The `init` program catches the following signals:
* `SIGCHLD`
* `SIGINT`
* `SIGTERM`
The `SIGCHLD` handler implements standard process reaping. If a terminated
process belongs to one of the supervised services, the configured action is
taken (e.g. restarting it).
When `SIGINT` is caugth, `init` transitions to the `reboot` target. Similarly,
`SIGTERM` causes `init` to transition to the `shutdown` target.
For more complex tasks, `init` creates a control socket that the command line
tools included in this package can use. For the time being, the control socket
can only tell the init daemon to transition to the `reboot` or `shutdown`
target.

83
docs/network.md Normal file
View file

@ -0,0 +1,83 @@
# Static Network Configuration
The default configuration provides multiple services that perform network
initialization and static configuration using helper scripts that require
programs from the `iproute2` package.
Configuration files are typically stored in `/etc/netcfg/` (depending on
configure options).
Please note that the loopback device is treated specially and not included in
any of the network configuration outlined below. The loopback device is brought
up and configured by a dedicated service long before the network configuration
is done.
## Interface Renaming
If the `ifrename` service is enabled (it is disabled by default), network
interfaces are renamed based on a rule set stored in the file `ifrename`.
The file contains comma separated shell globing patterns for the current
interface name, MAC address and a prefix for the new interface name.
For each network interface, rules are processed top to bottom. If the first two
globing patterns apply, the interface is renamed. Interfaces with the same
prefix are sorted by mac address and a running index is appended to the prefix.
If none of the rules apply, the interface name is left unchanged.
The intent is, to provide a way to configure persistent, deterministic names for
at least all network interfaces that are permanently installed on a board.
Extension cards or external network adapters should be given a different prefix
to avoid changes in the order as they come and go.
## Interface Configuration
After interface renaming, for each network interface, the configuration path is
scanned for files with the same name as the interface.
Each successfully found configuration file is processed line by line, top to
bottom. Each line may contain a keyword, followed by multiple arguments.
The following keywords can be used to add IPv4 or IPv6 network addresses to
an interface:
* address
* addr
* ip
* ip6
* ipv6
Those commands are expected to be followed by an IPv4 or IPv6 address and
network mask.
Furthermore, the following commands can be used for configuring interface
parameters:
* `arp {on|off}`
* `multicast {on|off}`
* `mtu <value>`
* `offload [rx {on|off}] [tx {on|off}] [sg {on|off}] [tso {on|off}]`
* `offload [gso {on|off}] [gro {on|off}] [lro {on|off}] [rxvlan {on|off}]`
* `offload [txvlan {on|off}] [ntuple {on|off}] [rxhash {on|off}]`
* `offload [ufo {on|off}]`
## Route Configuration
After interface configuration is done, routes and rules are restored from a
file named `routes` in the same configuration path.
The file may contain lines starting with `route` or `rule`. Everything that
follows is passed on to `ip route add` or `ip rule add` respectively.
## Net Filter Tables
An additional service is provided that restores the nft rule set from
`/etc/nftables.rules`.

193
docs/services.md
View file

@ -1,84 +1,31 @@
# Service Files
Services that can be started and managed by init are described by service
description files stored in `/usr/share/init`.
The init process reads service descriptions from `/etc/init.d` which usually
contains symlinks to the actual service files which can be conveniently removed
or added to disable or enable services.
The init process actually reads from `/etc/init.d` which contains symlinks to
the actual service files.
Default services provided by this package are installed in `/usr/share/init`,
i.e. this is where the symlinks point to.
Note that the actual locations may be different, depending on the configure
flags used.
Enabling a service means adding a symlink, disabling means removing a symlink.
Service descriptions can be parameterized. The arguments are extracted from the
name of the symlink. Currently only 1 parameter is supported. The argument
value is separated from the service name by an '@' character in the symlink
name.
The file name of the sysmlink, excluding any parameters, is accepted as the
canonical name of a service and used when referring to it via command line
utilities or when injecting dependencies from a service file.
Below is an annotated example for a simple, service description for a
generic, parameterized agetty service:
## Syntax
#
# The text that init should print out when the status of the
# service changes.
#
# The '%0' is replaced with the first argument extracted from the
# symlink name.
#
description "agetty on %0"
#
# How to run the service. 'respawn' means restart the service when it
# terminates, 'once' means run it only once and continue with other
# services in the mean while, 'wait' means run it once, but block until
# it exits.
#
type respawn
#
# When to start the service. 'boot' means when booting the system. Other
# options are 'reboot', 'shutdown' and 'ctrlaltdel'. The system always
# starts into the 'boot' target and then later transitions to one of the
# others.
#
target boot
#
# A list of service names that must be started before this service can
# be run, i.e. this services needs to be started after those.
#
# This can only refer to generic names, not specific instances. For
# instance, you can say "after getty" to make sure a service comes up after
# all gettys are started, but you cannot specify "after agetty@tty1".
#
# Similar to 'after', there is also a 'before' keyword for specifying
# dependencies.
#
after sysinit
#
# The 'tty' directive specifies a file to which all I/O of the process is
# redirected. The specified device file is used as a controlling tty for
# the process and a new session is created with the service process as
# session leader.
#
# In this example, we derive the controlling tty from the service
# description argument.
#
tty "/dev/%0"
#
# The 'exec' directive specifies the command to execute in order to start
# the service. See in the example below on how to run multiple commands.
#
# Again we use the argument to specify what terminal our getty
# should run on.
#
exec agetty %0 linux
As can be seen in this simple example, each line in a service description is
made up of a keyword, followed by one or more arguments and terminated by a
line break.
Each line in a service description is made up of a keyword, followed by one
or more arguments and terminated by a line break.
Blank lines are ignored and shell-style comments can be used.
@ -92,6 +39,115 @@ done using a '%' sign, followed by the argument index. A '%' sign can be
escaped by writing '%%'.
## Targets and Types
Service files specify a *target* which is basically like a SystemV runlevel
and can be one of the following:
* boot
* reboot
* shutdown
After parsing the configuration files, the init process starts running the
services for the `boot` target in a topological order as determined by their
dependencies.
Services can be of one of the following *types*:
* wait
* once
* respawn
Services of type `wait` are started exactly once and the init process waits
until they terminate before continuing with other services.
The type `once` also only runs services once, but immediately continues
starting other services in the mean time without waiting. The init process
only waits for `once` types when transitioning to another target.
Services of type `respawn` also don't stall the init process and are re-started
whenever they terminate.
The keyword `limit` can be used a after `respawn` to specify how often a service
may be restarted before giving up.
## Dependencies
A service description file can specify dependencies using the keywords `after`
and `before`, each followed by a space separated list of service names.
The init program executes a service after all the services specified by the
`after` keyword have been started (type once or respawn) or have been completed
(type wait).
The `before` keyword injects dependencies in reverse, i.e. all services
specified after the `before` keyword are only executed after the service in
question has been started.
If a service specified by `after` or `before` does not exist, it is simply
ignored. This can occur for instance if the specified service is not enabled
at all in the current configuration.
## Running Services
If a service contains an `exec` line, the init process attempts to run it
using the `runsvc` helper program that sets up the environment, attempts to
execute the specified command line and passes the exit status back to the init
process.
If multiple exec lines are specified, `runsvc` executes them sequentially and
stops if any one returns a non-zero exit status.
The environment variables visible to the service processes are read
from `/etc/initd.env`.
If the service description contains a `tty` field, the specified device file
is opened by runsvc and standard I/O is redirected to it and a new session
is created. The keyword `truncate` can be used to make `runsvc` truncate the
file to zero size first.
For convenience, multiple exec lines can be wrapped into braces, as can be
seen in one of the examples below.
## Example
Below is an annotated example for a simple, service description for a
generic, parameterized getty service:
#
# The text that init should print out when the status of the
# service changes.
#
# The '%0' is replaced with the first argument extracted from the
# symlink name.
#
description "agetty on %0"
# Restart the getty when it terminates.
type respawn
# We want to spawn gettys when booting the system
target boot
# Only run this service after the 'sysinit' service is done
after sysinit
#
# Redirect all I/O of the process to this file. The specified device file
# is used as a controlling tty for the process and a new session is created
# with the service process as session leader.
#
# In this example, we derive the controlling tty from the service
# description argument.
#
tty "/dev/%0"
# In order to run the service, simply fire up the agetty program
exec agetty %0 linux
If a service should sequentially run multiple commands, they can be grouped
inside braces as can be seen in the following, abbreviated example:
@ -107,4 +163,3 @@ inside braces as can be seen in the following, abbreviated example:
mkdir /var/tmp -m 0755
mount --bind /cfg/preserve/var_lib /var/lib
}

94
docs/usyslogd.md Normal file
View file

@ -0,0 +1,94 @@
# Syslogd Implementation
A tiny syslogd implementation `usyslogd` is provided as part of this package.
It opens a socket in `/dev/log`, processes syslog messages and forwards the
parsed message to a modular backend interface.
Currently, there is only one implementation of the backend interface that dumps
the log messages into files in the processes working directory (by default
`/var/log`).
A simple log rotation scheme has been implemented.
## Security Considerations
By default, the daemon switches its working directory to `/var/log`. The
directory is created if it doesn't exist and the daemon always tries to
change its mode to one that doesn't allow other users (except group members)
to access the directory.
If told to so on the command line, the daemon chroots to the log directory.
By default, the daemon then tries to drop privileges by switching to user and
group named `syslogd` if they exist (any other user or group can be specified
on the command line; doing so causes syslogd to fail if they don't exist).
On a system that hosts accounts for multiple users that may be more or less
trusted, one may consider only giving system services access to the syslog
socket and not allowing regular users. Otherwise, a user may flood the syslog
daemon with messages, possibly leading to resource starvation, or (in the case
of size limited log rotation outlined below) to the loss of otherwise critical
log messages. Since this is not the primary target of the Pygos system, such
a mechanism is not yet implemented.
In case of a system where only daemons are running, the above mentioned
security measure is useless. If a remote attacker manages to get regular user
privileges, you already have a different, much greater problem. Also, a remote
attacker would have to compromise a local daemon that already has special
access to the syslog socket, which is again your least concern in this
scenario.
## Logrotation
The backend can be configured to do log rotation in a continuous fashion (i.e.
in a way that log messages aren't lost), or in a way where it drops old
messages. Furthermore, the backend can be configured to automatically do a log
rotation if a certain size threshold is hit.
If the `usyslogd` receives a `SIGHUP`, it tells the backend to do log rotation.
In the case of the size threshold, the backend is expected to do the rotation
on its own if the predetermined limit is hit.
## File Based Backend
The file based backend writes log messages to files in the current working
directory (by default `/var/log`), named either after the ident string (if
specified) or the facility name.
Log messages are prefixed with an ISO 8601 time stamp, optionally the facility
name (unless part of the file name), the log level and the senders PID. Each
of those fields is enclosed in brackets.
Log rotation in a continuous fashion means renaming the existing log file to
one suffixed with the current time stamp. Overwriting old messages renaming
the log file by appending a constant `.1` suffix.
## Default Configuration
The default service configuration limits the log file size to 8 KiB and
configures the daemon to overwrite old messages when rotating log files,
effectively limiting the amount of log data to 16 KiB per source or facility.
The intended use case in the Pygos system is logging to a ramdisk without
exhausting available memory.
## Possible Future Directions
In the near term future, the daemon probably requires more fine grained control
over logging such as setting a minimum log level or a way to configure limits
per facility or service.
In the medium term future, extended resource control using c-groups might be
a possibility.
Future directions may include adding other backends, such as forwarding the
log messages to a central server, for instance using syslog over UDP/TCP or
using the front end of some time series database.