pygos/pkg-utils
1
0
Fork 0
mirror of https://github.com/pygos/pkg-utils.git synced 2024-06-16 09:18:43 +02:00
Code Issues Releases Wiki Activity

Compare commits

base: pygos:master
Branches Tags
pygos:master
pygos:v0.4.2
pygos:v0.4
pygos:v0.3
pygos:v0.2
pygos:v0.1
..
compare: pygos:v0.1
Branches Tags
pygos:master
pygos:v0.4.2
pygos:v0.4
pygos:v0.3
pygos:v0.2
pygos:v0.1

No commits in common. "master" and "v0.1" have entirely different histories.
master ... v0.1

62 changed files with 465 additions and 1752 deletions
Show stats Expand all files Collapse all files

1
.gitignore vendored
View file

@ -18,4 +18,3 @@ config.h
*.a
*~
pkg
pkg2sqfs

13
LICENSE
View file

@ -1,13 +0,0 @@
Copyright (c) 2019 David Oberhollenzer <goliath@infraroot.at>
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

2
Makefile.am
View file

@ -6,7 +6,7 @@ AM_CFLAGS = $(WARN_CFLAGS)
noinst_LIBRARIES =
bin_PROGRAMS =
EXTRA_DIST = autogen.sh LICENSE README.md
EXTRA_DIST = autogen.sh
include lib/Makemodule.am
include main/Makemodule.am

129
README.md
View file

@ -1,129 +0,0 @@
# About
This package contains the source code of a small archiving tool with support
for dependency management, aka a package manager, developed for the Pygos
build system.
Building this package produces a program called `pkg` that can:
* take a package description file and a set of input files and pack them to
a compressed archive.
* dump the contents and meta data of such an archive for inspection.
* install an archive and all its dependencies recursively, in topological
order, to a specified root directory.
* generate file listings from archives in formats suitable for `gensquashfs`
and Linux `CONFIG_INITRAMFS_SOURCE`.
* work out a build order for source packages given information on what source
packages provide what binary packages and what binary packages they need in
order to build.
* produce pretty dependency graphs.
## License
The source code in this package is provided under the OpenBSD flavored ISC
license. So you can practically do as you wish, as long as you retain the
original copyright notice. The software is provided "as is" (as usual) with
no warranty whatsoever (e.g. it might actually do what it was designed for,
but it could just as well set your carpet on fire).
The sub directory `m4` contains third party macro files used by the build
system which may be subject to their own, respective licenses.
## Package File Format
A writeup on the package file format can be found in
[doc/fileformat.md](doc/fileformat.md).
The format is currrently not finalized and will be frozen with the 1.0 release
of this package.
The `pkg` utility has an extensive built in help that can be accessed through
`pkg help` or `pkg help <command>`.
# Usage Example
An example package could be created from the following description file,
named `foobar.desc`:
requires coreutils libfrob basefiles
toc-compressor zlib
data-compressor lzma
In order to be installed, the package requires the other packages `coreutils`,
`libfrob` and `basefiles` to already be installed beforehand.
For whatever reason, we specify that the table of contents should be compressed
with zlib and the data with lzma, instead of using the built in default
choices.
In addition to the description file, we most likely also need want to include
some files in the package, so we create a file listing `foobar.files`:
# this is a comment, outlining commentary syntax and explaining
# that we create a bunch of directories with permission bits 755,
# owned by the user root
dir bin 0755 0 0
dir dev 0755 0 0
dir home 0755 0 0
# this directory is owned by a different user and group
dir home/goliath 0750 1000 1000
# create a symlink named /root that points to /home/goliath
slink root 0777 0 0 /home/goliath
# create a file /home/goliath/README.md from the input file
# README.md stored relative to the foobar.files
file home/goliath/README.md 0644 1000 1000 README.md
# create a file named /bin/foobar. Since we omit the input file path,
# assume the file ./bin/foobar also exists relative to foobar.files
file bin/foobar 0755 0 0
# create a character device with permissions 600, owned by 0:0,
# device number major 5, minor 1
nod dev/console 0600 0 0 c 5 1
Running the following command:
pkg pack -d foobar.desc -l foobar.files -r ./repodir
Creates a file `./repodir/foobar.pkg` from the package description and file
listing. Note that the files in the listing are the only things that actually
have to exist anywhere in the real file system to create the archive.
Lets say, we want to install `foobar` and all its dependencies recursively
into a staging root directory. Running the following command is sufficient:
pkg install -r ./rootfs -R ./repodir foobar
This installs `foobar.pkg` inside `./repodir`. Since `foobar` depends on
`coreutils`, `libfrob` and `basefiles`, the archive files `coreutils.pkg`,
`libfrob.pkg` and `basefiles.pkg` are also read from `./repodir` and
installed to `./rootfs`. The same way, all transitive dependencies are
installed recursively.
Assuming we are not running as root, the above command will tell us that it
cannot create device nodes while installing and it has trouble changing
permission/ownership on some file.
So instead, we could run the following command, to keep ownership of the user
that ran the command, use default permissions and not create device files:
pkg install -omD -r ./rootfs -R ./repodir foobar
For more information on possible options, simply run `pkg help install`.
Now lets assume we want to pack the staging root directory into a squashfs
file system.
The `mksquashfs` command is a bit silly in regards to managing user IDs of
files, but has half-assed support for what it calls "pseudo files" that let
us set arbitrary user IDs, permissions and create device files, even as a
regular user.
So we run the following command to generate a squashfs pseudo file for the
installed packages:
pkg install -l sqfs -r ./rootfs -R ./repodir foobar > pseudo.txt

17
configure.ac
View file

@ -1,5 +1,5 @@
AC_PREREQ([2.60])
AC_INIT([pkgtool], [0.4.2], [goliath@infraroot.at], pkgtool)
AC_INIT([pkgtool], [0.1], [david.oberhollenzer@tele2.at], pkgtool)
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign dist-xz subdir-objects])
AM_SILENT_RULES([yes])
@ -39,21 +39,6 @@ UL_WARN_ADD([-pedantic])
AC_SUBST([WARN_CFLAGS])
##### configuration options #####
AC_DEFINE([REPODIR], ["./"], [Default package repository directory])
AC_DEFINE([INSTALLROOT], ["./"], [Default package installation root])
AC_ARG_WITH([repo-dir],
[AS_HELP_STRING([--with-repo-dir],
[Set the default package repository directory])],
[AC_DEFINE_UNQUOTED([REPODIR], "$withval")])
AC_ARG_WITH([install-root],
[AS_HELP_STRING([--with-install-root],
[Set the default package installation directory])],
[AC_DEFINE_UNQUOTED([INSTALLROOT], "$withval")])
##### search for dependencies #####
have_zlib="no"

167
doc/fileformat.md
View file

@ -1,167 +0,0 @@
# Package File Format
## Record Structure
A package file consists of a series of records with possibly compressed
payload. Each record has a header, indicating the type of record, raw and
compressed size of the payload data and the compression algorithm used.
All multi byte integers are stored in little endian byte order.
Knowledge of the payload size lets a decoder skip unknown record types inserted
by an encoder that supports a newer version of the format, allowing for some
degree of backwards compatibility.
The diagram below illustrates what a record header looks like. The horizontally
arranged numbers indicate a byte offset inside a 32 bit word and the vertical
numbers count 32 bit words from the start of the header.
0 1 2 3
+-------+-------+-------+-------+
0 | magic |
+-------+-------+-------+-------+
1 | comp |reserved for future use|
+-------+-------+-------+-------+
2 | |
| compressed size |
3 | |
+-------+-------+-------+-------+
4 | |
| uncompressed size |
5 | |
+-------+-------+-------+-------+
6 | |
. | payload |
. | |
|/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
| |
|_______________________________|
The field `magic` holds a 32 bit magic number indicating the chunk type.
Currently, the following chunk types are supported:
* `PKG_MAGIC_HEADER` with the value `0x21676B70` (ASCII "pkg!"). The overall
package header with information about the package.
* `PKG_MAGIC_TOC` with the value `0x21636F74` (ASCII "toc!"). The table of
contents record.
* `PKG_MAGIC_DATA` with the value `0x21746164` (ASCII "dat!"). The package data
record.
The byte labeled `comp` holds a compression algorithm identifier. Currently, the
following compression algorithms are supported:
* `PKG_COMPRESSION_NONE` with the value 0. The payload is uncompressed. The
compressed size of the record payload must equal the uncompressed size.
* `PKG_COMPRESSION_ZLIB` with the value 1. The record payload area contains a
raw zlib stream.
* `PKG_COMPRESSION_LZMA` with the value 2. The record payload area contains
lzma compressed data.
The compressor ID is padded with 3 bytes that **must be set to zero** by an
encoder and are currently reserved for future use.
## Package Header Record
The header record must be present in ever package and must always be the first
record in a package file. If a decoder encounters a file which does not start
with the magic value of the header record, it must reject the file.
Future versions of the format that break backwards compatibility can simply
introduce a new magic value for the (possibly altered) header record. Older
decoders are expected to reject a file with the newer format, while newer
decoders can implement different behavior depending on what magic value they
find at the start.
The payload of header record is currently only used to store package
dependencies. It is byte aligned and contains a 16 bit integer, indicating the
number of dependencies, followed by a sequence of dependent packages.
Each dependent package is encoded as follows:
0 1 2 length - 1
+---------+---------+------- -+---------+
| type | length | name ... | |
+---------+---------+------- -+---------+
The type field must be set to 0, indicating that a dependency is required by a
package.
The length byte indicates the length of the package name that follows, allowing
for up to 255 bytes of package name afterwards.
If all dependencies have been processed, but there is still payload data left
in the header record, a decoder must ignore the extra data and skip to the end
of the record.
## Table of Contents Record
For each file, directory, symlink, et cetera contained in the package, the
table of contents contains an entry with the following common structure:
0 1 2 3
+-------+-------+-------+-------+
0 | mode | user id |
+-------+-------+-------+-------+
1 | group id | path length |
+-------+-------+-------+-------+
The mode field contains standard UNIX permissions. The user ID and group ID
fields contain the numeric IDs of the user and group respectively that own
the file. The path length field indicates the length of the byte aligned,
absolute file name that follows.
The file path is expected to neither start nor end with a slash, contain no
sequences of more than one slash and not contain the components `.` or `..`.
On the bit level, the mode field is structured as follows:
1 1 1 1 1 1
5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
type |U|G|S|r|w|x|r|w|x|r|w|x|
| | | | | | | | | | | | |
| | | | | | | | | | | | +--- others execute permission flag
| | | | | | | | | | | +----- others write permission flag
| | | | | | | | | | +------- others read permission flag
| | | | | | | | | +--------- group execute permission flag
| | | | | | | | +----------- group write permission flag
| | | | | | | +------------- group read permission flag
| | | | | | +--------------- owner execute permission flag
| | | | | +----------------- owner write permission flag
| | | | +------------------- owner read permission flag
| | | +--------------------- sticky bit
| | +----------------------- set GID bit
| +------------------------- set UID bit
+----------------------------- file type
Currently, the following file types are supported:
* `S_IFCHR` with a value of 2. The entry defines a character device.
* `S_IFDIR` with a value of 4. The entry defines a directory.
* `S_IFBLK` with a value of 6. The entry defines a block device.
* `S_IFREG` with a value of 8. The entry defines a regular file.
* `S_IFLNK` with a value of 10. The entry defines a symbolic link.
For character and block devices, the file path is followed by a byte aligned,
64 bit device number.
For regular files, the path is followed by a byte aligned, 64 bit integer
indicating the total size of the file in bytes, followed by a byte aligned,
32 bit integer containing a unique file ID.
For symlinks, the path is followed by a byte aligned, 16 bit integer holding
the length of the target path, followed by the byte aligned symlink target.
## Data Record
A package may contain multiple data records holding raw file data. Each data
record contains a sequence of a byte aligned, 32 bit file ID, followed by raw
data until the file is filled (size indicated by table of contents is reached).
A file may not span across multiple data records. A file ID must not occur
more than once in a data record and must only occur in a single data record.

1
main/command.h → include/command.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef COMMAND_H
#define COMMAND_H

14
include/comp/compressor.h → include/compressor.h
View file

@ -1,11 +1,10 @@
/* SPDX-License-Identifier: ISC */
#ifndef COMPRESSOR_H
#define COMPRESSOR_H
#include <stddef.h>
#include <sys/types.h>
#include "pkg/pkgformat.h"
#include "pkgformat.h"
typedef struct compressor_stream_t {
ssize_t (*write)(struct compressor_stream_t *stream, const uint8_t *in,
@ -24,14 +23,21 @@ typedef struct compressor_t {
const char *name;
PKG_COMPRESSION id;
compressor_stream_t *(*compression_stream)(struct compressor_t *cmp,
void *options);
compressor_stream_t *(*compression_stream)(struct compressor_t *cmp);
compressor_stream_t *(*uncompression_stream)(struct compressor_t *cmp);
} compressor_t;
void compressor_register(compressor_t *compressor);
compressor_t *compressor_by_name(const char *name);
compressor_t *compressor_by_id(PKG_COMPRESSION id);
#define REGISTER_COMPRESSOR(compressor) \
static void __attribute__((constructor)) register_##compressor(void) \
{ \
compressor_register((compressor_t *)&compressor); \
}
#endif /* COMPRESSOR_H */

2
include/filelist/image_entry.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef IMAGE_ENTRY_H
#define IMAGE_ENTRY_H
@ -31,6 +30,7 @@ typedef struct image_entry_t {
typedef enum {
TOC_FORMAT_PRETTY = 0,
TOC_FORMAT_SQFS = 1,
TOC_FORMAT_INITRD = 2,
TOC_FORMAT_PKG = 3,
} TOC_FORMAT;

8
include/pkg/pkgformat.h → include/pkgformat.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef PKG_FORMAT_H
#define PKG_FORMAT_H
@ -36,9 +35,10 @@ typedef struct {
} record_t;
typedef struct {
uint16_t mode;
uint16_t uid;
uint16_t gid;
uint32_t mode;
uint32_t uid;
uint32_t gid;
uint16_t path_length;
/* uint8_t path[]; */
} toc_entry_t;

1
include/pkg/pkgio.h → include/pkgio.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef PKGIO_H
#define PKGIO_H

1
include/pkg/pkgreader.h → include/pkgreader.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef PKGREADER_H
#define PKGREADER_H

3
include/pkg/pkgwriter.h → include/pkgwriter.h
View file

@ -1,11 +1,10 @@
/* SPDX-License-Identifier: ISC */
#ifndef PKGWRITER_H
#define PKGWRITER_H
#include <stdbool.h>
#include "comp/compressor.h"
#include "pkgformat.h"
#include "compressor.h"
typedef struct pkg_writer_t pkg_writer_t;

28
include/util/hashtable.h
View file

@ -1,28 +0,0 @@
/* SPDX-License-Identifier: ISC */
#ifndef HASH_TABLE_H
#define HASH_TABLE_H
typedef struct hash_bucket_t {
struct hash_bucket_t *next;
char *key;
void *value;
} hash_bucket_t;
typedef struct {
hash_bucket_t **buckets;
size_t num_buckets;
size_t count;
} hash_table_t;
int hash_table_init(hash_table_t *table, size_t size);
void hash_table_cleanup(hash_table_t *table);
void *hash_table_lookup(hash_table_t *table, const char *key);
int hash_table_set(hash_table_t *table, const char *key, void *value);
void hash_table_foreach(hash_table_t *table, void *usr,
int(*fun)(void *usr, const char *key, void *value));
#endif /* HASH_TABLE_H */

22
include/util/input_file.h
View file

@ -1,19 +1,27 @@
/* SPDX-License-Identifier: ISC */
#ifndef INPUT_FILE_H
#define INPUT_FILE_H
#include <stddef.h>
#include <stdio.h>
typedef struct {
FILE *f;
char *line;
const char *filename;
size_t linenum;
} input_file_t;
typedef struct {
const char *name;
int (*handle)(char *line, const char *filename,
size_t linenum, void *obj);
int (*handle)(input_file_t *f, void *obj);
} keyword_handler_t;
int process_file(const char *filename, const keyword_handler_t *handlers,
void cleanup_file(input_file_t *f);
int open_file(input_file_t *f, const char *filename);
int process_file(input_file_t *f, const keyword_handler_t *handlers,
size_t count, void *obj);
void input_file_complain(const char *filename, size_t linenum,
const char *msg);
void input_file_complain(const input_file_t *f, const char *msg);
#endif /* INPUT_FILE_H */

6
include/util/util.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef UTIL_H
#define UTIL_H
@ -12,9 +11,4 @@ ssize_t read_retry(int fd, void *buffer, size_t size);
int mkdir_p(const char *path);
typedef int (*linecb_t)(void *usr, const char *filename,
size_t linenum, char *line);
int foreach_line_in_file(const char *filename, void *usr, linecb_t fun);
#endif /* UTIL_H */

30
lib/Makemodule.am
View file

@ -2,37 +2,9 @@ libutil_a_SOURCES = lib/util/input_file.c lib/util/mkdir_p.c
libutil_a_SOURCES += lib/util/write_retry.c lib/util/read_retry.c
libutil_a_SOURCES += lib/util/canonicalize_name.c
libutil_a_SOURCES += include/util/util.h include/util/input_file.h
libutil_a_SOURCES += include/util/hashtable.h lib/util/hashtable.c
libutil_a_SOURCES += lib/util/fileproc.c
libfilelist_a_SOURCES = lib/filelist/dump_toc.c lib/filelist/image_entry.c
libfilelist_a_SOURCES += lib/filelist/image_entry_sort.c
libfilelist_a_SOURCES += include/filelist/image_entry.h
libcomp_a_SOURCES = lib/comp/compressor.c lib/comp/none.c
libcomp_a_SOURCES += include/comp/compressor.h lib/comp/internal.h
libcomp_a_CFLAGS = $(AM_CFLAGS)
libcomp_a_CPPFLAGS = $(AM_CPPFLAGS)
if WITH_ZLIB
libcomp_a_SOURCES += lib/comp/zlib.c
libcomp_a_CFLAGS += $(ZLIB_CFLAGS)
libcomp_a_CPPFLAGS += -DWITH_ZLIB
endif
if WITH_LZMA
libcomp_a_SOURCES += lib/comp/lzma.c
libcomp_a_CFLAGS += $(XZ_CFLAGS)
libcomp_a_CPPFLAGS += -DWITH_LZMA
endif
libpkg_a_SOURCES = include/pkg/pkgformat.h include/pkg/pkgreader.h
libpkg_a_SOURCES += include/pkg/pkgio.h include/pkg/pkgwriter.h
libpkg_a_SOURCES += include/pkg/pkglist.h
libpkg_a_SOURCES += lib/pkg/pkgreader.c lib/pkg/pkgwriter.c
libpkg_a_SOURCES += lib/pkg/pkg_unpack.c lib/pkg/pkgio_rd_image_entry.c
libpkg_a_SOURCES += lib/pkg/collect.c lib/pkg/pkglist.c lib/pkg/tsort.c
noinst_LIBRARIES += libutil.a libfilelist.a libcomp.a libpkg.a
noinst_LIBRARIES += libutil.a libfilelist.a

39
lib/comp/compressor.c
View file

@ -1,39 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include <string.h>
#include "internal.h"
static compressor_t *compressors[] = {
[PKG_COMPRESSION_NONE] = &comp_none,
#ifdef WITH_ZLIB
[PKG_COMPRESSION_ZLIB] = &comp_zlib,
#endif
#ifdef WITH_LZMA
[PKG_COMPRESSION_LZMA] = &comp_lzma,
#endif
};
compressor_t *compressor_by_name(const char *name)
{
size_t i;
for (i = 0; i < sizeof(compressors) / sizeof(compressors[0]); ++i) {
if (compressors[i] == NULL)
continue;
if (strcmp(compressors[i]->name, name) == 0)
return compressors[i];
}
return NULL;
}
compressor_t *compressor_by_id(PKG_COMPRESSION id)
{
if ((int)id < 0)
return NULL;
if ((size_t)id >= sizeof(compressors) / sizeof(compressors[0]))
return NULL;
return compressors[id];
}

11
lib/comp/internal.h
View file

@ -1,11 +0,0 @@
/* SPDX-License-Identifier: ISC */
#ifndef INTERNAL_H
#define INTERNAL_H
#include "comp/compressor.h"
extern compressor_t comp_lzma;
extern compressor_t comp_none;
extern compressor_t comp_zlib;
#endif /* INTERNAL_H */

38
lib/filelist/dump_toc.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <sys/sysmacros.h>
#include <sys/types.h>
#include <sys/stat.h>
@ -55,6 +54,40 @@ fail_type:
return -1;
}
static int print_sqfs(image_entry_t *ent, const char *root)
{
mode_t mode = ent->mode & (~S_IFMT);
(void)root;
switch (ent->mode & S_IFMT) {
case S_IFCHR:
printf("%s c %o %u %u %u %u\n", ent->name, mode,
(unsigned int)ent->uid, (unsigned int)ent->gid,
major(ent->data.device.devno),
minor(ent->data.device.devno));
break;
case S_IFBLK:
printf("%s b %o %u %u %u %u\n", ent->name, mode,
(unsigned int)ent->uid, (unsigned int)ent->gid,
major(ent->data.device.devno),
minor(ent->data.device.devno));
break;
case S_IFLNK:
mode = 0777;
/* fall-through */
case S_IFDIR:
case S_IFREG:
printf("%s m %o %u %u\n", ent->name, mode,
(unsigned int)ent->uid, (unsigned int)ent->gid);
break;
default:
fputs("unknown file type in table of contents\n", stderr);
return -1;
}
return 0;
}
static int print_initrd(image_entry_t *ent, const char *root)
{
mode_t mode = ent->mode & (~S_IFMT);
@ -122,7 +155,7 @@ static int print_pkg(image_entry_t *ent, const char *root)
return -1;
}
printf(" \"%s\" 0%o %u %u ", ent->name, mode,
printf(" %s 0%o %u %u ", ent->name, mode,
(unsigned int)ent->uid, (unsigned int)ent->gid);
switch (ent->mode & S_IFMT) {
@ -152,6 +185,7 @@ static int print_pkg(image_entry_t *ent, const char *root)
static print_fun_t printers[] = {
[TOC_FORMAT_PRETTY] = print_pretty,
[TOC_FORMAT_SQFS] = print_sqfs,
[TOC_FORMAT_INITRD] = print_initrd,
[TOC_FORMAT_PKG] = print_pkg,
};

1
lib/filelist/image_entry.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <sys/stat.h>
#include <stdlib.h>
#include <stdio.h>

36
lib/filelist/image_entry_sort.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <sys/stat.h>
#include <string.h>
@ -6,8 +5,6 @@
static int compare_ent(image_entry_t *a, image_entry_t *b)
{
int diff;
/* directories < .. < symlinks < devices < .. < files */
switch (a->mode & S_IFMT) {
case S_IFDIR:
@ -33,8 +30,7 @@ static int compare_ent(image_entry_t *a, image_entry_t *b)
return -1;
}
out_len:
diff = (int)strlen(a->name) - (int)strlen(b->name);
return diff ? diff : strcmp(a->name, b->name);
return (int)strlen(a->name) - (int)strlen(b->name);
out_fsize:
if (a->data.file.size > b->data.file.size)
return 1;
@ -68,34 +64,6 @@ static image_entry_t *insert_sorted(image_entry_t *list, image_entry_t *ent)
return list;
}
static void remove_duplicates(image_entry_t *list)
{
image_entry_t *it = list, *old;
int equal;
if (it == NULL)
return;
while (it->next != NULL) {
equal = 0;
if (S_ISDIR(it->mode) && S_ISDIR(it->next->mode)) {
equal = (strcmp(it->name, it->next->name) == 0);
equal = equal && (it->mode == it->next->mode);
equal = equal && (it->uid == it->next->uid);
equal = equal && (it->gid == it->next->gid);
}
if (equal) {
old = it->next;
it->next = old->next;
image_entry_free(old);
} else {
it = it->next;
}
}
}
image_entry_t *image_entry_sort(image_entry_t *list)
{
image_entry_t *sorted = NULL, *ent;
@ -107,7 +75,5 @@ image_entry_t *image_entry_sort(image_entry_t *list)
sorted = insert_sorted(sorted, ent);
}
remove_duplicates(sorted);
return sorted;
}

1
lib/util/canonicalize_name.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <string.h>
#include "util/util.h"

62
lib/util/fileproc.c
View file

@ -1,62 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <stdio.h>
#include <errno.h>
#include "util/util.h"
int foreach_line_in_file(const char *filename, void *usr, linecb_t fun)
{
FILE *f = fopen(filename, "r");
char *line = NULL, *ptr;
size_t n, lineno = 1;
ssize_t ret;
if (f == NULL) {
perror(filename);
return -1;
}
for (;;) {
free(line);
line = NULL;
errno = 0;
n = 0;
ret = getline(&line, &n, f);
if (ret < 0) {
if (errno == 0) {
ret = 0;
break;
}
perror(filename);
break;
}
for (ptr = line; isspace(*ptr); ++ptr)
;
if (ptr != line)
memmove(line, ptr, strlen(ptr) + 1);
ptr = line + strlen(line);
while (ptr > line && isspace(ptr[-1]))
--ptr;
*ptr = '\0';
if (*line == '\0')
continue;
if (fun(usr, filename, lineno++, line)) {
ret = -1;
break;
}
}
free(line);
fclose(f);
return ret;
}

148
lib/util/hashtable.c
View file

@ -1,148 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "util/hashtable.h"
/* R5 hash function (borrowed from reiserfs) */
static uint32_t strhash(const char *s)
{
const signed char *str = (const signed char *)s;
uint32_t a = 0;
while (*str != '\0') {
a += *str << 4;
a += *str >> 4;
a *= 11;
str++;
}
return a;
}
int hash_table_init(hash_table_t *table, size_t size)
{
table->num_buckets = size;
table->count = 0;
table->buckets = calloc(size, sizeof(table->buckets[0]));
if (table->buckets == NULL) {
fputs("out of memory\n", stderr);
return -1;
}
return 0;
}
void hash_table_cleanup(hash_table_t *table)
{
hash_bucket_t *bucket;
size_t i;
for (i = 0; i < table->num_buckets; ++i) {
while (table->buckets[i] != NULL) {
bucket = table->buckets[i];
table->buckets[i] = bucket->next;
free(bucket->key);
free(bucket);
}
}
free(table->buckets);
table->buckets = NULL;
table->num_buckets = 0;
table->count = 0;
}
void *hash_table_lookup(hash_table_t *table, const char *key)
{
hash_bucket_t *bucket;
uint32_t hash;
hash = strhash(key);
bucket = table->buckets[hash % table->num_buckets];
while (bucket != NULL) {
if (strcmp(bucket->key, key) == 0)
return bucket->value;
bucket = bucket->next;
}
return NULL;
}
int hash_table_set(hash_table_t *table, const char *key, void *value)
{
hash_bucket_t *bucket;
uint32_t hash;
size_t index;
hash = strhash(key);
index = hash % table->num_buckets;
bucket = table->buckets[index];
while (bucket != NULL) {
if (strcmp(bucket->key, key) == 0) {
bucket->value = value;
return 0;
}
bucket = bucket->next;
}
bucket = calloc(1, sizeof(*bucket));
if (bucket == NULL)
goto fail_oom;
bucket->key = strdup(key);
if (bucket->key == NULL)
goto fail_oom;
bucket->value = value;
bucket->next = table->buckets[index];
table->buckets[index] = bucket;
table->count += 1;
return 0;
fail_oom:
free(bucket);
fputs("out of memory\n", stderr);
return -1;
}
void hash_table_foreach(hash_table_t *table, void *usr,
int(*fun)(void *usr, const char *key, void *value))
{
hash_bucket_t *bucket, *prev;
size_t i;
for (i = 0; i < table->num_buckets; ++i) {
prev = NULL;
bucket = table->buckets[i];
while (bucket != NULL) {
if (fun(usr, bucket->key, bucket->value)) {
if (prev == NULL) {
table->buckets[i] = bucket->next;
free(bucket->key);
free(bucket);
bucket = table->buckets[i];
} else {
prev->next = bucket->next;
free(bucket->key);
free(bucket);
bucket = prev->next;
}
table->count -= 1;
} else {
prev = bucket;
bucket = bucket->next;
}
}
}
}

147
lib/util/input_file.c
View file

@ -1,59 +1,126 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
#include <ctype.h>
#include "util/input_file.h"
#include "util/util.h"
void input_file_complain(const char *filename, size_t linenum, const char *msg)
static int prefetch_line(input_file_t *f)
{
fprintf(stderr, "%s: %zu: %s\n", filename, linenum, msg);
}
char *line, *ptr;
ssize_t ret;
size_t n;
retry:
free(f->line);
f->line = NULL;
struct userdata {
const keyword_handler_t *handlers;
size_t count;
void *obj;
};
errno = 0;
line = NULL;
n = 0;
ret = getline(&line, &n, f->f);
static int handle_line(void *usr, const char *filename,
size_t linenum, char *line)
{
struct userdata *u = usr;
size_t i, len;
while (isspace(*line))
++line;
if (*line == '\0' || *line == '#')
return 0;
for (i = 0; i < u->count; ++i) {
len = strlen(u->handlers[i].name);
if (strncmp(line, u->handlers[i].name, len) != 0)
continue;
if (!isspace(line[len]) && line[len] != '\0')
continue;
for (line += len; isspace(*line); ++line)
;
break;
if (ret < 0) {
if (errno != 0) {
perror(f->filename);
free(line);
return -1;
}
free(line);
return 1;
}
if (i == u->count) {
fprintf(stderr, "%s: %zu: unknown keyword\n",
filename, linenum);
n = strlen(line);
while (n >0 && isspace(line[n - 1]))
--n;
line[n] = '\0';
f->line = line;
f->linenum += 1;
for (ptr = f->line; isspace(*ptr); ++ptr)
;
if (*ptr == '\0' || *ptr == '#')
goto retry;
if (ptr != f->line)
memmove(f->line, ptr, strlen(ptr) + 1);
ptr = f->line + strlen(f->line);
while (ptr > f->line && isspace(ptr[-1]))
--ptr;
*ptr = '\0';
if (f->line[0] == '\0')
goto retry;
return 0;
}
void cleanup_file(input_file_t *f)
{
fclose(f->f);
free(f->line);
}
int open_file(input_file_t *f, const char *filename)
{
memset(f, 0, sizeof(*f));
f->filename = filename;
f->f = fopen(filename, "r");
if (f->f == NULL) {
perror(f->filename);
return -1;
}
return u->handlers[i].handle(line, filename, linenum, u->obj);
return 0;
}
int process_file(const char *filename, const keyword_handler_t *handlers,
void input_file_complain(const input_file_t *f, const char *msg)
{
fprintf(stderr, "%s: %zu: %s\n", f->filename, f->linenum, msg);
}
int process_file(input_file_t *f, const keyword_handler_t *handlers,
size_t count, void *obj)
{
struct userdata u = { handlers, count, obj };
size_t i, len;
char *ptr;
int ret;
return foreach_line_in_file(filename, &u, handle_line);
for (;;) {
ret = prefetch_line(f);
if (ret < 0)
return -1;
if (ret > 0)
break;
ptr = f->line;
for (i = 0; i < count; ++i) {
len = strlen(handlers[i].name);
if (strncmp(ptr, handlers[i].name, len) != 0)
continue;
if (!isspace(ptr[len]) && ptr[len] != '\0')
continue;
for (ptr += len; isspace(*ptr); ++ptr)
;
memmove(f->line, ptr, strlen(ptr) + 1);
break;
}
if (i == count) {
fprintf(stderr, "%s: %zu: unknown keyword\n",
f->filename, f->linenum);
return -1;
}
if (handlers[i].handle(f, obj))
return -1;
}
return 0;
}

1
lib/util/mkdir_p.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <sys/types.h>
#include <sys/stat.h>
#include <string.h>

1
lib/util/read_retry.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <unistd.h>
#include <errno.h>

1
lib/util/write_retry.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <unistd.h>
#include <errno.h>

40
main/Makemodule.am
View file

@ -1,8 +1,9 @@
pkg_SOURCES = main/command.h main/pkg.c main/command.c
pkg_CPPFLAGS = $(AM_CPPFLAGS) -I$(top_srcdir)/main
pkg_SOURCES = include/pkgformat.h include/pkgreader.h include/pkgio.h
pkg_SOURCES += include/compressor.h include/command.h include/pkgwriter.h
pkg_SOURCES += main/pkg.c main/compressor.c main/command.c main/pkgreader.c
pkg_SOURCES += main/pkgwriter.c main/pkgio_rd_image_entry.c main/pkg_unpack.c
pkg_CFLAGS = $(AM_CFLAGS)
pkg_LDADD = libpkg.a libutil.a libfilelist.a libcomp.a
pkg_LDADD = libutil.a libfilelist.a
##### commands #####
@ -17,16 +18,9 @@ pkg_SOURCES += main/cmd/dump/dump.c main/cmd/dump/dump.h
pkg_SOURCES += main/cmd/dump/dump_header.c
# install command
pkg_SOURCES += main/cmd/install.c
# buildstrategy command
pkg_SOURCES += main/cmd/buildstrategy/buildstrategy.h
pkg_SOURCES += main/cmd/buildstrategy/buildstrategy.c
pkg_SOURCES += main/cmd/buildstrategy/srcpkg.c
pkg_SOURCES += main/cmd/buildstrategy/provider.c
# depgraph command
pkg_SOURCES += main/cmd/depgraph.c
pkg_SOURCES += main/cmd/install/collect.c main/cmd/install/install.c
pkg_SOURCES += main/cmd/install/install.h main/cmd/install/pkglist.c
pkg_SOURCES += main/cmd/install/tsort.c
# unpack command
pkg_SOURCES += main/cmd/unpack.c
@ -35,12 +29,24 @@ pkg_SOURCES += main/cmd/unpack.c
pkg_SOURCES += main/cmd/help.c
##### compressors #####
if WITH_LZMA
pkg_LDADD += $(XZ_LIBS)
endif
# dummy compressor
pkg_SOURCES += main/compressors/none.c
# zlib compressor
if WITH_ZLIB
pkg_SOURCES += main/compressors/zlib.c
pkg_CFLAGS += $(ZLIB_CFLAGS)
pkg_LDADD += $(ZLIB_LIBS)
endif
# lzma compressor
if WITH_LZMA
pkg_SOURCES += main/compressors/lzma.c
pkg_CFLAGS += $(XZ_CFLAGS)
pkg_LDADD += $(XZ_LIBS)
endif
bin_PROGRAMS += pkg

231
main/cmd/buildstrategy/buildstrategy.c
View file

@ -1,231 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include "buildstrategy.h"
typedef int (*line_handler_t)(const char *filename, size_t linenum,
const char *lhs, const char *rhs);
struct userdata {
line_handler_t cb;
};
static int handle_line(void *usr, const char *filename,
size_t linenum, char *line)
{
struct userdata *u = usr;
char *rhs;
rhs = strchr(line, ',');
if (rhs == NULL)
return 0;
*(rhs++) = '\0';
while (isspace(*rhs))
++rhs;
if (*line == '\0' || *rhs == '\0')
return 0;
return u->cb(filename, linenum, line, rhs);
}
static int foreach_line(const char *filename, line_handler_t cb)
{
struct userdata u = { cb };
return foreach_line_in_file(filename, &u, handle_line);
}
/*****************************************************************************/
static int handle_depends(const char *filename, size_t linenum,
const char *sourcepkg, const char *binpkg)
{
source_pkg_t *src, *dep;
size_t count;
void *new;
(void)filename; (void)linenum;
src = src_pkg_get(sourcepkg);
if (src == NULL)
return 0;
dep = provider_get(sourcepkg, binpkg);
if (dep == NULL)
return -1;
if ((src->num_depends % 10) == 0) {
count = src->num_depends + 10;
new = realloc(src->depends, sizeof(src->depends[0]) * count);
if (new == NULL) {
fputs("out of memory\n", stderr);
return -1;
}
src->depends = new;
}
src->depends[src->num_depends++] = dep;
return 0;
}
static int handle_provides(const char *filename, size_t linenum,
const char *sourcepkg, const char *binpkg)
{
(void)filename; (void)linenum;
return provider_add(sourcepkg, binpkg);
}
static int handle_prefere(const char *filename, size_t linenum,
const char *binpkg, const char *sourcepkg)
{
(void)filename; (void)linenum;
return provider_add_prefered(binpkg, sourcepkg);
}
/*****************************************************************************/
static const struct option long_opts[] = {
{ "prefere", required_argument, NULL, 'P' },
{ "provides", required_argument, NULL, 'p' },
{ "depends", required_argument, NULL, 'd' },
{ "graph", no_argument, NULL, 'g' },
{ NULL, 0, NULL, 0 },
};
static const char *short_opts = "p:d:P:g";
static void pkg_mark_deps(source_pkg_t *pkg)
{
size_t i;
if (pkg->flags & FLAG_BUILD_PKG)
return;
pkg->flags |= FLAG_BUILD_PKG;
for (i = 0; i < pkg->num_depends; ++i) {
if ((pkg->depends[i]->flags & FLAG_BUILD_PKG) == 0)
pkg_mark_deps(pkg->depends[i]);
}
}
static int cmd_buildstrategy(int argc, char **argv)
{
const char *provides = NULL, *depends = NULL, *prefere = NULL;
int i, ret = EXIT_FAILURE, mode = MODE_BUILD_ORDER;
source_pkg_t *pkg;
if (src_pkg_init())
return EXIT_FAILURE;
if (provider_init())
goto out_src;
for (;;) {
i = getopt_long(argc, argv, short_opts, long_opts, NULL);
if (i == -1)
break;
switch (i) {
case 'P':
prefere = optarg;
break;
case 'p':
provides = optarg;
break;
case 'd':
depends = optarg;
break;
case 'g':
mode = MODE_BUILD_GRAPH;
break;
default:
goto fail_arg;
}
}
if (optind >= argc) {
fputs("no packages specified\n", stderr);
goto fail_arg;
}
if (provides == NULL) {
fputs("no source package provides specified\n", stderr);
goto fail_arg;
}
if (prefere != NULL && foreach_line(prefere, handle_prefere) != 0)
goto out;
if (foreach_line(provides, handle_provides))
goto out;
if (depends != NULL && foreach_line(depends, handle_depends) != 0)
goto out;
for (i = optind; i < argc; ++i) {
pkg = provider_get(NULL, argv[i]);
if (pkg == NULL)
goto out;
pkg_mark_deps(pkg);
}
switch (mode) {
case MODE_BUILD_GRAPH:
fputs("digraph buildgraph {\n\tcompound=true;\n", stdout);
src_pkg_print_graph_cluster();
fputs("}\n", stdout);
break;
default:
if (src_pkg_output_build_order())
goto out;
break;
}
ret = EXIT_SUCCESS;
out:
provider_cleanup();
out_src:
src_pkg_cleanup();
return ret;
fail_arg:
tell_read_help(argv[0]);
goto out;
}
static command_t buildstrategy = {
.cmd = "buildstrategy",
.usage = "[OPTIONS...] <packages>",
.s_desc = "work out what source packages to build in what order",
.l_desc =
"The buildstrategy command takes as input a description of what binary\n"
"packages a source package needs in order to build and what binary packages\n"
"in turn produces in the process. The command then takes from the command\n"
"line arguments a list of desired binary packages and works out a minimal\n"
"set of source packages to build and in what order they should be built.\n"
"\n"
"Possible options:\n"
" --provides, -p <file> A two column CSV file. Each line contains the name\n"
" of a source package and a binary package that it\n"
" produces when built. If a source packages provides\n"
" multiple binary packages, use one line for each\n"
" binary package.\n"
" --depends, -d <file> A two column CSV file. Each line contains the name\n"
" of a source package and a binary package that it\n"
" requires to build.\n"
" --prefere, -P <file> A two column CSV file. Each line contains the name\n"
" of a binary package and a source package that\n"
" produces this binary. If the `--provides` file\n"
" specifies that more than one source pacakge\n"
" provides a binary package, this file contains the\n"
" prefered one we should use.\n"
" --graph, -g Instead of printing out an ordered list, produce\n"
" a dot graph of the source packages to be built.\n",
.run_cmd = cmd_buildstrategy,
};
REGISTER_COMMAND(buildstrategy)

55
main/cmd/buildstrategy/buildstrategy.h
View file

@ -1,55 +0,0 @@
/* SPDX-License-Identifier: ISC */
#ifndef BUILDSTRATEGY_H
#define BUILDSTRATEGY_H
#include <getopt.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <stdio.h>
#include <errno.h>
#include "command.h"
#include "util/util.h"
#include "util/hashtable.h"
enum {
FLAG_BUILD_PKG = 0x01,
};
enum {
MODE_BUILD_ORDER = 0,
MODE_BUILD_GRAPH,
};
typedef struct source_pkg_t {
struct source_pkg_t *next;
char *name;
struct source_pkg_t **depends;
size_t num_depends;
int flags;
} source_pkg_t;
int src_pkg_init(void);
void src_pkg_cleanup(void);
source_pkg_t *src_pkg_get(const char *name);
int src_pkg_output_build_order(void);
int provider_init(void);
void provider_cleanup(void);
int provider_add(const char *sourcepkg, const char *binpkg);
int provider_add_prefered(const char *binpkg, const char *sourcepkg);
source_pkg_t *provider_get(const char *parent, const char *binpkg);
void src_pkg_print_graph_cluster(void);
#endif /* BUILDSTRATEGY_H */

110
main/cmd/buildstrategy/provider.c
View file

@ -1,110 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include "buildstrategy.h"
static hash_table_t tbl_provides;
static hash_table_t tbl_preferes;
int provider_init(void)
{
if (hash_table_init(&tbl_provides, 1024))
return -1;
if (hash_table_init(&tbl_preferes, 1024)) {
hash_table_cleanup(&tbl_provides);
return -1;
}
return 0;
}
void provider_cleanup(void)
{
hash_table_cleanup(&tbl_provides);
hash_table_cleanup(&tbl_preferes);
}
int provider_add(const char *sourcepkg, const char *binpkg)
{
source_pkg_t *head = hash_table_lookup(&tbl_provides, binpkg), *src;
for (src = head; src != NULL; src = src->next) {
if (strcmp(src->name, sourcepkg) == 0)
return 0;
}
src = src_pkg_get(sourcepkg);
if (src == NULL)
return -1;
src->next = head;
return hash_table_set(&tbl_provides, binpkg, src);
}
int provider_add_prefered(const char *binpkg, const char *sourcepkg)
{
source_pkg_t *src = src_pkg_get(sourcepkg);
if (src == NULL)
return -1;
return hash_table_set(&tbl_preferes, binpkg, src);
}
source_pkg_t *provider_get(const char *parent, const char *binpkg)
{
source_pkg_t *spkg, *pref;
spkg = hash_table_lookup(&tbl_provides, binpkg);
if (spkg == NULL)
goto fail_none;
pref = hash_table_lookup(&tbl_preferes, binpkg);
if (spkg->next != NULL) {
if (pref == NULL)
goto fail_no_pref;
while (spkg != NULL && strcmp(spkg->name, pref->name) != 0) {
spkg = spkg->next;
}
if (spkg == NULL) {
spkg = hash_table_lookup(&tbl_provides, binpkg);
goto fail_provider;
}
} else {
if (pref != NULL && strcmp(spkg->name, pref->name) != 0)
goto fail_provider;
}
return spkg;
fail_none:
fprintf(stderr, "No source package provides binary package '%s'.\n\n",
binpkg);
goto fail;
fail_provider:
fprintf(stderr,
"Preferred provider for binary package '%s' is set to\n"
"source package '%s', which does not provide '%s'.\n\n",
binpkg, pref->name, binpkg);
goto fail_list_providers;
fail_no_pref:
fprintf(stderr, "No preferred provider set for "
"binary package '%s'.\n\n", binpkg);
goto fail_list_providers;
fail_list_providers:
fprintf(stderr, "The following source packages provide the "
"binary package '%s':\n\n", binpkg);
while (spkg != NULL) {
fprintf(stderr, "\t%s\n", spkg->name);
spkg = spkg->next;
}
fputc('\n', stderr);
goto fail;
fail:
if (parent != NULL) {
fprintf(stderr, "Binary package '%s' is required to "
"build source package '%s'\n", binpkg, parent);
}
return NULL;
}

145
main/cmd/buildstrategy/srcpkg.c
View file

@ -1,145 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include "buildstrategy.h"
static hash_table_t tbl_sourcepkgs;
static int src_pkg_destroy(void *usr, const char *name, void *p)
{
source_pkg_t *pkg = p;
(void)name; (void)usr;
free(pkg->name);
free(pkg->depends);
free(pkg);
return 1;
}
int src_pkg_init(void)
{
return hash_table_init(&tbl_sourcepkgs, 1024);
}
void src_pkg_cleanup(void)
{
hash_table_foreach(&tbl_sourcepkgs, NULL, src_pkg_destroy);
hash_table_cleanup(&tbl_sourcepkgs);
}
source_pkg_t *src_pkg_get(const char *name)
{
source_pkg_t *src = hash_table_lookup(&tbl_sourcepkgs, name);
if (src == NULL) {
src = calloc(1, sizeof(*src));
if (src == NULL)
goto fail_oom;
src->name = strdup(name);
if (src->name == NULL)
goto fail_oom;
if (hash_table_set(&tbl_sourcepkgs, name, src))
goto fail;
}
return src;
fail_oom:
fputs("out of memory\n", stderr);
fail:
if (src != NULL)
free(src->name);
free(src);
return NULL;
}
static int remove_untagged(void *usr, const char *name, void *p)
{
int mask = *((int *)usr);
source_pkg_t *pkg = p;
(void)name;
if ((pkg->flags & mask) == 0) {
free(pkg->name);
free(pkg->depends);
free(pkg);
return 1;
}
return 0;
}
static int find_no_dep(void *usr, const char *name, void *p)
{
source_pkg_t *pkg = p;
int *found = usr;
if (pkg->num_depends == 0) {
printf("%s\n", name);
pkg->flags &= ~FLAG_BUILD_PKG;
*found = 1;
}
return 0;
}
static int remove_unmarked_deps(void *usr, const char *name, void *p)
{
source_pkg_t *pkg = p;
size_t i = 0;
(void)usr; (void)name;
while (i < pkg->num_depends) {
if ((pkg->depends[i]->flags & FLAG_BUILD_PKG) == 0) {
pkg->depends[i] = pkg->depends[pkg->num_depends - 1];
pkg->num_depends -= 1;
} else {
++i;
}
}
return 0;
}
int src_pkg_output_build_order(void)
{
int i = FLAG_BUILD_PKG;
hash_table_foreach(&tbl_sourcepkgs, &i, remove_untagged);
while (tbl_sourcepkgs.count > 0) {
i = 0;
hash_table_foreach(&tbl_sourcepkgs, &i, find_no_dep);
if (!i) {
fputs("cycle detected in package dependencies!\n",
stderr);
return -1;
}
hash_table_foreach(&tbl_sourcepkgs, NULL, remove_unmarked_deps);
i = FLAG_BUILD_PKG;
hash_table_foreach(&tbl_sourcepkgs, &i, remove_untagged);
}
return 0;
}
static int print_cluster(void *usr, const char *name, void *p)
{
source_pkg_t *pkg = p;
size_t i;
(void)usr;
if (!(pkg->flags & FLAG_BUILD_PKG))
return 0;
for (i = 0; i < pkg->num_depends; ++i) {
printf("\t\"%s\" -> \"%s\"\n",
name, pkg->depends[i]->name);
}
return 0;
}
void src_pkg_print_graph_cluster(void)
{
hash_table_foreach(&tbl_sourcepkgs, NULL, print_cluster);
}

119
main/cmd/depgraph.c
View file

@ -1,119 +0,0 @@
/* SPDX-License-Identifier: ISC */
#include <stdbool.h>
#include <getopt.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include "pkg/pkglist.h"
#include "command.h"
#include "config.h"
static const struct option long_opts[] = {
{ "no-dependencies", no_argument, NULL, 'd' },
{ "repo-dir", required_argument, NULL, 'R' },
{ NULL, 0, NULL, 0 },
};
static const char *short_opts = "dR:";
static void print_dot_graph(struct pkg_dep_list *list)
{
struct pkg_dep_node *it;
size_t i;
printf("digraph dependencies {\n");
for (it = list->head; it != NULL; it = it->next) {
for (i = 0; i < it->num_deps; ++i) {
printf("\t\"%s\" -> \"%s\";\n",
it->name, it->deps[i]->name);
}
}
printf("}\n");
}
static int cmd_depgraph(int argc, char **argv)
{
int i, repofd = -1, ret = EXIT_FAILURE;
struct pkg_dep_list list;
bool resolve_deps = true;
memset(&list, 0, sizeof(list));
for (;;) {
i = getopt_long(argc, argv, short_opts, long_opts, NULL);
if (i == -1)
break;
switch (i) {
case 'd':
resolve_deps = false;
break;
case 'R':
if (repofd != -1) {
fputs("repo specified more than once\n",
stderr);
tell_read_help(argv[0]);
goto out;
}
repofd = open(optarg, O_RDONLY | O_DIRECTORY);
if (repofd < 0) {
perror(optarg);
goto out;
}
break;
default:
tell_read_help(argv[0]);
goto out;
}
}
if (repofd == -1) {
repofd = open(REPODIR, O_RDONLY | O_DIRECTORY);
if (repofd < 0) {
perror(REPODIR);
goto out;
}
}
for (i = optind; i < argc; ++i) {
if (append_pkg(&list, argv[i]) == NULL)
goto out;
}
if (resolve_deps) {
if (collect_dependencies(repofd, &list))
goto out;
}
print_dot_graph(&list);
ret = EXIT_SUCCESS;
out:
if (repofd != -1)
close(repofd);
pkg_list_cleanup(&list);
return ret;
}
static command_t depgraph = {
.cmd = "depgraph",
.usage = "[OPTIONS...] PACKAGES...",
.s_desc = "produces a DOT language dependency graph",
.l_desc =
"Fetches a number of packages from a repository directory by name and\n"
"produces a DOT language dependency graph for the packages.\n"
"\n"
"Possible options:\n"
" --repo-dir, -R <path> Specify the input repository path to fetch the\n"
" packages from.\n"
" If not set, defaults to " REPODIR ".\n"
" --no-dependencies, -d Do not resolve dependencies, only process the\n"
" packages listed on the command line.\n",
.run_cmd = cmd_depgraph,
};
REGISTER_COMMAND(depgraph)

46
main/cmd/dump/dump.c
View file

@ -1,14 +1,14 @@
/* SPDX-License-Identifier: ISC */
#include "dump.h"
static const struct option long_opts[] = {
{ "dependencies", no_argument, NULL, 'd' },
{ "list-files", required_argument, NULL, 'l' },
{ "list-files", no_argument, NULL, 'l' },
{ "format", required_argument, NULL, 'f' },
{ "root", required_argument, NULL, 'r' },
{ NULL, 0, NULL, 0 },
};
static const char *short_opts = "dl:r:";
static const char *short_opts = "dlf:r:";
static int cmd_dump(int argc, char **argv)
{
@ -25,14 +25,13 @@ static int cmd_dump(int argc, char **argv)
break;
switch (i) {
case 'l':
flags |= DUMP_TOC;
if (strcmp(optarg, "initrd") == 0) {
case 'f':
if (strcmp(optarg, "sqfs") == 0) {
format = TOC_FORMAT_SQFS;
} else if (strcmp(optarg, "initrd") == 0) {
format = TOC_FORMAT_INITRD;
} else if (strcmp(optarg, "pkg") == 0) {
format = TOC_FORMAT_PKG;
} else if (strcmp(optarg, "detail") == 0) {
format = TOC_FORMAT_PRETTY;
} else {
fprintf(stderr, "unknown format '%s'\n",
optarg);
@ -43,6 +42,9 @@ static int cmd_dump(int argc, char **argv)
case 'r':
root = optarg;
break;
case 'l':
flags |= DUMP_TOC;
break;
case 'd':
flags |= DUMP_DEPS;
break;
@ -100,21 +102,25 @@ static command_t dump = {
"Possible options:\n"
" --dependencies, -d Show dependency information of a package.\n"
"\n"
" --list-files, -l <format> Produce a list of files with a specified\n"
" formating.\n"
" --list-files, -l Produce a list of files. The format for the list\n"
" can be specified with --format.\n"
"\n"
" If \"detail\" is specified, a human readable,\n"
" pretty printed format with details is used.\n"
" --format, -f <format> Specify what format to use for printing the table\n"
" of contents. Default is a pretty printed, human\n"
" readable version.\n"
"\n"
" If \"initrd\" is specified, the format of\n"
" Linux gen_init_cpio is produced.\n"
" If \"sqfs\" is specified, a squashfs pseudo file\n"
" is genareated for setting permissions bits and\n"
" ownership appropriately.\n"
"\n"
" --root, -r <path> If a format is used that requires absoulute\n"
" input paths (e.g. initrd), prefix all file\n"
" paths with this. Can be used, for instance to\n"
" unpack a package to a staging directory and\n"
" generate a a listing for Linux\n"
" CONFIG_INITRAMFS_SOURCE.\n",
" If \"initrd\" is specified, a format appropriate\n"
" for Linux gen_init_cpio is produced.\n"
"\n"
" --root, -r <path> If a format is used that requires absoulute input\n"
" paths (e.g. initrd), prefix all file paths with\n"
" this. Can be used, for instance to unpack a\n"
" package to a staging directory and generate a\n"
" a listing for Linux CONFIG_INITRAMFS_SOURCE.\n",
.run_cmd = cmd_dump,
};

8
main/cmd/dump/dump.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef DUMP_H
#define DUMP_H
@ -12,11 +11,10 @@
#include "filelist/image_entry.h"
#include "pkg/pkgformat.h"
#include "pkg/pkgreader.h"
#include "pkg/pkgio.h"
#include "pkgformat.h"
#include "pkgreader.h"
#include "command.h"
#include "pkgio.h"
typedef enum {
DUMP_TOC = 0x01,

1
main/cmd/dump/dump_header.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "dump.h"
static const char *dependency_type[] = {

1
main/cmd/help.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <string.h>
#include <stdlib.h>
#include <ctype.h>

8
lib/pkg/collect.c → main/cmd/install/collect.c
View file

@ -1,10 +1,4 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "pkg/pkgreader.h"
#include "pkg/pkglist.h"
#include "install.h"
int collect_dependencies(int repofd, struct pkg_dep_list *list)
{

107
main/cmd/install.c → main/cmd/install/install.c
View file

@ -1,23 +1,4 @@
/* SPDX-License-Identifier: ISC */
#include <stdbool.h>
#include <getopt.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include "pkg/pkglist.h"
#include "pkg/pkgio.h"
#include "util/util.h"
#include "command.h"
#include "config.h"
enum {
INSTALL_MODE_INSTALL = 0,
INSTALL_MODE_LIST_PKG,
INSTALL_MODE_LIST_FILES,
};
#include "install.h"
static const struct option long_opts[] = {
{ "root", required_argument, NULL, 'r' },
@ -26,13 +7,14 @@ static const struct option long_opts[] = {
{ "no-dependencies", no_argument, NULL, 'd' },
{ "repo-dir", required_argument, NULL, 'R' },
{ "list-packages", no_argument, NULL, 'p' },
{ "list-files", required_argument, NULL, 'l' },
{ "list-files", no_argument, NULL, 'l' },
{ "format", required_argument, NULL, 'F' },
{ "no-symlinks", no_argument, NULL, 'L' },
{ "no-devices", no_argument, NULL, 'D' },
{ NULL, 0, NULL, 0 },
};
static const char *short_opts = "r:omdR:pl:F:LD";
static const char *short_opts = "r:omdR:plF:LD";
static int unpack_packages(int repofd, int rootfd, int flags,
struct pkg_dep_list *list)
@ -96,8 +78,8 @@ static int list_files(int repofd, const char *rootdir, TOC_FORMAT format,
static int cmd_install(int argc, char **argv)
{
int i, rootfd = AT_FDCWD, repofd = AT_FDCWD, flags = 0;
int ret = EXIT_FAILURE, mode = INSTALL_MODE_INSTALL;
int i, rootfd = -1, repofd = -1, flags = 0;
TOC_FORMAT format = TOC_FORMAT_PRETTY;
const char *rootdir = NULL;
struct pkg_dep_list list;
@ -119,12 +101,14 @@ static int cmd_install(int argc, char **argv)
break;
case 'l':
mode = INSTALL_MODE_LIST_FILES;
if (strcmp(optarg, "initrd") == 0) {
break;
case 'F':
if (strcmp(optarg, "sqfs") == 0) {
format = TOC_FORMAT_SQFS;
} else if (strcmp(optarg, "initrd") == 0) {
format = TOC_FORMAT_INITRD;
} else if (strcmp(optarg, "pkg") == 0) {
format = TOC_FORMAT_PKG;
} else if (strcmp(optarg, "detail")) {
format = TOC_FORMAT_PRETTY;
} else {
fprintf(stderr, "unknown format '%s'\n",
optarg);
@ -132,7 +116,7 @@ static int cmd_install(int argc, char **argv)
}
break;
case 'R':
if (repofd != -1) {
if (repofd != AT_FDCWD) {
fputs("repo specified more than once\n",
stderr);
tell_read_help(argv[0]);
@ -145,7 +129,7 @@ static int cmd_install(int argc, char **argv)
}
break;
case 'r':
if (rootfd != -1) {
if (rootfd != AT_FDCWD) {
fputs("root specified more than once\n",
stderr);
tell_read_help(argv[0]);
@ -181,22 +165,6 @@ static int cmd_install(int argc, char **argv)
}
}
if (repofd == -1) {
repofd = open(REPODIR, O_RDONLY | O_DIRECTORY);
if (repofd < 0) {
perror(REPODIR);
goto out;
}
}
if (rootfd == -1) {
rootfd = open(INSTALLROOT, O_RDONLY | O_DIRECTORY);
if (rootfd < 0) {
perror(INSTALLROOT);
goto out;
}
}
for (i = optind; i < argc; ++i) {
if (append_pkg(&list, argv[i]) == NULL)
goto out;
@ -226,9 +194,9 @@ static int cmd_install(int argc, char **argv)
ret = EXIT_SUCCESS;
out:
if (rootfd != -1)
if (rootfd != AT_FDCWD)
close(rootfd);
if (repofd != -1)
if (repofd != AT_FDCWD)
close(repofd);
pkg_list_cleanup(&list);
return ret;
@ -244,32 +212,31 @@ static command_t install = {
"packages are evaluated and also installed.\n"
"\n"
"Possible options:\n"
" --repo-dir, -R <path> Specify the input repository path to fetch the\n"
" packages from.\n"
" If not set, defaults to " REPODIR ".\n"
" --root, -r <path> A root directory to unpack the package.\n"
" If not set, defaults to " INSTALLROOT ".\n"
" --no-chown, -o Do not change ownership of the extracted data.\n"
" Keep the uid/gid of the user who runs the \n"
" program.\n"
" --no-chmod, -m Do not change permission flags of the extarcted\n"
" data. Use 0644 for all files and 0755 for all\n"
" directories.\n"
" --no-symlink, -L Do not create symlinks.\n"
" --no-devices, -D Do not create device files.\n"
" --no-dependencies, -d Do not resolve dependencies, only install files\n"
" packages listed on the command line.\n"
" --list-packages, -p Do not install packages, print out final\n"
" package list that would be installed.\n"
" --list-files, -l <format> Do not install packages, print out table of\n"
" contents for all packages that would be\n"
" installed in a specified format.\n"
" --repo-dir, -R <path> Specify the input repository path to fetch the\n"
" packages from.\n"
" --root, -r <path> A root directory to unpack the package. Defaults\n"
" to the current working directory if not set.\n"
" --no-chown, -o Do not change ownership of the extracted data.\n"
" Keep the uid/gid of the user who runs the program.\n"
" --no-chmod, -m Do not change permission flags of the extarcted\n"
" data. Use 0644 for all files and 0755 for all\n"
" directories.\n"
" --no-dependencies, -d Do not resolve dependencies, only install files\n"
" packages listed on the command line.\n"
" --list-packages, -p Do not install packages, print out final package\n"
" list that would be installed.\n"
" --list-files, -l Do not install packages, print out table of\n"
" contents for all packages that would be installed.\n"
" --format, -F <format> Specify what format to use for printing the table\n"
" of contents. Default is a pretty printed, human\n"
" readable version.\n"
"\n"
" If \"detail\" is specified, a human readable,\n"
" pretty printed format with details is used.\n"
" If \"sqfs\" is specified, a squashfs pseudo file\n"
" is genareated for setting permissions bits and\n"
" ownership appropriately.\n"
"\n"
" If \"initrd\" is specified, the format of Linux\n"
" gen_init_cpio is produced.\n",
" If \"initrd\" is specified, a format appropriate\n"
" for Linux gen_init_cpio is produced.\n",
.run_cmd = cmd_install,
};

29
include/pkg/pkglist.h → main/cmd/install/install.h
View file

@ -1,6 +1,27 @@
/* SPDX-License-Identifier: ISC */
#ifndef PKGLIST_H
#define PKGLIST_H
#ifndef INSTALL_H
#define INSTALL_H
#include <stdbool.h>
#include <getopt.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include "util/util.h"
#include "filelist/image_entry.h"
#include "pkgreader.h"
#include "command.h"
#include "pkgio.h"
enum {
INSTALL_MODE_INSTALL = 0,
INSTALL_MODE_LIST_PKG,
INSTALL_MODE_LIST_FILES,
};
struct pkg_dep_node {
char *name;
@ -30,4 +51,4 @@ int collect_dependencies(int repofd, struct pkg_dep_list *list);
int sort_by_dependencies(struct pkg_dep_list *list);
#endif /* PKGLIST_H */
#endif /* INSTALL_H */

7
lib/pkg/pkglist.c → main/cmd/install/pkglist.c
View file

@ -1,9 +1,4 @@
/* SPDX-License-Identifier: ISC */
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include "pkg/pkglist.h"
#include "install.h"
struct pkg_dep_node *append_pkg(struct pkg_dep_list *list, const char *name)
{

6
lib/pkg/tsort.c → main/cmd/install/tsort.c
View file

@ -1,8 +1,4 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <stdio.h>
#include "pkg/pkglist.h"
#include "install.h"
static void remove_dependency(struct pkg_dep_list *list,
struct pkg_dep_node *pkg)

70
main/cmd/pack/desc.c
View file

@ -1,10 +1,8 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
static int handle_requires(char *line, const char *filename,
size_t linenum, void *obj)
static int handle_requires(input_file_t *f, void *obj)
{
char *ptr = line, *end;
char *ptr = f->line, *end;
pkg_desc_t *desc = obj;
dependency_t *dep;
size_t len;
@ -21,14 +19,13 @@ static int handle_requires(char *line, const char *filename,
break;
if (len > 0xFF) {
input_file_complain(filename, linenum, "dependency name too long");
input_file_complain(f, "dependency name too long");
return -1;
}
dep = calloc(1, sizeof(*dep) + len + 1);
if (dep == NULL) {
input_file_complain(filename, linenum,
"out of memory");
input_file_complain(f, "out of memory");
return -1;
}
@ -43,30 +40,46 @@ static int handle_requires(char *line, const char *filename,
return 0;
}
static int handle_toc_compressor(char *line, const char *filename,
size_t linenum, void *obj)
static int handle_name(input_file_t *f, void *obj)
{
pkg_desc_t *desc = obj;
desc->toccmp = compressor_by_name(line);
if (desc->name != NULL) {
input_file_complain(f, "name redefined");
return -1;
}
if (desc->toccmp == NULL) {
input_file_complain(filename, linenum, "unkown compressor");
desc->name = strdup(f->line);
if (desc->name == NULL) {
input_file_complain(f, "out of memory");
return -1;
}
return 0;
}
static int handle_data_compressor(char *line, const char *filename,
size_t linenum, void *obj)
static int handle_toc_compressor(input_file_t *f, void *obj)
{
pkg_desc_t *desc = obj;
desc->datacmp = compressor_by_name(line);
desc->toccmp = compressor_by_name(f->line);
if (desc->toccmp == NULL) {
input_file_complain(f, "unkown compressor");
return -1;
}
return 0;
}
static int handle_data_compressor(input_file_t *f, void *obj)
{
pkg_desc_t *desc = obj;
desc->datacmp = compressor_by_name(f->line);
if (desc->datacmp == NULL) {
input_file_complain(filename, linenum, "unkown compressor");
input_file_complain(f, "unkown compressor");
return -1;
}
@ -77,6 +90,7 @@ static const keyword_handler_t line_hooks[] = {
{ "toc-compressor", handle_toc_compressor },
{ "data-compressor", handle_data_compressor },
{ "requires", handle_requires },
{ "name", handle_name },
};
#define NUM_LINE_HOOKS (sizeof(line_hooks) / sizeof(line_hooks[0]))
@ -102,13 +116,20 @@ static compressor_t *get_default_compressor(void)
int desc_read(const char *path, pkg_desc_t *desc)
{
char *ptr;
input_file_t f;
memset(desc, 0, sizeof(*desc));
if (process_file(path, line_hooks, NUM_LINE_HOOKS, desc))
if (open_file(&f, path))
return -1;
if (process_file(&f, line_hooks, NUM_LINE_HOOKS, desc)) {
cleanup_file(&f);
return -1;
}
cleanup_file(&f);
if (desc->datacmp == NULL)
desc->datacmp = get_default_compressor();
@ -121,19 +142,6 @@ int desc_read(const char *path, pkg_desc_t *desc)
return -1;
}
ptr = strrchr(path, '/');
desc->name = strdup((ptr == NULL) ? path : (ptr + 1));
if (desc->name == NULL) {
fputs("out of memory\n", stderr);
desc_free(desc);
return -1;
}
ptr = strrchr(desc->name, '.');
if (ptr != NULL)
*ptr = '\0';
return 0;
}

140
main/cmd/pack/filelist.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
static char *skipspace(char *str)
@ -8,89 +7,51 @@ static char *skipspace(char *str)
return str;
}
static void oom(const char *filename, size_t linenum)
static void oom(input_file_t *f)
{
input_file_complain(filename, linenum, "out of memory");
input_file_complain(f, "out of memory");
}
static void unescape_name(char *name)
{
char *dst = name, *src = name + 1;
while (*src != '"' && *src != '\0') {
if (src[0] == '\\' && (src[1] == '\\' || src[1] == '"')) {
*(dst++) = src[1];
src += 2;
} else {
*(dst++) = *(src++);
}
}
*dst = '\0';
}
static image_entry_t *filelist_mkentry(char *line, const char *filename,
size_t linenum, mode_t filetype)
static image_entry_t *filelist_mkentry(input_file_t *f, mode_t filetype)
{
char *line = f->line;
image_entry_t *ent;
char *start = line;
size_t i;
ent = calloc(1, sizeof(*ent));
if (ent == NULL) {
oom(filename, linenum);
oom(f);
return NULL;
}
/* name */
if (*line == '"') {
for (i = 1; line[i] != '"' && line[i] != '\0'; ++i) {
if (line[i] == '\\' &&
(line[i + 1] == '\\' || line[i + 1] == '"')) {
++i;
}
}
if (line[i++] != '"') {
input_file_complain(filename, linenum,
"missing \" after file name");
goto fail;
}
} else {
for (i = 0; !isspace(line[i]) && line[i] != '\0'; ++i)
;
}
for (i = 0; !isspace(line[i]) && line[i] != '\0'; ++i)
;
if (!isspace(line[i])) {
input_file_complain(filename, linenum,
"expected space after file name");
input_file_complain(f, "expected space after file name");
goto fail;
}
ent->name = calloc(1, i + 1);
if (ent->name == NULL) {
oom(filename, linenum);
oom(f);
goto fail;
}
memcpy(ent->name, line, i);
if (ent->name[0] == '"')
unescape_name(ent->name);
if (canonicalize_name(ent->name)) {
input_file_complain(filename, linenum, "invalid file name");
input_file_complain(f, "invalid file name");
goto fail;
}
if (ent->name[0] == '\0') {
input_file_complain(filename, linenum,
"refusing to add entry for '/'");
input_file_complain(f, "refusing to add entry for '/'");
goto fail;
}
if (strlen(ent->name) > 0xFFFF) {
input_file_complain(filename, linenum, "name too long");
input_file_complain(f, "name too long");
goto fail;
}
@ -98,23 +59,20 @@ static image_entry_t *filelist_mkentry(char *line, const char *filename,
/* mode */
if (!isdigit(*line)) {
input_file_complain(filename, linenum,
"expected numeric mode after file name");
input_file_complain(f, "expected numeric mode after file name");
goto fail;
}
while (isdigit(*line)) {
if (*line > '7') {
input_file_complain(filename, linenum,
"mode must be octal number");
input_file_complain(f, "mode must be octal number");
goto fail;
}
ent->mode = (ent->mode << 3) | (*(line++) - '0');
}
if (!isspace(*line)) {
input_file_complain(filename, linenum,
"expected space after file mode");
input_file_complain(f, "expected space after file mode");
goto fail;
}
@ -124,8 +82,7 @@ static image_entry_t *filelist_mkentry(char *line, const char *filename,
/* uid */
if (!isdigit(*line)) {
input_file_complain(filename, linenum,
"expected numeric UID after mode");
input_file_complain(f, "expected numeric UID after mode");
goto fail;
}
@ -133,8 +90,7 @@ static image_entry_t *filelist_mkentry(char *line, const char *filename,
ent->uid = (ent->uid * 10) + (*(line++) - '0');
if (!isspace(*line)) {
input_file_complain(filename, linenum,
"expected space after UID");
input_file_complain(f, "expected space after UID");
goto fail;
}
@ -142,8 +98,7 @@ static image_entry_t *filelist_mkentry(char *line, const char *filename,
/* gid */
if (!isdigit(*line)) {
input_file_complain(filename, linenum,
"expected numeric GID after UID");
input_file_complain(f, "expected numeric GID after UID");
goto fail;
}
@ -153,7 +108,7 @@ static image_entry_t *filelist_mkentry(char *line, const char *filename,
line = skipspace(line);
/* remove processed data */
memmove(start, line, strlen(line) + 1);
memmove(f->line, line, strlen(line) + 1);
return ent;
fail:
free(ent->name);
@ -161,10 +116,9 @@ fail:
return NULL;
}
int filelist_mkdir(char *line, const char *filename,
size_t linenum, void *obj)
int filelist_mkdir(input_file_t *f, void *obj)
{
image_entry_t *ent = filelist_mkentry(line,filename,linenum,S_IFDIR);
image_entry_t *ent = filelist_mkentry(f, S_IFDIR);
image_entry_t **listptr = obj;
if (ent == NULL)
@ -175,24 +129,22 @@ int filelist_mkdir(char *line, const char *filename,
return 0;
}
int filelist_mkslink(char *line, const char *filename,
size_t linenum, void *obj)
int filelist_mkslink(input_file_t *f, void *obj)
{
image_entry_t *ent = filelist_mkentry(line,filename,linenum,S_IFLNK);
image_entry_t *ent = filelist_mkentry(f, S_IFLNK);
image_entry_t **listptr = obj;
if (ent == NULL)
return -1;
ent->data.symlink.target = strdup(line);
ent->data.symlink.target = strdup(f->line);
if (ent->data.symlink.target == NULL) {
oom(filename, linenum);
oom(f);
goto fail;
}
if (strlen(ent->data.symlink.target) > 0xFFFF) {
input_file_complain(filename, linenum,
"symlink target too long");
input_file_complain(f, "symlink target too long");
goto fail;
}
@ -204,10 +156,9 @@ fail:
return -1;
}
int filelist_mkfile(char *line, const char *filename,
size_t linenum, void *obj)
int filelist_mkfile(input_file_t *f, void *obj)
{
image_entry_t *ent = filelist_mkentry(line,filename,linenum,S_IFREG);
image_entry_t *ent = filelist_mkentry(f, S_IFREG);
image_entry_t **listptr = obj;
const char *ptr;
struct stat sb;
@ -215,22 +166,22 @@ int filelist_mkfile(char *line, const char *filename,
if (ent == NULL)
return -1;
if (line[0] == '\0') {
ptr = strrchr(filename, '/');
if (f->line[0] == '\0') {
ptr = strrchr(f->filename, '/');
if (ptr == NULL) {
ent->data.file.location = strdup(ent->name);
} else {
asprintf(&ent->data.file.location, "%.*s/%s",
(int)(ptr - filename), filename,
(int)(ptr - f->filename), f->filename,
ent->name);
}
} else {
ent->data.file.location = strdup(line);
ent->data.file.location = strdup(f->line);
}
if (ent->data.file.location == NULL) {
oom(filename, linenum);
oom(f);
goto fail;
}
@ -241,8 +192,7 @@ int filelist_mkfile(char *line, const char *filename,
if (sizeof(off_t) > sizeof(uint64_t) &&
sb.st_size > (off_t)(~((uint64_t)0))) {
input_file_complain(filename, linenum,
"input file is too big");
input_file_complain(f, "input file is too big");
goto fail;
}
@ -256,18 +206,17 @@ fail:
return -1;
}
static int filelist_mknod(char *line, const char *filename,
size_t linenum, void *obj)
static int filelist_mknod(input_file_t *f, void *obj)
{
image_entry_t **listptr = obj, *ent;
unsigned int maj, min;
char *ptr;
ent = filelist_mkentry(line, filename, linenum, S_IFCHR);
ent = filelist_mkentry(f, S_IFCHR);
if (ent == NULL)
return -1;
switch (line[0]) {
switch (f->line[0]) {
case 'c':
case 'C':
break;
@ -279,10 +228,10 @@ static int filelist_mknod(char *line, const char *filename,
goto fail;
}
if (!isspace(line[1]))
if (!isspace(f->line[1]))
goto fail;
ptr = line + 1;
ptr = f->line + 1;
while (isspace(*ptr))
++ptr;
@ -296,8 +245,7 @@ static int filelist_mknod(char *line, const char *filename,
return 0;
fail:
image_entry_free(ent);
input_file_complain(filename, linenum,
"error in device specification");
input_file_complain(f, "error in device specification");
return -1;
}
@ -333,8 +281,12 @@ static int alloc_file_ids(image_entry_t *list)
int filelist_read(const char *filename, image_entry_t **out)
{
image_entry_t *list = NULL;
input_file_t f;
if (process_file(filename, line_hooks, NUM_LINE_HOOKS, &list))
if (open_file(&f, filename))
return -1;
if (process_file(&f, line_hooks, NUM_LINE_HOOKS, &list))
goto fail;
if (list != NULL) {
@ -344,9 +296,11 @@ int filelist_read(const char *filename, image_entry_t **out)
goto fail;
}
cleanup_file(&f);
*out = list;
return 0;
fail:
cleanup_file(&f);
image_entry_free_list(list);
return -1;
}

6
main/cmd/pack/pack.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
static const struct option long_opts[] = {
@ -65,7 +64,9 @@ static int cmd_pack(int argc, char **argv)
}
if (repodir == NULL) {
repodir = REPODIR;
fputs("missing argument: repository directory\n", stderr);
tell_read_help(argv[0]);
return EXIT_FAILURE;
}
if (optind < argc)
@ -115,7 +116,6 @@ static command_t pack = {
" --file-list, -l <path> Specify a file containing a list of input files.\n"
" --repo-dir, -r <path> Specify the output repository path to store the\n"
" package in.\n"
" If not set, defaults to " REPODIR ".\n"
" --description, -d <path> Specify a file containing a description of the\n"
" package, including information such as package\n"
" dependencies, the actual package name, etc.\n"

17
main/cmd/pack/pack.h
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#ifndef PACK_H
#define PACK_H
@ -21,11 +20,10 @@
#include "filelist/image_entry.h"
#include "comp/compressor.h"
#include "pkg/pkgformat.h"
#include "pkg/pkgwriter.h"
#include "compressor.h"
#include "pkgformat.h"
#include "pkgwriter.h"
#include "command.h"
#include "config.h"
typedef struct dependency_t {
struct dependency_t *next;
@ -40,14 +38,11 @@ typedef struct {
char *name;
} pkg_desc_t;
int filelist_mkdir(char *line, const char *filename,
size_t linenum, void *obj);
int filelist_mkdir(input_file_t *f, void *obj);
int filelist_mkslink(char *line, const char *filename,
size_t linenum, void *obj);
int filelist_mkslink(input_file_t *f, void *obj);
int filelist_mkfile(char *line, const char *filename,
size_t linenum, void *obj);
int filelist_mkfile(input_file_t *f, void *obj);
int filelist_read(const char *filename, image_entry_t **out);

1
main/cmd/pack/write_files.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
static int write_file(pkg_writer_t *wr, image_entry_t *ent)

1
main/cmd/pack/write_hdr.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
int write_header_data(pkg_writer_t *wr, pkg_desc_t *desc)

1
main/cmd/pack/write_toc.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include "pack.h"
static int write_entry(pkg_writer_t *wr, image_entry_t *it)

5
main/cmd/unpack.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <getopt.h>
#include <stdlib.h>
#include <unistd.h>
@ -7,9 +6,9 @@
#include "util/util.h"
#include "pkg/pkgreader.h"
#include "pkg/pkgio.h"
#include "pkgreader.h"
#include "command.h"
#include "pkgio.h"
static const struct option long_opts[] = {
{ "root", required_argument, NULL, 'r' },

1
main/command.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

31
main/compressor.c Normal file
View file

@ -0,0 +1,31 @@
#include <string.h>
#include "compressor.h"
static compressor_t *list = NULL;
void compressor_register(compressor_t *compressor)
{
compressor->next = list;
list = compressor;
}
compressor_t *compressor_by_name(const char *name)
{
compressor_t *it = list;
while (it != NULL && strcmp(it->name, name) != 0)
it = it->next;
return it;
}
compressor_t *compressor_by_id(PKG_COMPRESSION id)
{
compressor_t *it = list;
while (it != NULL && it->id != id)
it = it->next;
return it;
}

20
lib/comp/lzma.c → main/compressors/lzma.c
View file

@ -1,11 +1,10 @@
/* SPDX-License-Identifier: ISC */
#include <stdbool.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <lzma.h>
#include "internal.h"
#include "compressor.h"
#define CHUNK_SIZE 16384
@ -17,7 +16,6 @@ typedef struct {
int used;
bool eof;
bool error;
size_t dict_size;
} lzma_stream_t;
static ssize_t lzma_write(compressor_stream_t *base,
@ -101,7 +99,7 @@ static void lzma_destroy(compressor_stream_t *base)
free(lzma);
}
static compressor_stream_t *create_stream(bool compress, size_t dict_size)
static compressor_stream_t *create_stream(bool compress)
{
lzma_stream_t *lzma = calloc(1, sizeof(*lzma));
compressor_stream_t *base;
@ -114,7 +112,6 @@ static compressor_stream_t *create_stream(bool compress, size_t dict_size)
}
lzma->action = LZMA_RUN;
lzma->dict_size = dict_size;
base = (compressor_stream_t *)lzma;
base->write = lzma_write;
@ -128,9 +125,6 @@ static compressor_stream_t *create_stream(bool compress, size_t dict_size)
if (lzma_lzma_preset(&opt_lzma2, LZMA_PRESET_DEFAULT))
goto fail;
if (dict_size)
opt_lzma2.dict_size = dict_size;
filters[0].id = LZMA_FILTER_LZMA2;
filters[0].options = &opt_lzma2;
@ -153,21 +147,23 @@ fail:
return NULL;
}
static compressor_stream_t *lzma_compress(compressor_t *cmp, void *options)
static compressor_stream_t *lzma_compress(compressor_t *cmp)
{
(void)cmp;
return create_stream(true, options == NULL ? 0 : *((size_t *)options));
return create_stream(true);
}
static compressor_stream_t *lzma_uncompress(compressor_t *cmp)
{
(void)cmp;
return create_stream(false, 0);
return create_stream(false);
}
compressor_t comp_lzma = {
static compressor_t lzma = {
.name = "lzma",
.id = PKG_COMPRESSION_LZMA,
.compression_stream = lzma_compress,
.uncompression_stream = lzma_uncompress,
};
REGISTER_COMPRESSOR(lzma)

19
lib/comp/none.c → main/compressors/none.c
View file

@ -1,10 +1,9 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <zlib.h>
#include "internal.h"
#include "compressor.h"
#define CHUNK_SIZE 16384
@ -62,12 +61,11 @@ static void dummy_destroy(compressor_stream_t *base)
free(base);
}
static compressor_stream_t *create_dummy_stream(compressor_t *cmp,
void *options)
static compressor_stream_t *create_dummy_stream(compressor_t *cmp)
{
dummy_stream_t *dummy = calloc(1, sizeof(*dummy));
compressor_stream_t *base;
(void)cmp; (void)options;
(void)cmp;
if (dummy == NULL) {
perror("creating dummy compressor stream");
@ -82,14 +80,11 @@ static compressor_stream_t *create_dummy_stream(compressor_t *cmp,
return base;
}
static compressor_stream_t *create_dummy_uncompressor(compressor_t *cmp)
{
return create_dummy_stream(cmp, NULL);
}
compressor_t comp_none = {
static compressor_t none = {
.name = "none",
.id = PKG_COMPRESSION_NONE,
.compression_stream = create_dummy_stream,
.uncompression_stream = create_dummy_uncompressor,
.uncompression_stream = create_dummy_stream,
};
REGISTER_COMPRESSOR(none)

11
lib/comp/zlib.c → main/compressors/zlib.c
View file

@ -1,11 +1,10 @@
/* SPDX-License-Identifier: ISC */
#include <stdbool.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <zlib.h>
#include "internal.h"
#include "compressor.h"
#define CHUNK_SIZE 16384
@ -147,9 +146,9 @@ static compressor_stream_t *create_stream(bool compress)
return base;
}
static compressor_stream_t *zlib_compress(compressor_t *cmp, void *options)
static compressor_stream_t *zlib_compress(compressor_t *cmp)
{
(void)cmp; (void)options;
(void)cmp;
return create_stream(true);
}
@ -159,9 +158,11 @@ static compressor_stream_t *zlib_uncompress(compressor_t *cmp)
return create_stream(false);
}
compressor_t comp_zlib = {
static compressor_t zlib = {
.name = "zlib",
.id = PKG_COMPRESSION_ZLIB,
.compression_stream = zlib_compress,
.uncompression_stream = zlib_uncompress,
};
REGISTER_COMPRESSOR(zlib)

1
main/pkg.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <stdio.h>

3
lib/pkg/pkg_unpack.c → main/pkg_unpack.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <sys/types.h>
#include <sys/stat.h>
#include <stdbool.h>
@ -8,7 +7,7 @@
#include <errno.h>
#include <fcntl.h>
#include "pkg/pkgio.h"
#include "pkgio.h"
#include "util/util.h"
static int create_hierarchy(int dirfd, image_entry_t *list, int flags)

3
lib/pkg/pkgio_rd_image_entry.c → main/pkgio_rd_image_entry.c
View file

@ -1,10 +1,9 @@
/* SPDX-License-Identifier: ISC */
#include <sys/types.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <stdio.h>
#include "pkg/pkgio.h"
#include "pkgio.h"
#include "util/util.h"
static int read_extra(pkg_reader_t *pkg, image_entry_t *ent)

36
lib/pkg/pkgreader.c → main/pkgreader.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <stdbool.h>
#include <stdlib.h>
#include <unistd.h>
@ -8,9 +7,9 @@
#include <errno.h>
#include <ctype.h>
#include "comp/compressor.h"
#include "util/util.h"
#include "pkg/pkgreader.h"
#include "pkgreader.h"
#include "compressor.h"
struct pkg_reader_t {
int fd;
@ -235,29 +234,26 @@ record_t *pkg_reader_current_record_header(pkg_reader_t *rd)
ssize_t pkg_reader_read_payload(pkg_reader_t *rd, void *out, size_t size)
{
ssize_t ret, total = 0;
ssize_t ret;
do {
if (rd->have_error)
return -1;
if (rd->have_error)
return -1;
if (rd->have_eof || rd->offset_raw == rd->current.raw_size)
break;
if (rd->have_eof || rd->offset_raw == rd->current.raw_size)
return 0;
if (prefetch_compressed(rd))
return -1;
if (prefetch_compressed(rd))
return -1;
ret = rd->stream->read(rd->stream, out, size);
if (ret < 0)
return -1;
if (size >= (rd->current.raw_size - rd->offset_raw))
rd->stream->flush(rd->stream);
rd->offset_raw += ret;
out = (char *)out + ret;
size -= ret;
total += ret;
} while (size > 0);
ret = rd->stream->read(rd->stream, out, size);
if (ret < 0)
return -1;
return total;
rd->offset_raw += ret;
return ret;
}
int pkg_reader_rewind(pkg_reader_t *rd)

7
lib/pkg/pkgwriter.c → main/pkgwriter.c
View file

@ -1,4 +1,3 @@
/* SPDX-License-Identifier: ISC */
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
@ -6,7 +5,7 @@
#include <stdio.h>
#include <errno.h>
#include "pkg/pkgwriter.h"
#include "pkgwriter.h"
#include "util/util.h"
struct pkg_writer_t {
@ -96,7 +95,7 @@ pkg_writer_t *pkg_writer_open(const char *path, bool force)
return NULL;
}
wr->stream = cmp->compression_stream(cmp, NULL);
wr->stream = cmp->compression_stream(cmp);
if (wr->stream == NULL) {
fputs("error creating compressor stream for package header\n",
stderr);
@ -153,7 +152,7 @@ int pkg_writer_start_record(pkg_writer_t *wr, uint32_t magic,
if (write_header(wr))
return -1;
wr->stream = cmp->compression_stream(cmp, NULL);
wr->stream = cmp->compression_stream(cmp);
if (wr->stream == NULL)
return -1;