Commit Graph

105 Commits

Author SHA1 Message Date
David Oberhollenzer 2ba65d6cfa Bump version
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-07-17 18:13:29 +02:00
David Oberhollenzer 80f298de55 Always quote files when producing a listing
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-07-17 18:13:29 +02:00
David Oberhollenzer 4a7b1d3b3c cleanup: move package dependency management to libpkg.a
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-07-17 12:45:23 +02:00
David Oberhollenzer a632656a60 cleanup: remove compressor block processing functions
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-07-17 12:34:14 +02:00
David Oberhollenzer ec1020ffc6 Add support for filenames with spaces in them
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-07-17 11:17:57 +02:00
David Oberhollenzer 6d053ff213 Update README
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-06-15 18:29:38 +02:00
David Oberhollenzer f426e4ef91 Remove pkg2sqfs
Rewritten and moved to a seperate project.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-06-09 02:18:27 +02:00
David Oberhollenzer e08c819606 Bump version number
This is a patch level release that mainly contains a fix for pkg2sqfs.
It also contains some cleanups and a binary compatibility breaking
change to the package file format.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-05-03 17:09:58 +02:00
David Oberhollenzer ba034260d4 Minor cleanup
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-05-03 00:14:48 +02:00
David Oberhollenzer ab753a402f Shrink table of contents header
We aren't gonna need 2^32 uids/gids, this is a package manager, not a
file system. Also, mode only uses 16 of 32 bits. We can shrink the
entire header to 8 bytes.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-28 21:37:26 +02:00
David Oberhollenzer f95f02f6be Add options for default repo dir & default install dir
This should make working with the pkg program in the toolchain build
system much cleaner.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-28 21:37:26 +02:00
David Oberhollenzer 5df2d5f730 Remove squashfs pseudo file listing generator
We no longer need this, since we can now produce our own squashfs images
directly from package files.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-28 16:04:54 +02:00
David Oberhollenzer ed6841ed17 fix: flush SquashFS meta block if *exactely* filled
Otherwise we can get really funky corner cases where an inode offset is
indicated as just 1 byte after the end of a block. The unsquashfs program
doesn't complain but the kernel implementation really doesn't like that.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-23 13:54:17 +02:00
David Oberhollenzer 7a70868fcd cleanup: move generic squashfs code to library
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-23 12:36:12 +02:00
David Oberhollenzer 5f0ee09c83 Bump version number
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 22:00:22 +02:00
David Oberhollenzer 75e169d06a fixup: squashfs limits number of dirents to 256
The kernel implementation BAILS if a directory header is followed by
more than a hard-coded maximum of entries (currently 256).

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 22:00:22 +02:00
David Oberhollenzer 9402492c16 Add mention of pkg2sqfs to README
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:37:42 +02:00
David Oberhollenzer 377596f37e pkg2sqfs: remove debug print
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:37:42 +02:00
David Oberhollenzer 33997bdb9e pkg2sqfs: fix size computation of directories
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:37:42 +02:00
David Oberhollenzer 00db5541e9 file list sort: remove duplicate directory entries
Can happen when merging multiple packages. But make sure they actually
are duplicates.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:43 +02:00
David Oberhollenzer 96d4d353d4 Remove unnecessary flush from unpacking loop
The uncompressor will know when end of file is reached. Flushing
prematurely won't help. Especially the way the loop was constructed,
using an external buffer that is a lot larger than the internal
buffer of the compressor will cause multiple erroneous flushes.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:43 +02:00
David Oberhollenzer b30e912e97 pkg2sqfs: add command line option to set timestamp
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:43 +02:00
David Oberhollenzer 576491eae6 pkg2sqfs: add command line option to select compressor
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:41 +02:00
David Oberhollenzer ac8e457c96 pkg2sqfs: set lzma dictionary size to block size
It's the default the kernel implementation expects if no options
are set.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:03 +02:00
David Oberhollenzer 214a5e917e Add a way to pass options to compressors
Currently only used by lzma compressor to set the dictionary size.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:03 +02:00
David Oberhollenzer e123eaf433 pkg2sqfs: implement meta data compression
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:02 +02:00
David Oberhollenzer f80e4e8d29 pkg2sqfs: implement data & fragment compression
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:02 +02:00
David Oberhollenzer 26b3683f7a Add block processing function to compressors
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:02 +02:00
David Oberhollenzer 765c836c3d Remove dependency on libbsd
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-18 14:08:02 +02:00
David Oberhollenzer d94bd8ff17 pkg2sqfs: sort directory entries asciibetically
The squashfs kernel implementation relies on the following properties:
 - directory entries are ordered asciibetically by name
 - directory entries have consecutive inode numbers
 - inode numbers are assigned in that order

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 16:54:29 +02:00
David Oberhollenzer cd79a4fccf make file list sorting more robust
If entries are sorted by length and some have the same length, sort
them asciibetically.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 16:30:01 +02:00
David Oberhollenzer eadeee25d9 pkg2sqfs: move some functions to more approrpiate places
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 16:30:01 +02:00
David Oberhollenzer 025ab1007b pkg2sqfs: Add command line options for default inode settings
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 16:29:59 +02:00
David Oberhollenzer db25ddc108 pkg2sqfs: always padd image file to device block size
The kernel implementation apparently requires that.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 16:29:22 +02:00
David Oberhollenzer 858779e42c Add pkg2sqfs utility stub
The goal is to transform a package file to a squashfs image. The mksquashfs
utility is a PITA with all kinds of idiotic not-thought-through corner
cases (e.g. the way "pseudo file definitions" have been tacked on, the
inabillity to set UID/GID for / *AND* have files not owned by root, the
way xattrs are handled, .....).

It would be preferable to fix mksquashfs itself, but the source code itself
is a horrid, unmaintained garbage pile. Its easier to write a small utility
instead that can turn a pkg file into a squashfs image in a way that
propperly treats file permissions and ownership, and is deterministic
(i.e. reproducable).

The utility currently generates an uncompressed squashfs image that
unsquashfs can unpack, but the kernel refuses the mount. The exact
problem still needs to be determined.

Furthermore, compression has to be implemented and some cleanup shoul
be done.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-17 01:33:57 +02:00
David Oberhollenzer c78dc2255f pkg reader: try to read *entire* requested chunk
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-09 16:02:31 +02:00
David Oberhollenzer aa6bbf2ef4 Move pkg-tool internal headers out of global include directory
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-05 12:36:47 +02:00
David Oberhollenzer 6cfd9a37bf Move all the package I/O code to libpkg library
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-05 12:33:50 +02:00
David Oberhollenzer 1205ca0f3b Move compressors to utility library
- Remove compressor registration interface. Doesn't work in a static
   library, since all the files containing no exported members are
   optimized away at link time.
 - Repack as seperate utility library.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-04-05 12:22:47 +02:00
David Oberhollenzer 63bad9786a Release version 0.3
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-31 13:39:21 +02:00
David Oberhollenzer 4d48e374d8 Add option to buildstrategy to produce a dot graph
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-30 23:40:19 +01:00
David Oberhollenzer 52f7bdc5c8 minor cleanup
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-30 18:31:40 +01:00
David Oberhollenzer b7e81c081b cleanup: split package provider code out of buildstrategy.c
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-30 18:20:33 +01:00
David Oberhollenzer 46f340b20f cleanup: split source package code out of buildstrategy.c
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-30 17:57:34 +01:00
David Oberhollenzer 8a52ea6717 Allow multiple source packages to provide the same binary package
In build strategy, record a linked list of providers instead of just
one per binary package. Add an option to scan a preference list in
addition.

When looking up the provider and there is more than one, choose the
preferred one. Produce an error if none is given.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-29 23:55:12 +01:00
David Oberhollenzer b2f1b5ef58 cleanup: buildstrategy
Split utility function to get a source package by name out from
handle provides.

Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-29 21:57:48 +01:00
David Oberhollenzer 48d74777e7 Fix buildstrategy command argument handlin
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-29 21:54:08 +01:00
David Oberhollenzer 680b1bb7f9 Bump version number
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-09 15:25:34 +01:00
David Oberhollenzer db6c2118c4 Add missing colon to optarg listing flag
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-09 15:25:34 +01:00
David Oberhollenzer 19e2f8aa8c Fix file list removal of input line
Signed-off-by: David Oberhollenzer <goliath@infraroot.at>
2019-03-09 14:57:59 +01:00