Commit graph

1604 commits

Author SHA1 Message Date
Jeff Wong
5ed07ecc93
Imagemagick enable delegate build and disabled using shared libraries ()
compiles to a single portable binary

Create a base builder package with compile time dependencies. Create an
imagemagick_builder to build imagemagick.

The base builder can be extended for other builders.

Add imagemagick runtime dependencies to discourse_dependencies image
avoid -dev libs

Statically compile as much as possible with --disable-shared and
--enable-delegate-build flags.
References:
https://stackoverflow.com/questions/47031789/imagemagick-100-static-build-for-linux
https://www.imagemagick.org/discourse-server/viewtopic.php?t=14259

Add fonts-urw-base35 for NimbusSans-Regular, needed for letter avatar generation

Copy over the resulting magick bin, as well as etc and share files from the
compilation. etc is needed for magick to run, share is not, but contains
translations for errors which Discourse tests are dependent on reading from.

Create symlinks for other magick tooling - imagemagick creates symlink tool
names that Discourse uses. These *could* be dropped if Discourse decided to use
`magick {toolname}` rather than `{toolname}`.

Add nginx compile dependency - building nginx still needs libfreetype6
This was implicitly installed previously. Removing the imagemagick build from
base broke the next nginx build. Add this dependency back in. This dependency
can be removed once we build nginx separately as well.
2024-12-17 13:28:54 +08:00
Alan Guo Xiang Tan
e70c3596b8
DEV: Bump rust version to 1.82.0 ()
This is to resolve an error we are seeing in our arm64 dev builds

```
24 49.46 error: package `apple-xar v0.20.0` cannot be built because it requires rustc 1.81 or newer, while the currently active rustc version is 1.75.0
24 49.46 Either upgrade to rustc 1.81 or newer, or use
```
2024-12-17 07:57:38 +08:00
Alan Guo Xiang Tan
dfaf203e16
DEV: Periodic bump of launcher base image ()
Pulls in various fixes and updates
2024-12-10 10:39:20 +08:00
Alan Guo Xiang Tan
a0be7c92ce
Install postgresql-client-${PG_MAJOR} instead of postgresql-client ()
This ensures that the client tools we installed are compatible with the
version of the Postgres cluster.
2024-12-03 10:51:15 +08:00
Jeff Wong
69d7558c98
Move source for launcher2 to its own repository. () 2024-11-18 20:37:27 -08:00
Jeff Wong
b8f9edab75
jemalloc build with install_lib_shared libraries ()
Only output shared lib libraries, we should not need static libraries
2024-11-12 17:54:29 -08:00
Alan Guo Xiang Tan
58e0dce1a8
Bump Ruby to 3.3.6 ()
This is a routine update that includes minor bug fixes.

See https://www.ruby-lang.org/en/news/2024/11/05/ruby-3-3-6-released/
2024-11-13 08:49:34 +08:00
Jeff Wong
c77b53047b
FEATURE: print out compressed image sizes for amd64 ()
print out compressed image sizes for amd64 in a build step for PRs. Allows for PRs to be automatically reviewed for space efficiency
2024-11-10 23:10:49 -08:00
Jeff Wong
3792f7155c
DEV: slim image by dropping building jhead from source, and dropping optipng ()
jhead from apt pulls in a ton of packages, and optipng isn't used. Reduces compressed image size by 8MB.

main: 648.38MB
compiling jhead + dropping optipng: 640.14MB
2024-11-07 19:41:21 -08:00
Alan Guo Xiang Tan
1ae67ef87e
DEV: Remove cache mounts from image/base Dockerfile ()
There are no substantial benefits from using cache mounts here so we
are dropping those mounts.
2024-11-08 07:27:55 +08:00
Jeff Wong
785fda38c2
DEV: slim image, drop unused packages and use cache mounts ()
* DEV: slim image, drop unused packages and use cache mounts

apt using caches for /var/cache/apt, /var/cache/debconf, and var/lib/apt.

Drop autoconf and build-essential from installs, in favor of selective cmake,
g++, pkg-config, and patch packages.

drop apt-get -y upgrade in dockerfile. We should inherit upgrades from base
images. No need to apt-mark hold initscripts now that we're not running `upgrade`

Remove calls to vim as we no longer install vim here. Remove comment for
slimming locales as we have now done so.

merge all apt layers, minimizing amount of --mount arguments in dockerfile
2024-11-04 10:49:58 -08:00
Tealk
7d548ad4ae
Fix Warnung: the "listen ... http2" directive is deprecated ()
* Fix Warnung: the "listen ... http2" directive is deprecated

Signed-off-by: Tealk <tealk@anzah.email>

* Update templates/web.ssl.template.yml

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

* add on to http2

---------

Signed-off-by: Tealk <tealk@anzah.email>
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-10-16 10:17:43 +08:00
Jeff Wong
a1d8d0bbcb
DEV: generate only en_us locales ()
install locales-all installs 245MB
Generating common and en_us locales installs 22MB

update locales for postgres templates

`locale-gen $LANG` doesn't seem to actually do anything without updates to
`/etc/locale.gen`. Update the scripts to uncomment $LANG before running `update-locale`.
2024-10-14 23:19:13 -07:00
Jeff Wong
20e33fbfd9
DEV: force disable precompile on configure ()
Configure option does a precompile, which should forcefully disable
`PRECOMPILE_ON_BOOT` to prevent unnecessary precompile on boot when
`PRECOMPILE_ON_BOOT` env is set in templates.
2024-09-30 05:46:23 -07:00
Alan Guo Xiang Tan
9835070940
Fix the broken build ()
Follow-up to b47bd562cc

`Error response from daemon: No such image:
discourse/base:2.0.20240909-1149-arm6`
2024-09-09 20:47:50 +08:00
Alan Guo Xiang Tan
b47bd562cc
Release discourse/base images for discourse/discourse stable branch ()
This commit updates our `build` workflow to push the following manifests
which can be used to run Discourse against the stable branch.

1. discourse/base:2.0.$TIMESTAMP-stable
2. discourse/base:release-stable

Co-authored-by: David Taylor <david@taylorhq.com>
2024-09-09 19:49:14 +08:00
David Taylor
3384f2c71c
Improve node_modules cleanup logic for stable branch ()
- makes decision based on current state of directory, instead of `$version`
- cleans up the correct directories
- only cleans up the contents. This is important if node_modules directores are mounted volumes (e.g. in devcontainer)
2024-09-09 11:13:18 +01:00
Jeff Wong
34f016a7a6
DEV: export default base image ()
allows default base image to be more portable.
2024-09-08 23:11:55 -07:00
cheungtitus
d00c2e75d5
Added support of Linux on Mac with ARM processors ()
Co-authored-by: TItus Cheung <cheungtitus@kenrui-group.com>
2024-09-09 10:09:20 +08:00
Jeremy
6c37c5dacd
Update Discourse installation support in China (pnpm) ()
This commit updates 2 things.
1. Updates the yarn hook to replace the npm mirror before `pnpm install`.
2. Removes the `yarn.lock` patch as pnpm is now used.

After applying these modifications, I successfully installed Discourse on the Tencent Cloud China server. No more network problems.
2024-09-06 17:10:52 +01:00
David Taylor
928adfd09f
Suppress pnpm upgrade notices in logs () 2024-09-06 10:28:15 +01:00
Alan Guo Xiang Tan
189cfe2ac9
Clean up nginx folder after installation. () 2024-09-06 15:09:59 +08:00
Jeff Wong
e84629a754
DEV: update golang versions ()
update golang versions and bump dependencies
2024-09-05 08:08:17 +08:00
Jeff Wong
9e25686a99
FEATURE: add an option for custom namespaces ()
Add an option to select a targeted namespace, env var DISCOURSE_NAMESPACE

Configure:
Add `source-tag` to select which tag to configure from.
Rename old `tag` option to `target-tag` to differentiate from `source-tag` option

Migrate:
Add `tag` to select which tag to migrate with
2024-09-04 06:42:02 -07:00
Alan Guo Xiang Tan
3186478f51
Reduce min docker version to 20.10.0 ()
Follow up to 31931deae8

Docker 20.10.0 works for now so be less aggressive in forcing people to
upgrade.
2024-09-04 11:35:26 +08:00
Alan Guo Xiang Tan
31931deae8
DEV: Bump minimal Docker version to 24.0.7 ()
When running the newer Debian bookworm based images, we are seeing
`(ThreadError) can't create Thread: Operation not permitted` errors when
trying to spawn a thread in Ruby.

A similar issue was reported in https://github.com/docker-library/ruby/issues/429#issuecomment-1708908819
and the fix here is to upgrade Docker. Either way, we should probably
update because Docker 17 has been EOF for many many years.
2024-09-04 10:47:58 +08:00
Jeff Wong
fbfff14a83
add tmp directory to gitignore () 2024-09-03 12:12:34 +08:00
Jeff Wong
3e93a3e9b2
FEATURE: add autocomplete () 2024-09-02 18:54:14 -07:00
Jeff Wong
d87acd4ccd
FEATURE: add runtime features ()
adds commands for: start, run, stop, cleanup, destroy, logs, enter, restart, rebuild -- carrying over existing run commands from launcher1.

Rebuild will also do its best to minimize downtime with the following steps:

* Detect if Discourse is running as a single container or external DB
* Detect if db:migrate is configured to run on container boot
* Build initial container (keeping existing one online)
* Exit running containers if it's a single container (otherwise keeps existing online)
* Run migrations
  * Defer migrations if db:migrate is configured to run on container boot
  * Run migrations with SKIP_POST_DEPLOYMENT_MIGRATIONS=1 if it's a 2 container setup
  * Otherwise, run all migrations
* Destroy the old container (finally stopping the current, if it's still up here)
* Start the new container
* Run post-deploy migrations
  * Run migrations with SKIP_POST_DEPLOYMENT_MIGRATIONS=0 if it's a 2 container setup
2024-09-03 06:09:38 +08:00
Jeff Wong
de17bef722
DEV: install redis data folder if none exists () 2024-09-02 13:19:43 -07:00
David Taylor
87871bc430
Make compatible with pnpm ()
Switches behavior based on the presence of `yarn.lock`

See https://github.com/discourse/discourse/pull/28671
2024-09-02 20:55:57 +01:00
Alan Guo Xiang Tan
fe48fa2c5c
Switch to discourse/ruby:3.3.4-bookworm-slim () 2024-09-02 11:49:43 +08:00
Jeff Wong
44134a21bb
DEV: remove bundle cache ()
On the current image this removes ~78MB from the layer

Uncompressed image sizes:
before: 3.3GB
after: 3.23GB
2024-08-30 18:05:34 -07:00
Alan Guo Xiang Tan
cfe8aa0df0
Remove use of discourse/base:aarch64 in launcher2 ()
Since fc61b8a850, we started shipping
manifests for the `discourse/base` images so we don't have to
specifically pull the tag for arm64.
2024-08-29 19:03:17 -07:00
Alan Guo Xiang Tan
9cd8186278
Drop mac-os tests for launcher_go ()
The runners cost 10 times more and there are no macOS specific tests
which we are running.
2024-08-29 17:39:27 -07:00
Jeff Wong
85a6e38644
FEATURE: Add workflow for launcher_go () 2024-08-29 15:59:12 +08:00
Jeff Wong
b7e54edb04
FEATURE: add migrate and bootstrap commands ()
Adds command for migrate only. Adds batch commands for bootstrap (build+migrate+configure)
2024-08-29 11:01:28 +08:00
Jeff Wong
7c18fce1b4
DEV: remove oxipng archive in layer ()
* DEV: remove oxipng archive in layer

Saves ~52MB for the layer

* DEV: Remove nginx archives

Remove another 1.2MB from image
2024-08-28 19:34:24 -07:00
Alan Guo Xiang Tan
1740b529a6
DEV: Improve assertions of cli_build_test.go ()
This commit improves the assertions by testing against the entire
command string and env so that we can be sure of the full command we are
running.
2024-08-28 10:02:49 -07:00
David Taylor
83c7e4e587
Increase timeout for scheduled build job ()
Now that base & dev images are built in the same job, we need a little more time
2024-08-28 17:19:59 +08:00
Alan Guo Xiang Tan
595dd6af1c
DEV: Minor readability improvements to launcher2 () 2024-08-28 14:20:17 +08:00
Jeff Wong
d3ae996af6
FEATURE: add configure command ()
Add 'configure' command - If run after the "build" command, this is equivalent to today's 'bootstrap' command.

Note that unlike build command, a docker run+commit pattern needs to be used here as this requires a running database + mounted volumes.
2024-08-28 13:13:39 +08:00
Jeff Wong
0808c17a04
[WIP] Launcher2 ()
Partial merge of Launcher2's CLI

* FEATURE: merge launcher2 - build command

merge launcher2, only build commands
2024-08-26 12:36:41 -07:00
Alan Guo Xiang Tan
ceb92647b4
Fix timeout for arm64 builds not correctly set () 2024-08-26 15:05:20 +08:00
Alan Guo Xiang Tan
def0f9ec3f
Base image in launcher ()
This pulls in the Nginx 1.26.1 update, Ruby 3.3.4 update and Debian
bookworm update.
2024-08-26 13:22:33 +08:00
David Taylor
fc61b8a850
DEV: Introduce arm64 dev image and multi-arch manifests ()
- Combines dev image build into the `base` job
- Calculates a single timestamp for all builds
- Pushes timestamped per-arch images to Dockerhub for base, base-slim and dev images
- If both arch build jobs are successful, multiarch manifests are generated & pushed
2024-08-26 13:10:28 +08:00
David Taylor
b110891382
Update GitHub actions runners to ubuntu-22.04 () 2024-08-22 11:09:50 +01:00
Alan Guo Xiang Tan
84644fcbea
Bump Ruby to 3.3.4 ()
Ruby 3.3.4 has fixes for some regressions in 3.3.3. I don't think it
affects us but still good to upgrade.
2024-08-22 12:25:15 +08:00
Alan Guo Xiang Tan
4c58e2b75b
Switch to debian:bookworm-slim image ()
Debian Bullseye is EOL.
2024-08-22 10:57:56 +08:00
Alan Guo Xiang Tan
1d7323204b
Ensure uid, gid for postgres, redis and discourse stays consistent ()
When updating to `debian/base:bookworm` from `debian/base:bullseye`, we
noticed that the uid of the `postgres` and `redis` user changed leading
to permission issues when trying to access directories of mounted volumn
which was previously created with the old uid. The change is because the
`_apt` user is assigned a uid of `42` in Debian bookworm instead of `100`
in Debian bullseye. As a result, the `postgres` user created by the `postgres`
package is automatically assigned a uid of `100` in Debian bookworm instead of `101` in Debian
bullseye.

This commit updates the `slim.Dockefile` to manually add the `postgres`
user and group assigning it a gid of `104` and uid of `101`. The `redis`
user and group is assigned a uid of `103` and a gid of `106`. The
`discourse` user and group is assigned a uid of `1000` and gid of
`1000`.
2024-08-22 10:57:29 +08:00