diff --git a/README.md b/README.md index f7753d2..17a2466 100644 --- a/README.md +++ b/README.md @@ -6,73 +6,26 @@ Build those damn .deb's How this shit works ------------------- -![](doc/buildchain.png) +The build chain relies on `sbuild`, a wrapper tool for building the `.deb`, and `reprepro` to handle the apt repo part (signing and serving) -Content -------- +1. Initially, some chroots are prepared, one per available distribution (buster, bullseye, ...), pre-installed with build dependencies to speed up builds. See the `scripts/make-chroots` script. +2. `reprepro` is configured in [`/var/www/repo/debian/conf/distributions`](config/distributions). In this file, you'll find the supported distributions (aka codenames), and branches (aka components: stable, testing, unstable). It also declares what GPG key to use to sign the repo. +3. Upstream codes are kept in `gitrepos/` +4. Builds are launched, either manually via `./ynh-build` and `scripts/ynh-custom-builds` - or via a cron job (c.f. `./daily-build` which in fact runs every 15ish mins?) + - These scripts usually perform consistency checks, or tweak the changelog / version number, and then call `scripts/build_deb`, itself calling `sbuild` and `reprepro include` + - In the most simple case, the software being build is arch-independent, such as Python. In that case, it is pretty straightforward : `sbuild` starts the chroot, in which build dependencies are installed, and `dpkg-buildpackage` is called. + - In more complex cases, the software is arch-dependent. In that case, `sbuild` is called multiple times with `--host=$arch`. It will still run in the same amd64 chroot, but will install dependencies for the foreign architecture (such as the C compiler for armhf, or lib-foobar-dev for armhf), and build the package using these. + - For every `.deb`/`.changes` generated, `reprepro include` is called to add the new build to the apt repo. The builds are signed with the key declared in `conf/distributions`. +5. ??? +6. PROFIT! -The script `init.sh` is here to be able to easily redeploy the whole ecosystem on a new machine. -#### Scripts to handle common tasks - -- `ynh-build`, to build a new version of Yunohost packages from the git repositories -- `build_deb` (in `/scripts/`, which is used by `ynh-build` but can also be used to manually build stuff like metronome ...) - -#### Tools used - -- `pbuilder` build the packages (using `dpkg-buildpackage` I believe) -- `rebuildd` handles 'build jobs' (gets notified by -- `reprepro` manages the HTTP(S) repo in `/var/www/repo/debian/` -- `nginx` to serve the repo -- (not yet repaired?) a custom service to handle hooks from github - -#### Services - -These services should be running for everything to work properly -- `rebuildd` -- `rebuildd-httpd` (and `nginx`) to have "nice" interface to monitor and read build logs -- (not yet repaired?) the github webhook handling service - -Useful commands +Troubleshooting --------------- -- `rebuildd-job list` to list jobs -- `rebuildd` starts the rebuildd server/daemon - for now I have to start it manually and `disown` it. The service should be working but there's some weird stuff about lxc making it crashed ? -- `rebuildd-httpd 127.0.0.1:9998` starts the monitoring/log web interface - same as `rebuildd`, gotta start it manually for now :/ -- in `/var/www/repo/debian`, you can list available packages with `reprepro list jessie` - -Misc notes ----------- - -#### Hooks thingy - -At the start of the build, pbuilder will call the hooks in scripts/pbuilder/hooks - -#### If you need to rebuild custom packages (for instance, metronome ?) - -- See scripts/ynh-custom-builds - -#### Chroot images - -- To build stuff, pbuilder needs to chroot in environnement. -- These are contained in `images/$dist-$arch.tgz` -- You can rebuild them from `images/make-images` - -#### 'Packages' are generally 'source packages' for debian people - -Interesting note from [this page](http://tldp.org/HOWTO/html_single/Debian-Binary-Package-Building-HOWTO/) - ->an maintainers, packages are source packages, not binary packages. They never interact directly with the internals binary packages. In fact only 'dpkg-deb' and 'dpkg' developers need to know what they are. In fact it is not recommended to do so. -> ->If a developer were to explain someone how to build a Debian package, he will certainly explain how to make a source package and how to build it. -> ->On the other hand, not every developer wants to submit his software to Debian (yet), but still wants to profit from the advantages a packaging system like 'dpkg' offers without releasing package source code. Personally I will release my freeware projects still as tar.gz files with source code etc. for all kind of platforms, while I plan to offer more and more '.deb' packages for the convenience of Debian users who just want to install and use my software. - #### Relaunching a build manually with a shell ? -- Copy the 'shell after error' hook : `cp /usr/share/doc/pbuilder/examples/C10shell /home/vinaigrette/scripts/pbuilder/hooks/` -- cd /var/cache/rebuildd/build/ -- /home/vinaigrette/scripts/rebuildd/build-binaries stretch rspamd 1.6.4 armhf +If a build fails and needs to be debugged, you should run `export DEBUG=true`, and re-run the appropriate build command. This should add the option `--anything-failed-commands='%s'` to the `sbuild` command, which will then drop you in an interactive shell inside the chroot, right after the failure. This should help investigate what's happening. #### Removing "conflicting" sources @@ -97,13 +50,3 @@ reprepro remove reprepro -Vb /var/www/repo/debian remove buster python3-miniupnpc ``` (`-Vb /var/www/repo/debian` is where the repo is stored) - -### What do I do if a job crashed for some weird reason ? - -Sometimes build crash for random reason and just relaunching them fixes the -situation. But since that would be too simple, you can't just use the standard -'requeue' action of the `rebuildd-job` command because your job will end up in -WAIT_LOCKED indifinetely -.- ... - -Instead, you can re-add the job for the corresponding architecture. See -`./scripts/rebuildd-job add --help`. diff --git a/doc/.gitkeep b/doc/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/doc/buildchain.odg b/doc/buildchain.odg deleted file mode 100644 index 32dd046..0000000 Binary files a/doc/buildchain.odg and /dev/null differ diff --git a/doc/buildchain.png b/doc/buildchain.png deleted file mode 100644 index 24057df..0000000 Binary files a/doc/buildchain.png and /dev/null differ