Merge pull request #114 from fflorent/pre-upgrade

Pre upgrade scripts

README.md

@@ -8,6 +8,7 @@ The `package_check.sh` script perform a series of tests on a package for verify
 The test results are printed directly in the terminal and stored in the log file Test_results.log
 
 The script is able to perform the following tests:
+
 - [Linter](https://github.com/YunoHost/package_linter)
 - Install/remove/reinstall at the root of a domain (`domain.tld/`)
 - Install/remove/reinstall in a subpath (`domain.tld/foobar`)
@@ -64,10 +65,11 @@ to propagate (sigh))
 Then you shall initialize LXD which will ask you a bunch of question. Usually
 answering the default (just pressing enter) to all questions is fine. Just pay
 attention to :
+
 - the storage backend driver. Possibly `zfs` is the best, but requires a kernel >= 5.x
-and corresponding kernel module loaded. You can fallback to the `dir` driver.
+  and corresponding kernel module loaded. You can fallback to the `dir` driver.
 - the size of the default storage it'll create (the default is 5G but you may
-want 10G for heavy usage ?) (if you're using the 'dir' driver, this won't be asked)
+  want 10G for heavy usage ?) (if you're using the 'dir' driver, this won't be asked)
 
 ```bash
 lxd init
@@ -109,6 +111,7 @@ lxc image export images:debian/buster/armhf
 ```
 
 This command will create two files.
+
 - rootfs.squashfs
 - lxd.tar.xz
 
@@ -136,14 +139,24 @@ lxc exec inspired-lamprey -- dpkg --print-architecture
 If the `build_base_lxc.sh` script detects that you are trying a cross container architecture, it will try to perform this hack
 
 ## Syntax of `check_process`
+
 > Except spaces, the syntax of this file must be respected.
 
 ```
 ;; Default test serie
 # Comment ignored
 	; pre-install
+		set -euxo pipefail
 		echo -n "Here your commands to execute in the container"
 		echo ", before each installation of the app."
+	; pre-upgrade
+		set -euxo pipefail
+		if [ "$FROM_COMMIT" == '65c382d138596fcb32b4c97c39398815a1dcd4e8' ]
+		then
+			resolve=127.0.0.1:443:$DOMAIN
+			# Example of request using curl
+			curl -X POST --data '{"some-key": "some-value"}' https://$DOMAIN/$PATH --resolve "$resolve"
+		fi
 	; Manifest
         # You need to provide default values for installation parameters ...
         # EXCEPT for special args: domain, path, admin, and is_public
@@ -181,38 +194,46 @@ If the `build_base_lxc.sh` script detects that you are trying a cross container
 ```
 
 ### `;; Default test serie`
+
 A name for the series of tests to perform.
 It's possible to create multiple tests series, all with the same syntax.
 All different series will be performed sequentially.
 
 ### `; pre-install`
-*Optional instruction*  
+
+_Optional instruction_
 If you have to execute a command or a group of commands before the installation. You can use this instruction.
 All the commands added after the instruction `; pre-install` will be executed in the container before each installation of the app.
 
 ### `; Manifest`
+
 List of manifest keys.
 All manifest keys need to be filled to perform the installation.
+
 > The manifest keys already in the file here are simply examples. Check the package manifest.
 
 ### `; Actions`
+
 List of arguments for each action that needs an argument.
 `action_argument` is the name of the argument, as you can find at the end of [action.arguments.**action_argument**].
 `arg1|arg2` are the different arguments to use for the tests. You can have as many arguments as you want, each separated by `|`.
 
-*Only `actions.toml` can be tested by package_check, not `actions.json`.*
+_Only `actions.toml` can be tested by package_check, not `actions.json`._
 
 ### `; Config_panel`
+
 List of arguments for each config_panel configuration.
 `main.categorie.config_example` is the complete toml entry for the argument of a configuration.
 `arg1|arg2` are the different arguments to use for the tests. You can as many arguments as you want, each separated by `|`.
 
-*Only `config_panel.toml` can be tested by package_check, not `config_panel.json`.*
+_Only `config_panel.toml` can be tested by package_check, not `config_panel.json`._
 
 ### `; Checks`
+
 List of tests to perform.
 Each test set to 1 will be performed by the script.
 If a test is not in the list, it will be ignored. It's similar to set the test at 0.
+
 - `pkg_linter`: Check the package with [package linter](https://github.com/YunoHost/package_linter)
 - `setup_root`: Installation at the root of a domain.
 - `setup_sub_dir`: Installation in a path.
@@ -223,22 +244,26 @@ If a test is not in the list, it will be ignored. It's similar to set the test a
 - `backup_restore`: Backup then restore.
 - `multi_instance`: Install the application 2 times, to verify its ability to be multi-instanced.
 - `port_already_use`: Provoke an error by opening the port before.
-        The `port_already_use` test may eventually take as argument the port number.  
+  The `port_already_use` test may eventually take as argument the port number.
-        The port number must be written into parentheses.  
+  The port number must be written into parentheses.
 - `change_url`: Try to change the url by 6 different ways. Root to path, path to another path and path to root. And the same thing, to another domain.
 - `actions`: All actions available in actions.toml
 - `config_panel`: All configurations available in config_panel.toml
 
 ### `;;; Upgrade options`
-*Optional instruction*  
+
+_Optional instruction_
 For each specified commit for an upgrade, allow to give a name for this version and the manifest parameters which will be used for the preliminary installation.
 If there's no name specified, the commit will be used.
 And if there's no manifest arguments, the default arguments of the check process will be used.
+
 > 3 variables have to be used for the arguments of the manifest, DOMAIN, PATH and USER.
 
 ---
+
 The `package_check.sh` script accept 6 arguments in addition of the package to be checked.
+
 - `--branch=branch-name`: Check a branch of the repository instead of master. Allow to check a pull request.
-	You can use an url with a branch, https://github.com/YunoHost-Apps/APP_ynh/tree/my_branch, to implicitly use this argument.
+  You can use an url with a branch, https://github.com/YunoHost-Apps/APP_ynh/tree/my_branch, to implicitly use this argument.
 - `--interactive`: Wait for user input between each tests
 - `--help`: Display help.

lib/tests.sh

@@ -32,10 +32,10 @@ _PREINSTALL () {
         local preinstall_script="$TEST_CONTEXT/preinstall.sh"
         echo "$preinstall_template" > "$preinstall_script"
         # Hydrate the template with variables
-        sed -i "s/\$USER/$TEST_USER/" "$preinstall_script"
+        sed -i "s/\$USER/$TEST_USER/g" "$preinstall_script"
-        sed -i "s/\$DOMAIN/$DOMAIN/" "$preinstall_script"
+        sed -i "s/\$DOMAIN/$DOMAIN/g" "$preinstall_script"
-        sed -i "s/\$SUBDOMAIN/$SUBDOMAIN/" "$preinstall_script"
+        sed -i "s/\$SUBDOMAIN/$SUBDOMAIN/g" "$preinstall_script"
-        sed -i "s/\$PASSWORD/$YUNO_PWD/" "$preinstall_script"
+        sed -i "s/\$PASSWORD/$YUNO_PWD/g" "$preinstall_script"
         # Copy the pre-install script into the container.
         lxc file push "$preinstall_script" "$LXC_NAME/preinstall.sh"
         # Then execute the script to execute the pre-install commands.
@@ -43,6 +43,31 @@ _PREINSTALL () {
     fi
 }
 
+_PREUPGRADE () {
+    local preupgrade_template="$(jq -r '.preupgrade_template' $current_test_infos)"
+    local commit=${1:-HEAD}
+
+    # Exec the pre-upgrade instruction, if there one
+    if [ -n "$preupgrade_template" ]
+    then
+        log_small_title "Running pre-upgrade steps"
+        # Copy all the instructions into a script
+        local preupgrade_script="$TEST_CONTEXT/preupgrade.sh"
+        echo "$preupgrade_template" >> "$preupgrade_script"
+        # Hydrate the template with variables
+        sed -i "s/\$USER/$TEST_USER/g" "$preupgrade_script"
+        sed -i "s/\$DOMAIN/$DOMAIN/g" "$preupgrade_script"
+        sed -i "s/\$SUBDOMAIN/$SUBDOMAIN/g" "$preupgrade_script"
+        sed -i "s/\$PASSWORD/$YUNO_PWD/g" "$preupgrade_script"
+        sed -i "s/\$FROM_COMMIT/$commit/g" "$preupgrade_script"
+        # Copy the pre-upgrade script into the container.
+        lxc file push "$preupgrade_script" "$LXC_NAME/preupgrade.sh"
+        # Then execute the script to execute the pre-upgrade commands.
+        LXC_EXEC "bash /preupgrade.sh"
+        return $?
+    fi
+}
+
 _INSTALL_APP () {
     local install_args="$(jq -r '.install_args' $current_test_infos)"
 
@@ -435,6 +460,10 @@ TEST_UPGRADE () {
 
     log_small_title "Upgrade..."
 
+    _PREUPGRADE "${commit}"
+    ret=$?
+    [ $ret -eq 0 ] || { log_error "Pre-upgrade instruction failed"; return 1; }
+
     # Upgrade the application in a LXC container
     _RUN_YUNOHOST_CMD "app upgrade $app_id --file /app_folder --force" \
         && _VALIDATE_THAT_APP_CAN_BE_ACCESSED "$SUBDOMAIN" "$check_path" "upgrade"

lib/tests_coordination.sh

@@ -80,6 +80,7 @@ parse_check_process() {
         # during tests
         local install_args=$(       extract_check_process_section "^; Manifest"     "^; " $test_serie_rawconf | sed 's/\s*(.*)$//g' | tr -d '"' | tr '\n' '&')
         local preinstall_template=$(extract_check_process_section "^; pre-install"  "^; " $test_serie_rawconf)
+        local preupgrade_template=$(extract_check_process_section "^; pre-upgrade"  "^; " $test_serie_rawconf)
         local action_infos=$(       extract_check_process_section "^; Actions"      "^; " $test_serie_rawconf)
         local configpanel_infos=$(  extract_check_process_section "^; Config_panel" "^; " $test_serie_rawconf)
 
@@ -130,9 +131,10 @@ parse_check_process() {
                 --arg test_type "$test_type" \
                 --arg test_arg "$test_arg" \
                 --arg preinstall_template "$preinstall_template" \
+                --arg preupgrade_template "$preupgrade_template" \
                 --arg install_args "${_install_args//\"}" \
                 --argjson extra "$extra" \
-                '{ $test_serie, $test_type, $test_arg, $preinstall_template, $install_args, $extra }' \
+                '{ $test_serie, $test_type, $test_arg, $preinstall_template, $preupgrade_template, $install_args, $extra }' \
                 > "$TEST_CONTEXT/tests/$test_id.json"
         }