[enh] Refactor backup management to pave the way to borg (#275)

* [enh] Use a csv to list file to backup
* [enh] Use csv python module
* [wip] Backup refactoring
* [wip] Backup class refactoring
* [enh] Add archivemount dependencies
* [wip] Restore refactoring
* [fix] Some error in this refactoring
* [fix] Missing backup key translation
* [fix] Bad YNH_CWD in hook backup
* [fix] App backup part was broken
* [fix] Restore operation was broken
* [fix] No compressed backup
* [fix] Don't commit backup path into csv if app backup fail
* [fix] Default backup collect_dir should be in tmp subdir
* [enh] Simplify a copy code
* [enh] Build backup info from properties
* [enh] Improve comments presentation
* Adding first tests for backup/restore
* Adding more backup/restore app test scenario
* [enh] Separate BackupMethods in distinct class
* Adding test of restoring a wordpress archive from 2.4
* [fix] Be able to delete backup link too
* [fix] Bad internationalization key
* [fix] Edge case with empty mysql pwd restore
* [fix] Unset var in restore
* [fix] Edge case with empty mysql pwd restore
* Adding test for backup crash handling
* Cleaning tests + checking tmp dir is empty
* [fix] Missing tmp in backup path
* [fix] Error on reading backup csv
* Adding test of failed restore
* Adding tests when not enough space available
* Simplifying tests using markers
* [fix] ynh backup/restore helpers with only one arg
* [fix] Unmount subdir with python
* [enh] Improve backup size management
* [fix] None object in backup
* [enh] Remove dead code
* [fix] Missing locales
* [enh] Adapat test about needed space
* [fix] Pass some test
* [enh] Remove dead code
* [enh] Pass all test
* [enh] Adding test that backups contains what's expected
* Fix typo in tests
* [fix] Bad documentation
* [enh] Add comment
* [enh] Use len in place of implicit {} == False
* [enh] Add comment
* [enh] Add comment
* [enh] Refactoring on _collect_app_files
* Adding skeleton for remaining tests to write
* [enh] Use a csv to list file to backup
* [enh] Use csv python module
* [wip] Backup refactoring
* [wip] Backup class refactoring
* [enh] Add archivemount dependencies
* [wip] Restore refactoring
* [fix] Some error in this refactoring
* [fix] Missing backup key translation
* [fix] Bad YNH_CWD in hook backup
* [fix] App backup part was broken
* [fix] Restore operation was broken
* [fix] No compressed backup
* [fix] Don't commit backup path into csv if app backup fail
* [fix] Default backup collect_dir should be in tmp subdir
* [enh] Simplify a copy code
* [enh] Build backup info from properties
* [enh] Improve comments presentation
* Adding first tests for backup/restore
* Adding more backup/restore app test scenario
* [enh] Separate BackupMethods in distinct class
* Adding test of restoring a wordpress archive from 2.4
* [fix] Be able to delete backup link too
* [fix] Bad internationalization key
* [fix] Edge case with empty mysql pwd restore
* [fix] Unset var in restore
* [fix] Edge case with empty mysql pwd restore
* Adding test for backup crash handling
* Cleaning tests + checking tmp dir is empty
* [fix] Missing tmp in backup path
* [fix] Error on reading backup csv
* Adding test of failed restore
* Adding tests when not enough space available
* Simplifying tests using markers
* [fix] ynh backup/restore helpers with only one arg
* [fix] Unmount subdir with python
* [enh] Improve backup size management
* [fix] None object in backup
* [enh] Remove dead code
* [fix] Missing locales
* [enh] Adapat test about needed space
* [fix] Pass some test
* [enh] Remove dead code
* [enh] Pass all test
* [enh] Adding test that backups contains what's expected
* Fix typo in tests
* [fix] Bad documentation
* Adding skeleton for remaining tests to write
* [enh] Add comment
* [enh] Use len in place of implicit {} == False
* [enh] Add comment
* [enh] Add comment
* [enh] Refactoring on _collect_app_files
* [fix] Replay e1a507 deleted by rebase
* [fix] ynh_restore helper
* Renaming 'hooks' terminology to 'system' where it makes sense
* Propagating new --system/--ignore-system to actionmap
* Adding more tests + clarifying some functions and messages
* Factorize out the definition and validation of backup/restore targets
* Add missing key
* Use list comprehension instead of dirty loops
* [enh] Add docstring in BackupManager
* [enh] Add docstring on BackupMethod(s)
* [fix] Remove deadcode
* [fix] Remove debug message
* [enh] Add comments on RestoreManager
* [enh] Add comments on backup constants
* Adding a proper report/result for each backup target
* Skipping tests not implemented yet
* Fixing little mistake from merging
* [fix] Support different fs or archivemount error
* [enh] Backup helpers readability
* [fix] Copy backup method
* [fix] Deprecated warning always displayed
* [enh] Retrieve info.json file inside tar.gz
* Trying to reorganize methods with sections for readability
* [enh] Support archivemount failure
* [fix] Missing env var for system part restore helpers
* Clarifying disk usage / free space computation
* [enh] Refactoring around backup set_targets()
* Clarifying structure of backup_create and backup_restore
* Move RestoreManager between BackupManager and BackupMethods
* [fix] Missing locales
* [fix] System part restore if archivemount failure
* [enh] Extract all conf instead of specific code
* [fix] Other output directory (compressed archive)
* [enh] Add test for uncompressed backup
* [fix] Compressed backup in an existing output directory
* [fix] Return size for retro-compatibility
* [fix] Mountpoint check aborting script when called with -eu
* [fix] Avoid failure test with set -eu
* [fix] locale strings missing/bad arguments
* Check free space before mount
* [fix] ynh_restore_helpers with existing archive path
* Adding skeletons for moar tests
* Fixing some weird bug in _get_archive_path
* Adding a regen-conf at the end of system restore
* Adding tests of system restore from 2.4
* Have a class dedicated to target management
* Cleaning tests
* Misc formatting
* More meaningful variable names inside app restore
* [fix] can't call source ../settings/scripts/_common.sh in app backup
* [fix] ynh_install_app_dependencies is not compatible with readonly mount
* [fix] Remove temporary file

data/actionsmap/yunohost.yml

@@ -748,18 +748,25 @@ backup:
                     full: --no-compress
                     help: Do not create an archive file
                     action: store_true
-                --hooks:
+                --system:
-                    help: List of backup hooks names to execute
+                    help: List of system parts to backup (all by default)
                     nargs: "*"
-                --ignore-hooks:
-                    help: Do not execute backup hooks
-                    action: store_true
                 --apps:
-                    help: List of application names to backup
+                    help: List of application names to backup (all by default)
                     nargs: "*"
+                --hooks:
+                    help: (Deprecated) See --system
+                    nargs: "*"
+                --ignore-system:
+                    help: Do not backup system
+                    action: store_true
                 --ignore-apps:
                     help: Do not backup apps
                     action: store_true
+                --ignore-hooks:
+                    help: (Deprecated) See --ignore-system
+                    action: store_true
+
 
         ### backup_restore()
         restore:
@@ -772,17 +779,23 @@ backup:
             arguments:
                 name:
                     help: Name of the local backup archive
-                --hooks:
+                --system:
-                    help: List of restauration hooks names to execute
+                    help: List of system parts to restore (all by default)
                     nargs: "*"
                 --apps:
-                    help: List of application names to restore
+                    help: List of application names to restore (all by default)
                     nargs: "*"
+                --hooks:
+                    help: (Deprecated) See --system
+                    nargs: "*"
+                --ignore-system:
+                    help: Do not restore system parts
+                    action: store_true
                 --ignore-apps:
                     help: Do not restore apps
                     action: store_true
                 --ignore-hooks:
-                    help: Do not restore hooks
+                    help: (Deprecated) See --ignore-system
                     action: store_true
                 --force:
                     help: Force restauration on an already installed system

data/helpers.d/filesystem

@@ -1,57 +1,205 @@
 CAN_BIND=${CAN_BIND:-1}
 
-# Mark a file or a directory for backup
+# Add a file or a directory to the list of paths to backup
-# Note: currently, SRCPATH will be copied or binded to DESTPATH
+# 
+# Note: this helper could be used in backup hook or in backup script inside an
+# app package
+#
+# Details: ynh_backup writes SRC and the relative DEST into a CSV file. And it
+# creates the parent destination directory 
+# 
+# If DEST is ended by a slash it complete this path with the basename of SRC.
+#
+# usage: ynh_backup src [dest [is_big [arg]]]
+# | arg: src - file or directory to bind or symlink or copy. it shouldn't be in
+# the backup dir. 
+# | arg: dest - destination file or directory inside the
+# backup dir
+# | arg: is_big - 1 to indicate data are big (mail, video, image ...)
+# | arg: arg - Deprecated arg
+#
+# example:
+# # Wordpress app context
+#
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf"
+# # => This line will be added into CSV file
+# # "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/etc/nginx/conf.d/$domain.d/$app.conf"
+#
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "conf/nginx.conf"
+# # => "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/conf/nginx.conf"
+#
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "conf/"
+# # => "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/conf/$app.conf"
+# 
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "conf"
+# # => "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/conf"
+#
+# #Deprecated usages (maintained for retro-compatibility)
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "${backup_dir}/conf/nginx.conf"
+# # => "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/conf/nginx.conf"
+#
+# ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "/conf/"
+# # => "/etc/nginx/conf.d/$domain.d/$app.conf","apps/wordpress/conf/$app.conf"
 #
-# usage: ynh_backup srcdir destdir to_bind no_root
-# | arg: srcdir - directory to bind or copy
-# | arg: destdir - mountpoint or destination directory
-# | arg: to_bind - 1 to bind mounting the directory if possible
-# | arg: no_root - 1 to execute commands as current user
 ynh_backup() {
-    local SRCPATH=$1
+    # TODO find a way to avoid injection by file strange naming !
-    local DESTPATH=$2
+    local SRC_PATH="$1"
-    local TO_BIND=${3:-0}
+    local DEST_PATH="${2:-}"
-    local SUDO_CMD="sudo"
+    local IS_BIG="${3:-0}"
-    [[ "${4:-}" = "1" ]] && SUDO_CMD=
 
-    # validate arguments
+    # ==============================================================================
-    [[ -e "${SRCPATH}" ]] || {
+    # Format correctly source and destination paths
-        echo "Source path '${SRCPATH}' does not exist" >&2
+    # ==============================================================================
+    # Be sure the source path is not empty
+    [[ -e "${SRC_PATH}" ]] || {
+        echo "Source path '${SRC_PATH}' does not exist" >&2
         return 1
     }
 
-    # prepend the backup directory
+    # Transform the source path as an absolute path
-    [[ -n "${YNH_APP_BACKUP_DIR:-}" && "${DESTPATH:0:1}" != "/" ]] \
+    # If it's a dir remove the ending /
-        && DESTPATH="${YNH_APP_BACKUP_DIR}/${DESTPATH}"
+    SRC_PATH=$(realpath "$SRC_PATH")
-    [[ ! -e "${DESTPATH}" ]] || {
-        echo "Destination path '${DESTPATH}' already exist" >&2
-        return 1
-    }
 
-    # attempt to bind mounting the directory
+    # If there is no destination path, initialize it with the source path
-    if [[ "${CAN_BIND}" = "1" && "${TO_BIND}" = "1" ]]; then
+    # relative to "/".
-        eval $SUDO_CMD mkdir -p "${DESTPATH}"
+    # eg: SRC_PATH=/etc/yunohost -> DEST_PATH=etc/yunohost
+    if [[ -z "$DEST_PATH" ]]; then
 
-        if sudo mount --rbind "${SRCPATH}" "${DESTPATH}"; then
+        DEST_PATH="${SRC_PATH#/}"
-            # try to remount destination directory as read-only
+
-            sudo mount -o remount,ro,bind "${SRCPATH}" "${DESTPATH}" \
+    else
-              || true
+        if [[ "${DEST_PATH:0:1}" == "/" ]]; then
-            return 0
+
-        else
+            # If the destination path is an absolute path, transform it as a path
-            CAN_BIND=0
+            # relative to the current working directory ($YNH_CWD)
-            echo "Bind mounting seems to be disabled on your system."
+            #
-            echo "You have maybe to check your apparmor configuration."
+            # If it's an app backup script that run this helper, YNH_CWD is equal to
+            # $YNH_BACKUP_DIR/apps/APP_INSTANCE_NAME/backup/
+            #
+            # If it's a system part backup script, YNH_CWD is equal to $YNH_BACKUP_DIR
+            DEST_PATH="${DEST_PATH#$YNH_CWD/}"
+
+            # Case where $2 is an absolute dir but doesn't begin with $YNH_CWD
+            [[ "${DEST_PATH:0:1}" == "/" ]] \
+                && DEST_PATH="${DEST_PATH#/}"
         fi
 
-        # delete mountpoint directory safely
+        # Complete DEST_PATH if ended by a /
-        mountpoint -q "${DESTPATH}" && sudo umount -R "${DESTPATH}"
+        [[ "${DEST_PATH: -1}" == "/" ]] \
-        eval $SUDO_CMD rm -rf "${DESTPATH}"
+            &&  DEST_PATH="${DEST_PATH}/$(basename $SRC_PATH)"
     fi
 
-    # ... or just copy the directory
+    # Check if DEST_PATH already exists in tmp archive
-    eval $SUDO_CMD mkdir -p $(dirname "${DESTPATH}")
+    [[ ! -e "${DEST_PATH}" ]] || {
-    eval $SUDO_CMD cp -a "${SRCPATH}" "${DESTPATH}"
+        echo "Destination path '${DEST_PATH}' already exist" >&2
+        return 1
+    }
+
+    # Add the relative current working directory to the destination path
+    local REL_DIR="${YNH_CWD#$YNH_BACKUP_DIR}"
+    REL_DIR="${REL_DIR%/}/"
+    DEST_PATH="${REL_DIR}${DEST_PATH}"
+    DEST_PATH="${DEST_PATH#/}"
+    # ==============================================================================
+
+    # ==============================================================================
+    # Write file to backup into backup_list
+    # ==============================================================================
+    local SRC=$(echo "${SRC_PATH}" | sed -r 's/"/\"\"/g')
+    local DEST=$(echo "${DEST_PATH}" | sed -r 's/"/\"\"/g')
+    echo "\"${SRC}\",\"${DEST}\"" >> "${YNH_BACKUP_CSV}"
+
+    # ==============================================================================
+
+    # Create the parent dir of the destination path
+    # It's for retro compatibility, some script consider ynh_backup creates this dir
+    mkdir -p $(dirname "$YNH_BACKUP_DIR/${DEST_PATH}")
+}
+
+# Restore all files linked to the restore hook or to the restore app script
+#
+# usage: ynh_restore
+#
+ynh_restore () {
+    # Deduce the relative path of $YNH_CWD
+    local REL_DIR="${YNH_CWD#$YNH_BACKUP_DIR/}"
+    REL_DIR="${REL_DIR%/}/"
+
+    # For each destination path begining by $REL_DIR
+    cat ${YNH_BACKUP_CSV} | tr -d $'\r' | grep -ohP "^\".*\",\"$REL_DIR.*\"$" | \
+    while read line; do
+        local ORIGIN_PATH=$(echo "$line" | grep -ohP "^\"\K.*(?=\",\".*\"$)")
+        local ARCHIVE_PATH=$(echo "$line" | grep -ohP "^\".*\",\"$REL_DIR\K.*(?=\"$)")
+        ynh_restore_file "$ARCHIVE_PATH" "$ORIGIN_PATH"
+    done
+}
+
+# Return the path in the archive where has been stocked the origin path
+#
+# usage: _get_archive_path ORIGIN_PATH
+_get_archive_path () {
+    # For security reasons we use csv python library to read the CSV
+    sudo python -c "
+import sys
+import csv
+with open(sys.argv[1], 'r') as backup_file:
+    backup_csv = csv.DictReader(backup_file, fieldnames=['source', 'dest'])
+    for row in backup_csv:
+        if row['source']==sys.argv[2].strip('\"'):
+            print row['dest']
+            sys.exit(0)
+    raise Exception('Original path for %s not found' % sys.argv[2])
+    "  "${YNH_BACKUP_CSV}" "$1"
+    return $?
+}
+
+# Restore a file or a directory 
+#
+# Use the registered path in backup_list by ynh_backup to restore the file at
+# the good place.
+#
+# usage: ynh_restore_file ORIGIN_PATH [ DEST_PATH ]
+# | arg: ORIGIN_PATH - Path where was located the file or the directory before
+# to be backuped or relative path to $YNH_CWD where it is located in the backup archive
+# | arg: DEST_PATH   - Path where restore the file or the dir, if unspecified,
+# the destination will be ORIGIN_PATH or if the ORIGIN_PATH doesn't exist in
+# the archive, the destination will be searched into backup.csv
+#
+# examples:
+# ynh_restore_file "/etc/nginx/conf.d/$domain.d/$app.conf"
+# # if apps/wordpress/etc/nginx/conf.d/$domain.d/$app.conf exists, restore it into
+# # /etc/nginx/conf.d/$domain.d/$app.conf
+# # if no, search a correspondance in the csv (eg: conf/nginx.conf) and restore it into
+# # /etc/nginx/conf.d/$domain.d/$app.conf
+#
+# # DON'T GIVE THE ARCHIVE PATH:
+# ynh_restore_file "conf/nginx.conf"
+#
+ynh_restore_file () {
+    local ORIGIN_PATH="/${1#/}"
+    local ARCHIVE_PATH="$YNH_CWD${ORIGIN_PATH}"
+    # Default value for DEST_PATH = /$ORIGIN_PATH
+    local DEST_PATH="${2:-$ORIGIN_PATH}"
+    
+    # If ARCHIVE_PATH doesn't exist, search for a corresponding path in CSV
+    if [ ! -d "$ARCHIVE_PATH" ] && [ ! -f "$ARCHIVE_PATH" ] && [ ! -L "$ARCHIVE_PATH" ]; then
+        ARCHIVE_PATH="$YNH_BACKUP_DIR/$(_get_archive_path \"$ORIGIN_PATH\")"
+    fi
+    
+    # Restore ORIGIN_PATH into DEST_PATH
+    mkdir -p $(dirname "$DEST_PATH")
+
+    # Do a copy if it's just a mounting point
+    if mountpoint -q $YNH_BACKUP_DIR; then
+        if [[ -d "${ARCHIVE_PATH}" ]]; then
+            ARCHIVE_PATH="${ARCHIVE_PATH}/."
+            mkdir -p "$DEST_PATH"
+        fi
+        cp -a "$ARCHIVE_PATH" "${DEST_PATH}"
+    # Do a move if YNH_BACKUP_DIR is already a copy
+    else
+        mv "$ARCHIVE_PATH" "${DEST_PATH}"
+    fi
 }
 
 # Deprecated helper since it's a dangerous one!
@@ -60,7 +208,7 @@ ynh_bind_or_cp() {
     local NO_ROOT=0
     [[ "${AS_ROOT}" = "1" ]] || NO_ROOT=1
     echo "This helper is deprecated, you should use ynh_backup instead" >&2
-    ynh_backup "$1" "$2" 1 "$NO_ROOT"
+    ynh_backup "$1" "$2" 1
 }
 
 # Create a directory under /tmp

data/helpers.d/package

@@ -121,7 +121,7 @@ ynh_install_app_dependencies () {
     if ynh_package_is_installed "${dep_app}-ynh-deps"; then
 		echo "A package named ${dep_app}-ynh-deps is already installed" >&2
     else
-        cat > ./${dep_app}-ynh-deps.control << EOF	# Make a control file for equivs-build
+        cat > /tmp/${dep_app}-ynh-deps.control << EOF	# Make a control file for equivs-build
 Section: misc
 Priority: optional
 Package: ${dep_app}-ynh-deps
@@ -131,8 +131,9 @@ Architecture: all
 Description: Fake package for ${app} (YunoHost app) dependencies
  This meta-package is only responsible of installing its dependencies.
 EOF
-        ynh_package_install_from_equivs ./${dep_app}-ynh-deps.control \
+        ynh_package_install_from_equivs /tmp/${dep_app}-ynh-deps.control \
             || ynh_die "Unable to install dependencies"	# Install the fake package and its dependencies
+        rm /tmp/${dep_app}-ynh-deps.control
         ynh_app_setting_set $app apt_dependencies $dependencies
     fi
 }

data/hooks/post_backup_create/99-umount

@@ -1,13 +0,0 @@
-
-tmp_dir=$1
-retcode=$2
-
-FAILURE=0
-
-# Iterate over inverted ordered mountpoints to prevent issues
-for m in $(mount | grep " ${tmp_dir}" | awk '{ print $3 }' | tac); do
-    sudo umount $m
-    [[ $? != 0 ]] && FAILURE=1
-done
-
-exit $FAILURE

data/hooks/restore/11-conf_ynh_mysql

@@ -6,9 +6,13 @@ service mysql status >/dev/null 2>&1 \
 
 # retrieve current and new password
 [ -f /etc/yunohost/mysql ] \
-  && curr_pwd=$(sudo cat /etc/yunohost/mysql) \
+  && curr_pwd=$(sudo cat /etc/yunohost/mysql)
-  || curr_pwd="yunohost"
 new_pwd=$(sudo cat "${backup_dir}/root_pwd" || sudo cat "${backup_dir}/mysql")
+[ -z "$curr_pwd" ] && curr_pwd="yunohost"
+[ -z "$new_pwd" ] && {
+    . /usr/share/yunohost/helpers.d/string
+    new_pwd=$(ynh_string_random 10)
+}
 
 # attempt to change it
 sudo mysqladmin -s -u root -p"$curr_pwd" password "$new_pwd" || {

debian/control

@@ -26,6 +26,7 @@ Depends: ${python:Depends}, ${misc:Depends}
  , ssowat, metronome
  , rspamd (>= 1.2.0), rmilter (>=1.7.0), redis-server, opendkim-tools
  , haveged
+ , archivemount
 Recommends: yunohost-admin
  , openssh-server, ntp, inetutils-ping | iputils-ping
  , bash-completion, rsyslog, etckeeper

locales/en.json

@@ -54,29 +54,54 @@
     "ask_main_domain": "Main domain",
     "ask_new_admin_password": "New administration password",
     "ask_password": "Password",
+    "backup_abstract_method": "This backup method hasn't yet been implemented",
     "backup_action_required": "You must specify something to save",
     "backup_app_failed": "Unable to back up the app '{app:s}'",
+    "backup_applying_method_tar": "Creating the backup tar archive...",
+    "backup_applying_method_copy": "Copying all files to backup...",
+    "backup_applying_method_borg": "Sending all files to backup into borg-backup repository...",
+    "backup_applying_method_custom": "Calling the custom backup method '{method:s}'...",
     "backup_archive_app_not_found": "App '{app:s}' not found in the backup archive",
     "backup_archive_broken_link": "Unable to access backup archive (broken link to {path:s})",
-    "backup_archive_hook_not_exec": "Hook '{hook:s}' not executed in this backup",
+    "backup_archive_system_part_not_available": "System part '{part:s}' not available in this backup",
     "backup_archive_name_exists": "The backup's archive name already exists",
     "backup_archive_name_unknown": "Unknown local backup archive named '{name:s}'",
     "backup_archive_open_failed": "Unable to open the backup archive",
+    "backup_archive_mount_failed": "Mounting the backup archive failed",
+    "backup_archive_writing_error": "Unable to add files to backup into the compressed archive",
+    "backup_ask_for_copying_if_needed": "Your system don't support completely the quick method to organize files in the archive, do you want to organize its by copying {size:s}MB?",
+    "backup_borg_not_implemented": "Borg backup method is not yet implemented",
+    "backup_cant_mount_uncompress_archive": "Unable to mount in readonly mode the uncompress archive directory",
     "backup_cleaning_failed": "Unable to clean-up the temporary backup directory",
+    "backup_copying_to_organize_the_archive": "Copying {size:s}MB to organize the archive", 
     "backup_created": "Backup created",
     "backup_creating_archive": "Creating the backup archive...",
     "backup_creation_failed": "Backup creation failed",
+    "backup_csv_creation_failed": "Unable to create the CSV file needed for future restore operations",
+    "backup_csv_addition_failed": "Unable to add files to backup into the CSV file",
+    "backup_custom_need_mount_error": "Custom backup method failure on 'need_mount' step",
+    "backup_custom_backup_error": "Custom backup method failure on 'backup' step",
+    "backup_custom_mount_error": "Custom backup method failure on 'mount' step",
     "backup_delete_error": "Unable to delete '{path:s}'",
     "backup_deleted": "The backup has been deleted",
     "backup_extracting_archive": "Extracting the backup archive...",
     "backup_hook_unknown": "Backup hook '{hook:s}' unknown",
     "backup_invalid_archive": "Invalid backup archive",
+    "backup_no_uncompress_archive_dir": "Uncompress archive directory doesn't exist",
     "backup_nothings_done": "There is nothing to save",
+    "backup_method_tar_finished": "Backup tar archive created",
+    "backup_method_copy_finished": "Backup copy finished",
+    "backup_method_borg_finished": "Backup into borg finished",
+    "backup_method_custom_finished": "Custom backup method '{method:s}' finished",
     "backup_output_directory_forbidden": "Forbidden output directory. Backups can't be created in /bin, /boot, /dev, /etc, /lib, /root, /run, /sbin, /sys, /usr, /var or /home/yunohost.backup/archives sub-folders",
     "backup_output_directory_not_empty": "The output directory is not empty",
     "backup_output_directory_required": "You must provide an output directory for the backup",
     "backup_running_app_script": "Running backup script of app '{app:s}'...",
     "backup_running_hooks": "Running backup hooks...",
+    "backup_system_part_failed": "Unable to backup the '{part:s}' system part",
+    "backup_unable_to_organize_files": "Unable to organize files in the archive with the quick method", 
+    "backup_with_no_backup_script_for_app": "App {app:s} has no backup script. Ignoring.",
+    "backup_with_no_restore_script_for_app": "App {app:s} has no restore script, you won't be able to automatically restore the backup of this app.",
     "custom_app_url_required": "You must provide a URL to upgrade your custom app {app:s}",
     "custom_appslist_name_required": "You must provide a name for your custom app list",
     "diagnosis_debian_version_error": "Can't retrieve the Debian version: {error}",
@@ -199,14 +224,20 @@
     "restore_action_required": "You must specify something to restore",
     "restore_already_installed_app": "An app is already installed with the id '{app:s}'",
     "restore_app_failed": "Unable to restore the app '{app:s}'",
+    "restore_removing_tmp_dir_failed": "Unable to remove an old temporary directory",
     "restore_cleaning_failed": "Unable to clean-up the temporary restoration directory",
     "restore_complete": "Restore complete",
     "restore_confirm_yunohost_installed": "Do you really want to restore an already installed system? [{answers:s}]",
+    "restore_extracting": "Extracting needed files from the archive...",
     "restore_failed": "Unable to restore the system",
-    "restore_hook_unavailable": "Restoration hook '{hook:s}' not available on your system",
+    "restore_hook_unavailable": "Restoration script for '{part:s}' not available on your system and not in the archive either",
+    "restore_mounting_archive": "Mounting archive into '{path:s}'",
+    "restore_may_be_not_enough_disk_space": "Your system seems not to have enough disk space (freespace: {free_space:d} B, needed space: {needed_space:d} B, security margin: {margin:d} B)",
+    "restore_not_enough_disk_space": "Not enough disk space (freespace: {free_space:d} B, needed space: {needed_space:d} B, security margin: {margin:d} B)",
     "restore_nothings_done": "Nothing has been restored",
     "restore_running_app_script": "Running restore script of app '{app:s}'...",
     "restore_running_hooks": "Running restoration hooks...",
+    "restore_system_part_failed": "Unable to restore the '{part:s}' system part",
     "service_add_failed": "Unable to add service '{service:s}'",
     "service_added": "The service '{service:s}' has been added",
     "service_already_started": "Service '{service:s}' has already been started",

src/yunohost/tests/test_backuprestore.py

@@ -0,0 +1,637 @@
+import pytest
+import time
+import requests
+import os
+import shutil
+import subprocess
+from mock import ANY
+
+from moulinette.core import init_authenticator
+from yunohost.app import app_install, app_remove, app_ssowatconf
+from yunohost.app import _is_installed
+from yunohost.backup import backup_create, backup_restore, backup_list, backup_info, backup_delete
+from yunohost.domain import _get_maindomain
+from moulinette.core import MoulinetteError
+
+# Get main domain
+maindomain = _get_maindomain()
+
+# Instantiate LDAP Authenticator
+AUTH_IDENTIFIER = ('ldap', 'ldap-anonymous')
+AUTH_PARAMETERS = {'uri': 'ldap://localhost:389', 'base_dn': 'dc=yunohost,dc=org'}
+auth = None
+
+def setup_function(function):
+
+    print ""
+
+    global auth
+    auth = init_authenticator(AUTH_IDENTIFIER, AUTH_PARAMETERS)
+
+    assert backup_test_dependencies_are_met()
+
+    clean_tmp_backup_directory()
+    reset_ssowat_conf()
+    delete_all_backups()
+    uninstall_test_apps_if_needed()
+
+    assert len(backup_list()["archives"]) == 0
+
+    markers = function.__dict__.keys()
+
+    if "with_wordpress_archive_from_2p4" in markers:
+        add_archive_wordpress_from_2p4()
+        assert len(backup_list()["archives"]) == 1
+
+    if "with_backup_legacy_app_installed" in markers:
+        assert not app_is_installed("backup_legacy_app")
+        install_app("backup_legacy_app_ynh", "/yolo")
+        assert app_is_installed("backup_legacy_app")
+
+    if "with_backup_recommended_app_installed" in markers:
+        assert not app_is_installed("backup_recommended_app")
+        install_app("backup_recommended_app_ynh", "/yolo",
+                    "&helper_to_test=ynh_restore_file")
+        assert app_is_installed("backup_recommended_app")
+
+    if "with_backup_recommended_app_installed_with_ynh_restore" in markers:
+        assert not app_is_installed("backup_recommended_app")
+        install_app("backup_recommended_app_ynh", "/yolo",
+                    "&helper_to_test=ynh_restore")
+        assert app_is_installed("backup_recommended_app")
+
+    if "with_system_archive_from_2p4" in markers:
+        add_archive_system_from_2p4()
+        assert len(backup_list()["archives"]) == 1
+
+
+def teardown_function(function):
+
+    print ""
+    global auth
+    auth = init_authenticator(AUTH_IDENTIFIER, AUTH_PARAMETERS)
+
+    assert tmp_backup_directory_is_empty()
+
+    reset_ssowat_conf()
+    delete_all_backups()
+    uninstall_test_apps_if_needed()
+
+    markers = function.__dict__.keys()
+
+    if "clean_opt_dir" in markers:
+        shutil.rmtree("/opt/test_backup_output_directory")
+
+
+###############################################################################
+#  Helpers                                                                    #
+###############################################################################
+
+def app_is_installed(app):
+
+    # These are files we know should be installed by the app
+    app_files = []
+    app_files.append("/etc/nginx/conf.d/%s.d/%s.conf" % (maindomain, app))
+    app_files.append("/var/www/%s/index.html" % app)
+    app_files.append("/etc/importantfile")
+
+    return _is_installed(app) and all(os.path.exists(f) for f in app_files)
+
+
+def backup_test_dependencies_are_met():
+
+    # We need archivemount installed for the backup features to work
+    assert os.system("which archivemount >/dev/null") == 0
+
+    # Dummy test apps (or backup archives)
+    assert os.path.exists("./tests/apps/backup_wordpress_from_2p4")
+    assert os.path.exists("./tests/apps/backup_legacy_app_ynh")
+    assert os.path.exists("./tests/apps/backup_recommended_app_ynh")
+
+    return True
+
+def tmp_backup_directory_is_empty():
+
+    if not os.path.exists("/home/yunohost.backup/tmp/"):
+        return True
+    else:
+        return len(os.listdir('/home/yunohost.backup/tmp/')) == 0
+
+def clean_tmp_backup_directory():
+
+    if tmp_backup_directory_is_empty():
+        return
+
+    mount_lines = subprocess.check_output("mount").split("\n")
+
+    points_to_umount = [ line.split(" ")[2]
+                         for line in mount_lines
+                            if  len(line) >= 3
+                            and line.split(" ")[2].startswith("/home/yunohost.backup/tmp") ]
+
+    for point in reversed(points_to_umount):
+        os.system("umount %s" % point)
+
+    for f in os.listdir('/home/yunohost.backup/tmp/'):
+        shutil.rmtree("/home/yunohost.backup/tmp/%s" % f)
+
+    shutil.rmtree("/home/yunohost.backup/tmp/")
+
+def reset_ssowat_conf():
+
+    # Make sure we have a ssowat
+    os.system("mkdir -p /etc/ssowat/")
+    app_ssowatconf(auth)
+
+
+def delete_all_backups():
+
+    for archive in backup_list()["archives"]:
+        backup_delete(archive)
+
+
+def uninstall_test_apps_if_needed():
+
+    if _is_installed("backup_legacy_app"):
+        app_remove(auth, "backup_legacy_app")
+
+    if _is_installed("backup_recommended_app"):
+        app_remove(auth, "backup_recommended_app")
+
+    if _is_installed("wordpress"):
+        app_remove(auth, "wordpress")
+
+
+def install_app(app, path, additionnal_args=""):
+
+    app_install(auth, "./tests/apps/%s" % app,
+                args="domain=%s&path=%s%s" % (maindomain, path,
+                                              additionnal_args))
+
+
+def add_archive_wordpress_from_2p4():
+
+    os.system("mkdir -p /home/yunohost.backup/archives")
+
+    os.system("cp ./tests/apps/backup_wordpress_from_2p4/backup.info.json \
+               /home/yunohost.backup/archives/backup_wordpress_from_2p4.info.json")
+
+    os.system("cp ./tests/apps/backup_wordpress_from_2p4/backup.tar.gz \
+               /home/yunohost.backup/archives/backup_wordpress_from_2p4.tar.gz")
+
+
+def add_archive_system_from_2p4():
+
+    os.system("mkdir -p /home/yunohost.backup/archives")
+
+    os.system("cp ./tests/apps/backup_system_from_2p4/backup.info.json \
+               /home/yunohost.backup/archives/backup_system_from_2p4.info.json")
+
+    os.system("cp ./tests/apps/backup_system_from_2p4/backup.tar.gz \
+               /home/yunohost.backup/archives/backup_system_from_2p4.tar.gz")
+
+###############################################################################
+#  System backup                                                              #
+###############################################################################
+
+def test_backup_only_ldap():
+
+    # Create the backup
+    backup_create(ignore_system=False, ignore_apps=True, system=["conf_ldap"])
+
+    archives = backup_list()["archives"]
+    assert len(archives) == 1
+
+    archives_info = backup_info(archives[0], with_details=True)
+    assert archives_info["apps"] == {}
+    assert len(archives_info["system"].keys()) == 1
+    assert "conf_ldap" in archives_info["system"].keys()
+
+
+def test_backup_system_part_that_does_not_exists(mocker):
+
+    mocker.spy(m18n, "n")
+
+    # Create the backup
+    with pytest.raises(MoulinetteError):
+        backup_create(ignore_system=False, ignore_apps=True, system=["yolol"])
+
+    m18n.n.assert_any_call('backup_hook_unknown', hook="yolol")
+    m18n.n.assert_any_call('backup_nothings_done')
+
+###############################################################################
+#  System backup and restore                                                  #
+###############################################################################
+
+def test_backup_and_restore_all_sys():
+
+    # Create the backup
+    backup_create(ignore_system=False, ignore_apps=True)
+
+    archives = backup_list()["archives"]
+    assert len(archives) == 1
+
+    archives_info = backup_info(archives[0], with_details=True)
+    assert archives_info["apps"] == {}
+    assert (len(archives_info["system"].keys()) ==
+            len(os.listdir("/usr/share/yunohost/hooks/backup/")))
+
+    # Remove ssowat conf
+    assert os.path.exists("/etc/ssowat/conf.json")
+    os.system("rm -rf /etc/ssowat/")
+    assert not os.path.exists("/etc/ssowat/conf.json")
+
+    # Restore the backup
+    backup_restore(auth, name=archives[0], force=True,
+                   ignore_system=False, ignore_apps=True)
+
+    # Check ssowat conf is back
+    assert os.path.exists("/etc/ssowat/conf.json")
+
+
+def test_backup_and_restore_archivemount_failure(monkeypatch, mocker):
+
+    # Create the backup
+    backup_create(ignore_system=False, ignore_apps=True)
+
+    archives = backup_list()["archives"]
+    assert len(archives) == 1
+
+    archives_info = backup_info(archives[0], with_details=True)
+    assert archives_info["apps"] == {}
+    assert (len(archives_info["system"].keys()) ==
+            len(os.listdir("/usr/share/yunohost/hooks/backup/")))
+
+    # Remove ssowat conf
+    assert os.path.exists("/etc/ssowat/conf.json")
+    os.system("rm -rf /etc/ssowat/")
+    assert not os.path.exists("/etc/ssowat/conf.json")
+
+    def custom_subprocess_call(*args, **kwargs):
+        import subprocess as subprocess2
+        if args[0] and args[0][0]=="archivemount":
+            monkeypatch.undo()
+            return 1
+        return subprocess.call(*args, **kwargs)
+
+    monkeypatch.setattr("subprocess.call", custom_subprocess_call)
+    mocker.spy(m18n, "n")
+
+    # Restore the backup
+    backup_restore(auth, name=archives[0], force=True,
+                   ignore_system=False, ignore_apps=True)
+
+    # Check ssowat conf is back
+    assert os.path.exists("/etc/ssowat/conf.json")
+
+
+###############################################################################
+#  System restore from 2.4                                                    #
+###############################################################################
+
+@pytest.mark.with_system_archive_from_2p4
+def test_restore_system_from_Ynh2p4(monkeypatch, mocker):
+
+    # Backup current system
+    backup_create(ignore_system=False, ignore_apps=True)
+    archives = backup_list()["archives"]
+    assert len(archives) == 2
+
+    # Restore system archive from 2.4
+    try:
+        backup_restore(auth, name=backup_list()["archives"][1],
+                             ignore_system=False,
+                             ignore_apps=True,
+                             force=True)
+    finally:
+        # Restore system as it was
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=False,
+                             ignore_apps=True,
+                             force=True)
+
+
+@pytest.mark.with_system_archive_from_2p4
+def test_restore_system_from_Ynh2p4_archivemount_failure(monkeypatch, mocker):
+
+    # Backup current system
+    backup_create(ignore_system=False, ignore_apps=True)
+    archives = backup_list()["archives"]
+    assert len(archives) == 2
+
+    def custom_subprocess_call(*args, **kwargs):
+        import subprocess as subprocess2
+        if args[0] and args[0][0]=="archivemount":
+            monkeypatch.undo()
+            return 1
+        return subprocess.call(*args, **kwargs)
+
+    monkeypatch.setattr("subprocess.call", custom_subprocess_call)
+
+    try:
+        # Restore system from 2.4
+        backup_restore(auth, name=backup_list()["archives"][1],
+                             ignore_system=False,
+                             ignore_apps=True,
+                             force=True)
+    finally:
+        # Restore system as it was
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=False,
+                             ignore_apps=True,
+                             force=True)
+
+
+###############################################################################
+#  App backup                                                                 #
+###############################################################################
+
+@pytest.mark.with_backup_recommended_app_installed
+def test_backup_script_failure_handling(monkeypatch, mocker):
+
+    def custom_hook_exec(name, *args, **kwargs):
+
+        if os.path.basename(name).startswith("backup_"):
+            raise Exception
+        else:
+            return True
+
+    # Create a backup of this app and simulate a crash (patching the backup
+    # call with monkeypatch). We also patch m18n to check later it's been called
+    # with the expected error message key
+    monkeypatch.setattr("yunohost.backup.hook_exec", custom_hook_exec)
+    mocker.spy(m18n, "n")
+
+    with pytest.raises(MoulinetteError):
+        backup_create(ignore_system=True, ignore_apps=False, apps=["backup_recommended_app"])
+
+    m18n.n.assert_any_call('backup_app_failed', app='backup_recommended_app')
+
+@pytest.mark.with_backup_recommended_app_installed
+def test_backup_not_enough_free_space(monkeypatch, mocker):
+
+    def custom_disk_usage(path):
+        return 99999999999999999
+
+    def custom_free_space_in_directory(dirpath):
+        return 0
+
+    monkeypatch.setattr("yunohost.backup.disk_usage", custom_disk_usage)
+    monkeypatch.setattr("yunohost.backup.free_space_in_directory",
+                        custom_free_space_in_directory)
+
+    mocker.spy(m18n, "n")
+
+    with pytest.raises(MoulinetteError):
+        backup_create(ignore_system=True, ignore_apps=False, apps=["backup_recommended_app"])
+
+    m18n.n.assert_any_call('not_enough_disk_space', path=ANY)
+
+
+def test_backup_app_not_installed(mocker):
+
+    assert not _is_installed("wordpress")
+
+    mocker.spy(m18n, "n")
+
+    with pytest.raises(MoulinetteError):
+        backup_create(ignore_system=True, ignore_apps=False, apps=["wordpress"])
+
+    m18n.n.assert_any_call("unbackup_app", app="wordpress")
+    m18n.n.assert_any_call('backup_nothings_done')
+
+
+@pytest.mark.with_backup_recommended_app_installed
+def test_backup_app_with_no_backup_script(mocker):
+
+    backup_script = "/etc/yunohost/apps/backup_recommended_app/scripts/backup"
+    os.system("rm %s" % backup_script)
+    assert not os.path.exists(backup_script)
+
+    mocker.spy(m18n, "n")
+
+    with pytest.raises(MoulinetteError):
+        backup_create(ignore_system=True, ignore_apps=False, apps=["backup_recommended_app"])
+
+    m18n.n.assert_any_call("backup_with_no_backup_script_for_app", app="backup_recommended_app")
+    m18n.n.assert_any_call('backup_nothings_done')
+
+
+@pytest.mark.with_backup_recommended_app_installed
+def test_backup_app_with_no_restore_script(mocker):
+
+    restore_script = "/etc/yunohost/apps/backup_recommended_app/scripts/restore"
+    os.system("rm %s" % restore_script)
+    assert not os.path.exists(restore_script)
+
+    mocker.spy(m18n, "n")
+
+    # Backuping an app with no restore script will only display a warning to the
+    # user...
+
+    backup_create(ignore_system=True, ignore_apps=False, apps=["backup_recommended_app"])
+
+    m18n.n.assert_any_call("backup_with_no_restore_script_for_app", app="backup_recommended_app")
+
+
+@pytest.mark.clean_opt_dir
+def test_backup_with_different_output_directory():
+
+    # Create the backup
+    backup_create(ignore_system=False, ignore_apps=True, system=["conf_ssh"],
+                  output_directory="/opt/test_backup_output_directory",
+                  name="backup")
+
+    assert os.path.exists("/opt/test_backup_output_directory/backup.tar.gz")
+
+    archives = backup_list()["archives"]
+    assert len(archives) == 1
+
+    archives_info = backup_info(archives[0], with_details=True)
+    assert archives_info["apps"] == {}
+    assert len(archives_info["system"].keys()) == 1
+    assert "conf_ssh" in archives_info["system"].keys()
+
+@pytest.mark.clean_opt_dir
+def test_backup_with_no_compress():
+    # Create the backup
+    backup_create(ignore_system=False, ignore_apps=True, system=["conf_nginx"],
+                  output_directory="/opt/test_backup_output_directory",
+                  no_compress=True,
+                  name="backup")
+
+    assert os.path.exists("/opt/test_backup_output_directory/info.json")
+
+
+###############################################################################
+#  App restore                                                                #
+###############################################################################
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_wordpress_from_Ynh2p4():
+
+    backup_restore(auth, name=backup_list()["archives"][0],
+                         ignore_system=True,
+                         ignore_apps=False,
+                         apps=["wordpress"])
+
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_script_failure_handling(monkeypatch, mocker):
+
+    def custom_hook_exec(name, *args, **kwargs):
+        if os.path.basename(name).startswith("restore"):
+            monkeypatch.undo()
+            raise Exception
+
+    monkeypatch.setattr("yunohost.backup.hook_exec", custom_hook_exec)
+    mocker.spy(m18n, "n")
+
+    assert not _is_installed("wordpress")
+
+    with pytest.raises(MoulinetteError):
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=True,
+                             ignore_apps=False,
+                             apps=["wordpress"])
+
+    m18n.n.assert_any_call('restore_app_failed', app='wordpress')
+    m18n.n.assert_any_call('restore_nothings_done')
+    assert not _is_installed("wordpress")
+
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_not_enough_free_space(monkeypatch, mocker):
+
+    def custom_free_space_in_directory(dirpath):
+        return 0
+
+    monkeypatch.setattr("yunohost.backup.free_space_in_directory",
+                        custom_free_space_in_directory)
+    mocker.spy(m18n, "n")
+
+    assert not _is_installed("wordpress")
+
+    with pytest.raises(MoulinetteError):
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=True,
+                             ignore_apps=False,
+                             apps=["wordpress"])
+
+    m18n.n.assert_any_call('restore_not_enough_disk_space',
+        free_space=0,
+        margin=ANY,
+        needed_space=ANY)
+    assert not _is_installed("wordpress")
+
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_not_in_backup(mocker):
+
+    assert not _is_installed("wordpress")
+    assert not _is_installed("yoloswag")
+
+    mocker.spy(m18n, "n")
+
+    with pytest.raises(MoulinetteError):
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=True,
+                             ignore_apps=False,
+                             apps=["yoloswag"])
+
+    m18n.n.assert_any_call('backup_archive_app_not_found', app="yoloswag")
+    assert not _is_installed("wordpress")
+    assert not _is_installed("yoloswag")
+
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_archivemount_failure(monkeypatch, mocker):
+
+    def custom_subprocess_call(*args, **kwargs):
+        import subprocess as subprocess2
+        if args[0] and args[0][0]=="archivemount":
+            monkeypatch.undo()
+            return 1
+        return subprocess.call(*args, **kwargs)
+
+    monkeypatch.setattr("subprocess.call", custom_subprocess_call)
+    mocker.spy(m18n, "n")
+
+    assert not _is_installed("wordpress")
+
+    backup_restore(auth, name=backup_list()["archives"][0],
+                         ignore_system=True,
+                         ignore_apps=False,
+                         apps=["wordpress"])
+
+    assert _is_installed("wordpress")
+
+
+@pytest.mark.with_wordpress_archive_from_2p4
+def test_restore_app_already_installed(mocker):
+
+    assert not _is_installed("wordpress")
+
+    backup_restore(auth, name=backup_list()["archives"][0],
+                         ignore_system=True,
+                         ignore_apps=False,
+                         apps=["wordpress"])
+
+    assert _is_installed("wordpress")
+
+    mocker.spy(m18n, "n")
+    with pytest.raises(MoulinetteError):
+        backup_restore(auth, name=backup_list()["archives"][0],
+                             ignore_system=True,
+                             ignore_apps=False,
+                             apps=["wordpress"])
+
+    m18n.n.assert_any_call('restore_already_installed_app', app="wordpress")
+    m18n.n.assert_any_call('restore_nothings_done')
+
+    assert _is_installed("wordpress")
+
+
+@pytest.mark.with_backup_legacy_app_installed
+def test_backup_and_restore_legacy_app():
+
+    _test_backup_and_restore_app("backup_legacy_app")
+
+
+@pytest.mark.with_backup_recommended_app_installed
+def test_backup_and_restore_recommended_app():
+
+    _test_backup_and_restore_app("backup_recommended_app")
+
+
+@pytest.mark.with_backup_recommended_app_installed_with_ynh_restore
+def test_backup_and_restore_with_ynh_restore():
+
+    _test_backup_and_restore_app("backup_recommended_app")
+
+
+def _test_backup_and_restore_app(app):
+
+    # Create a backup of this app
+    backup_create(ignore_system=True, ignore_apps=False, apps=[app])
+
+    archives = backup_list()["archives"]
+    assert len(archives) == 1
+
+    archives_info = backup_info(archives[0], with_details=True)
+    assert archives_info["system"] == {}
+    assert len(archives_info["apps"].keys()) == 1
+    assert app in archives_info["apps"].keys()
+
+    # Uninstall the app
+    app_remove(auth, app)
+    assert not app_is_installed(app)
+
+    # Restore the app
+    backup_restore(auth, name=archives[0], ignore_system=True,
+                   ignore_apps=False, apps=[app])
+
+    assert app_is_installed(app)
+
+
+