Merge pull request #19 from YunoHost-Apps/testing

Testing new ynh_local_curl and ynh_handle_getopts_args

.gitignore

@@ -1 +1,2 @@
 README*
+scripts/test_local_curl.sh

ALL_README.md

@@ -1,7 +1,7 @@
 # All available README files by language
 
 - [Read the README in English](README.md)
-- [Lee el README en español](README_es.md)
+- [Lea el README en español](README_es.md)
 - [Irakurri README euskaraz](README_eu.md)
 - [Lire le README en français](README_fr.md)
 - [Le o README en galego](README_gl.md)

README.md

@@ -53,7 +53,7 @@ Generally the [wiki](https://codeberg.org/flohmarkt/flohmarkt/wiki) is a good so
 * [Service compatibility chart](https://codeberg.org/flohmarkt/flohmarkt/wiki/Service-compatibility-chart)
 
 
-**Shipped version:** 0.0~ynh5
+**Shipped version:** 0.0~ynh6
 
 **Demo:** <https://flohmarkt.ween.de/>
 

README_eu.md

@@ -53,7 +53,7 @@ Generally the [wiki](https://codeberg.org/flohmarkt/flohmarkt/wiki) is a good so
 * [Service compatibility chart](https://codeberg.org/flohmarkt/flohmarkt/wiki/Service-compatibility-chart)
 
 
-**Paketatutako bertsioa:** 0.0~ynh5
+**Paketatutako bertsioa:** 0.0~ynh6
 
 **Demoa:** <https://flohmarkt.ween.de/>
 

README_fr.md

@@ -53,7 +53,7 @@ Generally the [wiki](https://codeberg.org/flohmarkt/flohmarkt/wiki) is a good so
 * [Service compatibility chart](https://codeberg.org/flohmarkt/flohmarkt/wiki/Service-compatibility-chart)
 
 
-**Version incluse :** 0.0~ynh5
+**Version incluse :** 0.0~ynh6
 
 **Démo :** <https://flohmarkt.ween.de/>
 

README_gl.md

@@ -53,7 +53,7 @@ Generally the [wiki](https://codeberg.org/flohmarkt/flohmarkt/wiki) is a good so
 * [Service compatibility chart](https://codeberg.org/flohmarkt/flohmarkt/wiki/Service-compatibility-chart)
 
 
-**Versión proporcionada:** 0.0~ynh5
+**Versión proporcionada:** 0.0~ynh6
 
 **Demo:** <https://flohmarkt.ween.de/>
 

README_zh_Hans.md

@@ -53,7 +53,7 @@ Generally the [wiki](https://codeberg.org/flohmarkt/flohmarkt/wiki) is a good so
 * [Service compatibility chart](https://codeberg.org/flohmarkt/flohmarkt/wiki/Service-compatibility-chart)
 
 
-**分发版本：** 0.0~ynh5
+**分发版本：** 0.0~ynh6
 
 **演示：** <https://flohmarkt.ween.de/>
 

doc/ADMIN.md

@@ -18,10 +18,4 @@ More about development of **flohmarkt**s yunohost integration can be found in [D
 
 ## removing this app
 
-Removing a flohmarkt instance will **delete all of its data** for the time being. If you remove a flohmarkt **we strongly suggest to make a backup** beforehand. This [issue](https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/29) describes how we'll handle `remove` with and without `--purge` in the future.
-
-**Warning:** This might break any existing installation of couchdb (there's an couchdb app to install just couchdb and expose its port via nginx reverse-proxy and possibly other software installing a couchdb). This could happen if you installed the couchdb app after you installed flohmarkt.
-
-https://codeberg.org/flohmarkt/flohmarkt_ynh/src/commit/7721103bac61787f31a4b2f2ae695c65d4f26fc9/scripts/remove#L44
-
 When installing flohmarkt on a a domain and letting it talk to other ActivityPub instances it will propagate a key associated to your domain. If you remove your flohmarkt from that domain and loose that key other instances might not want to talk to you anymore after you installed flohmarkt again on the same domain generating a new key.

doc/PRE_INSTALL.md

@@ -13,11 +13,12 @@ flohmarkt will send email and you'll be asked for an address used as the sender
 
 This [issue](https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/50) contains information about the problem.
 
-## multiple flohmarkt on the same subdomain
+## multiple flohmarkt on the same subdomain will not work
 
-The installation will allow you to **test** this. The feature is not well tested, yet, and installing multiple productive flohmarkts in the same domain might not work on the fediverse.
+flohmarkt **needs its own subdomain** to be installed on. Some of the discussion about this can be found here:
 
-Feedback is more than welcome!
+https://codeberg.org/flohmarkt/flohmarkt/issues/251
+https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/53
 
 ## No integration in yunohost user database
 

manifest.toml

@@ -7,7 +7,7 @@ name = "flohmarkt"
 description.en = "A decentral federated small ads platform"
 description.fr = "Plateforme de petites annonces fédérées décentralisées"
 
-version = "0.0~ynh5"
+version = "0.0~ynh6"
 
 maintainers = ["Chris Vogel"]
 

scripts/_common.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+
+
 #=================================================
 # COMMON VARIABLES
 #=================================================
@@ -42,10 +44,514 @@ flohmarkt_old_app_dir="${flohmarkt_old_install}/flohmarkt"
 flohmarkt_old_log_dir="/var/log/flohmarkt/"
 flohmarkt_old_service="flohmarkt"
 
+#=================================================
+# HELPER DEVELOPMENT for yunohost core
+#=================================================
+
+# Redisgn of ynh_handle_getopts_args for flohmarkt to be tested as `flohmarkt_ynh_handle_getopts_args`
+# https://github.com/YunoHost/yunohost/pull/1856
+# Internal helper design to allow helpers to use getopts to manage their arguments
+#
+# [internal]
+#
+# example: function my_helper()
+# {
+#     local -A args_array=( [a]=arg1= [b]=arg2= [c]=arg3 )
+#     local arg1
+#     local arg2
+#     local arg3
+#     ynh_handle_getopts_args "$@"
+#
+#     [...]
+# }
+# my_helper --arg1 "val1" -b val2 -c
+#
+# usage: ynh_handle_getopts_args "$@"
+# | arg: $@    - Simply "$@" to tranfert all the positionnal arguments to the function
+#
+# This helper need an array, named "args_array" with all the arguments used by the helper
+#   that want to use ynh_handle_getopts_args
+# Be carreful, this array has to be an associative array, as the following example:
+# local -A args_array=( [a]=arg1 [b]=arg2= [c]=arg3 )
+# Let's explain this array:
+# a, b and c are short options, -a, -b and -c
+# arg1, arg2 and arg3 are the long options associated to the previous short ones. --arg1, --arg2 and --arg3
+# For each option, a short and long version has to be defined.
+# Let's see something more significant
+# local -A args_array=( [u]=user [f]=finalpath= [d]=database )
+#
+# NB: Because we're using 'declare' without -g, the array will be declared as a local variable.
+#
+# Please keep in mind that the long option will be used as a variable to store the values for this option.
+# For the previous example, that means that $finalpath will be fill with the value given as argument for this option.
+#
+# Also, in the previous example, finalpath has a '=' at the end. That means this option need a value.
+# So, the helper has to be call with --finalpath /final/path, --finalpath=/final/path or -f /final/path, 
+# the variable $finalpath will get the value /final/path
+# If there's many values for an option, -f /final /path, the value will be separated by a ';' $finalpath=/final;/path
+# For an option without value, like --user in the example, the helper can be called only with --user or -u. $user 
+# will then get the value 1.
+#
+# To keep a retrocompatibility, a package can still call a helper, using getopts, with positional arguments.
+# The "legacy mode" will manage the positional arguments and fill the variable in the same order than they are given 
+# in $args_array. e.g. for `my_helper "val1" val2`, arg1 will be filled with val1, and arg2 with val2.
+
+# Positional parameters (used to be the only way to use ynh_handle_getopts_args once upon a time) can be 
+# used also:
+# 
+# '--'          start processing the rest of the arguments as positional parameters
+# $legacy_args  The arguments positional parameters will be assign to
+#               Needs to be composed of array keys of args_array. If a key for a predefined variable
+#               is used multiple times the assigned values will be concatenated delimited by ';'.
+#               If the long option variable to contain the data is predefined as an array (e.g. using
+#               `local -a arg1` then multiple values will be assigned to its cells.
+#               If the last positional parameter defined in legacy_args is defined as an array all 
+#               the leftover positional parameters will be assigned to its cells.
+#               (it is named legacy_args, because the use of positional parameters was about to be
+#               deprecated before the last re-design of this sub)
+#
+# Requires YunoHost version 3.2.2 or higher.
+# flohmarkt_ynh_handle_getopts_args() {
+ynh_handle_getopts_args() {
+    # Manage arguments only if there's some provided
+    set +o xtrace # set +x
+    if [ $# -eq 0 ]; then
+      ynh_print_warn --message="ynh_handle_getopts_args called without arguments"
+      return
+    fi
+
+    # Store arguments in an array to keep each argument separated
+    local arguments=("$@")
+
+    # For each option in the array, reduce to short options for getopts (e.g. for [u]=user, --user will be -u)
+    # And built parameters string for getopts
+    # ${!args_array[@]} is the list of all option_flags in the array (An option_flag is 'u' in [u]=user, user is a value)
+    local getopts_parameters=""
+    local option_flag=""
+    ## go through all possible options and replace arguments with short versions
+    flohmarkt_print_debug "arguments = '${arguments[@]}"
+    flohmarkt_print_debug "args_array = (${!args_array[@]})"
+    for option_flag in "${!args_array[@]}"; do
+        # TODO refactor: Now I'm not sure anymore this part belongs here. To make the
+        # this all less hard to read and understand I'm thinking at the moment that it 
+        # would be good to split the different things done here into their own loops:
+        #
+        # 1. build the option string $getopts_parameters
+        # 2. go through the arguments and add empty arguments where needed to 
+        #    allow for cases like '--arg= --value' where 'value' is a valid option, too
+        # 3. replace long option names by short once
+        # 4. (possibly add empty parameters for '-a -v' in cases where -a expects a value
+        #    and -v is a valid option, too - but I dearly hope this will not be necessary)
+        flohmarkt_print_debug "option_flag = $option_flag"
+        # Concatenate each option_flags of the array to build the string of arguments for getopts
+        # Will looks like 'abcd' for -a -b -c -d
+        # If the value of an option_flag finish by =, it's an option with additionnal values. 
+        # (e.g. --user bob or -u bob)
+        # Check the last character of the value associate to the option_flag
+        flohmarkt_print_debug "compare to '${args_array[$option_flag]: -1}'"
+        if [ "${args_array[$option_flag]: -1}" = "=" ]; then
+            # For an option with additionnal values, add a ':' after the letter for getopts.
+            getopts_parameters="${getopts_parameters}${option_flag}:"
+        else
+            getopts_parameters="${getopts_parameters}${option_flag}"
+        fi
+        flohmarkt_print_debug "getopts_parameters = ${getopts_parameters}"
+        # Check each argument given to the function
+        local arg=""
+        # ${#arguments[@]} is the size of the array
+        ## for one possible option: look at each argument supplied:
+        for arg in $(seq 0 $((${#arguments[@]} - 1))); do
+            flohmarkt_print_debug "option_flag='$option_flag', arg = '$arg', argument = '${arguments[arg]}'"
+            # the following cases need to be taken care of
+            # '--arg=value'    → works
+            # '--arg= value'   → works
+            # '--arg=-value'   → works
+            # '--arg= -v'      or
+            # '--arg= --value' → works if not exists arg '[v]=value='
+            #                  → $arg will be set to '-v' or '--value'
+            #   but if exists '[v]=value=' this is not the expected behavior:
+            #   → then $arg is expected to contain an empty value and '-v' or '--value'
+            #     is expected to be interpreted as its own valid argument
+            #   (found in use of ynh_replace_string called by ynh_add_config)
+            #   solution:
+            #   insert an empty arg into array arguments to be later interpreted by 
+            #   getopts as the missing value to --arg=
+            if [[ -v arguments[arg+1] ]] && [[ ${arguments[arg]: -1} == '=' ]]; then
+                # arg ends with a '='
+                local this_argument=${arguments[arg]}
+                local next_argument=${arguments[arg+1]}
+                # for looking up next_argument in args_array remove optionally trailing '='
+                next_argument=$( printf '%s' "$next_argument" | cut -d'=' -f1 )
+                flohmarkt_print_debug "MISSING PARAMETER: this_argument='$this_argument', next_argument='$next_argument'"
+
+                # check if next_argument is a value in args_array
+                # → starts with '--' and the rest of the argument excluding optional trailing '=' 
+                #     of the string is a value in associative array args_array
+                # → or starts with '-' and the rest of the argument is a valid key in args_array
+                #   (long argument could already have been replaced by short version before)
+                flohmarkt_print_debug "args_array values='${args_array[@]}'"
+                flohmarkt_print_debug "args_array   keys='${!args_array[@]}'"
+                flohmarkt_print_debug "{next_argument:2}='${next_argument:2}'"
+                flohmarkt_print_debug "{next_argument:0:2}='${next_argument:0:2}'"
+                if ( [[ "${next_argument:0:2}" == '--' ]] \
+                    && printf '%s ' "${args_array[@]}" | fgrep -w "${next_argument:2}" > /dev/null ) \
+                || ( [[ "${next_argument:0:1}" == '-' ]] \
+                    && printf '%s ' "${!args_array[@]}" | fgrep -w "${next_argument:1:1}" > /dev/null )
+                then
+                    # insert an empty value to array arguments to be interpreted as the value
+                    # for argument[arg]
+                    arguments=( ${arguments[@]:0:arg+1} '' ${arguments[@]:arg+1})
+                    flohmarkt_print_debug "now arguments='${arguments[@]}'"
+                fi
+            fi
+
+            # Replace long option with = (match the beginning of the argument)
+            arguments[arg]="$(printf '%s\n' "${arguments[arg]}" \
+                | sed "s/^--${args_array[$option_flag]}/-${option_flag}/")"
+            # And long option without = (match the whole line)
+            arguments[arg]="$(printf '%s\n' "${arguments[arg]}" \
+                | sed "s/^--${args_array[$option_flag]%=}$/-${option_flag}/")"
+            flohmarkt_print_debug "                            arg = '$arg', argument = '${arguments[arg]}'"
+        done
+        flohmarkt_print_debug "====> end loop: arguments = '${arguments[@]}'"
+    done
+    flohmarkt_print_debug '================= end first loop ================='
+    
+    # Parse the first argument, return the number of arguments to be shifted off the arguments array
+    # The function call is necessary here to allow `getopts` to use $@
+    parse_arg() {
+        flohmarkt_print_debug "========= parse_arg started ======== , arguments='$@', getopts_parameters: '$getopts_parameters'"
+        # Initialize the index of getopts
+        OPTIND=1
+        # getopts will fill $parameter with the letter of the option it has read.
+        local parameter=""
+        getopts ":$getopts_parameters" parameter || true
+        flohmarkt_print_debug "after getopts - parameter='$parameter', OPTIND='${OPTIND:-none}', OPTARG='${OPTARG:-none}'"
+
+        if [ "$parameter" = "?" ]; then
+            ynh_die --message="Invalid argument: -${OPTARG:-}"
+            flohmarkt_print_debug "Invalid argument: -${OPTARG:-}"
+            exit 255
+        elif [ "$parameter" = ":" ]; then
+            ynh_die --message="-$OPTARG parameter requires an argument."
+            echo "-$OPTARG parameter requires an argument."
+            exit 255
+        else
+            # Use the long option, corresponding to the short option read by getopts, as a variable
+            # (e.g. for [u]=user, 'user' will be used as a variable)
+            # Also, remove '=' at the end of the long option
+            # The variable name will be stored in 'option_var' as a nameref
+            option_var="${args_array[$parameter]%=}"
+            flohmarkt_print_debug "option_var='$option_var'"
+            # if there's a '=' at the end of the long option name, this option takes values
+            if [ "${args_array[$parameter]: -1}" != "=" ]; then
+                # no argument expected for option - set option variable to '1'
+                option_value=1
+            else
+                # remove leading and trailing spaces from OPTARG
+                OPTARG="$( printf '%s' "${OPTARG}" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
+                option_value="${OPTARG}"
+            fi
+            flohmarkt_print_debug "option_value='$option_value'"
+            # set shift_value according to the number of options interpreted by getopts
+            shift_value=$(( $OPTIND - 1 ))
+            flohmarkt_print_debug "shift_value='$shift_value'"
+        fi
+    }
+
+    # iterate over the arguments: if first argument starts with a '-' feed arguments to getopts
+    # if first argument doesn't start with a '-' enter mode to read positional parameters
+    local argument
+    local positional_mode=0 # state is getopts mode at the beginning, not positional parameters
+    local positional_count=0 # counter for positional parameters
+    local option_var=''   # the variable name to be filled
+    # Try to use legacy_args as a list of option_flag of the array args_array
+    # Otherwise, fill it with getopts_parameters to get the option_flag. 
+    # (But an associative arrays isn't always sorted in the correct order...)
+    # Remove all ':' in getopts_parameters, if used.
+    legacy_args=${legacy_args:-${getopts_parameters//:/}}
+    while [[ -v 'arguments' ]] && [[ ${#arguments} -ne 0 ]]; do
+        flohmarkt_print_debug '======= start while loop ======='
+        local shift_value=0
+        local option_value='' # the value to be filled into ${!option_var}
+        argument=${arguments[0]}
+        flohmarkt_print_debug "argument='$argument'"
+        # if state once changed to positional parameter mode, all the rest of the arguments will
+        # be interpreted in positional parameter mode even if they start with a '-'
+        if [ $positional_mode == 0 ] && [ "${argument}" == '--' ];then
+            flohmarkt_print_debug "found '--', start positional parameter mode"
+            positional_mode=1
+            shift_value=1
+        elif [ $positional_mode == 0 ] && [ "${argument:0:1}" == '-' ]; then
+            flohmarkt_print_debug "getopts, arguments='${arguments[@]}', starting parse_arg"
+            parse_arg "${arguments[@]}"
+        else
+            positional_mode=1 # set state to positional parameter mode
+            flohmarkt_print_debug "positional parameter, argument='$argument'"
+
+            # Get the option_flag from getopts_parameters by using the option_flag according to the 
+            # position of the argument.
+            option_flag=${legacy_args:$positional_count:1}
+
+            # increment counter for legacy_args if still args left. If no args left check if the 
+            # last arg is a predefined array and let it cells be filled. Otherwise complain and 
+            # return.
+            flohmarkt_print_debug "positional_counter='$positional_count', max positional_counter='$(( ${#legacy_args} -1 ))'"
+            if [[ $positional_count -le $((${#legacy_args} - 1)) ]]; then
+                # set counter to for next option_flag to fill
+                positional_count=$((positional_count+1))
+                flohmarkt_print_debug "incremented positional_counter to '$positional_count'"
+
+                # Use the long option, corresponding to the option_flag, as a variable
+                # (e.g. for [u]=user, 'user' will be used as a variable)
+                # Also, remove '=' at the end of the long option
+                # The variable name will be stored in 'option_var'
+                option_var="${args_array[$option_flag]%=}"
+            elif [[ $positional_count -ge $((${#legacy_args} - 1)) ]] && 
+                ! declare -p ${option_var} | grep '^declare -a'
+            then
+                # no more legacy_args to fill - legacy behaviour: complain and return
+                ynh_print_warn --message="Too many arguments ! \"${arguments[$i]}\" will be ignored."
+                return
+            else
+                flohmarkt_print_debug "array found - keep going"
+            fi
+
+            # value to be assigned to ${!option_var}
+            option_value=$argument
+
+            # shift off one positional parameter
+            shift_value=1
+        fi
+
+        # fill option_var with value found
+        # if ${!option_var} is an array, fill mutiple values as array cells
+        # otherwise concatenate them seperated by ';'
+        # nameref is used to access the variable that is named $option_var
+        local -n option_ref=$option_var
+        # this defines option_ref as a reference to the variable named "$option_var"
+        # any operation on option_ref will be written to the variable named "$option_var"
+        # 'option_ref="hello world"' will work like as if '${!option_var}="hello world"'
+        # would be a valid syntax
+        # see also: `man bash` part about commands 'declare' option '-n'
+        flohmarkt_print_debug "option_ref declare: '$(declare -p option_ref)'"
+        flohmarkt_print_debug "option_value='$option_value'"
+        if [[ -v "$option_var" ]] && declare -p $option_var | grep '^declare -a ' > /dev/null; then
+          # hurray it's an array
+          flohmarkt_print_debug "hurray! '$option_var' is an array"
+          option_ref+=( "${option_value}" )
+        elif ! [[ -v "$option_var" ]] || [[ -z "$option_ref" ]]; then
+          flohmarkt_print_debug "'$option_var' is unset or empty"
+          option_ref=${option_value}
+        else
+          flohmarkt_print_debug "appending to string '$option_ref'"
+          option_ref+=";${option_value}"
+        fi
+        flohmarkt_print_debug "now declared $option_var: '$(declare -p $option_var)'"
+
+        # shift value off arguments array
+        flohmarkt_print_debug "shifting '$shift_value' off arguments='${arguments[@]}'"
+        arguments=("${arguments[@]:${shift_value}}")
+    done
+
+    # the former subroutine did this - no idea if it is expected somewhere
+    unset legacy_args
+
+    # re-enable trace
+    set -o xtrace # set -x
+}
+
+# local copy of ynh_local_curl() to test some improvement
+# https://github.com/YunoHost/yunohost/pull/1857
+# https://github.com/YunoHost/issues/issues/2396
+# https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/51
+ynh_local_curl() {
+# Curl abstraction to help with POST requests to local pages (such as installation forms)
+#
+# usage: ynh_local_curl [--option [-other_option […]]] "page" "key1=value1" "key2=value2" ...
+# | arg: -l --line_match: check answer against an extended regex
+# | arg: -m --method:     request method to use: POST (default), PUT, GET, DELETE
+# | arg: -H --header:     add a header to the request (can be used multiple times)
+# | arg: -d --data:       data to be PUT or POSTed. Can be used multiple times.
+# | arg: -s --seperator:  seperator used to concatenate POST/PUT --date or key=value ('none'=no seperator)
+# | arg:                  (default for POST: '&', default for PUT: ' ')
+# | arg: -u --user:       login username (requires --password)
+# | arg: -p --password:   login password
+# | arg: -n --no_sleep:   don't sleep 2 seconds (background: https://github.com/YunoHost/yunohost/pull/547)
+# | arg: page        - either the PAGE part in 'https://$domain/$path/PAGE' or an URI
+# | arg: key1=value1 - (Optional, POST only) legacy version of '--data' as positional parameter
+# | arg: key2=value2 - (Optional, POST only) Another POST key and corresponding value
+# | arg: ...         - (Optional, POST only) More POST keys and values
+#
+# example: ynh_local_curl "/install.php?installButton" "foo=$var1" "bar=$var2"
+#   → will open a POST request to "https://$domain/$path/install.php?installButton" posting "foo=$var1" and "bar=$var2"
+# example: ynh_local_curl -m POST --header "Accept: application/json" \ 
+#   -H "Content-Type: application/json" \
+#   --data "{\"members\":{\"names\": [\"${app}\"],\"roles\": [\"editor\"]}}" -l '"ok":true' \
+#   "http://localhost:5984/"
+#   → will open a POST request to "http://localhost:5984/" adding headers with "Accept: application/json"
+#     and "Content-Type: application/json" sending the data from the "--data" argument. ynh_local_curl will
+#     return with an error if the servers response does not match the extended regex '"ok":true'.
+#
+# For multiple calls, cookies are persisted between each call for the same app.
+#
+# `$domain` and `$path_url` need to be defined externally if the first form for the 'page' argument is used.
+# 
+# The return code of this function will vary depending of the use of --line_match:
+# 
+# If --line_match has been used the return code will be the one of the grep checking line_match
+# against the output of curl. The output of curl will not be returned.
+#
+# If --line_match has not been provided the return code will be the one of the curl command and
+# the output of curl will be echoed.
+#
+# Requires YunoHost version 2.6.4 or higher.
+
+    # Declare an array to define the options of this helper.a
+    local -A supported_methods=( [PUT]=1 [POST]=1 [GET]=1 [DELETE]=1 )
+    local legacy_args=Ld
+    local -A args_array=( [l]=line_match= [m]=method= [H]=header= [n]=no_sleep [L]=location= [d]=data= [u]=user= [p]=password= [s]=seperator= )
+    local line_match
+    local method
+    local -a header 
+    local no_sleep
+    local location
+    local user
+    local password
+    local seperator
+    local -a data
+    local -a curl_opt_args # optional arguments to `curl`
+    # Manage arguments with getopts
+    ynh_handle_getopts_args "$@"
+
+    # make sure method is a supported one
+    if ! [[ -v supported_methods[$method] ]]; then
+      ynh_die --message="method $method not supported by ynh_local_curl"
+    fi
+
+    # Define url of page to curl
+    # $location contains either an URL or just a page
+    local full_page_url
+    if [[ "$location" =~ ^https?:// ]]; then
+        # if $location starts with an http-protocol use value as a complete URL
+        full_page_url="$location"
+    elif [ "${path_url}" == "/" ]; then
+        # if $path_url points to the webserver root just append $location to localhost URL
+        full_page_url="https://localhost$(ynh_normalize_url_path $location)"
+    else
+        # else append $path_url and $location to localhost URL
+        full_page_url="https://localhost${path_url}$(ynh_normalize_url_path $location)"
+    fi
+    flohmarkt_print_debug "full_page_url='$full_page_url'"
+
+    # Concatenate data
+    # POST: all elements of array $data in one string seperated by '&'
+    # PUT:  all elements of $data concatenated in one string
+    # GET:  no data
+    # DELETE: no data
+    # if not defined by --seperator set default
+    if [[ -v seperator ]] && [[ "$seperator" == 'none' ]]; then
+        seperator=''
+    elif ! [[ -v seperator ]] && [[ "$method" == 'PUT' ]]; then
+        seperator=''
+    elif ! [[ -v seperator ]]; then
+        seperator='&'
+    fi
+    join_by() { local IFS="$1"; shift; echo "$*"; }
+    local P_DATA=$( join_by "$seperator" ${data[@]} )
+    if [[ "$P_DATA" != '' ]]; then curl_opt_args+=('--data'); curl_opt_args+=("$P_DATA"); fi
+
+    # prepend every element in header array with " -H "
+    local seq=0
+    while [[ -v header ]] && [[ $seq -lt ${#header[@]} ]]; do
+      curl_opt_args+=('-H')
+      curl_opt_args+=("${header[$seq]}")
+      seq=$(( $seq + 1 ))
+    done
+
+    # build --user for curl 
+    if [[ -n "$user" ]] && [[ -n "$password" ]]; then
+      curl_opt_args+=('--user' "$user:$password")
+    elif [[ -n "$user" ]] && [[ -z "$password" ]]; then
+      ynh_die --message="user provided via '-u/--user' needs password specified via '-p/--password'"
+    fi
+
+    flohmarkt_print_debug "long string curl_opt_args='${curl_opt_args[@]}'"
+    seq=0
+    while [[ $seq -lt ${#curl_opt_args[@]} ]]; do
+      flohmarkt_print_debug "  opt[$seq]='${curl_opt_args[$seq]}'"
+      seq=$(( $seq + 1 ))
+    done
+
+    # https://github.com/YunoHost/yunohost/pull/547
+    # Wait untils nginx has fully reloaded (avoid curl fail with http2) unless disabled
+    if ! [[ -v no_sleep ]]; then
+      sleep 2
+    fi
+
+    local app=${app:-testing}
+    local cookiefile=/tmp/ynh-$app-cookie.txt
+    touch $cookiefile
+    chown root $cookiefile
+    chmod 700 $cookiefile
+
+    # Temporarily enable visitors if needed...
+    # TODO maybe there's a way to do this using --user and --password instead?
+    #   would improve security
+    if ! [[ "$app" == "testing" ]]; then
+        local visitors_enabled=$(ynh_permission_has_user "main" "visitors" && echo yes || echo no)
+        if [[ $visitors_enabled == "no" ]]; then
+            ynh_permission_update --permission "main" --add "visitors"
+        fi
+    fi
+
+    flohmarkt_print_debug executing \'\
+    curl --silent --show-error --insecure --location --resolve "$domain:443:127.0.0.1" \
+      --header "Host: $domain" --cookie-jar $cookiefile --cookie $cookiefile \
+      "${curl_opt_args[@]}" "$full_page_url"\'
+    # Curl the URL
+    local curl_result=$( curl --request "$method" --silent --show-error --insecure --location \
+      --header "Host: $domain" --cookie-jar $cookiefile --cookie $cookiefile \
+      --resolve "$domain:443:127.0.0.1" "${curl_opt_args[@]}" "$full_page_url" )
+    local curl_error=$?
+    flohmarkt_print_debug "curl_result='$curl_result' ($curl_error)"
+    
+    # check result agains --line_match if provided
+    if [[ -v line_match ]] && [[ -n $line_match ]]; then
+      printf '%s' "$curl_result" | grep "$line_match" > /dev/null
+      # will return the error code of the above grep
+      curl_error=$?
+    else
+      # no --line_match, return curls error code and output
+      echo $curl_result
+    fi
+
+    # re-enable security
+    if [[ -v visitor_enabled ]] && [[ $visitors_enabled == "no" ]]; then
+       ynh_permission_update --permission "main" --remove "visitors"
+    fi
+    return $curl_error
+}
+
 #=================================================
 # PERSONAL HELPERS
 #=================================================
 
+# debug output for ynh_handle_getopts_args and ynh_local_curl
+# otherwise not needed TODO delete after development of the two is done
+flohmarkt_debug=0
+flohmarkt_print_debug() {
+    if [[ $flohmarkt_debug -eq 1 ]]; then echo "flohmarkt_debug: $*" >&2; fi
+}
+
+# create symlinks containing domain and path for install, data and log directories
+flohmarkt_ynh_create_symlinks() {
+  ln -s "$flohmarkt_install" "$flohmarkt_sym_install"
+  ln -s "$flohmarkt_data_dir" "$flohmarkt_sym_data_dir"
+  ln -s "$flohmarkt_log_dir" "$flohmarkt_sym_log_dir"
+  true
+}
+
 # set file permissions and owner for installation
 flohmarkt_ynh_set_permission() {
   # install dir - only root needs to write and $app reads
@@ -98,6 +604,22 @@ flohmarkt_ynh_up_inst_couchdb() {
           --package="couchdb"
 }
 
+# (re)initialise database during install and upgrade
+flohmarkt_ynh_initialize_couchdb() {
+    # run in a sub shell to not source the venv activate to the rest of the script
+    (
+        set +o nounset
+        source "$flohmarkt_venv_dir/bin/activate"
+        set -o nounset
+        # change directory to where flohmarkt.conf is located
+        cd $flohmarkt_app_dir
+        # initialize_couchdb seems to re-try on connect problems endlessly blocking the yunohost api
+        # give it 45 seconds to finish and then fail
+        # https://codeberg.org/ChriChri/flohmarkt_ynh/issues/13
+        timeout 45 python3 initialize_couchdb.py $password_couchdb_admin
+    )
+}
+
 flohmarkt_ynh_dump_couchdb() {
   ../settings/scripts/couchdb-dump/couchdb-dump.sh -b -H 127.0.0.1 -d "${app}" \
     -q -u admin -p "${password_couchdb_admin}" -f "${YNH_CWD}/${app}.json"
@@ -110,65 +632,80 @@ flohmarkt_ynh_import_couchdb() {
 }
 
 flohmarkt_ynh_delete_couchdb_user() {
-  # https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/46 - more than one revision?
+  local couchdb_user_revision=$( ynh_local_curl -n -m GET -u admin -p "$password_couchdb_admin" \
-  local couchdb_user_revision=$( curl -sX GET "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}" \
+    "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}" | jq -r ._rev )
-    --user "admin:${password_couchdb_admin}" | jq -r ._rev )
+  ynh_local_curl -n -m DELETE -u admin -p ${password_couchdb_admin} -l '"ok":true' \
-  curl -s -X DELETE "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}?rev=${couchdb_user_revision}" \
+    "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}?rev=${couchdb_user_revision}"
-    --user "admin:${password_couchdb_admin}"
-}
-
-flohmarkt_ynh_delete_couchdb_db() {
-  curl -s -X DELETE "http://127.0.0.1:5984/${app}" --user "admin:${password_couchdb_admin}"
 }
 
 flohmarkt_ynh_create_couchdb_user() {
-  curl -s -X PUT "http://127.0.0.1:5984/_users/org.couchdb.user:${app}" --user "admin:${password_couchdb_admin}"\
+  ynh_local_curl -n -m PUT -u admin -p "${password_couchdb_admin}" \
     -H "Accept: application/json" -H "Content-Type: application/json" \
-    -d "{\"name\": \"${app}\", \"password\": \"${password_couchdb_flohmarkt}\", \"roles\": [], \"type\": \"user\"}"
+    -d '{' \
-  # @@ check answer something like
+    -d "\"name\": \"${app}\", \"password\": \"${password_couchdb_flohmarkt}\"," \
-  # {"ok":true,"id":"org.couchdb.user:flohmarkt","rev":"35-9865694604ab384388eea0f978a6e728"}
+    -d '"roles": [], "type": "user"}' \
+    --line_match='"ok":true' \
+    "http://127.0.0.1:5984/_users/org.couchdb.user:${app}"
 }
 
 flohmarkt_ynh_couchdb_user_permissions() {
-  curl -s -X PUT "http://127.0.0.1:5984/${app}/_security" --user "admin:${password_couchdb_admin}"\
+  ynh_local_curl -n -m PUT -u admin -p "$password_couchdb_admin" \
     -H "Accept: application/json" -H "Content-Type: application/json" \
-    -d "{\"members\":{\"names\": [\"${app}\"],\"roles\": [\"editor\"]}}"
+    -d "{\"members\":{\"names\": [\"${app}\"],\"roles\": [\"editor\"]}}" \
-
+    --line_match='"ok":true' \
+    "http://127.0.0.1:5984/${app}/_security"
 }
 
 flohmarkt_ynh_exists_couchdb_user() {
-  if [[ $( curl -sX GET "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}" \
+  ynh_local_curl -n -m GET  -u admin -p "$password_couchdb_admin" -l "\"_id\":\"org.couchdb.user:${app}\"" \
-    --user "admin:${password_couchdb_admin}" | jq .error ) == '"not_found"' ]]
+    "http://127.0.0.1:5984/_users/org.couchdb.user%3A${app}"
-  then
-    false
-  else
-    true
-  fi
 }
 
 flohmarkt_ynh_exists_couchdb_db() {
-  if [[ $( curl -sX GET "http://127.0.0.1:5984/flohmarkt__22" --user "admin:${password_couchdb_admin}" \
+  ynh_local_curl -n -m GET -u admin -p "$password_couchdb_admin" -l "\"db_name\":\"${app}\"" \
-    | jq .error ) == '"not_found"' ]]
+    "http://127.0.0.1:5984/${app}"
-  then
-    false
-  else
-    true
-  fi
-  
 }
 
-# check whether old couchdb user or database exist before creating the new ones
+flohmarkt_ynh_delete_couchdb_db() {
-flohmarkt_ynh_check_old_couchdb() {
+  local legacy_args='n'
-  if flohmarkt_ynh_exists_couchdb_user; then
+  local -A args_array=( [n]=name= )
-    ynh_die --ret_code=100 --message="CouchDB user '$app' exists already. Stopping install."
+  ynh_handle_getopts_args "$@"
-  elif flohmarkt_ynh_exists_couchdb_db; then
+  local name=${name:-${app}}
-    ynh_die --ret_code=100 --message="CouchDB database '$app' exists already. Stopping install."
+
-  fi  
+  ynh_local_curl -n -m DELETE -u admin -p "$password_couchdb_admin" \
+    --line_match='"ok":true' "http://127.0.0.1:5984/${name}"
+}
+
+# replicate couchdb to a new database
+flohmarkt_ynh_copy_couchdb() {
+  local legacy_args='on'
+  local -A args_array=( [o]=old_name= [n]=new_name= )
+  local new_name
+  local old_name
+  ynh_handle_getopts_args "$@"
+  old_name=${old_name:-${app}}
+
+  ynh_local_curl -n -m POST -u admin -p "$password_couchdb_admin" \
+    -H "Accept: application/json" -H "Content-Type: application/json" \
+    -d '{ "create_target":true,"source" : "' -d"${old_name}" -d'",' \
+    -d '"target":"' -d "${new_name}" -d'"}' --seperator=none \
+    --line_match='"ok":true' "http://127.0.0.1:5984/_replicate"
+}
+
+# copy couchdb to new name and delete source
+flohmarkt_ynh_rename_couchdb() {
+  local legacy_args='on'
+  local -A args_array=( [o]=old_name= [n]=new_name= )
+  local new_name
+  local old_name
+  ynh_handle_getopts_args "$@"
+  old_name=${old_name:-${app}}
+
+  flohmarkt_ynh_copy_couchdb -o "$old_name" -n "$new_name"
+  flohmarkt_ynh_delete_couchdb_db "$old_name"
 }
 
 flohmarkt_ynh_restore_couchdb() {
-  flohmarkt_ynh_check_old_couchdb
-
   flohmarkt_ynh_import_couchdb
   flohmarkt_ynh_create_couchdb_user
   flohmarkt_ynh_couchdb_user_permissions

scripts/backup

@@ -12,10 +12,6 @@ ynh_backup --src_path="/etc/logrotate.d/$app"
 ynh_backup --src_path="/opt/couchdb/etc/local.d/05-flohmarkt.ini"
 ynh_backup --src_path="/etc/systemd/system/$flohmarkt_filename.service"
 ynh_backup --src_path="${flohmarkt_log_dir}"
-ynh_backup --src_path="$flohmarkt_sym_install"
-ynh_backup --src_path="$flohmarkt_sym_data_dir"
-ynh_backup --src_path="$flohmarkt_sym_log_dir"
-
 
 # for the following backups we'll want to stop flohmarkt and couchdb
 # to guarentee a consistant state

scripts/change_url

@@ -1,25 +0,0 @@
-#!/bin/bash
-
-#=================================================
-# GENERIC STARTING
-#=================================================
-# IMPORT GENERIC HELPERS
-#=================================================
-
-source _common.sh
-source /usr/share/yunohost/helpers
-
-#=================================================
-# STANDARD MODIFICATIONS
-#=================================================
-# MODIFY URL IN NGINX CONF
-#=================================================
-ynh_script_progression --message="Updating NGINX web server configuration..." --weight=2
-
-ynh_change_url_nginx_config
-
-#=================================================
-# END OF SCRIPT
-#=================================================
-
-ynh_script_progression --message="Change of URL completed for $app" --last

scripts/install

@@ -1,8 +1,8 @@
 #!/bin/bash
 
 # IMPORT GENERIC HELPERS
-source _common.sh
 source /usr/share/yunohost/helpers
+source _common.sh
 
 # https://codeberg.org/ChriChri/flohmarkt_ynh/issues/9
 # check if couchdb is already installed
@@ -17,7 +17,7 @@ then
 fi
 
 # INITIALIZE AND STORE SETTINGS
-# todo: do we need to store the password un-encrypted somewhere on the system?
+# TODO: do we need to store the password un-encrypted somewhere on the system?
 # → there's no way to get the admin password later if sometimes in the future deleting
 #   the app will delete its specific database only (instead of the whole couchdb server as
 #   of now in the earliest versions of the integration).
@@ -75,17 +75,7 @@ ynh_add_config --template="../conf/flohmarkt.conf" --destination="$flohmarkt_app
 
 # setup couchdb
 ynh_script_progression --message="Setup couchdb database and user for flohmarkt..." --weight=2
-flohmarkt_ynh_check_old_couchdb
+flohmarkt_ynh_initialize_couchdb
-(
-    set +o nounset
-    source "$flohmarkt_venv_dir/bin/activate"
-    set -o nounset
-    cd "$flohmarkt_app_dir"
-    # initialize_couchdb seems to re-try on connect problems endlessly blocking the yunohost api
-    # give it 45 seconds to finish and then fail
-    # https://codeberg.org/ChriChri/flohmarkt_ynh/issues/13
-    timeout 45 python3 initialize_couchdb.py $password_couchdb_admin
-)
 
 # SETUP LOGROTATE
 ynh_script_progression --message="Configuring log rotation..." --weight=2
@@ -123,9 +113,7 @@ ynh_add_fail2ban_config --logpath="$flohmarkt_logfile" --failregex='INFO: +<HOST
 # symlink data_dir and install_dir to make it easier to find the
 # files for a certain domain/path
 ynh_script_progression --message="Creating symlinks..." --weight=1
-ln -s "$flohmarkt_install" "$flohmarkt_sym_install"
+flohmarkt_ynh_create_symlinks
-ln -s "$flohmarkt_data_dir" "$flohmarkt_sym_data_dir"
-ln -s "$flohmarkt_log_dir" "$flohmarkt_sym_log_dir"
 
 # start service
 ynh_script_progression --message="Debug before starting flohmarkt..." --weight=1

scripts/remove

@@ -6,8 +6,8 @@
 # IMPORT GENERIC HELPERS
 #=================================================
 
-source _common.sh
 source /usr/share/yunohost/helpers
+source _common.sh
 
 #=================================================
 # STANDARD REMOVE
@@ -21,30 +21,31 @@ ynh_script_progression --message="Stopping $app..." --weight=2
 yunohost service stop $flohmarkt_filename
 
 # Remove the service from the list of services known by YunoHost (added from `yunohost service add`)
-if ynh_exec_warn_less yunohost service status $flohmarkt_filename >/dev/null
+if ynh_exec_warn_less yunohost service status $flohmarkt_filename >/dev/null; then
-then
 	ynh_script_progression --message="Removing $app service integration..." --weight=2
     ynh_remove_systemd_config --service=$flohmarkt_filename
     yunohost service remove $flohmarkt_filename
 fi
 
-# https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/12
+# remove DB user for this instance - a new install will create a new user and permissions
-ynh_script_progression --message="Removing database and database user..." --weight=2
+ynh_script_progression --message="Removing couchdb user..." --weight=2
-# remove DB
-flohmarkt_ynh_delete_couchdb_db
-# remove DB user for this instance:
 flohmarkt_ynh_delete_couchdb_user
 
+if [[ $YNH_APP_PURGE -eq 1 ]]; then
+  ynh_script_progression --message="Purging couchdb database ..." --weight=2
+  # remove DB
+  flohmarkt_ynh_delete_couchdb_db
+  # remove logfiles directory
+  ynh_script_progression --message="Purging logfiles..." --weight=2
+  ynh_secure_remove --file="${flohmarkt_log_dir}"
+else
+  ynh_script_progression --message="Purge not selected - not deleting couchdb and logfiles..." --weight=1
+fi
+
 # Remove the app-specific logrotate config
 ynh_script_progression --message="Removing logrotate configuration..." --weight=2
 ynh_remove_logrotate
 
-# remove logfiles directory
-# https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/29
-# can only remove a subdir or one file
-ynh_script_progression --message="Removing logfiles..." --weight=2
-ynh_secure_remove --file="${flohmarkt_log_dir}"
-
 # Remove the dedicated NGINX config
 ynh_script_progression --message="Removing nginx configuration..." --weight=2
 ynh_remove_nginx_config
@@ -53,22 +54,16 @@ ynh_remove_nginx_config
 ynh_script_progression --message="Removing fail2ban configuration..." --weight=2
 ynh_remove_fail2ban_config
 
-# Remove home
-# https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/29
-ynh_script_progression --message="Removing data directory..." --weight=2
-ynh_secure_remove --file=$data_dir
-
 # remove systemd service
-ynh_script_progression --message="Removing data directory..." --weight=2
+ynh_script_progression --message="Removing systemd.service directory..." --weight=2
 ynh_remove_systemd_config
 
 # remove symlinks
 ynh_script_progression --message="Removing symlinks..." --weight=2
-ynh_secure_remove --file="$flohmarkt_sym_install"
+# no ynh_secure_remove: 
+# https://codeberg.org/flohmarkt/flohmarkt_ynh/issues/52
 rm "$flohmarkt_sym_install" || true
-ynh_secure_remove --file="$flohmarkt_sym_data_dir"
 rm "$flohmarkt_sym_data_dir" || true
-ynh_secure_remove --file="$flohmarkt_sym_log_dir"
 rm "$flohmarkt_sym_log_dir" || true
 
 #=================================================

scripts/restore

@@ -1,7 +1,7 @@
 #!/bin/bash
 
-source ../settings/scripts/_common.sh
 source /usr/share/yunohost/helpers
+source ../settings/scripts/_common.sh
 
 # reinstall couchdb
 ynh_script_progression --message="Reinstalling couchdb..." --weight=40
@@ -23,6 +23,16 @@ else
   ynh_script_progression --message="CouchDB configuration file already exists." --weight=1
 fi
 
+# check if couchdb already exists and if so rename it to make room for the restore
+if flohmarkt_ynh_exists_couchdb_db; then
+  flohmarkt_couchdb_rename_to="${app}_$(date '+%Y-%m-%d_%H-%M-%S_%N')"
+  if flohmarkt_ynh_rename_couchdb "${app}" "${flohmarkt_couchdb_rename_to}"; then
+    ynh_print_warn --message="renamed existing database ${app} to ${flohmarkt_couchdb_rename_to}"
+  else
+    ynh_die --message="could not rename existing couchdb database and cannot overwrite it"
+  fi
+fi
+
 # restore couchdb from json
 ynh_script_progression --message="Importing couchdb from json backup..." --weight=8
 flohmarkt_ynh_restore_couchdb
@@ -74,4 +84,8 @@ flohmarkt_ynh_start_service
 ynh_script_progression --message="Reloading NGINX web server..." --weight=1
 ynh_systemd_action --service_name=nginx --action=reload
 
+# create symlinks
+ynh_script_progression --message="Creating symlinks..." --weight=1
+flohmarkt_ynh_create_symlinks
+
 ynh_script_progression --message="Restoration completed for $app" --last

scripts/upgrade

@@ -1,7 +1,7 @@
 #!/bin/bash
 
-source _common.sh
 source /usr/share/yunohost/helpers
+source _common.sh
 
 # UPGRADE_PACKAGE if only the YunoHost package has changed
 # UPGRADE_APP if the upstream app version has changed
@@ -59,17 +59,7 @@ ynh_script_progression --message="Upgrading flohmarkt configuration..." --weight
 ynh_add_config --template="../conf/flohmarkt.conf" --destination="$flohmarkt_app_dir/flohmarkt.conf"
 
 ynh_script_progression --message="Upgrading flohmarkt couchdb..." --weight=10
-# run initialize_couchdb.py
+flohmarkt_ynh_initialize_couchdb
-(
-    set +o nounset
-    source "$flohmarkt_venv_dir/bin/activate"
-    set -o nounset
-    cd "$flohmarkt_app_dir"
-    # initialize_couchdb seems to re-try on connect problems endlessly blocking the yunohost api
-    # give it 45 seconds to finish and then fail
-    # https://codeberg.org/ChriChri/flohmarkt_ynh/issues/13
-    timeout 45 python3 initialize_couchdb.py $password_couchdb_admin $password_couchdb_flohmarkt
-)
 
 # set file permissions for install dir
 flohmarkt_ynh_set_permission