YunoHost/yunohost
1
0
Fork 0
mirror of https://github.com/YunoHost/yunohost.git synced 2024-09-03 20:06:10 +02:00
Code Issues Projects Releases Packages Wiki Activity

Merge remote-tracking branch 'origin/dev' into bookworm

Browse source
This commit is contained in:
Alexandre Aubin 2024-06-30 00:23:58 +02:00
commit b661356783
54 changed files with 6019 additions and 1063 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
.gitlab/ci/bot.gitlab-ci.yml
View file

@ -7,11 +7,13 @@ generate-helpers-doc:
- git config --global user.name "$GITHUB_USER"
script:
- cd doc
- python3 generate_helper_doc.py
- python3 generate_helper_doc.py 2
- python3 generate_helper_doc.py 2.1
- python3 generate_resource_doc.py > resources.md
- python3 generate_configpanel_and_formoptions_doc.py > forms.md
- hub clone https://$GITHUB_TOKEN:x-oauth-basic@github.com/YunoHost/doc.git doc_repo
- cp helpers.md doc_repo/pages/06.contribute/10.packaging_apps/20.scripts/10.helpers/packaging_app_scripts_helpers.md
- cp helpers.v2.md doc_repo/pages/06.contribute/10.packaging_apps/20.scripts/10.helpers/packaging_app_scripts_helpers.md
- cp helpers.v2.1.md doc_repo/pages/06.contribute/10.packaging_apps/20.scripts/12.helpers21/packaging_app_scripts_helpers_v21.md
- cp resources.md doc_repo/pages/06.contribute/10.packaging_apps/10.manifest/10.appresources/packaging_app_manifest_resources.md
- cp forms doc_repo/pages/06.contribute/15.dev/03.forms/forms.md
- cd doc_repo

5
conf/postfix/main.cf
View file

@ -107,12 +107,11 @@ virtual_alias_domains =
virtual_minimum_uid = 100
virtual_uid_maps = static:vmail
virtual_gid_maps = static:mail
smtpd_sender_login_maps=
smtpd_sender_login_maps = unionmap:{
# Regular Yunohost accounts
ldap:/etc/postfix/ldap-accounts.cf,
# Extra maps for app system users who need to send emails
hash:/etc/postfix/app_senders_login_maps
hash:/etc/postfix/app_senders_login_maps }
# Dovecot LDA
virtual_transport = dovecot

74
debian/changelog vendored
View file

@ -4,6 +4,80 @@ yunohost (12.0.0) unstable; urgency=low
-- Alexandre Aubin <alex.aubin@mailoo.org> Thu, 04 May 2023 20:30:19 +0200
yunohost (11.2.19) stable; urgency=low
- apps: tweaks to be more robust and prevent the stupid flood of 'sh: 0: getcwd() failed: No such file or directory' when running an app upgrade/remove from /var/www/$app, sometimes making it look like the upgrade failed when it didnt (a349fc03)
- apps: be more robust when an app upgrade succeeds but for some reason is marked with 'broke the system' ... ending up in inconsistent state between the app settings vs the app scritpts (for example in v1->v2 transitions but not only) (e5b57590)
- helpers2.1: Fix getopts error handling ... (3e1c9eba)
- helpers2.1: also run _ynh_apply_default_permissions in ynh_restore to be consistent (also because the user uid on the new system may be different than in the archive etc) (eee84c5f)
-- Alexandre Aubin <alex.aubin@mailoo.org> Sat, 29 Jun 2024 23:55:52 +0200
yunohost (11.2.18) stable; urgency=low
- helpers2.1: Rework _ynh_apply_default_permissions to hopefully remove the necessity to chown/chmod in the app scripts ([#1883](http://github.com/YunoHost/yunohost/pull/1883))
- helpers2.1: in logrotate, make sure to also chown $app the log dir (1dfc47d1d)
- helpers2.1: forgot to rename the apt call in mongodb helpers (7b2959a3e)
- helpers2.1: in ynh_safe_rm, check if target is not a broken symlink before erorring out ([#1716](http://github.com/YunoHost/yunohost/pull/1716))
Thanks to all contributors <3 ! (Félix Piédallu)
-- Alexandre Aubin <alex.aubin@mailoo.org> Sat, 29 Jun 2024 18:05:04 +0200
yunohost (11.2.17.1) stable; urgency=low
- helpers2.1: fix __PATH__/ handling (997388dc)
- ci: Fix helpers 2.1 doc location (7347b08e)
- helpers/doc: De-hide some helpers v1 in documentation now that the structure is less bloated sort of ? (2a7fefae)
- helpers/doc: fix detail block, cant use the HTML <details> because grav doesnt interpret markdown in it (feb9a095)
-- Alexandre Aubin <alex.aubin@mailoo.org> Tue, 25 Jun 2024 14:19:58 +0200
yunohost (11.2.17) stable; urgency=low
- helpers: Misc cleaning / reorganizing to prepare new doc (2895d4d9)
- helpers: rework helper doc now that we have multiple versions of helpers in parallel + improve structure (group helper file in categories) (094cd9dd)
- helpers/mongo: less noisy output when checking the avx flag is here in /proc/cpuinfo (2af4c157)
- apps/helpers2.1: fix app env in resource upgrade context ending up in incorrect helper version being used (ed426f05)
- helpers2.1: forgot to propagate the 'goenv latest' fix from helpers v1 (d8c3ff4c)
- helpers2.1: drop ynh_apps helper because only a single app is using it ... (1fb80e5d)
- helpers2.1: other typo fixes
-- Alexandre Aubin <alex.aubin@mailoo.org> Mon, 24 Jun 2024 22:36:32 +0200
yunohost (11.2.16) stable; urgency=low
- apps/logs: fix some information not being redacted because of the packaging v2 flows (a25033bb)
- logs: misc ad-hoc tweaks to limit the noise in log sharing (06c8fbc8)
- helpers: (1/2/2.1) add a new ynh_app_setting_set_default to replace the unecessarily complex 'if [ -z ${foo:-} ]' trick ([#1873](http://github.com/YunoHost/yunohost/pull/1873))
- helpers2.1: drop unused 'local source' mechanism from ynh_setup_source (dd8db188)
- helpers2.1: fix positional arg parsing in ynh_psql_create_user (e5585136)
- helpers2.1: rework the fpm usage/footprint madness ([#1874](http://github.com/YunoHost/yunohost/pull/1874))
- helpers2.1: fix ynh_config_add_logrotate when no arg is passed (3942ea12)
- helpers2.1: sudo -u$app -> sudo -u $app (d4857834)
- helpers2.1: change default timeout of ynh_systemctl to 60s instead of 300s (262453f1)
- helpers2.1: display 100 lines instead of 20 in CI context when service fails to start (9298738d)
- helpers2.1: when using ynh_systemctl to reload/start/restart a service with a wait_until and it timesout, handle it as a failure rather than keep going (b3409729)
- helpers2.1: for some reason sudo -E doesn't preserve PATH even though it's exported, so we gotta explicitly use --preserve-env=PATH (5f6df6a8)
- helpers2.1: var rename / cosmetic etc for nodejs/ruby/go version and install directories (2b1f7426)
- i18n: Translations updated for Basque, Slovak
Thanks to all contributors <3 ! (alexAubin, Jose Riha, xabirequejo)
-- Alexandre Aubin <alex.aubin@mailoo.org> Sun, 23 Jun 2024 15:30:22 +0200
yunohost (11.2.15) stable; urgency=low
- apps: new experimentals "2.1" helpers ([#1855](http://github.com/YunoHost/yunohost/pull/1855))
- apps: when removing an app with --purge, also remove /var/log/{app}
- apps: drop clumsy auto-update of nodejs via cron job which fills up disk space with nodejs copies and doesnt actually restart the app services...
- apps: fix apt resources when multiple extras are set ([#1869](http://github.com/YunoHost/yunohost/pull/1869))
- mail: allow aliases for sender addresses of apps ([#1843](http://github.com/YunoHost/yunohost/pull/1843))
Thanks to all contributors <3 ! (alexAubin, Chris Vogel, Félix Piédallu)
-- Alexandre Aubin <alex.aubin@mailoo.org> Thu, 20 Jun 2024 21:20:47 +0200
yunohost (11.2.14.1) stable; urgency=low
- helpers: Fix typo in ynh_read_manifest documentation ([#1866](http://github.com/YunoHost/yunohost/pull/1866))

101
doc/generate_helper_doc.py
View file

@ -1,10 +1,55 @@
#!/usr/env/python3
import sys
import os
import glob
import datetime
import subprocess
tree = {
"sources": {
"title": "Sources",
"notes": "This is coupled to the 'sources' resource in the manifest.toml",
"subsections": ["sources"],
},
"tech": {
"title": "App technologies",
"notes": "These allow to install specific version of the technology required to run some apps",
"subsections": ["nodejs", "ruby", "go", "composer"],
},
"db": {
"title": "Databases",
"notes": "This is coupled to the 'database' resource in the manifest.toml - at least for mysql/postgresql. Mongodb/redis may have better integration in the future.",
"subsections": ["mysql", "postgresql", "mongodb", "redis"],
},
"conf": {
"title": "Configurations / templating",
"subsections": [
"templating",
"nginx",
"php",
"systemd",
"fail2ban",
"logrotate",
],
},
"misc": {
"title": "Misc tools",
"subsections": [
"utils",
"setting",
"string",
"backup",
"logging",
"multimedia",
],
},
"meh": {
"title": "Deprecated or handled by the core / app resources since v2",
"subsections": ["permission", "apt", "systemuser"],
},
}
def get_current_commit():
p = subprocess.Popen(
@ -19,14 +64,7 @@ def get_current_commit():
return current_commit
def render(helpers):
current_commit = get_current_commit()
data = {
"helpers": helpers,
"date": datetime.datetime.now().strftime("%d/%m/%Y"),
"version": open("../debian/changelog").readlines()[0].split()[1].strip("()"),
}
def render(tree, helpers_version):
from jinja2 import Template
from ansi2html import Ansi2HTMLConverter
@ -42,12 +80,15 @@ def render(helpers):
t = Template(template)
t.globals["now"] = datetime.datetime.utcnow
result = t.render(
current_commit=current_commit,
data=data,
tree=tree,
date=datetime.datetime.now().strftime("%d/%m/%Y"),
version=open("../debian/changelog").readlines()[0].split()[1].strip("()"),
helpers_version=helpers_version,
current_commit=get_current_commit(),
convert=shell_to_html,
shell_css=shell_css,
)
open("helpers.md", "w").write(result)
open(f"helpers.v{helpers_version}.md", "w").write(result)
##############################################################################
@ -87,7 +128,7 @@ class Parser:
# We're still in a comment bloc
assert line.startswith("# ") or line == "#", malformed_error(i)
current_block["comments"].append(line[2:])
elif line.strip() == "":
elif line.strip() == "" or line.startswith("_ynh"):
# Well eh that was not an actual helper definition ... start over ?
current_reading = "void"
current_block = {
@ -121,7 +162,11 @@ class Parser:
# (we ignore helpers containing [internal] ...)
if (
"[packagingv1]" not in current_block["comments"]
and "[internal]" not in current_block["comments"]
and not any(
line.startswith("[internal]")
for line in current_block["comments"]
)
and not current_block["name"].startswith("_")
):
self.blocks.append(current_block)
current_block = {
@ -212,23 +257,27 @@ def malformed_error(line_number):
def main():
helper_files = sorted(glob.glob("../helpers/*"))
helpers = []
for helper_file in helper_files:
if not os.path.isfile(helper_file):
continue
if len(sys.argv) == 1:
print("This script needs the helper version (1, 2, 2.1) as an argument")
sys.exit(1)
category_name = os.path.basename(helper_file)
print("Parsing %s ..." % category_name)
p = Parser(helper_file)
p.parse_blocks()
for b in p.blocks:
p.parse_block(b)
version = sys.argv[1]
helpers.append((category_name, p.blocks))
for section in tree.values():
section["helpers"] = {}
for subsection in section["subsections"]:
print(f"Parsing {subsection} ...")
helper_file = f"../helpers/helpers.v{version}.d/{subsection}"
assert os.path.isfile(helper_file), f"Uhoh, {file} doesn't exists?"
p = Parser(helper_file)
p.parse_blocks()
for b in p.blocks:
p.parse_block(b)
render(helpers)
section["helpers"][subsection] = p.blocks
render(tree, version)
main()

28
doc/helper_doc_template.md
View file

@ -1,18 +1,26 @@
---
title: App helpers
title: App helpers (v{{ helpers_version }})
template: docs
taxonomy:
category: docs
routes:
default: '/packaging_apps_helpers'
default: '/packaging_apps_helpers{% if helpers_version not in ["1", "2"] %}_v{{ helpers_version }}{% endif %}'
---
Doc auto-generated by [this script](https://github.com/YunoHost/yunohost/blob/{{ current_commit }}/doc/generate_helper_doc.py) on {{data.date}} (YunoHost version {{data.version}})
Doc auto-generated by [this script](https://github.com/YunoHost/yunohost/blob/{{ current_commit }}/doc/generate_helper_doc.py) on {{date}} (YunoHost version {{version}})
{% for section_id, section in tree.items() %}
## {{ section["title"].title() }}
{% if section['notes'] %}<p>{{ section['notes'] }}</p>{% endif %}
{% for subsection, helpers in section["helpers"].items() %}
### {{ subsection.upper() }}
{% for h in helpers %}
#### {{ h.name }}
{% for category, helpers in data.helpers %}
## {{ category.upper() }}
{% for h in helpers %}
### {{ h.name }}
[details summary="<i>{{ h.brief }}</i>" class="helper-card-subtitle text-muted"]
**Usage**: `{{ h.usage }}`
@ -51,9 +59,9 @@ Doc auto-generated by [this script](https://github.com/YunoHost/yunohost/blob/{{
**Details**:
{{ h.details }}
{%- endif %}
[Dude, show me the code!](https://github.com/YunoHost/yunohost/blob/{{ current_commit }}/helpers/{{ category }}#L{{ h.line + 1 }})
[Dude, show me the code!](https://github.com/YunoHost/yunohost/blob/{{ current_commit }}/helpers/helpers.v{{ helpers_version if helpers_version != "2" else "1" }}.d/{{ subsection }}#L{{ h.line + 1 }})
[/details]
{% endfor %}
---
{% endfor %}
{% endfor %}
{% endfor %}

2
helpers/helpers
View file

@ -11,7 +11,7 @@ set +x
YNH_HELPERS_DIR="$SCRIPT_DIR/helpers.v${YNH_HELPERS_VERSION}.d"
case "$YNH_HELPERS_VERSION" in
"1" | "2")
"1" | "2" | "2.1")
readarray -t HELPERS < <(find -L "$YNH_HELPERS_DIR" -mindepth 1 -maxdepth 1 -type f)
for helper in "${HELPERS[@]}"; do
[ -r "$helper" ] && source "$helper"

6
helpers/helpers.v1.d/apt
View file

@ -224,8 +224,6 @@ YNH_INSTALL_APP_DEPENDENCIES_REPLACE="true"
# Define and install dependencies with a equivs control file
#
# [packagingv1]
#
# This helper can/should only be called once per app
#
# example : ynh_install_app_dependencies dep1 dep2 "dep3|dep4|dep5"
@ -364,8 +362,6 @@ ynh_add_app_dependencies() {
# Remove fake package and its dependencies
#
# [packagingv1]
#
# Dependencies will removed only if no other package need them.
#
# usage: ynh_remove_app_dependencies
@ -398,8 +394,6 @@ ynh_remove_app_dependencies() {
# Install packages from an extra repository properly.
#
# [packagingv1]
#
# usage: ynh_install_extra_app_dependencies --repo="repo" --package="dep1 dep2" [--key=key_url] [--name=name]
# | arg: -r, --repo= - Complete url of the extra repository.
# | arg: -p, --package= - The packages to install from this extra repository

82
helpers/helpers.v1.d/composer Normal file
View file

@ -0,0 +1,82 @@
#!/bin/bash
readonly YNH_DEFAULT_COMPOSER_VERSION=1.10.17
# Declare the actual composer version to use.
# A packager willing to use another version of composer can override the variable into its _common.sh.
YNH_COMPOSER_VERSION=${YNH_COMPOSER_VERSION:-$YNH_DEFAULT_COMPOSER_VERSION}
# Execute a command with Composer
#
# usage: ynh_composer_exec [--phpversion=phpversion] [--workdir=$install_dir] --commands="commands"
# | arg: -v, --phpversion - PHP version to use with composer
# | arg: -w, --workdir - The directory from where the command will be executed. Default $install_dir or $final_path
# | arg: -c, --commands - Commands to execute.
#
# Requires YunoHost version 4.2 or higher.
ynh_composer_exec() {
local _globalphpversion=${phpversion-:}
# Declare an array to define the options of this helper.
local legacy_args=vwc
declare -Ar args_array=([v]=phpversion= [w]=workdir= [c]=commands=)
local phpversion
local workdir
local commands
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
workdir="${workdir:-${install_dir:-$final_path}}"
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
phpversion="${phpversion:-$YNH_PHP_VERSION}"
else
phpversion="${phpversion:-$_globalphpversion}"
fi
COMPOSER_HOME="$workdir/.composer" COMPOSER_MEMORY_LIMIT=-1 \
php${phpversion} "$workdir/composer.phar" $commands \
-d "$workdir" --no-interaction --no-ansi 2>&1
}
# Install and initialize Composer in the given directory
#
# usage: ynh_install_composer [--phpversion=phpversion] [--workdir=$install_dir] [--install_args="--optimize-autoloader"] [--composerversion=composerversion]
# | arg: -v, --phpversion - PHP version to use with composer
# | arg: -w, --workdir - The directory from where the command will be executed. Default $install_dir.
# | arg: -a, --install_args - Additional arguments provided to the composer install. Argument --no-dev already include
# | arg: -c, --composerversion - Composer version to install
#
# Requires YunoHost version 4.2 or higher.
ynh_install_composer() {
local _globalphpversion=${phpversion-:}
# Declare an array to define the options of this helper.
local legacy_args=vwac
declare -Ar args_array=([v]=phpversion= [w]=workdir= [a]=install_args= [c]=composerversion=)
local phpversion
local workdir
local install_args
local composerversion
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
workdir="${workdir:-$final_path}"
else
workdir="${workdir:-$install_dir}"
fi
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
phpversion="${phpversion:-$YNH_PHP_VERSION}"
else
phpversion="${phpversion:-$_globalphpversion}"
fi
install_args="${install_args:-}"
composerversion="${composerversion:-$YNH_COMPOSER_VERSION}"
curl -sS https://getcomposer.org/installer \
| COMPOSER_HOME="$workdir/.composer" \
php${phpversion} -- --quiet --install-dir="$workdir" --version=$composerversion \
|| ynh_die --message="Unable to install Composer."
# install dependencies
ynh_composer_exec --phpversion="${phpversion}" --workdir="$workdir" --commands="install --no-dev $install_args" \
|| ynh_die --message="Unable to install core dependencies with Composer."
}

4
helpers/helpers.v1.d/fail2ban
View file

@ -40,9 +40,7 @@
# ignoreregex =
# ```
#
# -----------------------------------------------------------------------------
#
# Note about the "failregex" option:
# ##### Note about the "failregex" option:
#
# regex to match the password failure messages in the logfile. The host must be
# matched by a group named "`host`". The tag "`<HOST>`" can be used for standard

2
helpers/helpers.v1.d/mongodb
View file

@ -310,7 +310,7 @@ ynh_install_mongo() {
ynh_print_info --message="Installing MongoDB Community Edition ..."
local mongo_debian_release=$(ynh_get_debian_release)
if [[ $(cat /proc/cpuinfo) != *"avx"* && "$mongo_version" != "4.4" ]]; then
if [[ "$(grep '^flags' /proc/cpuinfo | uniq)" != *"avx"* && "$mongo_version" != "4.4" ]]; then
ynh_print_warn --message="Installing Mongo 4.4 as $mongo_version is not compatible with your cpu (see https://docs.mongodb.com/manual/administration/production-notes/#x86_64)."
mongo_version="4.4"
fi

15
helpers/helpers.v1.d/mysql
View file

@ -174,6 +174,19 @@ ynh_mysql_user_exists() {
fi
}
# Check if a mysql database exists
#
# [internal]
#
# usage: ynh_mysql_database_exists database
# | arg: database - the database for which to check existence
# | exit: Return 1 if the database doesn't exist, 0 otherwise
#
ynh_mysql_database_exists() {
local database=$1
mysqlshow | grep -qE "^|\s+$database\s+|"
}
# Drop a user
#
# [internal]
@ -239,7 +252,7 @@ ynh_mysql_remove_db() {
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
if mysqlshow | grep -q "^| $db_name "; then
if ynh_mysql_database_exists "$db_name"; then
ynh_mysql_drop_db $db_name
else
ynh_print_warn --message="Database $db_name not found"

62
helpers/helpers.v1.d/nodejs
View file

@ -83,7 +83,7 @@ ynh_use_nodejs() {
# ynh_install_nodejs will install the version of node provided as argument by using n.
#
# usage: ynh_install_nodejs --nodejs_version=nodejs_version
# | arg: -n, --nodejs_version= - Version of node to install. When possible, your should prefer to use major version number (e.g. 8 instead of 8.10.0). The crontab will then handle the update of minor versions when needed.
# | arg: -n, --nodejs_version= - Version of node to install. When possible, your should prefer to use major version number (e.g. 8 instead of 8.10.0).
#
# `n` (Node version management) uses the `PATH` variable to store the path of the version of node it is going to use.
# That's how it changes the version
@ -149,9 +149,6 @@ ynh_install_nodejs() {
# Store nodejs_version into the config of this app
ynh_app_setting_set --app=$app --key=nodejs_version --value=$nodejs_version
# Build the update script and set the cronjob
ynh_cron_upgrade_node
ynh_use_nodejs
}
@ -180,62 +177,5 @@ ynh_remove_nodejs() {
ynh_secure_remove --file="$n_install_dir"
ynh_secure_remove --file="/usr/local/n"
sed --in-place "/N_PREFIX/d" /root/.bashrc
rm --force /etc/cron.daily/node_update
fi
}
# Set a cron design to update your node versions
#
# [internal]
#
# This cron will check and update all minor node versions used by your apps.
#
# usage: ynh_cron_upgrade_node
#
# Requires YunoHost version 2.7.12 or higher.
ynh_cron_upgrade_node() {
# Build the update script
cat >"$n_install_dir/node_update.sh" <<EOF
#!/bin/bash
version_path="$node_version_path"
n_install_dir="$n_install_dir"
# Log the date
date
# List all real installed version of node
all_real_version="\$(find \$version_path/* -maxdepth 0 -type d | sed "s@\$version_path/@@g")"
# Keep only the major version number of each line
all_real_version=\$(echo "\$all_real_version" | sed 's/\..*\$//')
# Remove double entries
all_real_version=\$(echo "\$all_real_version" | sort --unique)
# Read each major version
while read version
do
echo "Update of the version \$version"
sudo \$n_install_dir/bin/n \$version
# Find the last "real" version for this major version of node.
real_nodejs_version=\$(find \$version_path/\$version* -maxdepth 0 | sort --version-sort | tail --lines=1)
real_nodejs_version=\$(basename \$real_nodejs_version)
# Update the symbolic link for this version
sudo ln --symbolic --force --no-target-directory \$version_path/\$real_nodejs_version \$version_path/\$version
done <<< "\$(echo "\$all_real_version")"
EOF
chmod +x "$n_install_dir/node_update.sh"
# Build the cronjob
cat >"/etc/cron.daily/node_update" <<EOF
#!/bin/bash
$n_install_dir/node_update.sh >> $n_install_dir/node_update.log
EOF
chmod +x "/etc/cron.daily/node_update"
}

14
helpers/helpers.v1.d/permission
View file

@ -36,8 +36,6 @@
# | arg: -t, --show_tile= - (optional) Define if a tile will be shown in the SSO. If yes the name of the tile will be the 'label' parameter. Defaults to false for the permission different than 'main'.
# | arg: -P, --protected= - (optional) Define if this permission is protected. If it is protected the administrator won't be able to add or remove the visitors group of this permission. Defaults to 'false'.
#
# [packagingv1]
#
# If provided, 'url' or 'additional_urls' is assumed to be relative to the app domain/path if they
# start with '/'. For example:
# / -> domain.tld/app
@ -145,8 +143,6 @@ ynh_permission_create() {
# Remove a permission for the app (note that when the app is removed all permission is automatically removed)
#
# [packagingv1]
#
# example: ynh_permission_delete --permission=editors
#
# usage: ynh_permission_delete --permission="permission"
@ -165,8 +161,6 @@ ynh_permission_delete() {
# Check if a permission exists
#
# [packagingv1]
#
# usage: ynh_permission_exists --permission=permission
# | arg: -p, --permission= - the permission to check
# | exit: Return 1 if the permission doesn't exist, 0 otherwise
@ -185,8 +179,6 @@ ynh_permission_exists() {
# Redefine the url associated to a permission
#
# [packagingv1]
#
# usage: ynh_permission_url --permission "permission" [--url="url"] [--add_url="new-url" [ "other-new-url" ]] [--remove_url="old-url" [ "other-old-url" ]]
# [--auth_header=true|false] [--clear_urls]
# | arg: -p, --permission= - the name for the permission (by default a permission named "main" is removed automatically when the app is removed)
@ -255,8 +247,6 @@ ynh_permission_url() {
# Update a permission for the app
#
# [packagingv1]
#
# usage: ynh_permission_update --permission "permission" [--add="group" ["group" ...]] [--remove="group" ["group" ...]]
# [--label="label"] [--show_tile=true|false] [--protected=true|false]
# | arg: -p, --permission= - the name for the permission (by default a permission named "main" already exist)
@ -362,8 +352,6 @@ ynh_permission_has_user() {
# Check if a legacy permissions exist
#
# [packagingv1]
#
# usage: ynh_legacy_permissions_exists
# | exit: Return 1 if the permission doesn't exist, 0 otherwise
#
@ -379,8 +367,6 @@ ynh_legacy_permissions_exists() {
# Remove all legacy permissions
#
# [packagingv1]
#
# usage: ynh_legacy_permissions_delete_all
#
# example:

81
helpers/helpers.v1.d/php
View file

@ -359,84 +359,3 @@ ynh_get_scalable_phpfpm() {
fi
fi
}
readonly YNH_DEFAULT_COMPOSER_VERSION=1.10.17
# Declare the actual composer version to use.
# A packager willing to use another version of composer can override the variable into its _common.sh.
YNH_COMPOSER_VERSION=${YNH_COMPOSER_VERSION:-$YNH_DEFAULT_COMPOSER_VERSION}
# Execute a command with Composer
#
# usage: ynh_composer_exec [--phpversion=phpversion] [--workdir=$install_dir] --commands="commands"
# | arg: -v, --phpversion - PHP version to use with composer
# | arg: -w, --workdir - The directory from where the command will be executed. Default $install_dir or $final_path
# | arg: -c, --commands - Commands to execute.
#
# Requires YunoHost version 4.2 or higher.
ynh_composer_exec() {
local _globalphpversion=${phpversion-:}
# Declare an array to define the options of this helper.
local legacy_args=vwc
declare -Ar args_array=([v]=phpversion= [w]=workdir= [c]=commands=)
local phpversion
local workdir
local commands
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
workdir="${workdir:-${install_dir:-$final_path}}"
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
phpversion="${phpversion:-$YNH_PHP_VERSION}"
else
phpversion="${phpversion:-$_globalphpversion}"
fi
COMPOSER_HOME="$workdir/.composer" COMPOSER_MEMORY_LIMIT=-1 \
php${phpversion} "$workdir/composer.phar" $commands \
-d "$workdir" --no-interaction --no-ansi 2>&1
}
# Install and initialize Composer in the given directory
#
# usage: ynh_install_composer [--phpversion=phpversion] [--workdir=$install_dir] [--install_args="--optimize-autoloader"] [--composerversion=composerversion]
# | arg: -v, --phpversion - PHP version to use with composer
# | arg: -w, --workdir - The directory from where the command will be executed. Default $install_dir.
# | arg: -a, --install_args - Additional arguments provided to the composer install. Argument --no-dev already include
# | arg: -c, --composerversion - Composer version to install
#
# Requires YunoHost version 4.2 or higher.
ynh_install_composer() {
local _globalphpversion=${phpversion-:}
# Declare an array to define the options of this helper.
local legacy_args=vwac
declare -Ar args_array=([v]=phpversion= [w]=workdir= [a]=install_args= [c]=composerversion=)
local phpversion
local workdir
local install_args
local composerversion
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
workdir="${workdir:-$final_path}"
else
workdir="${workdir:-$install_dir}"
fi
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2; then
phpversion="${phpversion:-$YNH_PHP_VERSION}"
else
phpversion="${phpversion:-$_globalphpversion}"
fi
install_args="${install_args:-}"
composerversion="${composerversion:-$YNH_COMPOSER_VERSION}"
curl -sS https://getcomposer.org/installer \
| COMPOSER_HOME="$workdir/.composer" \
php${phpversion} -- --quiet --install-dir="$workdir" --version=$composerversion \
|| ynh_die --message="Unable to install Composer."
# install dependencies
ynh_composer_exec --phpversion="${phpversion}" --workdir="$workdir" --commands="install --no-dev $install_args" \
|| ynh_die --message="Unable to install core dependencies with Composer."
}

36
helpers/helpers.v1.d/setting
View file

@ -44,6 +44,42 @@ ynh_app_setting_set() {
ynh_app_setting "set" "$app" "$key" "$value"
}
# Set an application setting but only if the "$key" variable ain't set yet
#
# Note that it doesn't just define the setting but ALSO define the $foobar variable
#
# Hence it's meant as a replacement for this legacy overly complex syntax:
#
# if [ -z "${foo:-}" ]
# then
# foo="bar"
# ynh_app_setting_set --key="foo" --value="$foo"
# fi
#
# usage: ynh_app_setting_set_default --app=app --key=key --value=value
# | arg: -a, --app= - the application id
# | arg: -k, --key= - the setting name to set
# | arg: -v, --value= - the default setting value to set
#
# Requires YunoHost version 11.1.16 or higher.
ynh_app_setting_set_default() {
local _globalapp=${app-:}
# Declare an array to define the options of this helper.
local legacy_args=akv
local -A args_array=([a]=app= [k]=key= [v]=value=)
local app
local key
local value
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
app="${app:-$_globalapp}"
if [ -z "${!key:-}" ]; then
eval $key=\$value
ynh_app_setting "set" "$app" "$key" "$value"
fi
}
# Delete an application setting
#
# usage: ynh_app_setting_delete --app=app --key=key

300
helpers/helpers.v1.d/sources Normal file
View file

@ -0,0 +1,300 @@
#!/bin/bash
# Download, check integrity, uncompress and patch the source from app.src
#
# usage: ynh_setup_source --dest_dir=dest_dir [--source_id=source_id] [--keep="file1 file2"] [--full_replace]
# | arg: -d, --dest_dir= - Directory where to setup sources
# | arg: -s, --source_id= - Name of the source, defaults to `main` (when the sources resource exists in manifest.toml) or (legacy) `app` otherwise
# | arg: -k, --keep= - Space-separated list of files/folders that will be backup/restored in $dest_dir, such as a config file you don't want to overwrite. For example 'conf.json secrets.json logs' (no trailing `/` for folders)
# | arg: -r, --full_replace= - Remove previous sources before installing new sources (can be 1 or 0, default to 0)
#
# ##### New 'sources' resources
#
# (See also the resources documentation which may be more complete?)
#
# This helper will read infos from the 'sources' resources in the manifest.toml of the app
# and expect a structure like:
#
# ```toml
# [resources.sources]
# [resources.sources.main]
# url = "https://some.address.to/download/the/app/archive"
# sha256 = "0123456789abcdef" # The sha256 sum of the asset obtained from the URL
# ```
#
# ##### Optional flags
#
# ```text
# format = "tar.gz"/xz/bz2 # automatically guessed from the extension of the URL, but can be set explicitly. Will use `tar` to extract
# "zip" # automatically guessed from the extension of the URL, but can be set explicitly. Will use `unzip` to extract
# "docker" # useful to extract files from an already-built docker image (instead of rebuilding them locally). Will use `docker-image-extract` to extract
# "whatever" # an arbitrary value, not really meaningful except to imply that the file won't be extracted
#
# in_subdir = true # default, there's an intermediate subdir in the archive before accessing the actual files
# false # sources are directly in the archive root
# n # (special cases) an integer representing a number of subdirs levels to get rid of
#
# extract = true # default if file is indeed an archive such as .zip, .tar.gz, .tar.bz2, ...
# = false # default if file 'format' is not set and the file is not to be extracted because it is not an archive but a script or binary or whatever asset.
# # in which case the file will only be `mv`ed to the location possibly renamed using the `rename` value
#
# rename = "whatever_your_want" # to be used for convenience when `extract` is false and the default name of the file is not practical
# platform = "linux/amd64" # (defaults to "linux/$YNH_ARCH") to be used in conjonction with `format = "docker"` to specify which architecture to extract for
# ```
#
# You may also define assets url and checksum per-architectures such as:
# ```toml
# [resources.sources]
# [resources.sources.main]
# amd64.url = "https://some.address.to/download/the/app/archive/when/amd64"
# amd64.sha256 = "0123456789abcdef"
# armhf.url = "https://some.address.to/download/the/app/archive/when/armhf"
# armhf.sha256 = "fedcba9876543210"
# ```
#
# In which case `ynh_setup_source --dest_dir="$install_dir"` will automatically pick the appropriate source depending on the arch
#
# The helper will:
# - Download the specific URL if there is no local archive
# - Check the integrity with the specific sha256 sum
# - Uncompress the archive to `$dest_dir`.
# - If `in_subdir` is true, the first level directory of the archive will be removed.
# - If `in_subdir` is a numeric value, the N first level directories will be removed.
# - Patches named `sources/patches/${src_id}-*.patch` will be applied to `$dest_dir`
# - Extra files in `sources/extra_files/$src_id` will be copied to dest_dir
#
# Requires YunoHost version 2.6.4 or higher.
ynh_setup_source() {
# Declare an array to define the options of this helper.
local legacy_args=dsk
local -A args_array=([d]=dest_dir= [s]=source_id= [k]=keep= [r]=full_replace=)
local dest_dir
local source_id
local keep
local full_replace
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
keep="${keep:-}"
full_replace="${full_replace:-0}"
if test -e $YNH_APP_BASEDIR/manifest.toml && cat $YNH_APP_BASEDIR/manifest.toml | toml_to_json | jq -e '.resources.sources' >/dev/null
then
source_id="${source_id:-main}"
local sources_json=$(cat $YNH_APP_BASEDIR/manifest.toml | toml_to_json | jq ".resources.sources[\"$source_id\"]")
if jq -re ".url" <<< "$sources_json"
then
local arch_prefix=""
else
local arch_prefix=".$YNH_ARCH"
fi
local src_url="$(jq -r "$arch_prefix.url" <<< "$sources_json" | sed 's/^null$//')"
local src_sum="$(jq -r "$arch_prefix.sha256" <<< "$sources_json" | sed 's/^null$//')"
local src_sumprg="sha256sum"
local src_format="$(jq -r ".format" <<< "$sources_json" | sed 's/^null$//')"
local src_in_subdir="$(jq -r ".in_subdir" <<< "$sources_json" | sed 's/^null$//')"
local src_extract="$(jq -r ".extract" <<< "$sources_json" | sed 's/^null$//')"
local src_platform="$(jq -r ".platform" <<< "$sources_json" | sed 's/^null$//')"
local src_rename="$(jq -r ".rename" <<< "$sources_json" | sed 's/^null$//')"
[[ -n "$src_url" ]] || ynh_die "No URL defined for source $source_id$arch_prefix ?"
[[ -n "$src_sum" ]] || ynh_die "No sha256 sum defined for source $source_id$arch_prefix ?"
if [[ -z "$src_format" ]]
then
if [[ "$src_url" =~ ^.*\.zip$ ]] || [[ "$src_url" =~ ^.*/zipball/.*$ ]]
then
src_format="zip"
elif [[ "$src_url" =~ ^.*\.tar\.gz$ ]] || [[ "$src_url" =~ ^.*\.tgz$ ]] || [[ "$src_url" =~ ^.*/tar\.gz/.*$ ]] || [[ "$src_url" =~ ^.*/tarball/.*$ ]]
then
src_format="tar.gz"
elif [[ "$src_url" =~ ^.*\.tar\.xz$ ]]
then
src_format="tar.xz"
elif [[ "$src_url" =~ ^.*\.tar\.bz2$ ]]
then
src_format="tar.bz2"
elif [[ -z "$src_extract" ]]
then
src_extract="false"
fi
fi
else
source_id="${source_id:-app}"
local src_file_path="$YNH_APP_BASEDIR/conf/${source_id}.src"
# Load value from configuration file (see above for a small doc about this file
# format)
local src_url=$(grep 'SOURCE_URL=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_sum=$(grep 'SOURCE_SUM=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_sumprg=$(grep 'SOURCE_SUM_PRG=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_format=$(grep 'SOURCE_FORMAT=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_in_subdir=$(grep 'SOURCE_IN_SUBDIR=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_rename=$(grep 'SOURCE_FILENAME=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_extract=$(grep 'SOURCE_EXTRACT=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_platform=$(grep 'SOURCE_PLATFORM=' "$src_file_path" | cut --delimiter='=' --fields=2-)
fi
# Default value
src_sumprg=${src_sumprg:-sha256sum}
src_in_subdir=${src_in_subdir:-true}
src_format=${src_format:-tar.gz}
src_format=$(echo "$src_format" | tr '[:upper:]' '[:lower:]')
src_extract=${src_extract:-true}
if [[ "$src_extract" != "true" ]] && [[ "$src_extract" != "false" ]]
then
ynh_die "For source $source_id, expected either 'true' or 'false' for the extract parameter"
fi
# (Unused?) mecanism where one can have the file in a special local cache to not have to download it...
local local_src="/opt/yunohost-apps-src/${YNH_APP_ID}/${source_id}"
# Gotta use this trick with 'dirname' because source_id may contain slashes x_x
mkdir -p $(dirname /var/cache/yunohost/download/${YNH_APP_ID}/${source_id})
src_filename="/var/cache/yunohost/download/${YNH_APP_ID}/${source_id}"
if [ "$src_format" = "docker" ]; then
src_platform="${src_platform:-"linux/$YNH_ARCH"}"
else
if test -e "$local_src"; then
cp $local_src $src_filename
fi
[ -n "$src_url" ] || ynh_die "Couldn't parse SOURCE_URL from $src_file_path ?"
# If the file was prefetched but somehow doesn't match the sum, rm and redownload it
if [ -e "$src_filename" ] && ! echo "${src_sum} ${src_filename}" | ${src_sumprg} --check --status
then
rm -f "$src_filename"
fi
# Only redownload the file if it wasnt prefetched
if [ ! -e "$src_filename" ]
then
# NB. we have to declare the var as local first,
# otherwise 'local foo=$(false) || echo 'pwet'" does'nt work
# because local always return 0 ...
local out
# Timeout option is here to enforce the timeout on dns query and tcp connect (c.f. man wget)
out=$(wget --tries 3 --no-dns-cache --timeout 900 --no-verbose --output-document=$src_filename $src_url 2>&1) \
|| ynh_die --message="$out"
fi
# Check the control sum
if ! echo "${src_sum} ${src_filename}" | ${src_sumprg} --check --status
then
local actual_sum="$(${src_sumprg} ${src_filename} | cut --delimiter=' ' --fields=1)"
local actual_size="$(du -hs ${src_filename} | cut --fields=1)"
rm -f ${src_filename}
ynh_die --message="Corrupt source for ${src_url}: Expected sha256sum to be ${src_sum} but got ${actual_sum} (size: ${actual_size})."
fi
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
if [ -n "$keep" ] && [ -e "$dest_dir" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
mkdir -p $keep_dir
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$dest_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$keep_dir/$stuff_to_keep")"
cp --archive "$dest_dir/$stuff_to_keep" "$keep_dir/$stuff_to_keep"
fi
done
fi
if [ "$full_replace" -eq 1 ]; then
ynh_secure_remove --file="$dest_dir"
fi
# Extract source into the app dir
mkdir --parents "$dest_dir"
if [ -n "${install_dir:-}" ] && [ "$dest_dir" == "$install_dir" ]; then
_ynh_apply_default_permissions $dest_dir
fi
if [ -n "${final_path:-}" ] && [ "$dest_dir" == "$final_path" ]; then
_ynh_apply_default_permissions $dest_dir
fi
if [[ "$src_extract" == "false" ]]; then
if [[ -z "$src_rename" ]]
then
mv $src_filename $dest_dir
else
mv $src_filename $dest_dir/$src_rename
fi
elif [[ "$src_format" == "docker" ]]; then
"$YNH_HELPERS_DIR/vendor/docker-image-extract/docker-image-extract" -p $src_platform -o $dest_dir $src_url 2>&1
elif [[ "$src_format" == "zip" ]]; then
# Zip format
# Using of a temp directory, because unzip doesn't manage --strip-components
if $src_in_subdir; then
local tmp_dir=$(mktemp --directory)
unzip -quo $src_filename -d "$tmp_dir"
cp --archive $tmp_dir/*/. "$dest_dir"
ynh_secure_remove --file="$tmp_dir"
else
unzip -quo $src_filename -d "$dest_dir"
fi
ynh_secure_remove --file="$src_filename"
else
local strip=""
if [ "$src_in_subdir" != "false" ]; then
if [ "$src_in_subdir" == "true" ]; then
local sub_dirs=1
else
local sub_dirs="$src_in_subdir"
fi
strip="--strip-components $sub_dirs"
fi
if [[ "$src_format" =~ ^tar.gz|tar.bz2|tar.xz$ ]]; then
tar --extract --file=$src_filename --directory="$dest_dir" $strip
else
ynh_die --message="Archive format unrecognized."
fi
ynh_secure_remove --file="$src_filename"
fi
# Apply patches
if [ -d "$YNH_APP_BASEDIR/sources/patches/" ]; then
local patches_folder=$(realpath $YNH_APP_BASEDIR/sources/patches/)
if (($(find $patches_folder -type f -name "${source_id}-*.patch" 2>/dev/null | wc --lines) > "0")); then
pushd "$dest_dir"
for p in $patches_folder/${source_id}-*.patch; do
echo $p
patch --strip=1 <$p || ynh_print_warn --message="Packagers /!\\ patch $p failed to apply"
done
popd
fi
fi
# Add supplementary files
if test -e "$YNH_APP_BASEDIR/sources/extra_files/${source_id}"; then
cp --archive $YNH_APP_BASEDIR/sources/extra_files/$source_id/. "$dest_dir"
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
if [ -n "$keep" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$keep_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$dest_dir/$stuff_to_keep")"
# We add "--no-target-directory" (short option is -T) to handle the special case
# when we "keep" a folder, but then the new setup already contains the same dir (but possibly empty)
# in which case a regular "cp" will create a copy of the directory inside the directory ...
# resulting in something like /var/www/$app/data/data instead of /var/www/$app/data
# cf https://unix.stackexchange.com/q/94831 for a more elaborate explanation on the option
cp --archive --no-target-directory "$keep_dir/$stuff_to_keep" "$dest_dir/$stuff_to_keep"
fi
done
fi
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
}

58
helpers/helpers.v1.d/user → helpers/helpers.v1.d/systemuser
View file

@ -1,59 +1,5 @@
#!/bin/bash
# Check if a YunoHost user exists
#
# usage: ynh_user_exists --username=username
# | arg: -u, --username= - the username to check
# | ret: 0 if the user exists, 1 otherwise.
#
# example: ynh_user_exists 'toto' || echo "User does not exist"
#
# Requires YunoHost version 2.2.4 or higher.
ynh_user_exists() {
# Declare an array to define the options of this helper.
local legacy_args=u
local -A args_array=([u]=username=)
local username
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
yunohost user list --output-as json --quiet | jq -e ".users.\"${username}\"" >/dev/null
}
# Retrieve a YunoHost user information
#
# usage: ynh_user_get_info --username=username --key=key
# | arg: -u, --username= - the username to retrieve info from
# | arg: -k, --key= - the key to retrieve
# | ret: the value associate to that key
#
# example: mail=$(ynh_user_get_info --username="toto" --key=mail)
#
# Requires YunoHost version 2.2.4 or higher.
ynh_user_get_info() {
# Declare an array to define the options of this helper.
local legacy_args=uk
local -A args_array=([u]=username= [k]=key=)
local username
local key
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
yunohost user info "$username" --output-as json --quiet | jq -r ".$key"
}
# Get the list of YunoHost users
#
# usage: ynh_user_list
# | ret: one username per line as strings
#
# example: for u in $(ynh_user_list); do ... ; done
#
# Requires YunoHost version 2.4.0 or higher.
ynh_user_list() {
yunohost user list --output-as json --quiet | jq -r ".users | keys[]"
}
# Check if a user exists on the system
#
# [packagingv1]
@ -96,8 +42,6 @@ ynh_system_group_exists() {
# Create a system user
#
# [packagingv1]
#
# usage: ynh_system_user_create --username=user_name [--home_dir=home_dir] [--use_shell] [--groups="group1 group2"]
# | arg: -u, --username= - Name of the system user that will be create
# | arg: -h, --home_dir= - Path of the home dir for the user. Usually the final path of the app. If this argument is omitted, the user will be created without home
@ -152,8 +96,6 @@ ynh_system_user_create() {
# Delete a system user
#
# [packagingv1]
#
# usage: ynh_system_user_delete --username=user_name
# | arg: -u, --username= - Name of the system user that will be create
#

407
helpers/helpers.v1.d/templating Normal file
View file

@ -0,0 +1,407 @@
#!/bin/bash
# Create a dedicated config file from a template
#
# usage: ynh_add_config --template="template" --destination="destination"
# | arg: -t, --template= - Template config file to use
# | arg: -d, --destination= - Destination of the config file
# | arg: -j, --jinja - Use jinja template instead of the simple `__MY_VAR__` templating format
#
# examples:
# ynh_add_config --template=".env" --destination="$install_dir/.env" # (use the template file "conf/.env" from the app's package)
# ynh_add_config --jinja --template="config.j2" --destination="$install_dir/config" # (use the template file "conf/config.j2" from the app's package)
#
# The template can be by default the name of a file in the conf directory
# of a YunoHost Package, a relative path or an absolute path.
#
# The helper will use the template `template` to generate a config file
# `destination` by replacing the following keywords with global variables
# that should be defined before calling this helper :
# ```
# __PATH__ by $path_url
# __NAME__ by $app
# __NAMETOCHANGE__ by $app
# __USER__ by $app
# __FINALPATH__ by $final_path
# __PHPVERSION__ by $YNH_PHP_VERSION (packaging v1 only, packaging v2 uses phpversion setting implicitly set by apt resource)
# __YNH_NODE_LOAD_PATH__ by $ynh_node_load_PATH
# ```
# And any dynamic variables that should be defined before calling this helper like:
# ```
# __DOMAIN__ by $domain
# __APP__ by $app
# __VAR_1__ by $var_1
# __VAR_2__ by $var_2
# ```
#
# ##### When --jinja is enabled
#
# This option is meant for advanced use-cases where the "simple" templating
# mode ain't enough because you need conditional blocks or loops.
#
# For a full documentation of jinja's syntax you can refer to:
# https://jinja.palletsprojects.com/en/3.1.x/templates/
#
# Note that in YunoHost context, all variables are from shell variables and therefore are strings
#
# ##### Keeping track of manual changes by the admin
#
# The helper will verify the checksum and backup the destination file
# if it's different before applying the new template.
#
# And it will calculate and store the destination file checksum
# into the app settings when configuration is done.
#
# Requires YunoHost version 4.1.0 or higher.
ynh_add_config() {
# Declare an array to define the options of this helper.
local legacy_args=tdj
local -A args_array=([t]=template= [d]=destination= [j]=jinja)
local template
local destination
local jinja
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
local template_path
jinja="${jinja:-0}"
if [ -f "$YNH_APP_BASEDIR/conf/$template" ]; then
template_path="$YNH_APP_BASEDIR/conf/$template"
elif [ -f "$template" ]; then
template_path=$template
else
ynh_die --message="The provided template $template doesn't exist"
fi
ynh_backup_if_checksum_is_different --file="$destination"
# Make sure to set the permissions before we copy the file
# This is to cover a case where an attacker could have
# created a file beforehand to have control over it
# (cp won't overwrite ownership / modes by default...)
touch $destination
chmod 640 $destination
_ynh_apply_default_permissions $destination
if [[ "$jinja" == 1 ]]
then
# This is ran in a subshell such that the "export" does not "contaminate" the main process
(
export $(compgen -v)
j2 "$template_path" -f env -o $destination
)
else
cp -f "$template_path" "$destination"
ynh_replace_vars --file="$destination"
fi
ynh_store_file_checksum --file="$destination"
}
# Replace variables in a file
#
# [internal]
#
# usage: ynh_replace_vars --file="file"
# | arg: -f, --file= - File where to replace variables
#
# The helper will replace the following keywords with global variables
# that should be defined before calling this helper :
# __PATH__ by $path_url
# __NAME__ by $app
# __NAMETOCHANGE__ by $app
# __USER__ by $app
# __FINALPATH__ by $final_path
# __PHPVERSION__ by $YNH_PHP_VERSION (packaging v1 only, packaging v2 uses phpversion setting implicitly set by apt resource)
# __YNH_NODE_LOAD_PATH__ by $ynh_node_load_PATH
#
# And any dynamic variables that should be defined before calling this helper like:
# __DOMAIN__ by $domain
# __APP__ by $app
# __VAR_1__ by $var_1
# __VAR_2__ by $var_2
#
# Requires YunoHost version 4.1.0 or higher.
ynh_replace_vars() {
# Declare an array to define the options of this helper.
local legacy_args=f
local -A args_array=([f]=file=)
local file
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
# Replace specific YunoHost variables
if test -n "${path_url:-}"; then
# path_url_slash_less is path_url, or a blank value if path_url is only '/'
local path_url_slash_less=${path_url%/}
ynh_replace_string --match_string="__PATH__/" --replace_string="$path_url_slash_less/" --target_file="$file"
ynh_replace_string --match_string="__PATH__" --replace_string="$path_url" --target_file="$file"
fi
if test -n "${app:-}"; then
ynh_replace_string --match_string="__NAME__" --replace_string="$app" --target_file="$file"
ynh_replace_string --match_string="__NAMETOCHANGE__" --replace_string="$app" --target_file="$file"
ynh_replace_string --match_string="__USER__" --replace_string="$app" --target_file="$file"
fi
# Legacy
if test -n "${final_path:-}"; then
ynh_replace_string --match_string="__FINALPATH__" --replace_string="$final_path" --target_file="$file"
ynh_replace_string --match_string="__INSTALL_DIR__" --replace_string="$final_path" --target_file="$file"
fi
# Legacy / Packaging v1 only
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2 && test -n "${YNH_PHP_VERSION:-}"; then
ynh_replace_string --match_string="__PHPVERSION__" --replace_string="$YNH_PHP_VERSION" --target_file="$file"
fi
if test -n "${ynh_node_load_PATH:-}"; then
ynh_replace_string --match_string="__YNH_NODE_LOAD_PATH__" --replace_string="$ynh_node_load_PATH" --target_file="$file"
fi
# Replace others variables
# List other unique (__ __) variables in $file
local uniques_vars=($(grep -oP '__[A-Z0-9]+?[A-Z0-9_]*?[A-Z0-9]*?__' $file | sort --unique | sed "s@__\([^.]*\)__@\L\1@g"))
set +o xtrace # set +x
# Do the replacement
local delimit=@
for one_var in "${uniques_vars[@]}"; do
# Validate that one_var is indeed defined
# -v checks if the variable is defined, for example:
# -v FOO tests if $FOO is defined
# -v $FOO tests if ${!FOO} is defined
# More info: https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash/17538964#comment96392525_17538964
[[ -v "${one_var:-}" ]] || ynh_die --message="Variable \$$one_var wasn't initialized when trying to replace __${one_var^^}__ in $file"
# Escape delimiter in match/replace string
match_string="__${one_var^^}__"
match_string=${match_string//${delimit}/"\\${delimit}"}
replace_string="${!one_var}"
replace_string=${replace_string//\\/\\\\}
replace_string=${replace_string//${delimit}/"\\${delimit}"}
# Actually replace (sed is used instead of ynh_replace_string to avoid triggering an epic amount of debug logs)
sed --in-place "s${delimit}${match_string}${delimit}${replace_string}${delimit}g" "$file"
done
set -o xtrace # set -x
}
# Get a value from heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_read_var_in_file --file=PATH --key=KEY
# | arg: -f, --file= - the path to the file
# | arg: -k, --key= - the key to get
# | arg: -a, --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
#
# This helpers match several var affectation use case in several languages
# We don't use jq or equivalent to keep comments and blank space in files
# This helpers work line by line, it is not able to work correctly
# if you have several identical keys in your files
#
# Example of line this helpers can managed correctly
# .yml
# title: YunoHost documentation
# email: 'yunohost@yunohost.org'
# .json
# "theme": "colib'ris",
# "port": 8102
# "some_boolean": false,
# "user": null
# .ini
# some_boolean = On
# action = "Clear"
# port = 20
# .php
# $user=
# user => 20
# .py
# USER = 8102
# user = 'https://donate.local'
# CUSTOM['user'] = 'YunoHost'
#
# Requires YunoHost version 4.3 or higher.
ynh_read_var_in_file() {
# Declare an array to define the options of this helper.
local legacy_args=fka
local -A args_array=([f]=file= [k]=key= [a]=after=)
local file
local key
local after
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
after="${after:-}"
[[ -f $file ]] || ynh_die --message="File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local line_number=1
if [[ -n "$after" ]]; then
line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
echo YNH_NULL
return 0
fi
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
local first_char="${expression:0:1}"
if [[ "$first_char" == '"' ]]; then
echo "$expression" | grep -m1 -o -P '"\K([^"](\\")?)*[^\\](?=")' | head -n1 | sed 's/\\"/"/g'
elif [[ "$first_char" == "'" ]]; then
echo "$expression" | grep -m1 -o -P "'\K([^'](\\\\')?)*[^\\\\](?=')" | head -n1 | sed "s/\\\\'/'/g"
else
echo "$expression"
fi
set -o xtrace # set -x
}
# Set a value into heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_write_var_in_file --file=PATH --key=KEY --value=VALUE
# | arg: -f, --file= - the path to the file
# | arg: -k, --key= - the key to set
# | arg: -v, --value= - the value to set
# | arg: -a, --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
#
# Requires YunoHost version 4.3 or higher.
ynh_write_var_in_file() {
# Declare an array to define the options of this helper.
local legacy_args=fkva
local -A args_array=([f]=file= [k]=key= [v]=value= [a]=after=)
local file
local key
local value
local after
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
after="${after:-}"
[[ -f $file ]] || ynh_die --message="File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local after_line_number=1
if [[ -n "$after" ]]; then
after_line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$after_line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$after_line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
return 1
fi
local value_line_number="$(tail +$after_line_number ${file} | grep -m1 -n -i -P $var_part'\K.*$' | cut -d: -f1)"
value_line_number=$((after_line_number + value_line_number))
local range="${after_line_number},${value_line_number} "
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
endline=${expression_with_comment#"$expression"}
endline="$(echo "$endline" | sed 's/\\/\\\\/g')"
value="$(echo "$value" | sed 's/\\/\\\\/g')"
value=${value//&/"\&"}
local first_char="${expression:0:1}"
delimiter=$'\001'
if [[ "$first_char" == '"' ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# So we need \\\\ to go through 2 sed
value="$(echo "$value" | sed 's/"/\\\\"/g')"
sed -ri "${range}s$delimiter"'(^'"${var_part}"'")([^"]|\\")*("[\s;,]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}"'"'"${endline}${delimiter}i" ${file}
elif [[ "$first_char" == "'" ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# However double quotes implies to double \\ to
# So we need \\\\\\\\ to go through 2 sed and 1 double quotes str
value="$(echo "$value" | sed "s/'/\\\\\\\\'/g")"
sed -ri "${range}s$delimiter(^${var_part}')([^']|\\')*('"'[\s,;]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}'${endline}${delimiter}i" ${file}
else
if [[ "$value" == *"'"* ]] || [[ "$value" == *'"'* ]] || [[ "$ext" =~ ^php|py|json|js$ ]]; then
value='\"'"$(echo "$value" | sed 's/"/\\\\"/g')"'\"'
fi
if [[ "$ext" =~ ^yaml|yml$ ]]; then
value=" $value"
fi
sed -ri "${range}s$delimiter(^${var_part}).*\$$delimiter\1${value}${endline}${delimiter}i" ${file}
fi
set -o xtrace # set -x
}
# Render templates with Jinja2
#
# [internal]
#
# Attention : Variables should be exported before calling this helper to be
# accessible inside templates.
#
# usage: ynh_render_template some_template output_path
# | arg: some_template - Template file to be rendered
# | arg: output_path - The path where the output will be redirected to
ynh_render_template() {
local template_path=$1
local output_path=$2
mkdir -p "$(dirname $output_path)"
# Taken from https://stackoverflow.com/a/35009576
python3 -c 'import os, sys, jinja2; sys.stdout.write(
jinja2.Template(sys.stdin.read()
).render(os.environ));' <$template_path >$output_path
}

799
helpers/helpers.v1.d/utils
View file

@ -72,322 +72,6 @@ then
ynh_abort_if_errors
fi
# Download, check integrity, uncompress and patch the source from app.src
#
# usage: ynh_setup_source --dest_dir=dest_dir [--source_id=source_id] [--keep="file1 file2"] [--full_replace]
# | arg: -d, --dest_dir= - Directory where to setup sources
# | arg: -s, --source_id= - Name of the source, defaults to `main` (when the sources resource exists in manifest.toml) or (legacy) `app` otherwise
# | arg: -k, --keep= - Space-separated list of files/folders that will be backup/restored in $dest_dir, such as a config file you don't want to overwrite. For example 'conf.json secrets.json logs' (no trailing `/` for folders)
# | arg: -r, --full_replace= - Remove previous sources before installing new sources (can be 1 or 0, default to 0)
#
# #### New 'sources' resources
#
# (See also the resources documentation which may be more complete?)
#
# This helper will read infos from the 'sources' resources in the manifest.toml of the app
# and expect a structure like:
#
# ```toml
# [resources.sources]
# [resources.sources.main]
# url = "https://some.address.to/download/the/app/archive"
# sha256 = "0123456789abcdef" # The sha256 sum of the asset obtained from the URL
# ```
#
# ##### Optional flags
#
# ```text
# format = "tar.gz"/xz/bz2 # automatically guessed from the extension of the URL, but can be set explicitly. Will use `tar` to extract
# "zip" # automatically guessed from the extension of the URL, but can be set explicitly. Will use `unzip` to extract
# "docker" # useful to extract files from an already-built docker image (instead of rebuilding them locally). Will use `docker-image-extract` to extract
# "whatever" # an arbitrary value, not really meaningful except to imply that the file won't be extracted
#
# in_subdir = true # default, there's an intermediate subdir in the archive before accessing the actual files
# false # sources are directly in the archive root
# n # (special cases) an integer representing a number of subdirs levels to get rid of
#
# extract = true # default if file is indeed an archive such as .zip, .tar.gz, .tar.bz2, ...
# = false # default if file 'format' is not set and the file is not to be extracted because it is not an archive but a script or binary or whatever asset.
# # in which case the file will only be `mv`ed to the location possibly renamed using the `rename` value
#
# rename = "whatever_your_want" # to be used for convenience when `extract` is false and the default name of the file is not practical
# platform = "linux/amd64" # (defaults to "linux/$YNH_ARCH") to be used in conjonction with `format = "docker"` to specify which architecture to extract for
# ```
#
# You may also define assets url and checksum per-architectures such as:
# ```toml
# [resources.sources]
# [resources.sources.main]
# amd64.url = "https://some.address.to/download/the/app/archive/when/amd64"
# amd64.sha256 = "0123456789abcdef"
# armhf.url = "https://some.address.to/download/the/app/archive/when/armhf"
# armhf.sha256 = "fedcba9876543210"
# ```
#
# In which case ynh_setup_source --dest_dir="$install_dir" will automatically pick the appropriate source depending on the arch
#
#
#
# #### Legacy format '.src'
#
# This helper will read `conf/${source_id}.src`, download and install the sources.
#
# The src file need to contains:
# ```
# SOURCE_URL=Address to download the app archive
# SOURCE_SUM=Sha256 sum
# SOURCE_FORMAT=tar.gz
# SOURCE_IN_SUBDIR=false
# SOURCE_FILENAME=example.tar.gz
# SOURCE_EXTRACT=(true|false)
# SOURCE_PLATFORM=linux/arm64/v8
# ```
#
# The helper will:
# - Download the specific URL if there is no local archive
# - Check the integrity with the specific sha256 sum
# - Uncompress the archive to `$dest_dir`.
# - If `in_subdir` is true, the first level directory of the archive will be removed.
# - If `in_subdir` is a numeric value, the N first level directories will be removed.
# - Patches named `sources/patches/${src_id}-*.patch` will be applied to `$dest_dir`
# - Extra files in `sources/extra_files/$src_id` will be copied to dest_dir
#
# Requires YunoHost version 2.6.4 or higher.
ynh_setup_source() {
# Declare an array to define the options of this helper.
local legacy_args=dsk
local -A args_array=([d]=dest_dir= [s]=source_id= [k]=keep= [r]=full_replace=)
local dest_dir
local source_id
local keep
local full_replace
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
keep="${keep:-}"
full_replace="${full_replace:-0}"
if test -e $YNH_APP_BASEDIR/manifest.toml && cat $YNH_APP_BASEDIR/manifest.toml | toml_to_json | jq -e '.resources.sources' >/dev/null
then
source_id="${source_id:-main}"
local sources_json=$(cat $YNH_APP_BASEDIR/manifest.toml | toml_to_json | jq ".resources.sources[\"$source_id\"]")
if jq -re ".url" <<< "$sources_json"
then
local arch_prefix=""
else
local arch_prefix=".$YNH_ARCH"
fi
local src_url="$(jq -r "$arch_prefix.url" <<< "$sources_json" | sed 's/^null$//')"
local src_sum="$(jq -r "$arch_prefix.sha256" <<< "$sources_json" | sed 's/^null$//')"
local src_sumprg="sha256sum"
local src_format="$(jq -r ".format" <<< "$sources_json" | sed 's/^null$//')"
local src_in_subdir="$(jq -r ".in_subdir" <<< "$sources_json" | sed 's/^null$//')"
local src_extract="$(jq -r ".extract" <<< "$sources_json" | sed 's/^null$//')"
local src_platform="$(jq -r ".platform" <<< "$sources_json" | sed 's/^null$//')"
local src_rename="$(jq -r ".rename" <<< "$sources_json" | sed 's/^null$//')"
[[ -n "$src_url" ]] || ynh_die "No URL defined for source $source_id$arch_prefix ?"
[[ -n "$src_sum" ]] || ynh_die "No sha256 sum defined for source $source_id$arch_prefix ?"
if [[ -z "$src_format" ]]
then
if [[ "$src_url" =~ ^.*\.zip$ ]] || [[ "$src_url" =~ ^.*/zipball/.*$ ]]
then
src_format="zip"
elif [[ "$src_url" =~ ^.*\.tar\.gz$ ]] || [[ "$src_url" =~ ^.*\.tgz$ ]] || [[ "$src_url" =~ ^.*/tar\.gz/.*$ ]] || [[ "$src_url" =~ ^.*/tarball/.*$ ]]
then
src_format="tar.gz"
elif [[ "$src_url" =~ ^.*\.tar\.xz$ ]]
then
src_format="tar.xz"
elif [[ "$src_url" =~ ^.*\.tar\.bz2$ ]]
then
src_format="tar.bz2"
elif [[ -z "$src_extract" ]]
then
src_extract="false"
fi
fi
else
source_id="${source_id:-app}"
local src_file_path="$YNH_APP_BASEDIR/conf/${source_id}.src"
# Load value from configuration file (see above for a small doc about this file
# format)
local src_url=$(grep 'SOURCE_URL=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_sum=$(grep 'SOURCE_SUM=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_sumprg=$(grep 'SOURCE_SUM_PRG=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_format=$(grep 'SOURCE_FORMAT=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_in_subdir=$(grep 'SOURCE_IN_SUBDIR=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_rename=$(grep 'SOURCE_FILENAME=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_extract=$(grep 'SOURCE_EXTRACT=' "$src_file_path" | cut --delimiter='=' --fields=2-)
local src_platform=$(grep 'SOURCE_PLATFORM=' "$src_file_path" | cut --delimiter='=' --fields=2-)
fi
# Default value
src_sumprg=${src_sumprg:-sha256sum}
src_in_subdir=${src_in_subdir:-true}
src_format=${src_format:-tar.gz}
src_format=$(echo "$src_format" | tr '[:upper:]' '[:lower:]')
src_extract=${src_extract:-true}
if [[ "$src_extract" != "true" ]] && [[ "$src_extract" != "false" ]]
then
ynh_die "For source $source_id, expected either 'true' or 'false' for the extract parameter"
fi
# (Unused?) mecanism where one can have the file in a special local cache to not have to download it...
local local_src="/opt/yunohost-apps-src/${YNH_APP_ID}/${source_id}"
# Gotta use this trick with 'dirname' because source_id may contain slashes x_x
mkdir -p $(dirname /var/cache/yunohost/download/${YNH_APP_ID}/${source_id})
src_filename="/var/cache/yunohost/download/${YNH_APP_ID}/${source_id}"
if [ "$src_format" = "docker" ]; then
src_platform="${src_platform:-"linux/$YNH_ARCH"}"
else
if test -e "$local_src"; then
cp $local_src $src_filename
fi
[ -n "$src_url" ] || ynh_die "Couldn't parse SOURCE_URL from $src_file_path ?"
# If the file was prefetched but somehow doesn't match the sum, rm and redownload it
if [ -e "$src_filename" ] && ! echo "${src_sum} ${src_filename}" | ${src_sumprg} --check --status
then
rm -f "$src_filename"
fi
# Only redownload the file if it wasnt prefetched
if [ ! -e "$src_filename" ]
then
# NB. we have to declare the var as local first,
# otherwise 'local foo=$(false) || echo 'pwet'" does'nt work
# because local always return 0 ...
local out
# Timeout option is here to enforce the timeout on dns query and tcp connect (c.f. man wget)
out=$(wget --tries 3 --no-dns-cache --timeout 900 --no-verbose --output-document=$src_filename $src_url 2>&1) \
|| ynh_die --message="$out"
fi
# Check the control sum
if ! echo "${src_sum} ${src_filename}" | ${src_sumprg} --check --status
then
local actual_sum="$(${src_sumprg} ${src_filename} | cut --delimiter=' ' --fields=1)"
local actual_size="$(du -hs ${src_filename} | cut --fields=1)"
rm -f ${src_filename}
ynh_die --message="Corrupt source for ${src_url}: Expected sha256sum to be ${src_sum} but got ${actual_sum} (size: ${actual_size})."
fi
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
if [ -n "$keep" ] && [ -e "$dest_dir" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
mkdir -p $keep_dir
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$dest_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$keep_dir/$stuff_to_keep")"
cp --archive "$dest_dir/$stuff_to_keep" "$keep_dir/$stuff_to_keep"
fi
done
fi
if [ "$full_replace" -eq 1 ]; then
ynh_secure_remove --file="$dest_dir"
fi
# Extract source into the app dir
mkdir --parents "$dest_dir"
if [ -n "${install_dir:-}" ] && [ "$dest_dir" == "$install_dir" ]; then
_ynh_apply_default_permissions $dest_dir
fi
if [ -n "${final_path:-}" ] && [ "$dest_dir" == "$final_path" ]; then
_ynh_apply_default_permissions $dest_dir
fi
if [[ "$src_extract" == "false" ]]; then
if [[ -z "$src_rename" ]]
then
mv $src_filename $dest_dir
else
mv $src_filename $dest_dir/$src_rename
fi
elif [[ "$src_format" == "docker" ]]; then
"$YNH_HELPERS_DIR/vendor/docker-image-extract/docker-image-extract" -p $src_platform -o $dest_dir $src_url 2>&1
elif [[ "$src_format" == "zip" ]]; then
# Zip format
# Using of a temp directory, because unzip doesn't manage --strip-components
if $src_in_subdir; then
local tmp_dir=$(mktemp --directory)
unzip -quo $src_filename -d "$tmp_dir"
cp --archive $tmp_dir/*/. "$dest_dir"
ynh_secure_remove --file="$tmp_dir"
else
unzip -quo $src_filename -d "$dest_dir"
fi
ynh_secure_remove --file="$src_filename"
else
local strip=""
if [ "$src_in_subdir" != "false" ]; then
if [ "$src_in_subdir" == "true" ]; then
local sub_dirs=1
else
local sub_dirs="$src_in_subdir"
fi
strip="--strip-components $sub_dirs"
fi
if [[ "$src_format" =~ ^tar.gz|tar.bz2|tar.xz$ ]]; then
tar --extract --file=$src_filename --directory="$dest_dir" $strip
else
ynh_die --message="Archive format unrecognized."
fi
ynh_secure_remove --file="$src_filename"
fi
# Apply patches
if [ -d "$YNH_APP_BASEDIR/sources/patches/" ]; then
local patches_folder=$(realpath $YNH_APP_BASEDIR/sources/patches/)
if (($(find $patches_folder -type f -name "${source_id}-*.patch" 2>/dev/null | wc --lines) > "0")); then
pushd "$dest_dir"
for p in $patches_folder/${source_id}-*.patch; do
echo $p
patch --strip=1 <$p || ynh_print_warn --message="Packagers /!\\ patch $p failed to apply"
done
popd
fi
fi
# Add supplementary files
if test -e "$YNH_APP_BASEDIR/sources/extra_files/${source_id}"; then
cp --archive $YNH_APP_BASEDIR/sources/extra_files/$source_id/. "$dest_dir"
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
if [ -n "$keep" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$keep_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$dest_dir/$stuff_to_keep")"
# We add "--no-target-directory" (short option is -T) to handle the special case
# when we "keep" a folder, but then the new setup already contains the same dir (but possibly empty)
# in which case a regular "cp" will create a copy of the directory inside the directory ...
# resulting in something like /var/www/$app/data/data instead of /var/www/$app/data
# cf https://unix.stackexchange.com/q/94831 for a more elaborate explanation on the option
cp --archive --no-target-directory "$keep_dir/$stuff_to_keep" "$dest_dir/$stuff_to_keep"
fi
done
fi
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
}
# Curl abstraction to help with POST requests to local pages (such as installation forms)
#
# usage: ynh_local_curl "page_uri" "key1=value1" "key2=value2" ...
@ -447,435 +131,6 @@ ynh_local_curl() {
fi
}
# Create a dedicated config file from a template
#
# usage: ynh_add_config --template="template" --destination="destination"
# | arg: -t, --template= - Template config file to use
# | arg: -d, --destination= - Destination of the config file
# | arg: -j, --jinja - Use jinja template instead of legacy __MY_VAR__
#
# examples:
# ynh_add_config --template=".env" --destination="$install_dir/.env" use the template file "../conf/.env"
# ynh_add_config --jinja --template="config.j2" --destination="$install_dir/config" use the template file "../conf/config.j2"
# ynh_add_config --template="/etc/nginx/sites-available/default" --destination="etc/nginx/sites-available/mydomain.conf"
#
##
## How it works in "legacy" mode
##
# The template can be by default the name of a file in the conf directory
# of a YunoHost Package, a relative path or an absolute path.
#
# The helper will use the template `template` to generate a config file
# `destination` by replacing the following keywords with global variables
# that should be defined before calling this helper :
# ```
# __PATH__ by $path_url
# __NAME__ by $app
# __NAMETOCHANGE__ by $app
# __USER__ by $app
# __FINALPATH__ by $final_path
# __PHPVERSION__ by $YNH_PHP_VERSION (packaging v1 only, packaging v2 uses phpversion setting implicitly set by apt resource)
# __YNH_NODE_LOAD_PATH__ by $ynh_node_load_PATH
# ```
# And any dynamic variables that should be defined before calling this helper like:
# ```
# __DOMAIN__ by $domain
# __APP__ by $app
# __VAR_1__ by $var_1
# __VAR_2__ by $var_2
# ```
#
##
## When --jinja is enabled
##
# For a full documentation of the template you can refer to: https://jinja.palletsprojects.com/en/3.1.x/templates/
# In Yunohost context there are no really some specificity except that all variable passed are of type string.
# So here are some example of recommended usage:
#
# If you need a conditional block
#
# {% if should_my_block_be_shown == 'true' %}
# ...
# {% endif %}
#
# or
#
# {% if should_my_block_be_shown == '1' %}
# ...
# {% endif %}
#
# If you need to iterate with loop:
#
# {% for yolo in var_with_multiline_value.splitlines() %}
# ...
# {% endfor %}
#
# or
#
# {% for jail in my_var_with_coma.split(',') %}
# ...
# {% endfor %}
#
# The helper will verify the checksum and backup the destination file
# if it's different before applying the new template.
#
# And it will calculate and store the destination file checksum
# into the app settings when configuration is done.
#
# Requires YunoHost version 4.1.0 or higher.
ynh_add_config() {
# Declare an array to define the options of this helper.
local legacy_args=tdj
local -A args_array=([t]=template= [d]=destination= [j]=jinja)
local template
local destination
local jinja
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
local template_path
jinja="${jinja:-0}"
if [ -f "$YNH_APP_BASEDIR/conf/$template" ]; then
template_path="$YNH_APP_BASEDIR/conf/$template"
elif [ -f "$template" ]; then
template_path=$template
else
ynh_die --message="The provided template $template doesn't exist"
fi
ynh_backup_if_checksum_is_different --file="$destination"
# Make sure to set the permissions before we copy the file
# This is to cover a case where an attacker could have
# created a file beforehand to have control over it
# (cp won't overwrite ownership / modes by default...)
touch $destination
chmod 640 $destination
_ynh_apply_default_permissions $destination
if [[ "$jinja" == 1 ]]
then
# This is ran in a subshell such that the "export" does not "contaminate" the main process
(
export $(compgen -v)
j2 "$template_path" -f env -o $destination
)
else
cp -f "$template_path" "$destination"
ynh_replace_vars --file="$destination"
fi
ynh_store_file_checksum --file="$destination"
}
# Replace variables in a file
#
# [internal]
#
# usage: ynh_replace_vars --file="file"
# | arg: -f, --file= - File where to replace variables
#
# The helper will replace the following keywords with global variables
# that should be defined before calling this helper :
# __PATH__ by $path_url
# __NAME__ by $app
# __NAMETOCHANGE__ by $app
# __USER__ by $app
# __FINALPATH__ by $final_path
# __PHPVERSION__ by $YNH_PHP_VERSION (packaging v1 only, packaging v2 uses phpversion setting implicitly set by apt resource)
# __YNH_NODE_LOAD_PATH__ by $ynh_node_load_PATH
#
# And any dynamic variables that should be defined before calling this helper like:
# __DOMAIN__ by $domain
# __APP__ by $app
# __VAR_1__ by $var_1
# __VAR_2__ by $var_2
#
# Requires YunoHost version 4.1.0 or higher.
ynh_replace_vars() {
# Declare an array to define the options of this helper.
local legacy_args=f
local -A args_array=([f]=file=)
local file
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
# Replace specific YunoHost variables
if test -n "${path_url:-}"; then
# path_url_slash_less is path_url, or a blank value if path_url is only '/'
local path_url_slash_less=${path_url%/}
ynh_replace_string --match_string="__PATH__/" --replace_string="$path_url_slash_less/" --target_file="$file"
ynh_replace_string --match_string="__PATH__" --replace_string="$path_url" --target_file="$file"
fi
if test -n "${app:-}"; then
ynh_replace_string --match_string="__NAME__" --replace_string="$app" --target_file="$file"
ynh_replace_string --match_string="__NAMETOCHANGE__" --replace_string="$app" --target_file="$file"
ynh_replace_string --match_string="__USER__" --replace_string="$app" --target_file="$file"
fi
# Legacy
if test -n "${final_path:-}"; then
ynh_replace_string --match_string="__FINALPATH__" --replace_string="$final_path" --target_file="$file"
ynh_replace_string --match_string="__INSTALL_DIR__" --replace_string="$final_path" --target_file="$file"
fi
# Legacy / Packaging v1 only
if dpkg --compare-versions ${YNH_APP_PACKAGING_FORMAT:-0} lt 2 && test -n "${YNH_PHP_VERSION:-}"; then
ynh_replace_string --match_string="__PHPVERSION__" --replace_string="$YNH_PHP_VERSION" --target_file="$file"
fi
if test -n "${ynh_node_load_PATH:-}"; then
ynh_replace_string --match_string="__YNH_NODE_LOAD_PATH__" --replace_string="$ynh_node_load_PATH" --target_file="$file"
fi
# Replace others variables
# List other unique (__ __) variables in $file
local uniques_vars=($(grep -oP '__[A-Z0-9]+?[A-Z0-9_]*?[A-Z0-9]*?__' $file | sort --unique | sed "s@__\([^.]*\)__@\L\1@g"))
set +o xtrace # set +x
# Do the replacement
local delimit=@
for one_var in "${uniques_vars[@]}"; do
# Validate that one_var is indeed defined
# -v checks if the variable is defined, for example:
# -v FOO tests if $FOO is defined
# -v $FOO tests if ${!FOO} is defined
# More info: https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash/17538964#comment96392525_17538964
[[ -v "${one_var:-}" ]] || ynh_die --message="Variable \$$one_var wasn't initialized when trying to replace __${one_var^^}__ in $file"
# Escape delimiter in match/replace string
match_string="__${one_var^^}__"
match_string=${match_string//${delimit}/"\\${delimit}"}
replace_string="${!one_var}"
replace_string=${replace_string//\\/\\\\}
replace_string=${replace_string//${delimit}/"\\${delimit}"}
# Actually replace (sed is used instead of ynh_replace_string to avoid triggering an epic amount of debug logs)
sed --in-place "s${delimit}${match_string}${delimit}${replace_string}${delimit}g" "$file"
done
set -o xtrace # set -x
}
# Get a value from heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_read_var_in_file --file=PATH --key=KEY
# | arg: -f, --file= - the path to the file
# | arg: -k, --key= - the key to get
# | arg: -a, --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
#
# This helpers match several var affectation use case in several languages
# We don't use jq or equivalent to keep comments and blank space in files
# This helpers work line by line, it is not able to work correctly
# if you have several identical keys in your files
#
# Example of line this helpers can managed correctly
# .yml
# title: YunoHost documentation
# email: 'yunohost@yunohost.org'
# .json
# "theme": "colib'ris",
# "port": 8102
# "some_boolean": false,
# "user": null
# .ini
# some_boolean = On
# action = "Clear"
# port = 20
# .php
# $user=
# user => 20
# .py
# USER = 8102
# user = 'https://donate.local'
# CUSTOM['user'] = 'YunoHost'
#
# Requires YunoHost version 4.3 or higher.
ynh_read_var_in_file() {
# Declare an array to define the options of this helper.
local legacy_args=fka
local -A args_array=([f]=file= [k]=key= [a]=after=)
local file
local key
local after
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
after="${after:-}"
[[ -f $file ]] || ynh_die --message="File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local line_number=1
if [[ -n "$after" ]]; then
line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
echo YNH_NULL
return 0
fi
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
local first_char="${expression:0:1}"
if [[ "$first_char" == '"' ]]; then
echo "$expression" | grep -m1 -o -P '"\K([^"](\\")?)*[^\\](?=")' | head -n1 | sed 's/\\"/"/g'
elif [[ "$first_char" == "'" ]]; then
echo "$expression" | grep -m1 -o -P "'\K([^'](\\\\')?)*[^\\\\](?=')" | head -n1 | sed "s/\\\\'/'/g"
else
echo "$expression"
fi
set -o xtrace # set -x
}
# Set a value into heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_write_var_in_file --file=PATH --key=KEY --value=VALUE
# | arg: -f, --file= - the path to the file
# | arg: -k, --key= - the key to set
# | arg: -v, --value= - the value to set
# | arg: -a, --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
#
# Requires YunoHost version 4.3 or higher.
ynh_write_var_in_file() {
# Declare an array to define the options of this helper.
local legacy_args=fkva
local -A args_array=([f]=file= [k]=key= [v]=value= [a]=after=)
local file
local key
local value
local after
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
after="${after:-}"
[[ -f $file ]] || ynh_die --message="File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local after_line_number=1
if [[ -n "$after" ]]; then
after_line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$after_line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$after_line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
return 1
fi
local value_line_number="$(tail +$after_line_number ${file} | grep -m1 -n -i -P $var_part'\K.*$' | cut -d: -f1)"
value_line_number=$((after_line_number + value_line_number))
local range="${after_line_number},${value_line_number} "
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
endline=${expression_with_comment#"$expression"}
endline="$(echo "$endline" | sed 's/\\/\\\\/g')"
value="$(echo "$value" | sed 's/\\/\\\\/g')"
value=${value//&/"\&"}
local first_char="${expression:0:1}"
delimiter=$'\001'
if [[ "$first_char" == '"' ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# So we need \\\\ to go through 2 sed
value="$(echo "$value" | sed 's/"/\\\\"/g')"
sed -ri "${range}s$delimiter"'(^'"${var_part}"'")([^"]|\\")*("[\s;,]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}"'"'"${endline}${delimiter}i" ${file}
elif [[ "$first_char" == "'" ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# However double quotes implies to double \\ to
# So we need \\\\\\\\ to go through 2 sed and 1 double quotes str
value="$(echo "$value" | sed "s/'/\\\\\\\\'/g")"
sed -ri "${range}s$delimiter(^${var_part}')([^']|\\')*('"'[\s,;]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}'${endline}${delimiter}i" ${file}
else
if [[ "$value" == *"'"* ]] || [[ "$value" == *'"'* ]] || [[ "$ext" =~ ^php|py|json|js$ ]]; then
value='\"'"$(echo "$value" | sed 's/"/\\\\"/g')"'\"'
fi
if [[ "$ext" =~ ^yaml|yml$ ]]; then
value=" $value"
fi
sed -ri "${range}s$delimiter(^${var_part}).*\$$delimiter\1${value}${endline}${delimiter}i" ${file}
fi
set -o xtrace # set -x
}
# Render templates with Jinja2
#
# [internal]
#
# Attention : Variables should be exported before calling this helper to be
# accessible inside templates.
#
# usage: ynh_render_template some_template output_path
# | arg: some_template - Template file to be rendered
# | arg: output_path - The path where the output will be redirected to
ynh_render_template() {
local template_path=$1
local output_path=$2
mkdir -p "$(dirname $output_path)"
# Taken from https://stackoverflow.com/a/35009576
python3 -c 'import os, sys, jinja2; sys.stdout.write(
jinja2.Template(sys.stdin.read()
).render(os.environ));' <$template_path >$output_path
}
# Fetch the Debian release codename
#
# [packagingv1]
@ -1142,3 +397,57 @@ int_to_bool() {
toml_to_json() {
python3 -c 'import toml, json, sys; print(json.dumps(toml.load(sys.stdin)))'
}
# Check if a YunoHost user exists
#
# usage: ynh_user_exists --username=username
# | arg: -u, --username= - the username to check
# | ret: 0 if the user exists, 1 otherwise.
#
# example: ynh_user_exists 'toto' || echo "User does not exist"
#
# Requires YunoHost version 2.2.4 or higher.
ynh_user_exists() {
# Declare an array to define the options of this helper.
local legacy_args=u
local -A args_array=([u]=username=)
local username
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
yunohost user list --output-as json --quiet | jq -e ".users.\"${username}\"" >/dev/null
}
# Retrieve a YunoHost user information
#
# usage: ynh_user_get_info --username=username --key=key
# | arg: -u, --username= - the username to retrieve info from
# | arg: -k, --key= - the key to retrieve
# | ret: the value associate to that key
#
# example: mail=$(ynh_user_get_info --username="toto" --key=mail)
#
# Requires YunoHost version 2.2.4 or higher.
ynh_user_get_info() {
# Declare an array to define the options of this helper.
local legacy_args=uk
local -A args_array=([u]=username= [k]=key=)
local username
local key
# Manage arguments with getopts
ynh_handle_getopts_args "$@"
yunohost user info "$username" --output-as json --quiet | jq -r ".$key"
}
# Get the list of YunoHost users
#
# usage: ynh_user_list
# | ret: one username per line as strings
#
# example: for u in $(ynh_user_list); do ... ; done
#
# Requires YunoHost version 2.4.0 or higher.
ynh_user_list() {
yunohost user list --output-as json --quiet | jq -r ".users | keys[]"
}

317
helpers/helpers.v2.1.d/apt Normal file
View file

@ -0,0 +1,317 @@
#!/bin/bash
YNH_APT_INSTALL_DEPENDENCIES_REPLACE="true"
# Define and install dependencies with a equivs control file
#
# example : ynh_install_app_dependencies dep1 dep2 "dep3|dep4|dep5"
#
# usage: ynh_install_app_dependencies dep [dep [...]]
# | arg: dep - the package name to install in dependence.
# | arg: "dep1|dep2|…" - You can specify alternatives. It will require to install (dep1 or dep2, etc).
#
ynh_apt_install_dependencies() {
# Add a comma for each space between packages. But not add a comma if the space separate a version specification. (See below)
local dependencies="$(sed 's/\([^\<=\>]\)\ \([^(]\)/\1, \2/g' <<< "$@" | sed 's/|/ | /')"
local version=$(ynh_read_manifest "version")
local app_ynh_deps="${app//_/-}-ynh-deps" # Replace all '_' by '-', and append -ynh-deps
# Handle specific versions
if grep '[<=>]' <<< "$dependencies"; then
# Replace version specifications by relationships syntax
# https://www.debian.org/doc/debian-policy/ch-relationships.html
# Sed clarification
# [^(\<=\>] ignore if it begins by ( or < = >. To not apply twice.
# [\<=\>] matches < = or >
# \+ matches one or more occurence of the previous characters, for >= or >>.
# [^,]\+ matches all characters except ','
# Ex: 'package>=1.0' will be replaced by 'package (>= 1.0)'
dependencies="$(sed 's/\([^(\<=\>]\)\([\<=\>]\+\)\([^,]\+\)/\1 (\2 \3)/g' <<< "$dependencies")"
fi
# ############################## #
# Specific tweaks related to PHP #
# ############################## #
# Check for specific php dependencies which requires sury
# This grep will for example return "7.4" if dependencies is "foo bar php7.4-pwet php-gni"
# The (?<=php) syntax corresponds to lookbehind ;)
local specific_php_version=$(grep -oP '(?<=php)[0-9.]+(?=-|\>|)' <<< "$dependencies" | sort -u)
if [[ -n "$specific_php_version" ]]
then
# Cover a small edge case where a packager could have specified "php7.4-pwet php5-gni" which is confusing
[[ $(echo $specific_php_version | wc -l) -eq 1 ]] \
|| ynh_die "Inconsistent php versions in dependencies ... found : $specific_php_version"
dependencies+=", php${specific_php_version}, php${specific_php_version}-fpm, php${specific_php_version}-common"
local old_php_version=$(ynh_app_setting_get --key=php_version)
# If the PHP version changed, remove the old fpm conf
if [ -n "$old_php_version" ] && [ "$old_php_version" != "$specific_php_version" ]; then
if [[ -f "/etc/php/$php_version/fpm/pool.d/$app.conf" ]]
then
ynh_backup_if_checksum_is_different "/etc/php/$php_version/fpm/pool.d/$app.conf"
ynh_remove_fpm_config
fi
fi
# Store php_version into the config of this app
ynh_app_setting_set --key=php_version --value=$specific_php_version
# Set the default php version back as the default version for php-cli.
if test -e /usr/bin/php$YNH_DEFAULT_PHP_VERSION
then
update-alternatives --set php /usr/bin/php$YNH_DEFAULT_PHP_VERSION
fi
elif grep --quiet 'php' <<< "$dependencies"; then
ynh_app_setting_set --key=php_version --value=$YNH_DEFAULT_PHP_VERSION
fi
# Specific tweak related to Postgresql (cf end of the helper)
local psql_installed="$(_ynh_apt_package_is_installed "postgresql-$PSQL_VERSION" && echo yes || echo no)"
# The first time we run ynh_install_app_dependencies, we will replace the
# entire control file (This is in particular meant to cover the case of
# upgrade script where ynh_install_app_dependencies is called with this
# expected effect) Otherwise, any subsequent call will add dependencies
# to those already present in the equivs control file.
if [[ $YNH_APT_INSTALL_DEPENDENCIES_REPLACE == "true" ]]
then
YNH_APT_INSTALL_DEPENDENCIES_REPLACE="false"
else
local current_dependencies=""
if _ynh_apt_package_is_installed "${app_ynh_deps}"
then
current_dependencies="$(dpkg-query --show --showformat='${Depends}' ${app_ynh_deps}) "
current_dependencies=${current_dependencies// | /|}
fi
dependencies="$current_dependencies, $dependencies"
fi
# #############################
# Actual install using equivs #
# #############################
# Prepare the virtual-dependency control file for equivs
local TMPDIR=$(mktemp --directory)
cat >${TMPDIR}/control <<EOF # Make a control file for equivs-build
Section: misc
Priority: optional
Package: ${app_ynh_deps}
Version: ${version}
Depends: ${dependencies}
Architecture: all
Description: Fake package for ${app} (YunoHost app) dependencies
This meta-package is only responsible of installing its dependencies.
EOF
# Make sure to delete equivs' legacy compat file
# It's now handle somewhat magically through the control file
rm -f /usr/share/equivs/template/debian/compat
_ynh_apt update
_ynh_wait_dpkg_free
(
# NB: this is in a subshell (though not sure why exactly not just use pushd/popd...)
cd "$TMPDIR"
# Install the fake package without its dependencies with dpkg --force-depends
LC_ALL=C equivs-build ./control > ./equivs_log 2>&1 || { cat ./equivs_log; false; }
LC_ALL=C dpkg --force-depends --install "./${app_ynh_deps}_${version}_all.deb" > ./dpkg_log 2>&1
)
# Then install the missing dependencies with apt install
_ynh_apt_install --fix-broken || {
# If the installation failed
# (the following is ran inside { } to not start a subshell otherwise ynh_die wouldnt exit the original process)
# Parse the list of problematic dependencies from dpkg's log ...
# (relevant lines look like: "foo-ynh-deps depends on bar; however:")
cat $TMPDIR/dpkg_log
local problematic_dependencies="$(grep -oP '(?<=-ynh-deps depends on ).*(?=; however)' $TMPDIR/dpkg_log | tr '\n' ' ')"
# Fake an install of those dependencies to see the errors
# The sed command here is, Print only from 'Reading state info' to the end.
[[ -n "$problematic_dependencies" ]] && _ynh_apt_install $problematic_dependencies --dry-run 2>&1 | sed --quiet '/Reading state info/,$p' | grep -v "fix-broken\|Reading state info" >&2
ynh_die "Unable to install dependencies"
}
rm --recursive --force "$TMPDIR" # Remove the temp dir.
# check if the package is actually installed
_ynh_apt_package_is_installed "${app_ynh_deps}" || ynh_die "Unable to install dependencies"
# Specific tweak related to Postgresql
# -> trigger postgresql regenconf if we may have just installed postgresql
local psql_installed2="$(_ynh_apt_package_is_installed "postgresql-$PSQL_VERSION" && echo yes || echo no)"
if [[ "$psql_installed" != "$psql_installed2" ]]
then
yunohost tools regen-conf postgresql
fi
}
# Remove fake package and its dependencies
#
# Dependencies will removed only if no other package need them.
#
# usage: ynh_apt_remove_dependencies
ynh_apt_remove_dependencies() {
local app_ynh_deps="${app//_/-}-ynh-deps" # Replace all '_' by '-', and append -ynh-deps
local current_dependencies=""
if _ynh_apt_package_is_installed "${app_ynh_deps}"; then
current_dependencies="$(dpkg-query --show --showformat='${Depends}' ${app_ynh_deps}) "
current_dependencies=${current_dependencies// | /|}
fi
# Edge case where the app dep may be on hold,
# cf https://forum.yunohost.org/t/migration-error-cause-of-ffsync/20675/4
if apt-mark showhold | grep -q -w ${app_ynh_deps}
then
apt-mark unhold ${app_ynh_deps}
fi
# Remove the fake package and its dependencies if they not still used.
# (except if dpkg doesn't know anything about the package,
# which should be symptomatic of a failed install, and we don't want bash to report an error)
if dpkg-query --show ${app_ynh_deps} &>/dev/null
then
_ynh_apt autoremove --purge ${app_ynh_deps}
fi
}
# Install packages from an extra repository properly.
#
# usage: ynh_apt_install_dependencies_from_extra_repository --repo="repo" --package="dep1 dep2" --key=key_url
# | arg: --repo= - Complete url of the extra repository.
# | arg: --package= - The packages to install from this extra repository
# | arg: --key= - url to get the public key.
#
ynh_apt_install_dependencies_from_extra_repository() {
# ============ Argument parsing =============
local -A args_array=([r]=repo= [p]=package= [k]=key=)
local repo
local package
local key
ynh_handle_getopts_args "$@"
# ===========================================
# Split the repository into uri, suite and components.
repo="${repo#deb }"
local uri="$(echo "$repo" | awk '{ print $1 }')"
local suite="$(echo "$repo" | awk '{ print $2 }')"
local component="${repo##$uri $suite }"
# Add the new repo in sources.list.d
mkdir --parents "/etc/apt/sources.list.d"
echo "deb $uri $suite $component" > "/etc/apt/sources.list.d/$app.list"
# Pin the new repo with the default priority, so it won't be used for upgrades.
# Build $pin from the uri without http and any sub path
local pin="${uri#*://}"
pin="${pin%%/*}"
# Pin repository
mkdir --parents "/etc/apt/preferences.d"
cat << EOF > "/etc/apt/preferences.d/$app"
Package: *
Pin: origin $pin
Pin-Priority: 995
EOF
mkdir --parents "/etc/apt/trusted.gpg.d"
# Timeout option is here to enforce the timeout on dns query and tcp connect (c.f. man wget)
wget --timeout 900 --quiet "$key" --output-document=- | gpg --dearmor > /etc/apt/trusted.gpg.d/$app.gpg
# Update the list of package with the new repo NB: we use -o
# Dir::Etc::sourcelist to only refresh this repo, because
# ynh_apt_install_dependencies will also call an ynh_apt update on its own
# and it's good to limit unecessary requests ... Here we mainly want to
# validate that the url+key is correct before going further
_ynh_apt update -o Dir::Etc::sourcelist="/etc/apt/sources.list.d/$app.list"
# Install requested dependencies from this extra repository.
# NB: because of the mechanism with $ynh_apt_install_DEPENDENCIES_REPLACE,
# this will usually only *append* to the existing list of dependency, not
# replace the existing $app-ynh-deps
ynh_apt_install_dependencies "$package"
# Force to upgrade to the last version...
# Without doing apt install, an already installed dep is not upgraded
local apps_auto_installed="$(apt-mark showauto $package)"
_ynh_apt_install "$package"
[ -z "$apps_auto_installed" ] || apt-mark auto $apps_auto_installed
# Remove this extra repository after packages are installed
ynh_safe_rm "/etc/apt/sources.list.d/$app.list"
ynh_safe_rm "/etc/apt/preferences.d/$app"
ynh_safe_rm "/etc/apt/trusted.gpg.d/$app.gpg"
_ynh_apt update
}
# #####################
# Internal misc utils #
# #####################
# Check if apt is free to use, or wait, until timeout.
_ynh_wait_dpkg_free() {
local try
set +o xtrace # set +x
# With seq 1 17, timeout will be almost 30 minutes
for try in $(seq 1 17); do
# Check if /var/lib/dpkg/lock is used by another process
if lsof /var/lib/dpkg/lock >/dev/null; then
echo "apt is already in use..."
# Sleep an exponential time at each round
sleep $((try * try))
else
# Check if dpkg hasn't been interrupted and is fully available.
# See this for more information: https://sources.debian.org/src/apt/1.4.9/apt-pkg/deb/debsystem.cc/#L141-L174
local dpkg_dir="/var/lib/dpkg/updates/"
# For each file in $dpkg_dir
while read dpkg_file <&9; do
# Check if the name of this file contains only numbers.
if echo "$dpkg_file" | grep --perl-regexp --quiet "^[[:digit:]]+$"; then
# If so, that a remaining of dpkg.
ynh_print_warn "dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem."
set -o xtrace # set -x
return 1
fi
done 9<<<"$(ls -1 $dpkg_dir)"
set -o xtrace # set -x
return 0
fi
done
echo "apt still used, but timeout reached !"
set -o xtrace # set -x
}
# Check either a package is installed or not
_ynh_apt_package_is_installed() {
local package=$1
dpkg-query --show --showformat='${db:Status-Status}' "$package" 2>/dev/null \
| grep --quiet "^installed$" &>/dev/null
}
# Return the installed version of an apt package, if installed
_ynh_apt_package_version() {
if _ynh_apt_package_is_installed "$package"; then
dpkg-query --show --showformat='${Version}' "$package" 2>/dev/null
else
echo ''
fi
}
# APT wrapper for non-interactive operation
_ynh_apt() {
_ynh_wait_dpkg_free
LC_ALL=C DEBIAN_FRONTEND=noninteractive apt-get --assume-yes --quiet -o=Acquire::Retries=3 -o=Dpkg::Use-Pty=0 $@
}
# Wrapper around "apt install" with the appropriate options
_ynh_apt_install() {
_ynh_apt --no-remove --option Dpkg::Options::=--force-confdef \
--option Dpkg::Options::=--force-confold install $@
}

277
helpers/helpers.v2.1.d/backup Normal file
View file

@ -0,0 +1,277 @@
#!/bin/bash
CAN_BIND=${CAN_BIND:-1}
# Add a file or a directory to the list of paths to backup
#
# usage: ynh_backup /path/to/stuff
#
# NB : note that this helper does *NOT* perform any copy in itself, it only
# declares stuff to be backuped via a CSV which is later picked up by the core
#
# NB 2 : there is a specific behavior for $data_dir (or childs of $data_dir) and
# /var/log/$app which are *NOT* backedup during safety-backup-before-upgrade,
# OR if the setting "do_not_backup_data" is equals 1 for that app
#
# The rationale is that these directories are usually too heavy to be integrated in every backup
# (think for example about Nextcloud with quite a lot of data, or an app with a lot of media files...)
#
# This is coupled to the fact that $data_dir and the log dir won't be (and
# should NOT) be deleted during remove, unless --purge is used. Hence, if the
# upgrade fails and the script is removed prior to restoring the backup, the
# data/logs are not destroyed.
#
ynh_backup() {
local target="$1"
local is_data=false
# If the path starts with /var/log/$app or $data_dir
if ([[ -n "${app:-}" ]] && [[ "$target" == "/var/log/$app*" ]]) || ([[ -n "${data_dir:-}" ]] && [[ "$target" == "$data_dir*" ]])
then
is_data=true
fi
if [[ -n "${app:-}" ]]
then
local do_not_backup_data=$(ynh_app_setting_get --key=do_not_backup_data)
fi
# If backing up core only (used by ynh_backup_before_upgrade),
# don't backup big data items
if [[ "$is_data" == true ]] && ([[ ${do_not_backup_data:-0} -eq 1 ]] || [[ ${BACKUP_CORE_ONLY:-0} -eq 1 ]]); then
if [ $BACKUP_CORE_ONLY -eq 1 ]; then
ynh_print_info "$target will not be saved, because 'BACKUP_CORE_ONLY' is set."
else
ynh_print_info "$target will not be saved, because 'do_not_backup_data' is set."
fi
return 1
fi
# ==============================================================================
# Format correctly source and destination paths
# ==============================================================================
# Be sure the source path is not empty
if [ ! -e "$target" ]; then
ynh_print_warn "File or folder '${target}' to be backed up does not exist"
return 1
fi
# Transform the source path as an absolute path
# If it's a dir remove the ending /
src_path=$(realpath "$target")
# Initialize the dest path with the source path relative to "/".
# eg: src_path=/etc/yunohost -> dest_path=etc/yunohost
dest_path="${src_path#/}"
# Check if dest_path already exists in tmp archive
if [[ -e "${dest_path}" ]]; then
ynh_print_warn "Destination path '${dest_path}' already exist"
return 1
fi
# Add the relative current working directory to the destination path
local rel_dir="${YNH_CWD#$YNH_BACKUP_DIR}"
rel_dir="${rel_dir%/}/"
dest_path="${rel_dir}${dest_path}"
dest_path="${dest_path#/}"
# ==============================================================================
# ==============================================================================
# Write file to backup into backup_list
# ==============================================================================
local src=$(echo "${src_path}" | sed --regexp-extended 's/"/\"\"/g')
local dest=$(echo "${dest_path}" | sed --regexp-extended 's/"/\"\"/g')
echo "\"${src}\",\"${dest}\"" >>"${YNH_BACKUP_CSV}"
# ==============================================================================
# Create the parent dir of the destination path
# It's for retro compatibility, some script consider ynh_backup creates this dir
mkdir --parents $(dirname "$YNH_BACKUP_DIR/${dest_path}")
}
# Return the path in the archive where has been stocked the origin path
#
# [internal]
#
# usage: _get_archive_path ORIGIN_PATH
_get_archive_path() {
# For security reasons we use csv python library to read the CSV
python3 -c "
import sys
import csv
with open(sys.argv[1], 'r') as backup_file:
backup_csv = csv.DictReader(backup_file, fieldnames=['source', 'dest'])
for row in backup_csv:
if row['source']==sys.argv[2].strip('\"'):
print(row['dest'])
sys.exit(0)
raise Exception('Original path for %s not found' % sys.argv[2])
" "${YNH_BACKUP_CSV}" "$1"
return $?
}
# Restore a file or a directory from the backup archive
#
# usage: ynh_restore /path/to/stuff
#
# examples:
# ynh_restore "/etc/nginx/conf.d/$domain.d/$app.conf"
#
# If the file or dir to be restored already exists on the system and is lighter
# than 500 Mo, it is backed up in `/var/cache/yunohost/appconfbackup/`.
# Otherwise, the existing file or dir is removed.
#
# if `apps/$app/etc/nginx/conf.d/$domain.d/$app.conf` exists, restore it into
# `/etc/nginx/conf.d/$domain.d/$app.conf`
# otheriwse, search for a match in the csv (eg: conf/nginx.conf) and restore it into
# `/etc/nginx/conf.d/$domain.d/$app.conf`
ynh_restore() {
target="$1"
local archive_path="$YNH_CWD${target}"
# If the path starts with /var/log/$app or $data_dir
local is_data=false
if ([[ -n "${app:-}" ]] && [[ "$target" == "/var/log/$app*" ]]) || ([[ -n "${data_dir:-}" ]] && [[ "$target" == "$data_dir*" ]])
then
is_data=true
fi
# If archive_path doesn't exist, search for a corresponding path in CSV
if [ ! -d "$archive_path" ] && [ ! -f "$archive_path" ] && [ ! -L "$archive_path" ]; then
if [[ "$is_data" == true ]]
then
ynh_print_info "Skipping $target which doesn't exists in the archive, probably because restoring from a safety-backup-before-upgrade"
# Assume it's not a big deal, we may be restoring a safety-backup-before-upgrade which doesnt contain those
return 0
else
# (get_archive_path will raise an exception if no match found)
archive_path="$YNH_BACKUP_DIR/$(_get_archive_path \"$target\")"
fi
fi
# Move the old directory if it already exists
if [[ -e "${target}" ]]; then
# Check if the file/dir size is less than 500 Mo
if [[ $(du --summarize --bytes ${target} | cut --delimiter="/" --fields=1) -le "500000000" ]]; then
local backup_file="/var/cache/yunohost/appconfbackup/${target}.backup.$(date '+%Y%m%d.%H%M%S')"
mkdir --parents "$(dirname "$backup_file")"
mv "${target}" "$backup_file" # Move the current file or directory
else
ynh_safe_rm "${target}"
fi
fi
# Restore target into target
mkdir --parents $(dirname "$target")
# Do a copy if it's just a mounting point
if mountpoint --quiet $YNH_BACKUP_DIR; then
if [[ -d "${archive_path}" ]]; then
archive_path="${archive_path}/."
mkdir --parents "$target"
fi
cp --archive "$archive_path" "${target}"
# Do a move if YNH_BACKUP_DIR is already a copy
else
mv "$archive_path" "${target}"
fi
_ynh_apply_default_permissions "$target"
}
# Restore all files that were previously backuped in an app backup script
#
# usage: ynh_restore_everything
ynh_restore_everything() {
# Deduce the relative path of $YNH_CWD
local REL_DIR="${YNH_CWD#$YNH_BACKUP_DIR/}"
REL_DIR="${REL_DIR%/}/"
# For each destination path begining by $REL_DIR
cat ${YNH_BACKUP_CSV} | tr --delete $'\r' | grep --only-matching --no-filename --perl-regexp "^\".*\",\"$REL_DIR.*\"$" \
| while read line; do
local ARCHIVE_PATH=$(echo "$line" | grep --only-matching --no-filename --perl-regexp "^\".*\",\"$REL_DIR\K.*(?=\"$)")
ynh_restore "$ARCHIVE_PATH"
done
}
_ynh_file_checksum_exists() {
local file=$1
local checksum_setting_name=checksum_${file//[\/ ]/_} # Replace all '/' and ' ' by '_'
[[ -n "$(ynh_app_setting_get --key=$checksum_setting_name)" ]]
}
# Calculate and store a file checksum into the app settings
#
# usage: ynh_store_file_checksum /path/to/file
ynh_store_file_checksum() {
set +o xtrace # set +x
local file=$1
local checksum_setting_name=checksum_${file//[\/ ]/_} # Replace all '/' and ' ' by '_'
ynh_app_setting_set --key=$checksum_setting_name --value=$(md5sum "$file" | cut --delimiter=' ' --fields=1)
if ynh_in_ci_tests; then
# Using a base64 is in fact more reversible than "replace / and space by _" ... So we can in fact obtain the original file path in an easy reliable way ...
local file_path_base64=$(echo "$file" | base64 -w0)
mkdir -p /var/cache/yunohost/appconfbackup/
cat $file > /var/cache/yunohost/appconfbackup/original_${file_path_base64}
fi
# If backup_file_checksum isn't empty, ynh_backup_if_checksum_is_different has made a backup
if [ -n "${backup_file_checksum-}" ]; then
# Print the diff between the previous file and the new one.
# diff return 1 if the files are different, so the || true
diff --report-identical-files --unified --color=always $backup_file_checksum $file >&2 || true
fi
# Unset the variable, so it wouldn't trig a ynh_store_file_checksum without a ynh_backup_if_checksum_is_different before it.
unset backup_file_checksum
set -o xtrace # set -x
}
# Verify the checksum and backup the file if it's different
#
# usage: ynh_backup_if_checksum_is_different /path/to/file
#
# This helper is primarily meant to allow to easily backup personalised/manually
# modified config files.
ynh_backup_if_checksum_is_different() {
set +o xtrace # set +x
local file=$1
local checksum_setting_name=checksum_${file//[\/ ]/_} # Replace all '/' and ' ' by '_'
local checksum_value=$(ynh_app_setting_get --key=$checksum_setting_name)
# backup_file_checksum isn't declare as local, so it can be reuse by ynh_store_file_checksum
backup_file_checksum=""
if [ -n "$checksum_value" ]; then # Proceed only if a value was stored into the app settings
if [ -e $file ] && ! echo "$checksum_value $file" | md5sum --check --status; then # If the checksum is now different
backup_file_checksum="/var/cache/yunohost/appconfbackup/$file.backup.$(date '+%Y%m%d.%H%M%S')"
mkdir --parents "$(dirname "$backup_file_checksum")"
cp --archive "$file" "$backup_file_checksum" # Backup the current file
ynh_print_warn "File $file has been manually modified since the installation or last upgrade. So it has been duplicated in $backup_file_checksum"
echo "$backup_file_checksum" # Return the name of the backup file
if ynh_in_ci_tests; then
local file_path_base64=$(echo "$file" | base64 -w0)
if test -e /var/cache/yunohost/appconfbackup/original_${file_path_base64}
then
ynh_print_warn "Diff with the original file:"
diff --report-identical-files --unified --color=always /var/cache/yunohost/appconfbackup/original_${file_path_base64} $file >&2 || true
fi
fi
fi
fi
set -o xtrace # set -x
}
# Delete a file checksum from the app settings
#
# usage: ynh_delete_file_checksum /path/to/file
ynh_delete_file_checksum() {
local file=$1
local checksum_setting_name=checksum_${file//[\/ ]/_} # Replace all '/' and ' ' by '_'
ynh_app_setting_delete --key=$checksum_setting_name
}

45
helpers/helpers.v2.1.d/composer Normal file
View file

@ -0,0 +1,45 @@
#!/bin/bash
# Install and initialize Composer in the given directory
#
# The installed version is defined by `$composer_version` which should be defined
# as global prior to calling this helper.
#
# Will use `$install_dir` as workdir unless `$composer_workdir` exists (but that shouldnt be necessary)
#
# usage: ynh_composer_install
ynh_composer_install() {
local workdir="${composer_workdir:-$install_dir}"
[[ -n "${composer_version}" ]] || ynh_die "\$composer_version should be defined before calling ynh_composer_install. (In the past, this was called \$YNH_COMPOSER_VERSION)"
[[ ! -e "$workdir/composer.phar" ]] || ynh_safe_rm $workdir/composer.phar
local composer_url="https://getcomposer.org/download/$composer_version/composer.phar"
# NB. we have to declare the var as local first,
# otherwise 'local foo=$(false) || echo 'pwet'" does'nt work
# because local always return 0 ...
local out
# Timeout option is here to enforce the timeout on dns query and tcp connect (c.f. man wget)
out=$(wget --tries 3 --no-dns-cache --timeout 900 --no-verbose --output-document=$workdir/composer.phar $composer_url 2>&1) \
|| ynh_die "$out"
}
# Execute a command with Composer
#
# Will use `$install_dir` as workdir unless `$composer_workdir` exists (but that shouldnt be necessary)
#
# You may also define `composer_user=root` prior to call this helper if you
# absolutely need composer to run as root, but this is discouraged...
#
# usage: ynh_composer_exec commands
ynh_composer_exec() {
local workdir="${composer_workdir:-$install_dir}"
COMPOSER_HOME="$workdir/.composer" \
COMPOSER_MEMORY_LIMIT=-1 \
sudo -E -u "${composer_user:-$app}" \
php${php_version} "$workdir/composer.phar" $@ \
-d "$workdir" --no-interaction --no-ansi 2>&1
}

363
helpers/helpers.v2.1.d/config Normal file
View file

@ -0,0 +1,363 @@
#!/bin/bash
_ynh_app_config_get_one() {
local short_setting="$1"
local type="$2"
local bind="$3"
local getter="get__${short_setting}"
# Get value from getter if exists
if type -t $getter 2>/dev/null | grep -q '^function$' 2>/dev/null; then
old[$short_setting]="$($getter)"
formats[${short_setting}]="yaml"
elif [[ "$bind" == *"("* ]] && type -t "get__${bind%%(*}" 2>/dev/null | grep -q '^function$' 2>/dev/null; then
old[$short_setting]="$("get__${bind%%(*}" $short_setting $type $bind)"
formats[${short_setting}]="yaml"
elif [[ "$bind" == "null" ]]; then
old[$short_setting]="YNH_NULL"
# Get value from app settings or from another file
elif [[ "$type" == "file" ]]; then
if [[ "$bind" == "settings" ]]; then
ynh_die "File '${short_setting}' can't be stored in settings"
fi
old[$short_setting]="$(ls "$(echo $bind | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/)" 2>/dev/null || echo YNH_NULL)"
file_hash[$short_setting]="true"
# Get multiline text from settings or from a full file
elif [[ "$type" == "text" ]]; then
if [[ "$bind" == "settings" ]]; then
old[$short_setting]="$(ynh_app_setting_get $app $short_setting)"
elif [[ "$bind" == *":"* ]]; then
ynh_die "For technical reasons, multiline text '${short_setting}' can't be stored automatically in a variable file, you have to create custom getter/setter"
else
old[$short_setting]="$(cat $(echo $bind | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/) 2>/dev/null || echo YNH_NULL)"
fi
# Get value from a kind of key/value file
else
local bind_after=""
if [[ "$bind" == "settings" ]]; then
bind=":/etc/yunohost/apps/$app/settings.yml"
fi
local bind_key_="$(echo "$bind" | cut -d: -f1)"
bind_key_=${bind_key_:-$short_setting}
if [[ "$bind_key_" == *">"* ]]; then
bind_after="$(echo "${bind_key_}" | cut -d'>' -f1)"
bind_key_="$(echo "${bind_key_}" | cut -d'>' -f2)"
fi
local bind_file="$(echo "$bind" | cut -d: -f2 | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/)"
old[$short_setting]="$(ynh_read_var_in_file --file="${bind_file}" --key="${bind_key_}" --after="${bind_after}")"
fi
}
_ynh_app_config_apply_one() {
local short_setting="$1"
local setter="set__${short_setting}"
local bind="${binds[$short_setting]}"
local type="${types[$short_setting]}"
if [ "${changed[$short_setting]}" == "true" ]; then
# Apply setter if exists
if type -t $setter 2>/dev/null | grep -q '^function$' 2>/dev/null; then
$setter
elif [[ "$bind" == *"("* ]] && type -t "set__${bind%%(*}" 2>/dev/null | grep -q '^function$' 2>/dev/null; then
"set__${bind%%(*}" $short_setting $type $bind
elif [[ "$bind" == "null" ]]; then
return
# Save in a file
elif [[ "$type" == "file" ]]; then
if [[ "$bind" == "settings" ]]; then
ynh_die "File '${short_setting}' can't be stored in settings"
fi
local bind_file="$(echo "$bind" | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/)"
if [[ "${!short_setting}" == "" ]]; then
ynh_backup_if_checksum_is_different "$bind_file"
ynh_safe_rm "$bind_file"
ynh_delete_file_checksum "$bind_file"
ynh_print_info "File '$bind_file' removed"
else
ynh_backup_if_checksum_is_different "$bind_file"
if [[ "${!short_setting}" != "$bind_file" ]]; then
cp "${!short_setting}" "$bind_file"
fi
if _ynh_file_checksum_exists "$bind_file"
then
ynh_store_file_checksum "$bind_file"
fi
ynh_print_info "File '$bind_file' overwritten with ${!short_setting}"
fi
# Save value in app settings
elif [[ "$bind" == "settings" ]]; then
ynh_app_setting_set --key=$short_setting --value="${!short_setting}"
ynh_print_info "Configuration key '$short_setting' edited in app settings"
# Save multiline text in a file
elif [[ "$type" == "text" ]]; then
if [[ "$bind" == *":"* ]]; then
ynh_die "For technical reasons, multiline text '${short_setting}' can't be stored automatically in a variable file, you have to create custom getter/setter"
fi
local bind_file="$(echo "$bind" | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/)"
ynh_backup_if_checksum_is_different "$bind_file"
echo "${!short_setting}" >"$bind_file"
if _ynh_file_checksum_exists "$bind_file"
then
ynh_store_file_checksum "$bind_file"
fi
ynh_print_info "File '$bind_file' overwritten with the content provided in question '${short_setting}'"
# Set value into a kind of key/value file
else
local bind_after=""
local bind_key_="$(echo "$bind" | cut -d: -f1)"
if [[ "$bind_key_" == *">"* ]]; then
bind_after="$(echo "${bind_key_}" | cut -d'>' -f1)"
bind_key_="$(echo "${bind_key_}" | cut -d'>' -f2)"
fi
bind_key_=${bind_key_:-$short_setting}
local bind_file="$(echo "$bind" | cut -d: -f2 | sed s@__INSTALL_DIR__@${install_dir:-}@ | sed s/__APP__/$app/)"
ynh_backup_if_checksum_is_different "$bind_file"
ynh_write_var_in_file --file="${bind_file}" --key="${bind_key_}" --value="${!short_setting}" --after="${bind_after}"
if _ynh_file_checksum_exists "$bind_file"
then
ynh_store_file_checksum "$bind_file"
fi
# We stored the info in settings in order to be able to upgrade the app
ynh_app_setting_set --key=$short_setting --value="${!short_setting}"
ynh_print_info "Configuration key '$bind_key_' edited into $bind_file"
fi
fi
}
_ynh_app_config_get() {
# From settings
local lines
lines=$(
python3 <<EOL
import toml
from collections import OrderedDict
with open("../config_panel.toml", "r") as f:
file_content = f.read()
loaded_toml = toml.loads(file_content, _dict=OrderedDict)
for panel_name, panel in loaded_toml.items():
if not isinstance(panel, dict): continue
bind_panel = panel.get('bind')
for section_name, section in panel.items():
if not isinstance(section, dict): continue
bind_section = section.get('bind')
if not bind_section:
bind_section = bind_panel
elif bind_section[-1] == ":" and bind_panel and ":" in bind_panel:
regex, bind_panel_file = bind_panel.split(":")
if ">" in bind_section:
bind_section = bind_section + bind_panel_file
else:
bind_section = regex + bind_section + bind_panel_file
for name, param in section.items():
if not isinstance(param, dict):
continue
bind = param.get('bind')
if not bind:
if bind_section:
bind = bind_section
else:
bind = 'settings'
elif bind[-1] == ":" and bind_section and ":" in bind_section:
regex, bind_file = bind_section.split(":")
if ">" in bind:
bind = bind + bind_file
else:
bind = regex + bind + bind_file
if bind == "settings" and param.get('type', 'string') == 'file':
bind = 'null'
print('|'.join([
name,
param.get('type', 'string'),
bind
]))
EOL
)
for line in $lines; do
# Split line into short_setting, type and bind
IFS='|' read short_setting type bind <<<"$line"
binds[${short_setting}]="$bind"
types[${short_setting}]="$type"
file_hash[${short_setting}]=""
formats[${short_setting}]=""
ynh_app_config_get_one $short_setting $type $bind
done
}
_ynh_app_config_apply() {
for short_setting in "${!old[@]}"; do
ynh_app_config_apply_one $short_setting
done
}
_ynh_app_config_show() {
for short_setting in "${!old[@]}"; do
if [[ "${old[$short_setting]}" != YNH_NULL ]]; then
if [[ "${formats[$short_setting]}" == "yaml" ]]; then
ynh_return "${short_setting}:"
ynh_return "$(echo "${old[$short_setting]}" | sed 's/^/ /g')"
else
ynh_return "${short_setting}: '$(echo "${old[$short_setting]}" | sed "s/'/''/g" | sed ':a;N;$!ba;s/\n/\n\n/g')'"
fi
fi
done
}
_ynh_app_config_validate() {
# Change detection
ynh_script_progression "Checking what changed in the new configuration..."
local nothing_changed=true
local changes_validated=true
for short_setting in "${!old[@]}"; do
changed[$short_setting]=false
if [ -z ${!short_setting+x} ]; then
# Assign the var with the old value in order to allows multiple
# args validation
declare -g "$short_setting"="${old[$short_setting]}"
continue
fi
if [ ! -z "${file_hash[${short_setting}]}" ]; then
file_hash[old__$short_setting]=""
file_hash[new__$short_setting]=""
if [ -f "${old[$short_setting]}" ]; then
file_hash[old__$short_setting]=$(sha256sum "${old[$short_setting]}" | cut -d' ' -f1)
if [ -z "${!short_setting}" ]; then
changed[$short_setting]=true
nothing_changed=false
fi
fi
if [ -f "${!short_setting}" ]; then
file_hash[new__$short_setting]=$(sha256sum "${!short_setting}" | cut -d' ' -f1)
if [[ "${file_hash[old__$short_setting]}" != "${file_hash[new__$short_setting]}" ]]; then
changed[$short_setting]=true
nothing_changed=false
fi
fi
else
if [[ "${!short_setting}" != "${old[$short_setting]}" ]]; then
changed[$short_setting]=true
nothing_changed=false
fi
fi
done
if [[ "$nothing_changed" == "true" ]]; then
ynh_print_info "Nothing has changed"
exit 0
fi
# Run validation if something is changed
ynh_script_progression "Validating the new configuration..."
for short_setting in "${!old[@]}"; do
[[ "${changed[$short_setting]}" == "false" ]] && continue
local result=""
if type -t validate__$short_setting | grep -q '^function$' 2>/dev/null; then
result="$(validate__$short_setting)"
elif [[ "$bind" == *"("* ]] && type -t "validate__${bind%%(*}" 2>/dev/null | grep -q '^function$' 2>/dev/null; then
"validate__${bind%%(*}" $short_setting
fi
if [ -n "$result" ]; then
#
# Return a yaml such as:
#
# validation_errors:
# some_key: "An error message"
# some_other_key: "Another error message"
#
# We use changes_validated to know if this is
# the first validation error
if [[ "$changes_validated" == true ]]; then
ynh_return "validation_errors:"
fi
ynh_return " ${short_setting}: \"$result\""
changes_validated=false
fi
done
# If validation failed, exit the script right now (instead of going into apply)
# Yunohost core will pick up the errors returned via ynh_return previously
if [[ "$changes_validated" == "false" ]]; then
exit 0
fi
}
ynh_app_config_get_one() {
_ynh_app_config_get_one $1 $2 $3
}
ynh_app_config_get() {
_ynh_app_config_get
}
ynh_app_config_show() {
_ynh_app_config_show
}
ynh_app_config_validate() {
_ynh_app_config_validate
}
ynh_app_config_apply_one() {
_ynh_app_config_apply_one $1
}
ynh_app_config_apply() {
_ynh_app_config_apply
}
ynh_app_action_run() {
local runner="run__$1"
# Get value from getter if exists
if type -t "$runner" 2>/dev/null | grep -q '^function$' 2>/dev/null; then
$runner
#ynh_return "result:"
#ynh_return "$(echo "${result}" | sed 's/^/ /g')"
else
ynh_die "No handler defined in app's script for action $1. If you are the maintainer of this app, you should define '$runner'"
fi
}
ynh_app_config_run() {
declare -Ag old=()
declare -Ag changed=()
declare -Ag file_hash=()
declare -Ag binds=()
declare -Ag types=()
declare -Ag formats=()
case $1 in
show)
ynh_app_config_get
ynh_app_config_show
;;
apply)
max_progression=4
ynh_script_progression "Reading config panel description and current configuration..."
ynh_app_config_get
ynh_app_config_validate
ynh_script_progression "Applying the new configuration..."
ynh_app_config_apply
ynh_script_progression "Configuration of $app completed"
;;
*)
ynh_app_action_run $1
esac
}

118
helpers/helpers.v2.1.d/fail2ban Normal file
View file

@ -0,0 +1,118 @@
#!/bin/bash
# Create a dedicated fail2ban config (jail and filter conf files)
#
# usage: ynh_config_add_fail2ban --logpath=log_file --failregex=filter
# | arg: --logpath= - Log file to be checked by fail2ban
# | arg: --failregex= - Failregex to be looked for by fail2ban
#
# If --logpath / --failregex are provided, the helper will generate the appropriate conf using these.
#
# Otherwise, it will assume that the app provided templates, namely
# `../conf/f2b_jail.conf` and `../conf/f2b_filter.conf`
#
# They will typically look like (for example here for synapse):
# ```
# f2b_jail.conf:
# [__APP__]
# enabled = true
# port = http,https
# filter = __APP__
# logpath = /var/log/__APP__/logfile.log
# maxretry = 5
# ```
# ```
# f2b_filter.conf:
# [INCLUDES]
# before = common.conf
# [Definition]
#
# # Part of regex definition (just used to make more easy to make the global regex)
# __synapse_start_line = .? \- synapse\..+ \-
#
# # Regex definition.
# failregex = ^%(__synapse_start_line)s INFO \- POST\-(\d+)\- <HOST> \- \d+ \- Received request\: POST /_matrix/client/r0/login\??<SKIPLINES>%(__synapse_start_line)s INFO \- POST\-\1\- Got login request with identifier: \{u'type': u'm.id.user', u'user'\: u'(.+?)'\}, medium\: None, address: None, user\: u'\5'<SKIPLINES>%(__synapse_start_line)s WARNING \- \- (Attempted to login as @\5\:.+ but they do not exist|Failed password login for user @\5\:.+)$
#
# ignoreregex =
# ```
#
# ##### Regarding the the `failregex` option:
#
# regex to match the password failure messages in the logfile. The host must be
# matched by a group named "`host`". The tag "`<HOST>`" can be used for standard
# IP/hostname matching and is only an alias for `(?:::f{4,6}:)?(?P<host>[\w\-.^_]+)`
#
# You can find some more explainations about how to make a regex here :
# https://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Filters
#
# To validate your regex you can test with this command:
# ```
# fail2ban-regex /var/log/YOUR_LOG_FILE_PATH /etc/fail2ban/filter.d/YOUR_APP.conf
# ```
ynh_config_add_fail2ban() {
# ============ Argument parsing =============
local -A args_array=([l]=logpath= [r]=failregex=)
local logpath
local failregex
ynh_handle_getopts_args "$@"
# ===========================================
# If failregex is provided, Build a config file on-the-fly using $logpath and $failregex
if [[ -n "${failregex:-}" ]]; then
test -n "$logpath" || ynh_die "ynh_config_add_fail2ban expects a logfile path as first argument and received nothing."
echo "
[__APP__]
enabled = true
port = http,https
filter = __APP__
logpath = __LOGPATH__
maxretry = 5
" >"$YNH_APP_BASEDIR/conf/f2b_jail.conf"
echo "
[INCLUDES]
before = common.conf
[Definition]
failregex = __FAILREGEX__
ignoreregex =
" >"$YNH_APP_BASEDIR/conf/f2b_filter.conf"
fi
ynh_config_add --template="f2b_jail.conf" --destination="/etc/fail2ban/jail.d/$app.conf"
ynh_config_add --template="f2b_filter.conf" --destination="/etc/fail2ban/filter.d/$app.conf"
# if "$logpath" doesn't exist (as if using --use_template argument), assign
# "$logpath" using the one in the previously generated fail2ban conf file
if [ -z "${logpath:-}" ]; then
# the first sed deletes possibles spaces and the second one extract the path
logpath=$(grep "^logpath" "/etc/fail2ban/jail.d/$app.conf" | sed "s/ //g" | sed "s/logpath=//g")
fi
# Create the folder and logfile if they doesn't exist,
# as fail2ban require an existing logfile before configuration
mkdir -p "/var/log/$app"
if [ ! -f "$logpath" ]; then
touch "$logpath"
fi
# Make sure log folder's permissions are correct
chown -R "$app:$app" "/var/log/$app"
chmod -R u=rwX,g=rX,o= "/var/log/$app"
ynh_systemctl --service=fail2ban --action=reload --wait_until="(Started|Reloaded) Fail2Ban Service" --log_path=systemd
local fail2ban_error="$(journalctl --no-hostname --unit=fail2ban | tail --lines=50 | grep "WARNING.*$app.*")"
if [[ -n "$fail2ban_error" ]]; then
ynh_print_warn "Fail2ban failed to load the jail for $app"
ynh_print_warn "${fail2ban_error#*WARNING}"
fi
}
# Remove the dedicated fail2ban config (jail and filter conf files)
#
# usage: ynh_config_remove_fail2ban
ynh_config_remove_fail2ban() {
ynh_safe_rm "/etc/fail2ban/jail.d/$app.conf"
ynh_safe_rm "/etc/fail2ban/filter.d/$app.conf"
ynh_systemctl --service=fail2ban --action=reload
}

187
helpers/helpers.v2.1.d/getopts Normal file
View file

@ -0,0 +1,187 @@
#!/bin/bash
# Internal helper design to allow helpers to use getopts to manage their arguments
#
# [internal]
#
# example: function my_helper()
# {
# local -A args_array=( [a]=arg1= [b]=arg2= [c]=arg3 )
# local arg1
# local arg2
# local arg3
# ynh_handle_getopts_args "$@"
#
# [...]
# }
# my_helper --arg1 "val1" -b val2 -c
#
# usage: ynh_handle_getopts_args "$@"
# | arg: $@ - Simply "$@" to tranfert all the positionnal arguments to the function
#
# This helper need an array, named "args_array" with all the arguments used by the helper
# that want to use ynh_handle_getopts_args
# Be carreful, this array has to be an associative array, as the following example:
# local -A args_array=( [a]=arg1 [b]=arg2= [c]=arg3 )
# Let's explain this array:
# a, b and c are short options, -a, -b and -c
# arg1, arg2 and arg3 are the long options associated to the previous short ones. --arg1, --arg2 and --arg3
# For each option, a short and long version has to be defined.
# Let's see something more significant
# local -A args_array=( [u]=user [f]=finalpath= [d]=database )
#
# NB: Because we're using 'declare' without -g, the array will be declared as a local variable.
#
# Please keep in mind that the long option will be used as a variable to store the values for this option.
# For the previous example, that means that $finalpath will be fill with the value given as argument for this option.
#
# Also, in the previous example, finalpath has a '=' at the end. That means this option need a value.
# So, the helper has to be call with --finalpath /final/path, --finalpath=/final/path or -f /final/path, the variable $finalpath will get the value /final/path
# If there's many values for an option, -f /final /path, the value will be separated by a ';' $finalpath=/final;/path
# For an option without value, like --user in the example, the helper can be called only with --user or -u. $user will then get the value 1.
#
ynh_handle_getopts_args() {
# Trick to only re-enable debugging if it was set before
local xtrace_enable=$(set +o | grep xtrace)
# Manage arguments only if there's some provided
set +o xtrace # set +x
if [ $# -eq 0 ]; then
eval "$xtrace_enable"
return
# Validate that the first char is - because it should be something like --option=value or -o ...
elif [[ "${1:0:1}" != "-" ]]
then
ynh_die "It looks like you called the helper using positional arguments instead of keyword arguments ?"
fi
# Store arguments in an array to keep each argument separated
local arguments=("$@")
# For each option in the array, reduce to short options for getopts (e.g. for [u]=user, --user will be -u)
# And built parameters string for getopts
# ${!args_array[@]} is the list of all option_flags in the array (An option_flag is 'u' in [u]=user, user is a value)
local getopts_parameters=""
local option_flag=""
for option_flag in "${!args_array[@]}"; do
# Concatenate each option_flags of the array to build the string of arguments for getopts
# Will looks like 'abcd' for -a -b -c -d
# If the value of an option_flag finish by =, it's an option with additionnal values. (e.g. --user bob or -u bob)
# Check the last character of the value associate to the option_flag
if [ "${args_array[$option_flag]: -1}" = "=" ]; then
# For an option with additionnal values, add a ':' after the letter for getopts.
getopts_parameters="${getopts_parameters}${option_flag}:"
else
getopts_parameters="${getopts_parameters}${option_flag}"
fi
# Check each argument given to the function
local arg=""
# ${#arguments[@]} is the size of the array
for arg in $(seq 0 $((${#arguments[@]} - 1))); do
# Escape options' values starting with -. Otherwise the - will be considered as another option.
arguments[arg]="${arguments[arg]//--${args_array[$option_flag]}-/--${args_array[$option_flag]}\\TOBEREMOVED\\-}"
# And replace long option (value of the option_flag) by the short option, the option_flag itself
# (e.g. for [u]=user, --user will be -u)
# Replace long option with = (match the beginning of the argument)
arguments[arg]="$(printf '%s\n' "${arguments[arg]}" | sed "s/^--${args_array[$option_flag]}/-${option_flag} /")"
# And long option without = (match the whole line)
arguments[arg]="$(printf '%s\n' "${arguments[arg]}" | sed "s/^--${args_array[$option_flag]%=}$/-${option_flag} /")"
done
done
# Read and parse all the arguments
# Use a function here, to use standart arguments $@ and be able to use shift.
parse_arg() {
# Read all arguments, until no arguments are left
while [ $# -ne 0 ]; do
# Initialize the index of getopts
OPTIND=1
# Parse with getopts only if the argument begin by -, that means the argument is an option
# getopts will fill $parameter with the letter of the option it has read.
local parameter=""
getopts ":$getopts_parameters" parameter || true
if [ "$parameter" = "?" ]; then
ynh_die "Invalid argument: ${1:-}"
elif [ "$parameter" = ":" ]; then
ynh_die "${1:-} parameter requires an argument."
else
local shift_value=1
# Use the long option, corresponding to the short option read by getopts, as a variable
# (e.g. for [u]=user, 'user' will be used as a variable)
# Also, remove '=' at the end of the long option
# The variable name will be stored in 'option_var'
local option_var="${args_array[$parameter]%=}"
# If this option doesn't take values
# if there's a '=' at the end of the long option name, this option takes values
if [ "${args_array[$parameter]: -1}" != "=" ]; then
# 'eval ${option_var}' will use the content of 'option_var'
eval ${option_var}=1
else
# Read all other arguments to find multiple value for this option.
# Load args in a array
local all_args=("$@")
# If the first argument is longer than 2 characters,
# There's a value attached to the option, in the same array cell
if [ ${#all_args[0]} -gt 2 ]; then
# Remove the option and the space, so keep only the value itself.
all_args[0]="${all_args[0]#-${parameter} }"
# At this point, if all_args[0] start with "-", then the argument is not well formed
if [ "${all_args[0]:0:1}" == "-" ]; then
ynh_die "Argument \"${all_args[0]}\" not valid! Did you use a single \"-\" instead of two?"
fi
# Reduce the value of shift, because the option has been removed manually
shift_value=$((shift_value - 1))
fi
# Declare the content of option_var as a variable.
eval ${option_var}=""
# Then read the array value per value
local i
for i in $(seq 0 $((${#all_args[@]} - 1))); do
# If this argument is an option, end here.
if [ "${all_args[$i]:0:1}" == "-" ]; then
# Ignore the first value of the array, which is the option itself
if [ "$i" -ne 0 ]; then
break
fi
else
# Ignore empty parameters
if [ -n "${all_args[$i]}" ]; then
# Else, add this value to this option
# Each value will be separated by ';'
if [ -n "${!option_var}" ]; then
# If there's already another value for this option, add a ; before adding the new value
eval ${option_var}+="\;"
fi
# Remove the \ that escape - at beginning of values.
all_args[i]="${all_args[i]//\\TOBEREMOVED\\/}"
# For the record.
# We're using eval here to get the content of the variable stored itself as simple text in $option_var...
# Other ways to get that content would be to use either ${!option_var} or declare -g ${option_var}
# But... ${!option_var} can't be used as left part of an assignation.
# declare -g ${option_var} will create a local variable (despite -g !) and will not be available for the helper itself.
# So... Stop fucking arguing each time that eval is evil... Go find an other working solution if you can find one!
eval ${option_var}+='"${all_args[$i]}"'
fi
shift_value=$((shift_value + 1))
fi
done
fi
fi
# Shift the parameter and its argument(s)
shift $shift_value
done
}
# Call parse_arg and pass the modified list of args as an array of arguments.
parse_arg "${arguments[@]}"
eval "$xtrace_enable"
}

195
helpers/helpers.v2.1.d/go Normal file
View file

@ -0,0 +1,195 @@
#!/bin/bash
readonly GOENV_INSTALL_DIR="/opt/goenv"
# goenv_ROOT is the directory of goenv, it needs to be loaded as a environment variable.
export GOENV_ROOT="$GOENV_INSTALL_DIR"
_ynh_load_go_in_path_and_other_tweaks() {
# Get the absolute path of this version of go
go_dir="$GOENV_INSTALL_DIR/versions/$app/bin"
# Load the path of this version of go in $PATH
if [[ :$PATH: != *":$go_dir"* ]]; then
PATH="$go_dir:$PATH"
fi
# Export PATH such that it's available through sudo -E / ynh_exec_as $app
export PATH
# This is in full lowercase such that it gets replaced in templates
path_with_go="$PATH"
PATH_with_go="$PATH"
# Sets the local application-specific go version
pushd ${install_dir}
$GOENV_INSTALL_DIR/bin/goenv local $go_version
popd
}
# Install a specific version of Go using goenv
#
# The installed version is defined by `$go_version` which should be defined as global prior to calling this helper
#
# usage: ynh_go_install
#
# The helper adds the appropriate, specific version of go to the `$PATH` variable (which
# is preserved when calling `ynh_exec_as_app`). Also defines:
# - `$path_with_go` (the value of the modified `$PATH`, but you dont really need it?)
# - `$go_dir` (the directory containing the specific go version)
#
# This helper also creates a /etc/profile.d/goenv.sh that configures PATH environment for goenv
ynh_go_install () {
[[ -n "${go_version:-}" ]] || ynh_die "\$go_version should be defined prior to calling ynh_go_install"
# Load goenv path in PATH
local CLEAR_PATH="$GOENV_INSTALL_DIR/bin:$PATH"
# Remove /usr/local/bin in PATH in case of Go prior installation
PATH=$(echo $CLEAR_PATH | sed 's@/usr/local/bin:@@')
# Move an existing Go binary, to avoid to block goenv
test -x /usr/bin/go && mv /usr/bin/go /usr/bin/go_goenv
# Install or update goenv
mkdir -p $GOENV_INSTALL_DIR
pushd "$GOENV_INSTALL_DIR"
if ! [ -x "$GOENV_INSTALL_DIR/bin/goenv" ]; then
ynh_print_info "Downloading goenv..."
git init -q
git remote add origin https://github.com/syndbg/goenv.git
else
ynh_print_info "Updating goenv..."
fi
git fetch -q --tags --prune origin
local git_latest_tag=$(git describe --tags "$(git rev-list --tags --max-count=1)")
git checkout -q "$git_latest_tag"
_ynh_go_try_bash_extension
goenv=$GOENV_INSTALL_DIR/bin/goenv
popd
# Install or update xxenv-latest
mkdir -p "$GOENV_INSTALL_DIR/plugins/xxenv-latest"
pushd "$GOENV_INSTALL_DIR/plugins/xxenv-latest"
if ! [ -x "$GOENV_INSTALL_DIR/plugins/xxenv-latest/bin/goenv-latest" ]; then
ynh_print_info "Downloading xxenv-latest..."
git init -q
git remote add origin https://github.com/momo-lab/xxenv-latest.git
else
ynh_print_info "Updating xxenv-latest..."
fi
git fetch -q --tags --prune origin
local git_latest_tag=$(git describe --tags "$(git rev-list --tags --max-count=1)")
git checkout -q "$git_latest_tag"
popd
# Enable caching
mkdir -p "${GOENV_INSTALL_DIR}/cache"
# Create shims directory if needed
mkdir -p "${GOENV_INSTALL_DIR}/shims"
# Restore /usr/local/bin in PATH
PATH=$CLEAR_PATH
# And replace the old Go binary
test -x /usr/bin/go_goenv && mv /usr/bin/go_goenv /usr/bin/go
# Install the requested version of Go
local final_go_version=$("$GOENV_INSTALL_DIR/plugins/xxenv-latest/bin/goenv-latest" --print "$go_version")
ynh_print_info "Installation of Go-$final_go_version"
goenv install --quiet --skip-existing "$final_go_version" 2>&1
# Store go_version into the config of this app
ynh_app_setting_set --app="$app" --key="go_version" --value="$final_go_version"
go_version=$final_go_version
# Cleanup Go versions
_ynh_go_cleanup
# Set environment for Go users
echo "#goenv
export GOENV_ROOT=$GOENV_INSTALL_DIR
export PATH=\"$GOENV_INSTALL_DIR/bin:$PATH\"
eval \"\$(goenv init -)\"
#goenv" > /etc/profile.d/goenv.sh
# Load the environment
eval "$(goenv init -)"
_ynh_load_go_in_path_and_other_tweaks
}
# Remove the version of Go used by the app.
#
# This helper will also cleanup Go versions
#
# usage: ynh_go_remove
ynh_go_remove () {
local go_version=$(ynh_app_setting_get --key="go_version")
# Load goenv path in PATH
local CLEAR_PATH="$GOENV_INSTALL_DIR/bin:$PATH"
# Remove /usr/local/bin in PATH in case of Go prior installation
PATH=$(echo $CLEAR_PATH | sed 's@/usr/local/bin:@@')
# Remove the line for this app
ynh_app_setting_delete --key="go_version"
# Cleanup Go versions
_ynh_go_cleanup
}
# Remove no more needed versions of Go used by the app.
#
# [internal]
#
# This helper will check what Go version are no more required,
# and uninstall them
# If no app uses Go, goenv will be also removed.
#
# usage: _ynh_go_cleanup
_ynh_go_cleanup () {
# List required Go versions
local installed_apps=$(yunohost app list --output-as json --quiet | jq -r .apps[].id)
local required_go_versions=""
for installed_app in $installed_apps
do
local installed_app_go_version=$(ynh_app_setting_get --app=$installed_app --key="go_version")
if [[ $installed_app_go_version ]]
then
required_go_versions="${installed_app_go_version}\n${required_go_versions}"
fi
done
# Remove no more needed Go versions
local installed_go_versions=$(goenv versions --bare --skip-aliases | grep -Ev '/')
for installed_go_version in $installed_go_versions
do
if ! `echo ${required_go_versions} | grep "${installed_go_version}" 1>/dev/null 2>&1`
then
ynh_print_info "Removing of Go-$installed_go_version"
$GOENV_INSTALL_DIR/bin/goenv uninstall --force "$installed_go_version"
fi
done
# If none Go version is required
if [[ ! $required_go_versions ]]
then
# Remove goenv environment configuration
ynh_print_info "Removing of goenv"
ynh_safe_rm "$GOENV_INSTALL_DIR"
ynh_safe_rm "/etc/profile.d/goenv.sh"
fi
}
_ynh_go_try_bash_extension() {
if [ -x src/configure ]; then
src/configure && make -C src || {
ynh_print_info "Optional bash extension failed to build, but things will still work normally."
}
fi
}

112
helpers/helpers.v2.1.d/logging Normal file
View file

@ -0,0 +1,112 @@
#!/bin/bash
# Print a message to stderr and terminate the current script
#
# usage: ynh_die "Some message"
ynh_die() {
echo "$1" 1>&2
exit 1
}
# Print an "INFO" message
#
# usage: ynh_print_info "Some message"
ynh_print_info() {
echo "$1" >&$YNH_STDINFO
}
# Print a warning on stderr
#
# usage: ynh_print_warn "Some message"
ynh_print_warn() {
echo "$1" >&2
}
# Execute a command and redirect stderr to stdout
#
# usage: ynh_hide_warnings your command and args
# | arg: command - command to execute
#
ynh_hide_warnings() {
# Note that "$@" is used and not $@, c.f. https://unix.stackexchange.com/a/129077
"$@" 2>&1
}
# Execute a command and redirect stderr in /dev/null. Print stderr on error.
#
# usage: ynh_exec_and_print_stderr_only_if_error your command and args
# | arg: command - command to execute
#
# Note that you should NOT quote the command but only prefix it with ynh_exec_and_print_stderr_only_if_error
ynh_exec_and_print_stderr_only_if_error() {
logfile="$(mktemp)"
rc=0
# Note that "$@" is used and not $@, c.f. https://unix.stackexchange.com/a/129077
"$@" 2> "$logfile" || rc="$?"
if (( rc != 0 )); then
cat "$logfile" >&2
ynh_safe_rm "$logfile"
return "$rc"
fi
}
# Return data to the YunoHost core for later processing
# (to be used by special hooks like app config panel and core diagnosis)
#
# usage: ynh_return somedata
ynh_return() {
echo "$1" >>"$YNH_STDRETURN"
}
# Initial definitions for ynh_script_progression
increment_progression=0
previous_weight=0
max_progression=-1
# Set the scale of the progression bar
# progress_string(0,1,2) should have the size of the scale.
progress_scale=20
progress_string2="####################"
progress_string1="++++++++++++++++++++"
progress_string0="...................."
# Print a progress bar showing the progression of an app script
#
# usage: ynh_script_progression "Some message"
ynh_script_progression() {
set +o xtrace # set +x
# Compute $max_progression (if we didn't already)
if [ "$max_progression" = -1 ]; then
# Get the number of occurrences of 'ynh_script_progression' in the script. Except those are commented.
local helper_calls=
max_progression="$(grep --count "^[^#]*ynh_script_progression" $0)"
fi
# Increment each execution of ynh_script_progression in this script by the weight of the previous call.
increment_progression=$(($increment_progression + $previous_weight))
# Store the weight of the current call in $previous_weight for next call
previous_weight=1
# Reduce $increment_progression to the size of the scale
local effective_progression=$(($increment_progression * $progress_scale / $max_progression))
# If last is specified, fill immediately the progression_bar
# Build $progression_bar from progress_string(0,1,2) according to $effective_progression and the weight of the current task
# expected_progression is the progression expected after the current task
local expected_progression="$((($increment_progression + 1) * $progress_scale / $max_progression - $effective_progression))"
# Hack for the "--last" message
if grep -qw 'completed' <<< "$1";
then
effective_progression=$progress_scale
expected_progression=0
fi
# left_progression is the progression not yet done
local left_progression="$(($progress_scale - $effective_progression - $expected_progression))"
# Build the progression bar with $effective_progression, work done, $expected_progression, current work and $left_progression, work to be done.
local progression_bar="${progress_string2:0:$effective_progression}${progress_string1:0:$expected_progression}${progress_string0:0:$left_progression}"
echo "[$progression_bar] > ${1}" >&$YNH_STDINFO
set -o xtrace # set -x
}

75
helpers/helpers.v2.1.d/logrotate Normal file
View file

@ -0,0 +1,75 @@
#!/bin/bash
FIRST_CALL_TO_LOGROTATE="true"
# Add a logrotate configuration to manage log files / log directory
#
# usage: ynh_config_add_logrotate [/path/to/log/file/or/folder]
#
# If not argument is provided, `/var/log/$app/*.log` is used as default.
#
# The configuration is autogenerated by YunoHost
# (ie it doesnt come from a specific app template like nginx or systemd conf)
ynh_config_add_logrotate() {
local logfile="${1:-}"
set -o noglob
if [[ -z "$logfile" ]]; then
logfile="/var/log/${app}/*.log"
elif [[ "${logfile##*.}" != "log" ]] && [[ "${logfile##*.}" != "txt" ]]; then
logfile="$logfile/*.log"
fi
set +o noglob
for stuff in $logfile
do
# Make sure the permissions of the parent dir are correct (otherwise the config file could be ignored and the corresponding logs never rotated)
local dir=$(dirname "$stuff")
mkdir --parents $dir
chmod 750 $dir
chown $app:$app $dir
done
local tempconf="$(mktemp)"
cat << EOF >$tempconf
$logfile {
# Rotate if the logfile exceeds 100Mo
size 100M
# Keep 12 old log maximum
rotate 12
# Compress the logs with gzip
compress
# Compress the log at the next cycle. So keep always 2 non compressed logs
delaycompress
# Copy and truncate the log to allow to continue write on it. Instead of moving the log.
copytruncate
# Do not trigger an error if the log is missing
missingok
# Do not rotate if the log is empty
notifempty
# Keep old logs in the same dir
noolddir
}
EOF
if [[ "$FIRST_CALL_TO_LOGROTATE" == "true" ]]
then
cat $tempconf > /etc/logrotate.d/$app
else
cat $tempconf >> /etc/logrotate.d/$app
fi
FIRST_CALL_TO_LOGROTATE="false"
chmod 644 "/etc/logrotate.d/$app"
}
# Remove the app's logrotate config.
#
# usage: ynh_remove_logrotate
ynh_config_remove_logrotate() {
if [ -e "/etc/logrotate.d/$app" ]; then
rm "/etc/logrotate.d/$app"
fi
}

274
helpers/helpers.v2.1.d/mongodb Normal file
View file

@ -0,0 +1,274 @@
#!/bin/bash
# Execute a mongo command
#
# example: ynh_mongo_exec --command='db.getMongo().getDBNames().indexOf("wekan")'
# example: ynh_mongo_exec --command="db.getMongo().getDBNames().indexOf(\"wekan\")"
#
# usage: ynh_mongo_exec [--database=database] --command="command"
# | arg: --database= - The database to connect to
# | arg: --command= - The command to evaluate
#
#
ynh_mongo_exec() {
# ============ Argument parsing =============
local -A args_array=( [d]=database= [c]=command= )
local database
local command
ynh_handle_getopts_args "$@"
database="${database:-}"
# ===========================================
if [ -n "$database" ]
then
mongosh --quiet <<EOF
use $database
${command}
quit()
EOF
else
mongosh --quiet --eval="$command"
fi
}
# Drop a database
#
# [internal]
#
# If you intend to drop the database *and* the associated user,
# consider using ynh_mongo_remove_db instead.
#
# usage: ynh_mongo_drop_db --database=database
# | arg: --database= - The database name to drop
#
#
ynh_mongo_drop_db() {
# ============ Argument parsing =============
local -A args_array=( [d]=database= )
local database
ynh_handle_getopts_args "$@"
# ===========================================
ynh_mongo_exec --database="$database" --command='db.runCommand({dropDatabase: 1})'
}
# Dump a database
#
# example: ynh_mongo_dump_db --database=wekan > ./dump.bson
#
# usage: ynh_mongo_dump_db --database=database
# | arg: --database= - The database name to dump
# | ret: the mongodump output
#
#
ynh_mongo_dump_db() {
# ============ Argument parsing =============
local -A args_array=( [d]=database= )
local database
ynh_handle_getopts_args "$@"
# ===========================================
mongodump --quiet --db="$database" --archive
}
# Create a user
#
# [internal]
#
# usage: ynh_mongo_create_user --db_user=user --db_pwd=pwd --db_name=name
# | arg: --db_user= - The user name to create
# | arg: --db_pwd= - The password to identify user by
# | arg: --db_name= - Name of the database to grant privilegies
#
#
ynh_mongo_create_user() {
# ============ Argument parsing =============
local -A args_array=( [u]=db_user= [n]=db_name= [p]=db_pwd= )
local db_user
local db_name
local db_pwd
ynh_handle_getopts_args "$@"
# ===========================================
# Create the user and set the user as admin of the db
ynh_mongo_exec --database="$db_name" --command='db.createUser( { user: "'${db_user}'", pwd: "'${db_pwd}'", roles: [ { role: "readWrite", db: "'${db_name}'" } ] } );'
# Add clustermonitoring rights
ynh_mongo_exec --database="$db_name" --command='db.grantRolesToUser("'${db_user}'",[{ role: "clusterMonitor", db: "admin" }]);'
}
# Check if a mongo database exists
#
# usage: ynh_mongo_database_exists --database=database
# | arg: --database= - The database for which to check existence
# | exit: Return 1 if the database doesn't exist, 0 otherwise
#
#
ynh_mongo_database_exists() {
# ============ Argument parsing =============
local -A args_array=([d]=database=)
local database
ynh_handle_getopts_args "$@"
# ===========================================
if [ $(ynh_mongo_exec --command='db.getMongo().getDBNames().indexOf("'${database}'")') -lt 0 ]
then
return 1
else
return 0
fi
}
# Restore a database
#
# example: ynh_mongo_restore_db --database=wekan < ./dump.bson
#
# usage: ynh_mongo_restore_db --database=database
# | arg: --database= - The database name to restore
#
#
ynh_mongo_restore_db() {
# ============ Argument parsing =============
local -A args_array=( [d]=database= )
local database
ynh_handle_getopts_args "$@"
# ===========================================
mongorestore --quiet --db="$database" --archive
}
# Drop a user
#
# [internal]
#
# usage: ynh_mongo_drop_user --db_user=user --db_name=name
# | arg: --db_user= - The user to drop
# | arg: --db_name= - Name of the database
#
#
ynh_mongo_drop_user() {
# ============ Argument parsing =============
local -A args_array=( [u]=db_user= [n]=db_name= )
local db_user
local db_name
ynh_handle_getopts_args "$@"
# ===========================================
ynh_mongo_exec --database="$db_name" --command='db.dropUser("'$db_user'", {w: "majority", wtimeout: 5000})'
}
# Create a database, an user and its password. Then store the password in the app's config
#
# usage: ynh_mongo_setup_db --db_user=user --db_name=name [--db_pwd=pwd]
# | arg: --db_user= - Owner of the database
# | arg: --db_name= - Name of the database
# | arg: --db_pwd= - Password of the database. If not provided, a password will be generated
#
# After executing this helper, the password of the created database will be available in $db_pwd
# It will also be stored as "mongopwd" into the app settings.
#
#
ynh_mongo_setup_db() {
# ============ Argument parsing =============
local -A args_array=( [u]=db_user= [n]=db_name= [p]=db_pwd= )
local db_user
local db_name
db_pwd=""
ynh_handle_getopts_args "$@"
# ===========================================
local new_db_pwd=$(ynh_string_random) # Generate a random password
# If $db_pwd is not provided, use new_db_pwd instead for db_pwd
db_pwd="${db_pwd:-$new_db_pwd}"
# Create the user and grant access to the database
ynh_mongo_create_user --db_user="$db_user" --db_pwd="$db_pwd" --db_name="$db_name"
# Store the password in the app's config
ynh_app_setting_set --key=db_pwd --value=$db_pwd
}
# Remove a database if it exists, and the associated user
#
# usage: ynh_mongo_remove_db --db_user=user --db_name=name
# | arg: --db_user= - Owner of the database
# | arg: --db_name= - Name of the database
#
#
ynh_mongo_remove_db() {
# ============ Argument parsing =============
local -A args_array=( [u]=db_user= [n]=db_name= )
local db_user
local db_name
ynh_handle_getopts_args "$@"
# ===========================================
if ynh_mongo_database_exists --database=$db_name; then # Check if the database exists
ynh_mongo_drop_db --database=$db_name # Remove the database
else
ynh_print_warn "Database $db_name not found"
fi
# Remove mongo user if it exists
ynh_mongo_drop_user --db_user=$db_user --db_name=$db_name
}
# Install MongoDB and integrate MongoDB service in YunoHost
#
# The installed version is defined by $mongo_version which should be defined as global prior to calling this helper
#
# usage: ynh_install_mongo
#
ynh_install_mongo() {
[[ -n "${mongo_version:-}" ]] || ynh_die "\$mongo_version should be defined prior to calling ynh_install_mongo"
ynh_print_info "Installing MongoDB Community Edition ..."
local mongo_debian_release=$YNH_DEBIAN_VERSION
if [[ "$(grep '^flags' /proc/cpuinfo | uniq)" != *"avx"* && "$mongo_version" != "4.4" ]]; then
ynh_print_warn "Installing Mongo 4.4 as $mongo_version is not compatible with your cpu (see https://docs.mongodb.com/manual/administration/production-notes/#x86_64)."
mongo_version="4.4"
fi
if [[ "$mongo_version" == "4.4" ]]; then
ynh_print_warn "Switched to buster install as Mongo 4.4 is not compatible with $mongo_debian_release."
mongo_debian_release=buster
fi
ynh_apt_install_dependencies_from_extra_repository \
--repo="deb http://repo.mongodb.org/apt/debian $mongo_debian_release/mongodb-org/$mongo_version main" \
--package="mongodb-org mongodb-org-server mongodb-org-tools mongodb-mongosh" \
--key="https://www.mongodb.org/static/pgp/server-$mongo_version.asc"
mongodb_servicename=mongod
# Make sure MongoDB is started and enabled
systemctl enable $mongodb_servicename --quiet
systemctl daemon-reload --quiet
ynh_systemctl --service=$mongodb_servicename --action=restart --wait_until="aiting for connections" --log_path="/var/log/mongodb/$mongodb_servicename.log"
# Integrate MongoDB service in YunoHost
yunohost service add $mongodb_servicename --description="MongoDB daemon" --log="/var/log/mongodb/$mongodb_servicename.log"
# Store mongo_version into the config of this app
ynh_app_setting_set --key=mongo_version --value=$mongo_version
}
# Remove MongoDB
# Only remove the MongoDB service integration in YunoHost for now
# if MongoDB package as been removed
#
# usage: ynh_remove_mongo
#
#
ynh_remove_mongo() {
# Only remove the mongodb service if it is not installed.
if ! _ynh_apt_package_is_installed "mongodb*"
then
ynh_print_info "Removing MongoDB service..."
mongodb_servicename=mongod
# Remove the mongodb service
yunohost service remove $mongodb_servicename
ynh_safe_rm "/var/lib/mongodb"
ynh_safe_rm "/var/log/mongodb"
fi
}

96
helpers/helpers.v2.1.d/multimedia Normal file
View file

@ -0,0 +1,96 @@
#!/bin/bash
readonly MEDIA_GROUP=multimedia
readonly MEDIA_DIRECTORY=/home/yunohost.multimedia
# Initialize the multimedia directory system
#
# usage: ynh_multimedia_build_main_dir
ynh_multimedia_build_main_dir() {
## Création du groupe multimedia
groupadd -f $MEDIA_GROUP
## Création des dossiers génériques
mkdir -p "$MEDIA_DIRECTORY"
mkdir -p "$MEDIA_DIRECTORY/share"
mkdir -p "$MEDIA_DIRECTORY/share/Music"
mkdir -p "$MEDIA_DIRECTORY/share/Picture"
mkdir -p "$MEDIA_DIRECTORY/share/Video"
mkdir -p "$MEDIA_DIRECTORY/share/eBook"
## Création des dossiers utilisateurs
for user in $(yunohost user list --output-as json | jq -r '.users | keys[]'); do
mkdir -p "$MEDIA_DIRECTORY/$user"
mkdir -p "$MEDIA_DIRECTORY/$user/Music"
mkdir -p "$MEDIA_DIRECTORY/$user/Picture"
mkdir -p "$MEDIA_DIRECTORY/$user/Video"
mkdir -p "$MEDIA_DIRECTORY/$user/eBook"
ln -sfn "$MEDIA_DIRECTORY/share" "$MEDIA_DIRECTORY/$user/Share"
# Création du lien symbolique dans le home de l'utilisateur.
#link will only be created if the home directory of the user exists and if it's located in '/home' folder
local user_home="$(getent passwd $user | cut -d: -f6 | grep '^/home/')"
if [[ -d "$user_home" ]]; then
ln -sfn "$MEDIA_DIRECTORY/$user" "$user_home/Multimedia"
fi
# Propriétaires des dossiers utilisateurs.
chown -R $user "$MEDIA_DIRECTORY/$user"
done
# Default yunohost hooks for post_user_create,delete will take care
# of creating/deleting corresponding multimedia folders when users
# are created/deleted in the future...
## Application des droits étendus sur le dossier multimedia.
# Droit d'écriture pour le groupe et le groupe multimedia en acl et droit de lecture pour other:
setfacl -RnL -m g:$MEDIA_GROUP:rwX,g::rwX,o:r-X "$MEDIA_DIRECTORY" || true
# Application de la même règle que précédemment, mais par défaut pour les nouveaux fichiers.
setfacl -RnL -m d:g:$MEDIA_GROUP:rwX,g::rwX,o:r-X "$MEDIA_DIRECTORY" || true
# Réglage du masque par défaut. Qui garantie (en principe...) un droit maximal à rwx. Donc pas de restriction de droits par l'acl.
setfacl -RL -m m::rwx "$MEDIA_DIRECTORY" || true
}
# Add a directory in yunohost.multimedia
#
# usage: ynh_multimedia_addfolder --source_dir="source_dir" --dest_dir="dest_dir"
#
# | arg: --source_dir= - Source directory - The real directory which contains your medias.
# | arg: --dest_dir= - Destination directory - The name and the place of the symbolic link, relative to "/home/yunohost.multimedia"
#
# This "directory" will be a symbolic link to a existing directory.
ynh_multimedia_addfolder() {
# ============ Argument parsing =============
local -A args_array=([s]=source_dir= [d]=dest_dir=)
local source_dir
local dest_dir
ynh_handle_getopts_args "$@"
# ===========================================
# Ajout d'un lien symbolique vers le dossier à partager
ln -sfn "$source_dir" "$MEDIA_DIRECTORY/$dest_dir"
## Application des droits étendus sur le dossier ajouté
# Droit d'écriture pour le groupe et le groupe multimedia en acl et droit de lecture pour other:
setfacl -RnL -m g:$MEDIA_GROUP:rwX,g::rwX,o:r-X "$source_dir"
# Application de la même règle que précédemment, mais par défaut pour les nouveaux fichiers.
setfacl -RnL -m d:g:$MEDIA_GROUP:rwX,g::rwX,o:r-X "$source_dir"
# Réglage du masque par défaut. Qui garantie (en principe...) un droit maximal à rwx. Donc pas de restriction de droits par l'acl.
setfacl -RL -m m::rwx "$source_dir"
}
# Allow an user to have an write authorisation in multimedia directories
#
# usage: ynh_multimedia_addaccess user_name
#
# | arg: --user_name= - The name of the user which gain this access.
ynh_multimedia_addaccess() {
# ============ Argument parsing =============
local -A args_array=([u]=user_name=)
local user_name
ynh_handle_getopts_args "$@"
# ===========================================
groupadd -f multimedia
usermod -a -G multimedia $user_name
}

116
helpers/helpers.v2.1.d/mysql Normal file
View file

@ -0,0 +1,116 @@
#!/bin/bash
# Run SQL instructions in a database ($db_name by default)
#
# usage: ynh_mysql_db_shell [database] <<< "instructions"
# | arg: database= - the database to connect to (by default, $db_name)
#
# examples:
# ynh_mysql_db_shell $db_name <<< "UPDATE ...;"
# ynh_mysql_db_shell < /path/to/file.sql
#
ynh_mysql_db_shell() {
local database=${1:-$db_name}
mysql -B $database
}
# Create a database and grant optionnaly privilegies to a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_mysql_create_db db [user [pwd]]
# | arg: db - the database name to create
# | arg: user - the user to grant privilegies
# | arg: pwd - the password to identify user by
#
ynh_mysql_create_db() {
local db=$1
local sql="CREATE DATABASE ${db};"
# grant all privilegies to user
if [[ $# -gt 1 ]]; then
sql+=" GRANT ALL PRIVILEGES ON ${db}.* TO '${2}'@'localhost'"
if [[ -n ${3:-} ]]; then
sql+=" IDENTIFIED BY '${3}'"
fi
sql+=" WITH GRANT OPTION;"
fi
mysql -B <<< "$sql"
}
# Drop a database
#
# [internal] ... handled by the core / "database resource"
#
# If you intend to drop the database *and* the associated user,
# consider using ynh_mysql_remove_db instead.
#
# usage: ynh_mysql_drop_db db
# | arg: db - the database name to drop
#
ynh_mysql_drop_db() {
mysql -B <<< "DROP DATABASE ${1};"
}
# Dump a database
#
# usage: ynh_mysql_dump_db database
# | arg: database - the database name to dump (by default, $db_name)
# | ret: The mysqldump output
#
# example: ynh_mysql_dump_db "roundcube" > ./dump.sql
#
ynh_mysql_dump_db() {
local database=${1:-$db_name}
mysqldump --single-transaction --skip-dump-date --routines "$database"
}
# Create a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_mysql_create_user user pwd [host]
# | arg: user - the user name to create
# | arg: pwd - the password to identify user by
#
ynh_mysql_create_user() {
mysql -B <<< "CREATE USER '${1}'@'localhost' IDENTIFIED BY '${2}';"
}
# Check if a mysql user exists
#
# [internal]
#
# usage: ynh_mysql_user_exists user
# | arg: user - the user for which to check existence
# | ret: 0 if the user exists, 1 otherwise.
ynh_mysql_user_exists() {
local user=$1
[[ -n "$(mysql -B <<< "SELECT User from mysql.user WHERE User = '$user';")" ]]
}
# Check if a mysql database exists
#
# [internal]
#
# usage: ynh_mysql_database_exists database
# | arg: database - the database for which to check existence
# | exit: Return 1 if the database doesn't exist, 0 otherwise
#
ynh_mysql_database_exists() {
local database=$1
mysqlshow | grep -q "^| $database "
}
# Drop a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_mysql_drop_user user
# | arg: user - the user name to drop
#
ynh_mysql_drop_user() {
mysql -B <<< "DROP USER '${1}'@'localhost';"
}

59
helpers/helpers.v2.1.d/nginx Normal file
View file

@ -0,0 +1,59 @@
#!/bin/bash
# Create a dedicated nginx config
#
# usage: ynh_config_add_nginx
#
# This will use a template in `../conf/nginx.conf`
# See the documentation of `ynh_config_add` for a description of the template
# format and how placeholders are replaced with actual variables.
#
# Additionally, ynh_config_add_nginx will replace:
# - `#sub_path_only` by empty string if `path` is not `'/'`
# - `#root_path_only` by empty string if `path` *is* `'/'`
#
# This allows to enable/disable specific behaviors dependenging on the install
# location
ynh_config_add_nginx() {
local finalnginxconf="/etc/nginx/conf.d/$domain.d/$app.conf"
ynh_config_add --template="nginx.conf" --destination="$finalnginxconf"
if [ "${path:-}" != "/" ]; then
ynh_replace --match="^#sub_path_only" --replace="" --file="$finalnginxconf"
else
ynh_replace --match="^#root_path_only" --replace="" --file="$finalnginxconf"
fi
ynh_store_file_checksum "$finalnginxconf"
ynh_systemctl --service=nginx --action=reload
}
# Remove the dedicated nginx config
#
# usage: ynh_config_remove_nginx
ynh_config_remove_nginx() {
ynh_safe_rm "/etc/nginx/conf.d/$domain.d/$app.conf"
ynh_systemctl --service=nginx --action=reload
}
# Regen the nginx config in a change url context
#
# usage: ynh_config_change_url_nginx
ynh_config_change_url_nginx() {
# Make a backup of the original NGINX config file if manually modified
# (nb: this is possibly different from the same instruction called by
# ynh_config_add inside ynh_config_add_nginx because the path may have
# changed if we're changing the domain too...)
local old_nginx_conf_path=/etc/nginx/conf.d/$old_domain.d/$app.conf
ynh_backup_if_checksum_is_different "$old_nginx_conf_path"
ynh_delete_file_checksum "$old_nginx_conf_path"
ynh_safe_rm "$old_nginx_conf_path"
# Regen the nginx conf
ynh_config_add_nginx
}

124
helpers/helpers.v2.1.d/nodejs Normal file
View file

@ -0,0 +1,124 @@
#!/bin/bash
readonly N_INSTALL_DIR="/opt/node_n"
# N_PREFIX is the directory of n, it needs to be loaded as a environment variable.
export N_PREFIX="$N_INSTALL_DIR"
# [internal]
_ynh_load_nodejs_in_path_and_other_tweaks() {
# Get the absolute path of this version of node
nodejs_dir="$N_INSTALL_DIR/n/versions/node/$nodejs_version/bin"
# Load the path of this version of node in $PATH
if [[ :$PATH: != *":$nodejs_dir"* ]]; then
PATH="$nodejs_dir:$PATH"
fi
# Export PATH such that it's available through sudo -E / ynh_exec_as $app
export PATH
# This is in full lowercase such that it gets replaced in templates
path_with_nodejs="$PATH"
PATH_with_nodejs="$PATH"
# Prevent yet another Node and Corepack madness, with Corepack wanting the user to confirm download of Yarn
export COREPACK_ENABLE_DOWNLOAD_PROMPT=0
}
# Install a specific version of nodejs, using 'n'
#
# The installed version is defined by `$nodejs_version` which should be defined as global prior to calling this helper
#
# usage: ynh_nodejs_install
#
# `n` (Node version management) uses the `PATH` variable to store the path of the version of node it is going to use.
# That's how it changes the version
#
# The helper adds the appropriate, specific version of nodejs to the `$PATH` variable (which
# is preserved when calling ynh_exec_as_app). Also defines:
# - `$path_with_nodejs` to be used in the systemd config (`Environment="PATH=__PATH_WITH_NODEJS__"`)
# - `$nodejs_dir`, the directory containing the specific version of nodejs, which may be used in the systemd config too (e.g. `ExecStart=__NODEJS_DIR__/node foo bar`)
ynh_nodejs_install() {
# Use n, https://github.com/tj/n to manage the nodejs versions
[[ -n "${nodejs_version:-}" ]] || ynh_die "\$nodejs_version should be defined prior to calling ynh_nodejs_install"
# Create $N_INSTALL_DIR
mkdir --parents "$N_INSTALL_DIR"
# Load n path in PATH
CLEAR_PATH="$N_INSTALL_DIR/bin:$PATH"
# Remove /usr/local/bin in PATH in case of node prior installation
PATH=$(echo $CLEAR_PATH | sed 's@/usr/local/bin:@@')
# Move an existing node binary, to avoid to block n.
test -x /usr/bin/node && mv /usr/bin/node /usr/bin/node_n
test -x /usr/bin/npm && mv /usr/bin/npm /usr/bin/npm_n
# Install (or update if YunoHost vendor/ folder updated since last install) n
mkdir -p $N_INSTALL_DIR/bin/
cp "$YNH_HELPERS_DIR/vendor/n/n" $N_INSTALL_DIR/bin/n
# Tweak for n to understand it's installed in $N_PREFIX
ynh_replace --match="^N_PREFIX=\${N_PREFIX-.*}$" --replace="N_PREFIX=\${N_PREFIX-$N_PREFIX}" --file="$N_INSTALL_DIR/bin/n"
# Restore /usr/local/bin in PATH
PATH=$CLEAR_PATH
# And replace the old node binary.
test -x /usr/bin/node_n && mv /usr/bin/node_n /usr/bin/node
test -x /usr/bin/npm_n && mv /usr/bin/npm_n /usr/bin/npm
# Install the requested version of nodejs
uname=$(uname --machine)
if [[ $uname =~ aarch64 || $uname =~ arm64 ]]; then
n $nodejs_version --arch=arm64
else
n $nodejs_version
fi
# Find the last "real" version for this major version of node.
real_nodejs_version=$(find $N_INSTALL_DIR/n/versions/node/$nodejs_version* -maxdepth 0 | sort --version-sort | tail --lines=1)
real_nodejs_version=$(basename $real_nodejs_version)
# Create a symbolic link for this major version if the file doesn't already exist
if [ ! -e "$N_INSTALL_DIR/n/versions/node/$nodejs_version" ]; then
ln --symbolic --force --no-target-directory \
$N_INSTALL_DIR/n/versions/node/$real_nodejs_version \
$N_INSTALL_DIR/n/versions/node/$nodejs_version
fi
# Store the ID of this app and the version of node requested for it
echo "$YNH_APP_INSTANCE_NAME:$nodejs_version" | tee --append "$N_INSTALL_DIR/ynh_app_version"
# Store nodejs_version into the config of this app
ynh_app_setting_set --key=nodejs_version --value=$nodejs_version
_ynh_load_nodejs_in_path_and_other_tweaks
}
# Remove the version of node used by the app.
#
# usage: ynh_nodejs_remove
#
# This helper will check if another app uses the same version of node.
# - If not, this version of node will be removed.
# - If no other app uses node, n will be also removed.
ynh_nodejs_remove() {
[[ -n "${nodejs_version:-}" ]] || ynh_die "\$nodejs_version should be defined prior to calling ynh_nodejs_remove"
# Remove the line for this app
sed --in-place "/$YNH_APP_INSTANCE_NAME:$nodejs_version/d" "$N_INSTALL_DIR/ynh_app_version"
# If no other app uses this version of nodejs, remove it.
if ! grep --quiet "$nodejs_version" "$N_INSTALL_DIR/ynh_app_version"; then
$N_INSTALL_DIR/bin/n rm $nodejs_version
fi
# If no other app uses n, remove n
if [ ! -s "$N_INSTALL_DIR/ynh_app_version" ]; then
ynh_safe_rm "$N_INSTALL_DIR"
sed --in-place "/N_PREFIX/d" /root/.bashrc
fi
}

310
helpers/helpers.v2.1.d/permission Normal file
View file

@ -0,0 +1,310 @@
#!/bin/bash
# Create a new permission for the app
#
# Example 1: `ynh_permission_create --permission=admin --url=/admin --additional_urls=domain.tld/admin /superadmin --allowed=alice bob \
# --label="My app admin" --show_tile=true`
#
# This example will create a new permission permission with this following effect:
# - A tile named "My app admin" in the SSO will be available for the users alice and bob. This tile will point to the relative url '/admin'.
# - Only the user alice and bob will have the access to theses following url: /admin, domain.tld/admin, /superadmin
#
#
# Example 2:
#
# ynh_permission_create --permission=api --url=domain.tld/api --auth_header=false --allowed=visitors \
# --label="MyApp API" --protected=true
#
# This example will create a new protected permission. So the admin won't be able to add/remove the visitors group of this permission.
# In case of an API with need to be always public it avoid that the admin break anything.
# With this permission all client will be allowed to access to the url 'domain.tld/api'.
# Note that in this case no tile will be show on the SSO.
# Note that the auth_header parameter is to 'false'. So no authentication header will be passed to the application.
# Generally the API is requested by an application and enabling the auth_header has no advantage and could bring some issues in some case.
# So in this case it's better to disable this option for all API.
#
#
# usage: ynh_permission_create --permission="permission" [--url="url"] [--additional_urls="second-url" [ "third-url" ]] [--auth_header=true|false]
# [--allowed=group1 [ group2 ]] [--label="label"] [--show_tile=true|false]
# [--protected=true|false]
# | arg: --permission= - the name for the permission (by default a permission named "main" already exist)
# | arg: --url= - (optional) URL for which access will be allowed/forbidden. Note that if 'show_tile' is enabled, this URL will be the URL of the tile.
# | arg: --additional_urls= - (optional) List of additional URL for which access will be allowed/forbidden
# | arg: --auth_header= - (optional) Define for the URL of this permission, if SSOwat pass the authentication header to the application. Default is true
# | arg: --allowed= - (optional) A list of group/user to allow for the permission
# | arg: --label= - (optional) Define a name for the permission. This label will be shown on the SSO and in the admin. Default is "APP_LABEL (permission name)".
# | arg: --show_tile= - (optional) Define if a tile will be shown in the SSO. If yes the name of the tile will be the 'label' parameter. Defaults to false for the permission different than 'main'.
# | arg: --protected= - (optional) Define if this permission is protected. If it is protected the administrator won't be able to add or remove the visitors group of this permission. Defaults to 'false'.
#
# [packagingv1]
#
# If provided, 'url' or 'additional_urls' is assumed to be relative to the app domain/path if they
# start with '/'. For example:
# / -> domain.tld/app
# /admin -> domain.tld/app/admin
# domain.tld/app/api -> domain.tld/app/api
#
# 'url' or 'additional_urls' can be treated as a PCRE (not lua) regex if it starts with "re:".
# For example:
# re:/api/[A-Z]*$ -> domain.tld/app/api/[A-Z]*$
# re:domain.tld/app/api/[A-Z]*$ -> domain.tld/app/api/[A-Z]*$
#
# Note that globally the parameter 'url' and 'additional_urls' are same. The only difference is:
# - 'url' is only one url, 'additional_urls' can be a list of urls. There are no limitation of 'additional_urls'
# - 'url' is used for the url of tile in the SSO (if enabled with the 'show_tile' parameter)
#
#
# About the authentication header (auth_header parameter).
# The SSO pass (by default) to the application theses following HTTP header (linked to the authenticated user) to the application:
# - "Auth-User": username
# - "Remote-User": username
# - "Email": user email
#
# Generally this feature is usefull to authenticate automatically the user in the application but in some case the application don't work with theses header and theses header need to be disabled to have the application to work correctly.
# See https://github.com/YunoHost/issues/issues/1420 for more informations
ynh_permission_create() {
# ============ Argument parsing =============
local -A args_array=([p]=permission= [u]=url= [A]=additional_urls= [h]=auth_header= [a]=allowed= [l]=label= [t]=show_tile= [P]=protected=)
local permission
local url
local additional_urls
local auth_header
local allowed
local label
local show_tile
local protected
ynh_handle_getopts_args "$@"
url=${url:-}
additional_urls=${additional_urls:-}
auth_header=${auth_header:-}
allowed=${allowed:-}
label=${label:-}
show_tile=${show_tile:-}
protected=${protected:-}
# ===========================================
if [[ -n $url ]]; then
url=",url='$url'"
fi
if [[ -n $additional_urls ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# By example:
# --additional_urls /urlA /urlB
# will be:
# additional_urls=['/urlA', '/urlB']
additional_urls=",additional_urls=['${additional_urls//;/\',\'}']"
fi
if [[ -n $auth_header ]]; then
if [ $auth_header == "true" ]; then
auth_header=",auth_header=True"
else
auth_header=",auth_header=False"
fi
fi
if [[ -n $allowed ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# By example:
# --allowed alice bob
# will be:
# allowed=['alice', 'bob']
allowed=",allowed=['${allowed//;/\',\'}']"
fi
if [[ -n ${label:-} ]]; then
label=",label='$label'"
else
label=",label='$permission'"
fi
if [[ -n ${show_tile:-} ]]; then
if [ $show_tile == "true" ]; then
show_tile=",show_tile=True"
else
show_tile=",show_tile=False"
fi
fi
if [[ -n ${protected:-} ]]; then
if [ $protected == "true" ]; then
protected=",protected=True"
else
protected=",protected=False"
fi
fi
yunohost tools shell -c "from yunohost.permission import permission_create; permission_create('$app.$permission' $url $additional_urls $auth_header $allowed $label $show_tile $protected)"
}
# Remove a permission for the app (note that when the app is removed all permission is automatically removed)
#
# example: ynh_permission_delete --permission=editors
#
# usage: ynh_permission_delete --permission="permission"
# | arg: --permission= - the name for the permission (by default a permission named "main" is removed automatically when the app is removed)
ynh_permission_delete() {
# ============ Argument parsing =============
local -A args_array=([p]=permission=)
local permission
ynh_handle_getopts_args "$@"
# ===========================================
yunohost tools shell -c "from yunohost.permission import permission_delete; permission_delete('$app.$permission')"
}
# Check if a permission exists
#
# usage: ynh_permission_exists --permission=permission
# | arg: --permission= - the permission to check
# | exit: Return 1 if the permission doesn't exist, 0 otherwise
ynh_permission_exists() {
# ============ Argument parsing =============
local -A args_array=([p]=permission=)
local permission
ynh_handle_getopts_args "$@"
# ===========================================
yunohost user permission list "$app" --output-as json --quiet \
| jq -e --arg perm "$app.$permission" '.permissions[$perm]' >/dev/null
}
# Redefine the url associated to a permission
#
# usage: ynh_permission_url --permission "permission" [--url="url"] [--add_url="new-url" [ "other-new-url" ]] [--remove_url="old-url" [ "other-old-url" ]]
# [--auth_header=true|false] [--clear_urls]
# | arg: --permission= - the name for the permission (by default a permission named "main" is removed automatically when the app is removed)
# | arg: --url= - (optional) URL for which access will be allowed/forbidden. Note that if you want to remove url you can pass an empty sting as arguments ("").
# | arg: --add_url= - (optional) List of additional url to add for which access will be allowed/forbidden.
# | arg: --remove_url= - (optional) List of additional url to remove for which access will be allowed/forbidden
# | arg: --auth_header= - (optional) Define for the URL of this permission, if SSOwat pass the authentication header to the application
# | arg: --clear_urls - (optional) Clean all urls (url and additional_urls)
ynh_permission_url() {
# ============ Argument parsing =============
local -A args_array=([p]=permission= [u]=url= [a]=add_url= [r]=remove_url= [h]=auth_header= [c]=clear_urls)
local permission
local url
local add_url
local remove_url
local auth_header
local clear_urls
ynh_handle_getopts_args "$@"
url=${url:-}
add_url=${add_url:-}
remove_url=${remove_url:-}
auth_header=${auth_header:-}
clear_urls=${clear_urls:-}
# ===========================================
if [[ -n $url ]]; then
url=",url='$url'"
fi
if [[ -n $add_url ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# For example:
# --add_url /urlA /urlB
# will be:
# add_url=['/urlA', '/urlB']
add_url=",add_url=['${add_url//;/\',\'}']"
fi
if [[ -n $remove_url ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# For example:
# --remove_url /urlA /urlB
# will be:
# remove_url=['/urlA', '/urlB']
remove_url=",remove_url=['${remove_url//;/\',\'}']"
fi
if [[ -n $auth_header ]]; then
if [ $auth_header == "true" ]; then
auth_header=",auth_header=True"
else
auth_header=",auth_header=False"
fi
fi
if [[ -n $clear_urls ]] && [ $clear_urls -eq 1 ]; then
clear_urls=",clear_urls=True"
fi
yunohost tools shell -c "from yunohost.permission import permission_url; permission_url('$app.$permission' $url $add_url $remove_url $auth_header $clear_urls)"
}
# Update a permission for the app
#
# usage: ynh_permission_update --permission "permission" [--add="group" ["group" ...]] [--remove="group" ["group" ...]]
#
# | arg: --permission= - the name for the permission (by default a permission named "main" already exist)
# | arg: --add= - the list of group or users to enable add to the permission
# | arg: --remove= - the list of group or users to remove from the permission
ynh_permission_update() {
# ============ Argument parsing =============
local -A args_array=([p]=permission= [a]=add= [r]=remove=)
local permission
local add
local remove
ynh_handle_getopts_args "$@"
add=${add:-}
remove=${remove:-}
# ===========================================
if [[ -n $add ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# For example:
# --add alice bob
# will be:
# add=['alice', 'bob']
add=",add=['${add//';'/"','"}']"
fi
if [[ -n $remove ]]; then
# Convert a list from getopts to python list
# Note that getopts separate the args with ';'
# For example:
# --remove alice bob
# will be:
# remove=['alice', 'bob']
remove=",remove=['${remove//';'/"','"}']"
fi
yunohost tools shell -c "from yunohost.permission import user_permission_update; user_permission_update('$app.$permission' $add $remove , force=True)"
}
# Check if a permission has an user
#
# example: ynh_permission_has_user --permission=main --user=visitors
#
# usage: ynh_permission_has_user --permission=permission --user=user
# | arg: --permission= - the permission to check
# | arg: --user= - the user seek in the permission
# | exit: Return 1 if the permission doesn't have that user or doesn't exist, 0 otherwise
ynh_permission_has_user() {
# ============ Argument parsing =============
local -A args_array=([p]=permission= [u]=user=)
local permission
local user
ynh_handle_getopts_args "$@"
# ===========================================
if ! ynh_permission_exists --permission=$permission; then
return 1
fi
# Check both allowed and corresponding_users sections in the json
for section in "allowed" "corresponding_users"; do
if yunohost user permission info "$app.$permission" --output-as json --quiet \
| jq -e --arg user $user --arg section $section '.[$section] | index($user)' >/dev/null; then
return 0
fi
done
return 1
}

161
helpers/helpers.v2.1.d/php Normal file
View file

@ -0,0 +1,161 @@
#!/bin/bash
# (this is used in the apt helpers, big meh ...)
readonly YNH_DEFAULT_PHP_VERSION=7.4
# Legacy: auto-convert phpversion to php_version (for consistency with nodejs_version, ruby_version, ...)
if [[ -n "${app:-}" ]] && [[ -n "${phpversion:-}" ]]
then
if [[ -z "${php_version:-}" ]]
then
php_version=$phpversion
ynh_app_setting_set --key=php_version --value=$php_version
fi
ynh_app_setting_delete --key=phpversion
unset phpversion
fi
# Create a dedicated PHP-FPM config
#
# usage: ynh_config_add_phpfpm
#
# This will automatically generate an appropriate PHP-FPM configuration for this app.
#
# The resulting configuration will be deployed to the appropriate place:
# `/etc/php/$php_version/fpm/pool.d/$app.conf`
#
# If the app provides a `conf/extra_php-fpm.conf` template, it will be appended
# to the generated configuration. (In the vast majority of cases, this shouldnt
# be necessary)
#
# $php_version should be defined prior to calling this helper, but there should
# be no reason to manually set it, as it is automatically set by the apt
# helpers/resources when installing phpX.Y dependencies (PHP apps should at
# least install phpX.Y-fpm using the `apt` helper/resource)
#
# `$php_group` can be defined as a global (from `_common.sh`) if the worker
# processes should run with a different group than `$app`
#
# Additional "pm" and "php_admin_value" settings which are meant to be possibly
# configurable by admins from a future standard config panel at some point,
# related to performance and availability of the app, for which tweaking may be
# required if the app is used by "plenty" of users and other memory/CPU load
# considerations....
#
# If you have good reasons to be willing to use different
# defaults than the one set by this helper (while still allowing admin to
# override it) you should use `ynh_app_setting_set_default`
#
# - `$php_upload_max_filezise`: corresponds upload_max_filesize and post_max_size. Defaults to 50M
# - `$php_process_management`: corresponds to "pm" (ondemand, dynamic, static). Defaults to ondemand
# - `$php_max_children`: by default, computed from "total RAM" divided by 40, cf `_default_php_max_children`
# - `$php_memory_limit`: by default, 128M (from global php.ini)
#
# Note that if $php_process_management is set to "dynamic", then these
# variables MUST be defined prior to calling the helper (no default value) ...
# Check PHP-FPM's manual for more info on what these are (: ...
#
# - `$php_start_servers`
# - `$php_min_spare_servers`
# - `$php_max_spare_servers`
#
ynh_config_add_phpfpm() {
[[ -n "${php_version:-}" ]] || ynh_die "\$php_version should be defined prior to calling ynh_config_add_phpfpm. You should not need to define it manually, it is automatically set by the apt helper when installing the phpX.Y- depenencies"
# Apps may define $php_group as a global (e.g. from _common.sh) to change this
# (this is not meant to be overridable by users)
local php_group=${php_group:-$app}
# Meant to be overridable by users from a standard config panel at some point ...
# Apps willing to tweak these should use ynh_setting_set_default_value (in install and upgrade?)
#
local php_upload_max_filesize=${php_upload_max_filesize:-50M}
local php_process_management=${php_process_management:-ondemand} # alternatively 'dynamic' or 'static'
local php_max_children=${php_max_children:-$(_default_php_max_children)}
local php_memory_limit=${php_memory_limit:-128M} # default value is from global php.ini
local phpfpm_template=$(mktemp)
cat << EOF > $phpfpm_template
[__APP__]
user = __APP__
group = __PHP_GROUP__
chdir = __INSTALL_DIR__
listen = /var/run/php/php__PHP_VERSION__-fpm-__APP__.sock
listen.owner = www-data
listen.group = www-data
pm = __PHP_PROCESS_MANAGEMENT__
pm.max_children = __PHP_MAX_CHILDREN__
pm.max_requests = 500
request_terminate_timeout = 1d
EOF
if [ "$php_process_management" = "dynamic" ]; then
cat << EOF >> $phpfpm_template
pm.start_servers = __PHP_START_SERVERS__
pm.min_spare_servers = __PHP_MIN_SPARE_SERVERS__
pm.max_spare_servers = __PHP_MAX_SPARE_SERVERS__
EOF
elif [ "$php_process_management" = "ondemand" ]; then
cat << EOF >> $phpfpm_template
pm.process_idle_timeout = 10s
EOF
fi
cat << EOF >> $phpfpm_template
php_admin_value[upload_max_filesize] = __PHP_UPLOAD_MAX_FILESIZE__
php_admin_value[post_max_size] = __PHP_UPLOAD_MAX_FILESIZE__
php_admin_value[memory_limit] = __PHP_MEMORY_LIMIT__
EOF
# Concatene the extra config
if [ -e $YNH_APP_BASEDIR/conf/extra_php-fpm.conf ]; then
cat $YNH_APP_BASEDIR/conf/extra_php-fpm.conf >>"$phpfpm_template"
fi
# Make sure the fpm pool dir exists
mkdir --parents "/etc/php/$php_version/fpm/pool.d"
# And hydrate configuration
ynh_config_add --template="$phpfpm_template" --destination="/etc/php/$php_version/fpm/pool.d/$app.conf"
# Validate that the new php conf doesn't break php-fpm entirely
if ! php-fpm${php_version} --test 2>/dev/null; then
php-fpm${php_version} --test || true
ynh_safe_rm "/etc/php/$php_version/fpm/pool.d/$app.conf"
ynh_die "The new configuration broke php-fpm?"
fi
ynh_systemctl --service=php${php_version}-fpm --action=reload
}
# Remove the dedicated PHP-FPM config
#
# usage: ynh_config_remove_phpfpm
ynh_config_remove_phpfpm() {
ynh_safe_rm "/etc/php/$php_version/fpm/pool.d/$app.conf"
ynh_systemctl --service="php${php_version}-fpm" --action=reload
}
_default_php_max_children() {
# Get the total of RAM available
local total_ram=$(ynh_get_ram --total)
# The value of pm.max_children is the total amount of ram divide by 2,
# divide again by 20MB (= a default, classic worker footprint) This is
# designed such that if PHP-FPM start the maximum of children, it won't
# exceed half of the ram.
local php_max_children="$(($total_ram / 40))"
# Make sure we get at least max_children = 1
if [ $php_max_children -le 0 ]; then
php_max_children=1
# To not overload the proc, limit the number of children to 4 times the number of cores.
elif [ $php_max_children -gt "$(($(nproc) * 4))" ]; then
php_max_children="$(($(nproc) * 4))"
fi
echo "$php_max_children"
}

124
helpers/helpers.v2.1.d/postgresql Normal file
View file

@ -0,0 +1,124 @@
#!/bin/bash
PSQL_ROOT_PWD_FILE=/etc/yunohost/psql
PSQL_VERSION=13
# Run SQL instructions in a database ($db_name by default)
#
# usage: ynh_psql_db_shell database <<< "instructions"
# | arg: database - the database to connect to (by default, $db_name)
#
# examples:
# ynh_psql_db_shell $db_name <<< "UPDATE ...;"
# ynh_psql_db_shell < /path/to/file.sql
#
ynh_psql_db_shell() {
local database="${1:-$db_name}"
sudo --login --user=postgres psql "$database"
}
# Create a database and grant optionnaly privilegies to a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_psql_create_db db [user]
# | arg: db - the database name to create
# | arg: user - the user to grant privilegies
#
ynh_psql_create_db() {
local db=$1
local user=${2:-}
local sql="CREATE DATABASE ${db};"
# grant all privilegies to user
if [ -n "$user" ]; then
sql+="ALTER DATABASE ${db} OWNER TO ${user};"
sql+="GRANT ALL PRIVILEGES ON DATABASE ${db} TO ${user} WITH GRANT OPTION;"
fi
sudo --login --user=postgres psql <<< "$sql"
}
# Drop a database
#
# [internal] ... handled by the core / "database resource"
#
# If you intend to drop the database *and* the associated user,
# consider using ynh_psql_remove_db instead.
#
# usage: ynh_psql_drop_db db
# | arg: db - the database name to drop
#
ynh_psql_drop_db() {
local db=$1
# First, force disconnection of all clients connected to the database
# https://stackoverflow.com/questions/17449420/postgresql-unable-to-drop-database-because-of-some-auto-connections-to-db
sudo --login --user=postgres psql $db <<< "REVOKE CONNECT ON DATABASE $db FROM public;"
sudo --login --user=postgres psql $db <<< "SELECT pg_terminate_backend (pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = '$db' AND pid <> pg_backend_pid();"
sudo --login --user=postgres dropdb $db
}
# Dump a database
#
# usage: ynh_psql_dump_db database
# | arg: database - the database name to dump (by default, $db_name)
# | ret: the psqldump output
#
# example: ynh_psql_dump_db 'roundcube' > ./dump.sql
#
ynh_psql_dump_db() {
local database="${1:-$db_name}"
sudo --login --user=postgres pg_dump "$database"
}
# Create a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_psql_create_user user pwd
# | arg: user - the user name to create
# | arg: pwd - the password to identify user by
#
ynh_psql_create_user() {
local user=$1
local pwd=$2
sudo --login --user=postgres psql <<< "CREATE USER $user WITH ENCRYPTED PASSWORD '$pwd'"
}
# Check if a psql user exists
#
# [internal]
#
# usage: ynh_psql_user_exists user
# | arg: user= - the user for which to check existence
# | exit: Return 1 if the user doesn't exist, 0 otherwise
#
ynh_psql_user_exists() {
local user=$1
sudo --login --user=postgres psql -tAc "SELECT rolname FROM pg_roles WHERE rolname='$user';" | grep --quiet "$user"
}
# Check if a psql database exists
#
# [internal]
#
# usage: ynh_psql_database_exists database
# | arg: database - the database for which to check existence
# | exit: Return 1 if the database doesn't exist, 0 otherwise
#
ynh_psql_database_exists() {
local database=$1
sudo --login --user=postgres psql -tAc "SELECT datname FROM pg_database WHERE datname='$database';" | grep --quiet "$database"
}
# Drop a user
#
# [internal] ... handled by the core / "database resource"
#
# usage: ynh_psql_drop_user user
# | arg: user - the user name to drop
#
ynh_psql_drop_user() {
sudo --login --user=postgres psql <<< "DROP USER ${1};"
}

39
helpers/helpers.v2.1.d/redis Normal file
View file

@ -0,0 +1,39 @@
#!/bin/bash
# get the first available redis database
#
# usage: ynh_redis_get_free_db
# | returns: the database number to use
ynh_redis_get_free_db() {
local result max db
result=$(redis-cli INFO keyspace)
# get the num
max=$(cat /etc/redis/redis.conf | grep ^databases | grep -Eow "[0-9]+")
db=0
# default Debian setting is 15 databases
for i in $(seq 0 "$max")
do
if ! echo "$result" | grep -q "db$i"
then
db=$i
break 1
fi
db=-1
done
test "$db" -eq -1 && ynh_die "No available Redis databases..."
echo "$db"
}
# Create a master password and set up global settings
# Please always call this script in install and restore scripts
#
# usage: ynh_redis_remove_db database
# | arg: database - the database to erase
ynh_redis_remove_db() {
local db=$1
redis-cli -n "$db" flushdb
}

252
helpers/helpers.v2.1.d/ruby Normal file
View file

@ -0,0 +1,252 @@
#!/bin/bash
readonly RBENV_INSTALL_DIR="/opt/rbenv"
# RBENV_ROOT is the directory of rbenv, it needs to be loaded as a environment variable.
export RBENV_ROOT="$RBENV_INSTALL_DIR"
export rbenv_root="$RBENV_INSTALL_DIR"
_ynh_load_ruby_in_path_and_other_tweaks() {
# Get the absolute path of this version of Ruby
ruby_dir="$RBENV_INSTALL_DIR/versions/$app/bin"
# Load the path of this version of ruby in $PATH
if [[ :$PATH: != *":$ruby_dir"* ]]; then
PATH="$ruby_dir:$PATH"
fi
# Export PATH such that it's available through sudo -E / ynh_exec_as $app
export PATH
# This is in full lowercase such that it gets replaced in templates
path_with_ruby="$PATH"
PATH_with_ruby="$PATH"
# Sets the local application-specific Ruby version
pushd ${install_dir}
$RBENV_INSTALL_DIR/bin/rbenv local $ruby_version
popd
}
# Install a specific version of Ruby using rbenv
#
# The installed version is defined by `$ruby_version` which should be defined as global prior to calling this helper
#
# usage: ynh_ruby_install
#
# The helper adds the appropriate, specific version of ruby to the `$PATH` variable (which
# is preserved when calling ynh_exec_as_app). Also defines:
# - `$path_with_ruby` to be used in the systemd config (`Environment="PATH=__PATH_WITH_RUBY__"`)
# - `$ruby_dir`, the directory containing the specific version of ruby, which may be used in the systemd config too (e.g. `ExecStart=__RUBY_DIR__/ruby foo bar`)
#
# This helper also creates a /etc/profile.d/rbenv.sh that configures PATH environment for rbenv
ynh_ruby_install () {
[[ -n "${ruby_version:-}" ]] || ynh_die "\$ruby_version should be defined prior to calling ynh_ruby_install"
# Load rbenv path in PATH
local CLEAR_PATH="$RBENV_INSTALL_DIR/bin:$PATH"
# Remove /usr/local/bin in PATH in case of Ruby prior installation
PATH=$(echo $CLEAR_PATH | sed 's@/usr/local/bin:@@')
# Move an existing Ruby binary, to avoid to block rbenv
test -x /usr/bin/ruby && mv /usr/bin/ruby /usr/bin/ruby_rbenv
# Install or update rbenv
mkdir -p $RBENV_INSTALL_DIR
rbenv="$(command -v rbenv $RBENV_INSTALL_DIR/bin/rbenv | grep "$RBENV_INSTALL_DIR/bin/rbenv" | head -1)"
if [ -n "$rbenv" ]; then
pushd "${rbenv%/*/*}"
if git remote -v 2>/dev/null | grep "https://github.com/rbenv/rbenv.git"; then
echo "Updating rbenv..."
git pull -q --tags origin master
_ynh_ruby_try_bash_extension
else
echo "Reinstalling rbenv..."
cd ..
ynh_safe_rm $RBENV_INSTALL_DIR
mkdir -p $RBENV_INSTALL_DIR
cd $RBENV_INSTALL_DIR
git init -q
git remote add -f -t master origin https://github.com/rbenv/rbenv.git > /dev/null 2>&1
git checkout -q -b master origin/master
_ynh_ruby_try_bash_extension
rbenv=$RBENV_INSTALL_DIR/bin/rbenv
fi
popd
else
echo "Installing rbenv..."
pushd $RBENV_INSTALL_DIR
git init -q
git remote add -f -t master origin https://github.com/rbenv/rbenv.git > /dev/null 2>&1
git checkout -q -b master origin/master
_ynh_ruby_try_bash_extension
rbenv=$RBENV_INSTALL_DIR/bin/rbenv
popd
fi
mkdir -p "${RBENV_INSTALL_DIR}/plugins"
ruby_build="$(command -v "$RBENV_INSTALL_DIR"/plugins/*/bin/rbenv-install rbenv-install | head -1)"
if [ -n "$ruby_build" ]; then
pushd "${ruby_build%/*/*}"
if git remote -v 2>/dev/null | grep "https://github.com/rbenv/ruby-build.git"; then
echo "Updating ruby-build..."
git pull -q origin master
fi
popd
else
echo "Installing ruby-build..."
git clone -q https://github.com/rbenv/ruby-build.git "${RBENV_INSTALL_DIR}/plugins/ruby-build"
fi
rbenv_alias="$(command -v "$RBENV_INSTALL_DIR"/plugins/*/bin/rbenv-alias rbenv-alias | head -1)"
if [ -n "$rbenv_alias" ]; then
pushd "${rbenv_alias%/*/*}"
if git remote -v 2>/dev/null | grep "https://github.com/tpope/rbenv-aliases.git"; then
echo "Updating rbenv-aliases..."
git pull -q origin master
fi
popd
else
echo "Installing rbenv-aliases..."
git clone -q https://github.com/tpope/rbenv-aliases.git "${RBENV_INSTALL_DIR}/plugins/rbenv-aliase"
fi
rbenv_latest="$(command -v "$RBENV_INSTALL_DIR"/plugins/*/bin/rbenv-latest rbenv-latest | head -1)"
if [ -n "$rbenv_latest" ]; then
pushd "${rbenv_latest%/*/*}"
if git remote -v 2>/dev/null | grep "https://github.com/momo-lab/xxenv-latest.git"; then
echo "Updating xxenv-latest..."
git pull -q origin master
fi
popd
else
echo "Installing xxenv-latest..."
git clone -q https://github.com/momo-lab/xxenv-latest.git "${RBENV_INSTALL_DIR}/plugins/xxenv-latest"
fi
# Enable caching
mkdir -p "${RBENV_INSTALL_DIR}/cache"
# Create shims directory if needed
mkdir -p "${RBENV_INSTALL_DIR}/shims"
# Restore /usr/local/bin in PATH
PATH=$CLEAR_PATH
# And replace the old Ruby binary
test -x /usr/bin/ruby_rbenv && mv /usr/bin/ruby_rbenv /usr/bin/ruby
# Install the requested version of Ruby
local final_ruby_version=$(rbenv latest --print $ruby_version)
if ! [ -n "$final_ruby_version" ]; then
final_ruby_version=$ruby_version
fi
echo "Installing Ruby $final_ruby_version"
RUBY_CONFIGURE_OPTS="--disable-install-doc --with-jemalloc" MAKE_OPTS="-j2" rbenv install --skip-existing $final_ruby_version > /dev/null 2>&1
# Store ruby_version into the config of this app
ynh_app_setting_set --key=ruby_version --value=$final_ruby_version
ruby_version=$final_ruby_version
# Remove app virtualenv
if rbenv alias --list | grep --quiet "$app "
then
rbenv alias $app --remove
fi
# Create app virtualenv
rbenv alias $app $final_ruby_version
# Cleanup Ruby versions
_ynh_ruby_cleanup
# Set environment for Ruby users
echo "#rbenv
export RBENV_ROOT=$RBENV_INSTALL_DIR
export PATH=\"$RBENV_INSTALL_DIR/bin:$PATH\"
eval \"\$(rbenv init -)\"
#rbenv" > /etc/profile.d/rbenv.sh
# Load the environment
eval "$(rbenv init -)"
_ynh_load_ruby_in_path_and_other_tweaks
}
# Remove the version of Ruby used by the app.
#
# This helper will also cleanup unused Ruby versions
#
# usage: ynh_ruby_remove
ynh_ruby_remove () {
[[ -n "${ruby_version:-}" ]] || ynh_die "\$ruby_version should be defined prior to calling ynh_ruby_remove"
# Load rbenv path in PATH
local CLEAR_PATH="$RBENV_INSTALL_DIR/bin:$PATH"
# Remove /usr/local/bin in PATH in case of Ruby prior installation
PATH=$(echo $CLEAR_PATH | sed 's@/usr/local/bin:@@')
rbenv alias $app --remove
# Remove the line for this app
ynh_app_setting_delete --key=ruby_version
# Cleanup Ruby versions
_ynh_ruby_cleanup
}
# Remove no more needed versions of Ruby used by the app.
#
# [internal]
#
# This helper will check what Ruby version are no more required,
# and uninstall them
# If no app uses Ruby, rbenv will be also removed.
_ynh_ruby_cleanup () {
# List required Ruby versions
local installed_apps=$(yunohost app list | grep -oP 'id: \K.*$')
local required_ruby_versions=""
for installed_app in $installed_apps
do
local installed_app_ruby_version=$(ynh_app_setting_get --app=$installed_app --key="ruby_version")
if [[ -n "$installed_app_ruby_version" ]]
then
required_ruby_versions="${installed_app_ruby_version}\n${required_ruby_versions}"
fi
done
# Remove no more needed Ruby versions
local installed_ruby_versions=$(rbenv versions --bare --skip-aliases | grep -Ev '/')
for installed_ruby_version in $installed_ruby_versions
do
if ! echo ${required_ruby_versions} | grep -q "${installed_ruby_version}"
then
echo "Removing Ruby-$installed_ruby_version"
$RBENV_INSTALL_DIR/bin/rbenv uninstall --force $installed_ruby_version
fi
done
# If none Ruby version is required
if [[ -z "$required_ruby_versions" ]]
then
# Remove rbenv environment configuration
echo "Removing rbenv"
ynh_safe_rm "$RBENV_INSTALL_DIR"
ynh_safe_rm "/etc/profile.d/rbenv.sh"
fi
}
_ynh_ruby_try_bash_extension() {
if [ -x src/configure ]; then
src/configure && make -C src 2>&1 || {
ynh_print_info "Optional bash extension failed to build, but things will still work normally."
}
fi
}

124
helpers/helpers.v2.1.d/setting Normal file
View file

@ -0,0 +1,124 @@
#!/bin/bash
# Get an application setting
#
# usage: ynh_app_setting_get --key=key
# | arg: --app= - the application id (global $app by default)
# | arg: --key= - the setting to get
ynh_app_setting_get() {
# ============ Argument parsing =============
local _globalapp=${app-:}
local -A args_array=([a]=app= [k]=key=)
local app
local key
ynh_handle_getopts_args "$@"
app="${app:-$_globalapp}"
# ===========================================
ynh_app_setting "get" "$app" "$key"
}
# Set an application setting
#
# usage: ynh_app_setting_set --key=key --value=value
# | arg: --app= - the application id (global $app by default)
# | arg: --key= - the setting name to set
# | arg: --value= - the setting value to set
ynh_app_setting_set() {
# ============ Argument parsing =============
local _globalapp=${app-:}
local -A args_array=([a]=app= [k]=key= [v]=value=)
local app
local key
local value
ynh_handle_getopts_args "$@"
app="${app:-$_globalapp}"
# ===========================================
ynh_app_setting "set" "$app" "$key" "$value"
}
# Set an application setting but only if the "$key" variable ain't set yet
#
# Note that it doesn't just define the setting but ALSO define the $foobar variable
#
# Hence it's meant as a replacement for this legacy overly complex syntax:
#
# if [ -z "${foo:-}" ]
# then
# foo="bar"
# ynh_app_setting_set --key="foo" --value="$foo"
# fi
#
# usage: ynh_app_setting_set_default --key=key --value=value
# | arg: --app= - the application id (global $app by default)
# | arg: --key= - the setting name to set
# | arg: --value= - the default setting value to set
ynh_app_setting_set_default() {
# ============ Argument parsing =============
local _globalapp=${app-:}
local -A args_array=([a]=app= [k]=key= [v]=value=)
local app
local key
local value
ynh_handle_getopts_args "$@"
app="${app:-$_globalapp}"
# ===========================================
if [ -z "${!key:-}" ]; then
eval $key=\$value
ynh_app_setting "set" "$app" "$key" "$value"
fi
}
# Delete an application setting
#
# usage: ynh_app_setting_delete --key=key
# | arg: --app= - the application id (global $app by default)
# | arg: --key= - the setting to delete
ynh_app_setting_delete() {
# ============ Argument parsing =============
local _globalapp=${app-:}
local -A args_array=([a]=app= [k]=key=)
local app
local key
ynh_handle_getopts_args "$@"
app="${app:-$_globalapp}"
# ===========================================
ynh_app_setting "delete" "$app" "$key"
}
# Small "hard-coded" interface to avoid calling "yunohost app" directly each
# time dealing with a setting is needed (which may be so slow on ARM boards)
#
# [internal]
#
ynh_app_setting() {
# Trick to only re-enable debugging if it was set before
local xtrace_enable=$(set +o | grep xtrace)
set +o xtrace # set +x
ACTION="$1" APP="$2" KEY="$3" VALUE="${4:-}" python3 - <<EOF
import os, yaml, sys
app, action = os.environ['APP'], os.environ['ACTION'].lower()
key, value = os.environ['KEY'], os.environ.get('VALUE', None)
setting_file = "/etc/yunohost/apps/%s/settings.yml" % app
assert os.path.exists(setting_file), "Setting file %s does not exists ?" % setting_file
with open(setting_file) as f:
settings = yaml.safe_load(f)
if action == "get":
if key in settings:
print(settings[key])
else:
if action == "delete":
if key in settings:
del settings[key]
elif action == "set":
settings[key] = value
else:
raise ValueError("action should either be get, set or delete")
with open(setting_file, "w") as f:
yaml.safe_dump(settings, f, default_flow_style=False)
EOF
eval "$xtrace_enable"
}

261
helpers/helpers.v2.1.d/sources Normal file
View file

@ -0,0 +1,261 @@
#!/bin/bash
# Download, check integrity, uncompress and patch upstream sources
#
# usage: ynh_setup_source --dest_dir=dest_dir [--source_id=source_id] [--keep="file1 file2"] [--full_replace]
# | arg: --dest_dir= - Directory where to setup sources
# | arg: --source_id= - Name of the source, defaults to `main` (when the sources resource exists in manifest.toml) or (legacy) `app` otherwise
# | arg: --keep= - Space-separated list of files/folders that will be backup/restored in $dest_dir, such as a config file you don't want to overwrite. For example 'conf.json secrets.json logs' (no trailing `/` for folders)
# | arg: --full_replace= - Remove previous sources before installing new sources (can be 1 or 0, default to 0)
#
# ##### New 'sources' resources
#
# (See also the resources documentation which may be more complete?)
#
# This helper will read infos from the 'sources' resources in the manifest.toml of the app
# and expect a structure like:
#
# ```toml
# [resources.sources]
# [resources.sources.main]
# url = "https://some.address.to/download/the/app/archive"
# sha256 = "0123456789abcdef" # The sha256 sum of the asset obtained from the URL
# ```
#
# ##### Optional flags
#
# ```text
# format = "tar.gz"/xz/bz2 # automatically guessed from the extension of the URL, but can be set explicitly. Will use `tar` to extract
# "zip" # automatically guessed from the extension of the URL, but can be set explicitly. Will use `unzip` to extract
# "docker" # useful to extract files from an already-built docker image (instead of rebuilding them locally). Will use `docker-image-extract` to extract
# "whatever" # an arbitrary value, not really meaningful except to imply that the file won't be extracted
#
# in_subdir = true # default, there's an intermediate subdir in the archive before accessing the actual files
# false # sources are directly in the archive root
# n # (special cases) an integer representing a number of subdirs levels to get rid of
#
# extract = true # default if file is indeed an archive such as .zip, .tar.gz, .tar.bz2, ...
# = false # default if file 'format' is not set and the file is not to be extracted because it is not an archive but a script or binary or whatever asset.
# # in which case the file will only be `mv`ed to the location possibly renamed using the `rename` value
#
# rename = "whatever_your_want" # to be used for convenience when `extract` is false and the default name of the file is not practical
# platform = "linux/amd64" # (defaults to "linux/$YNH_ARCH") to be used in conjonction with `format = "docker"` to specify which architecture to extract for
# ```
#
# You may also define assets url and checksum per-architectures such as:
# ```toml
# [resources.sources]
# [resources.sources.main]
# amd64.url = "https://some.address.to/download/the/app/archive/when/amd64"
# amd64.sha256 = "0123456789abcdef"
# armhf.url = "https://some.address.to/download/the/app/archive/when/armhf"
# armhf.sha256 = "fedcba9876543210"
# ```
#
# In which case `ynh_setup_source --dest_dir="$install_dir"` will automatically pick the appropriate source depending on the arch
#
# The helper will:
# - Download the specific URL if there is no local archive
# - Check the integrity with the specific sha256 sum
# - Uncompress the archive to `$dest_dir`.
# - If `in_subdir` is true, the first level directory of the archive will be removed.
# - If `in_subdir` is a numeric value, the N first level directories will be removed.
# - Patches named `patches/${src_id}-*.patch` will be applied to `$dest_dir`
ynh_setup_source() {
# ============ Argument parsing =============
local -A args_array=([d]=dest_dir= [s]=source_id= [k]=keep= [r]=full_replace)
local dest_dir
local source_id
local keep
local full_replace
ynh_handle_getopts_args "$@"
keep="${keep:-}"
full_replace="${full_replace:-0}"
source_id="${source_id:-main}"
# ===========================================
local sources_json=$(ynh_read_manifest "resources.sources[\"$source_id\"]")
if jq -re ".url" <<< "$sources_json"
then
local arch_prefix=""
else
local arch_prefix=".$YNH_ARCH"
fi
local src_url="$(jq -r "$arch_prefix.url" <<< "$sources_json" | sed 's/^null$//')"
local src_sum="$(jq -r "$arch_prefix.sha256" <<< "$sources_json" | sed 's/^null$//')"
local src_format="$(jq -r ".format" <<< "$sources_json" | sed 's/^null$//')"
local src_in_subdir="$(jq -r ".in_subdir" <<< "$sources_json" | sed 's/^null$//')"
src_in_subdir=${src_in_subdir:-true}
local src_extract="$(jq -r ".extract" <<< "$sources_json" | sed 's/^null$//')"
local src_platform="$(jq -r ".platform" <<< "$sources_json" | sed 's/^null$//')"
local src_rename="$(jq -r ".rename" <<< "$sources_json" | sed 's/^null$//')"
[[ -n "$src_url" ]] || ynh_die "No URL defined for source $source_id$arch_prefix ?"
[[ -n "$src_sum" ]] || ynh_die "No sha256 sum defined for source $source_id$arch_prefix ?"
if [[ -z "$src_format" ]]
then
if [[ "$src_url" =~ ^.*\.zip$ ]] || [[ "$src_url" =~ ^.*/zipball/.*$ ]]
then
src_format="zip"
elif [[ "$src_url" =~ ^.*\.tar\.gz$ ]] || [[ "$src_url" =~ ^.*\.tgz$ ]] || [[ "$src_url" =~ ^.*/tar\.gz/.*$ ]] || [[ "$src_url" =~ ^.*/tarball/.*$ ]]
then
src_format="tar.gz"
elif [[ "$src_url" =~ ^.*\.tar\.xz$ ]]
then
src_format="tar.xz"
elif [[ "$src_url" =~ ^.*\.tar\.bz2$ ]]
then
src_format="tar.bz2"
elif [[ -z "$src_extract" ]]
then
src_extract="false"
fi
fi
src_format=${src_format:-tar.gz}
src_format=$(echo "$src_format" | tr '[:upper:]' '[:lower:]')
src_extract=${src_extract:-true}
if [[ "$src_extract" != "true" ]] && [[ "$src_extract" != "false" ]]
then
ynh_die "For source $source_id, expected either 'true' or 'false' for the extract parameter"
fi
# Gotta use this trick with 'dirname' because source_id may contain slashes x_x
mkdir -p $(dirname /var/cache/yunohost/download/${YNH_APP_ID}/${source_id})
src_filename="/var/cache/yunohost/download/${YNH_APP_ID}/${source_id}"
if [ "$src_format" = "docker" ]; then
src_platform="${src_platform:-"linux/$YNH_ARCH"}"
else
[ -n "$src_url" ] || ynh_die "Couldn't parse SOURCE_URL from $src_file_path ?"
# If the file was prefetched but somehow doesn't match the sum, rm and redownload it
if [ -e "$src_filename" ] && ! echo "${src_sum} ${src_filename}" | sha256sum --check --status
then
rm -f "$src_filename"
fi
# Only redownload the file if it wasnt prefetched
if [ ! -e "$src_filename" ]
then
# NB. we have to declare the var as local first,
# otherwise 'local foo=$(false) || echo 'pwet'" does'nt work
# because local always return 0 ...
local out
# Timeout option is here to enforce the timeout on dns query and tcp connect (c.f. man wget)
out=$(wget --tries 3 --no-dns-cache --timeout 900 --no-verbose --output-document=$src_filename $src_url 2>&1) \
|| ynh_die "$out"
fi
# Check the control sum
if ! echo "${src_sum} ${src_filename}" | sha256sum --check --status
then
local actual_sum="$(sha256sum ${src_filename} | cut --delimiter=' ' --fields=1)"
local actual_size="$(du -hs ${src_filename} | cut --fields=1)"
rm -f ${src_filename}
ynh_die "Corrupt source for ${src_url}: Expected sha256sum to be ${src_sum} but got ${actual_sum} (size: ${actual_size})."
fi
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
if [ -n "$keep" ] && [ -e "$dest_dir" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
mkdir -p $keep_dir
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$dest_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$keep_dir/$stuff_to_keep")"
cp --archive "$dest_dir/$stuff_to_keep" "$keep_dir/$stuff_to_keep"
fi
done
fi
if [ "$full_replace" -eq 1 ]; then
ynh_safe_rm "$dest_dir"
fi
# Extract source into the app dir
mkdir --parents "$dest_dir"
if [[ "$src_extract" == "false" ]]; then
if [[ -z "$src_rename" ]]
then
mv $src_filename $dest_dir
else
mv $src_filename $dest_dir/$src_rename
fi
elif [[ "$src_format" == "docker" ]]; then
"$YNH_HELPERS_DIR/vendor/docker-image-extract/docker-image-extract" -p $src_platform -o $dest_dir $src_url 2>&1
elif [[ "$src_format" == "zip" ]]; then
# Zip format
# Using of a temp directory, because unzip doesn't manage --strip-components
if $src_in_subdir; then
local tmp_dir=$(mktemp --directory)
unzip -quo $src_filename -d "$tmp_dir"
cp --archive $tmp_dir/*/. "$dest_dir"
ynh_safe_rm "$tmp_dir"
else
unzip -quo $src_filename -d "$dest_dir"
fi
ynh_safe_rm "$src_filename"
else
local strip=""
if [ "$src_in_subdir" != "false" ]; then
if [ "$src_in_subdir" == "true" ]; then
local sub_dirs=1
else
local sub_dirs="$src_in_subdir"
fi
strip="--strip-components $sub_dirs"
fi
if [[ "$src_format" =~ ^tar.gz|tar.bz2|tar.xz$ ]]; then
tar --extract --file=$src_filename --directory="$dest_dir" $strip
else
ynh_die "Archive format unrecognized."
fi
ynh_safe_rm "$src_filename"
fi
# Apply patches
if [ -d "$YNH_APP_BASEDIR/patches/" ]; then
local patches_folder=$(realpath $YNH_APP_BASEDIR/patches/)
# Check if any file matching the pattern exists, cf https://stackoverflow.com/a/34195247
if compgen -G "$patches_folder/${source_id}-*.patch" >/dev/null; then
pushd "$dest_dir"
for p in $patches_folder/${source_id}-*.patch; do
echo $p
patch --strip=1 <$p || ynh_print_warn "Packagers /!\\ patch $p failed to apply"
done
popd
fi
fi
# Keep files to be backup/restored at the end of the helper
# Assuming $dest_dir already exists
if [ -n "$keep" ]; then
local keep_dir=/var/cache/yunohost/files_to_keep_during_setup_source/${YNH_APP_ID}
local stuff_to_keep
for stuff_to_keep in $keep; do
if [ -e "$keep_dir/$stuff_to_keep" ]; then
mkdir --parents "$(dirname "$dest_dir/$stuff_to_keep")"
# We add "--no-target-directory" (short option is -T) to handle the special case
# when we "keep" a folder, but then the new setup already contains the same dir (but possibly empty)
# in which case a regular "cp" will create a copy of the directory inside the directory ...
# resulting in something like /var/www/$app/data/data instead of /var/www/$app/data
# cf https://unix.stackexchange.com/q/94831 for a more elaborate explanation on the option
cp --archive --no-target-directory "$keep_dir/$stuff_to_keep" "$dest_dir/$stuff_to_keep"
fi
done
fi
rm -rf /var/cache/yunohost/files_to_keep_during_setup_source/
if [ -n "${install_dir:-}" ] && [ "$dest_dir" == "$install_dir" ]; then
_ynh_apply_default_permissions $dest_dir
fi
}

129
helpers/helpers.v2.1.d/string Normal file
View file

@ -0,0 +1,129 @@
#!/bin/bash
# Generate a random string
#
# usage: ynh_string_random [--length=string_length]
# | arg: --length= - the string length to generate (default: 24)
# | arg: --filter= - the kind of characters accepted in the output (default: 'A-Za-z0-9')
# | ret: the generated string
#
# example: pwd=$(ynh_string_random --length=8)
ynh_string_random() {
# ============ Argument parsing =============
local -A args_array=([l]=length= [f]=filter=)
local length
local filter
ynh_handle_getopts_args "$@"
length=${length:-24}
filter=${filter:-'A-Za-z0-9'}
# ===========================================
dd if=/dev/urandom bs=1 count=1000 2>/dev/null \
| tr --complement --delete "$filter" \
| sed --quiet 's/\(.\{'"$length"'\}\).*/\1/p'
}
# Substitute/replace a string (or expression) by another in a file
#
# usage: ynh_replace --match=match --replace=replace --file=file
# | arg: --match= - String to be searched and replaced in the file
# | arg: --replace= - String that will replace matches
# | arg: --file= - File in which the string will be replaced.
#
# As this helper is based on sed command, regular expressions and references to
# sub-expressions can be used (see sed manual page for more information)
ynh_replace() {
# ============ Argument parsing =============
local -A args_array=([m]=match= [r]=replace= [f]=file=)
local match
local replace
local file
ynh_handle_getopts_args "$@"
# ===========================================
set +o xtrace # set +x
local delimit=$'\001'
# Escape the delimiter if it's in the string.
match=${match//${delimit}/"\\${delimit}"}
replace=${replace//${delimit}/"\\${delimit}"}
set -o xtrace # set -x
sed --in-place "s${delimit}${match}${delimit}${replace}${delimit}g" "$file"
}
# Substitute/replace a regex in a file
#
# usage: ynh_replace_regex --match=match --replace=replace --file=file
# | arg: --match= - String to be searched and replaced in the file
# | arg: --replace= - String that will replace matches
# | arg: --file= - File in which the string will be replaced.
#
# This helper will use ynh_replace, but as you can use special
# characters, you can't use some regular expressions and sub-expressions.
ynh_replace_regex() {
# ============ Argument parsing =============
local -A args_array=([m]=match= [r]=replace= [f]=file=)
local match
local replace
local file
ynh_handle_getopts_args "$@"
# ===========================================
# Escape any backslash to preserve them as simple backslash.
match=${match//\\/"\\\\"}
replace=${replace//\\/"\\\\"}
# Escape the & character, who has a special function in sed.
match=${match//&/"\&"}
replace=${replace//&/"\&"}
ynh_replace --match="$match" --replace="$replace" --file="$file"
}
# Sanitize a string intended to be the name of a database
#
# [packagingv1]
#
# usage: ynh_sanitize_dbid --db_name=name
# | arg: --db_name= - name to correct/sanitize
# | ret: the corrected name
#
# example: dbname=$(ynh_sanitize_dbid $app)
#
# Underscorify the string (replace - and . by _)
ynh_sanitize_dbid() {
# ============ Argument parsing =============
local -A args_array=([n]=db_name=)
local db_name
ynh_handle_getopts_args "$@"
# ===========================================
# We should avoid having - and . in the name of databases. They are replaced by _
echo ${db_name//[-.]/_}
}
# Normalize the url path syntax
#
# Handle the slash at the beginning of path and its absence at ending
# Return a normalized url path
#
# examples:
# url_path=$(ynh_normalize_url_path $url_path)
# ynh_normalize_url_path example # -> /example
# ynh_normalize_url_path /example # -> /example
# ynh_normalize_url_path /example/ # -> /example
# ynh_normalize_url_path / # -> /
#
# usage: ynh_normalize_url_path path_to_normalize
ynh_normalize_url_path() {
local path_url=$1
test -n "$path_url" || ynh_die "ynh_normalize_url_path expect a URL path as first argument and received nothing."
if [ "${path_url:0:1}" != "/" ]; then # If the first character is not a /
path_url="/$path_url" # Add / at begin of path variable
fi
if [ "${path_url:${#path_url}-1}" == "/" ] && [ ${#path_url} -gt 1 ]; then # If the last character is a / and that not the only character.
path_url="${path_url:0:${#path_url}-1}" # Delete the last character
fi
echo $path_url
}

181
helpers/helpers.v2.1.d/systemd Normal file
View file

@ -0,0 +1,181 @@
#!/bin/bash
# Create a dedicated systemd config
#
# usage: ynh_config_add_systemd [--service=service] [--template=template]
# | arg: --service= - Service name (optionnal, `$app` by default)
# | arg: --template= - Name of template file (optionnal, this is 'systemd' by default, meaning `../conf/systemd.service` will be used as template)
#
# This will use the template `../conf/<templatename>.service`.
#
# See the documentation of `ynh_config_add` for a description of the template
# format and how placeholders are replaced with actual variables.
ynh_config_add_systemd() {
# ============ Argument parsing =============
local -A args_array=([s]=service= [t]=template=)
local service
local template
ynh_handle_getopts_args "$@"
service="${service:-$app}"
template="${template:-systemd.service}"
# ===========================================
ynh_config_add --template="$template" --destination="/etc/systemd/system/$service.service"
systemctl enable $service --quiet
systemctl daemon-reload
}
# Remove the dedicated systemd config
#
# usage: ynh_config_remove_systemd service
# | arg: service - Service name (optionnal, $app by default)
ynh_config_remove_systemd() {
local service="${1:-$app}"
if [ -e "/etc/systemd/system/$service.service" ]; then
ynh_systemctl --service=$service --action=stop
systemctl disable $service --quiet
ynh_safe_rm "/etc/systemd/system/$service.service"
systemctl daemon-reload
fi
}
# Start (or other actions) a service, print a log in case of failure and optionnaly wait until the service is completely started
#
# usage: ynh_systemctl [--service=service] [--action=action] [ [--wait_until="line to match"] [--log_path=log_path] [--timeout=300] [--length=20] ]
# | arg: --service= - Name of the service to start. Default : `$app`
# | arg: --action= - Action to perform with systemctl. Default: start
# | arg: --wait_until= - The pattern to find in the log to attest the service is effectively fully started.
# | arg: --log_path= - Log file - Path to the log file. Default : `/var/log/$app/$app.log`
# | arg: --timeout= - Timeout - The maximum time to wait before ending the watching. Default : 60 seconds.
# | arg: --length= - Length of the error log displayed for debugging : Default : 20
ynh_systemctl() {
# ============ Argument parsing =============
local -A args_array=([n]=service= [a]=action= [w]=wait_until= [p]=log_path= [t]=timeout= [e]=length=)
local service
local action
local wait_until
local length
local log_path
local timeout
ynh_handle_getopts_args "$@"
service="${service:-$app}"
action=${action:-start}
wait_until=${wait_until:-}
length=${length:-20}
log_path="${log_path:-/var/log/$service/$service.log}"
timeout=${timeout:-60}
# ===========================================
# On CI, use length=100 because it's sometime hell to debug otherwise for super-long output
if ynh_in_ci_tests && [ $length -le 20 ]
then
length=100
fi
# Manage case of service already stopped
if [ "$action" == "stop" ] && ! systemctl is-active --quiet $service; then
return 0
fi
# Start to read the log
if [[ -n "$wait_until" ]]; then
local templog="$(mktemp)"
# Following the starting of the app in its log
if [ "$log_path" == "systemd" ]; then
# Read the systemd journal
journalctl --unit=$service --follow --since=-0 --quiet >"$templog" &
# Get the PID of the journalctl command
local pid_tail=$!
else
# Read the specified log file
tail --follow=name --retry --lines=0 "$log_path" >"$templog" 2>&1 &
# Get the PID of the tail command
local pid_tail=$!
fi
fi
# Use reload-or-restart instead of reload. So it wouldn't fail if the service isn't running.
if [ "$action" == "reload" ]; then
action="reload-or-restart"
fi
local time_start="$(date --utc --rfc-3339=seconds | cut -d+ -f1) UTC"
# If the service fails to perform the action
if ! systemctl $action $service; then
# Show syslog for this service
journalctl --quiet --no-hostname --no-pager --lines=$length --unit=$service >&2
# If a log is specified for this service, show also the content of this log
if [ -e "$log_path" ]; then
tail --lines=$length "$log_path" >&2
fi
_ynh_clean_check_starting
return 1
fi
# Start the timeout and try to find wait_until
if [[ -n "${wait_until:-}" ]]; then
set +o xtrace # set +x
local i=0
local starttime=$(date +%s)
for i in $(seq 1 $timeout); do
# Read the log until the sentence is found, that means the app finished to start. Or run until the timeout
if [ "$log_path" == "systemd" ]; then
# For systemd services, we in fact dont rely on the templog, which for some reason is not reliable, but instead re-read journalctl every iteration, starting at the timestamp where we triggered the action
if journalctl --unit=$service --since="$time_start" --quiet --no-pager --no-hostname | grep --extended-regexp --quiet "$wait_until"; then
ynh_print_info "The service $service has correctly executed the action ${action}."
break
fi
else
if grep --extended-regexp --quiet "$wait_until" "$templog"; then
ynh_print_info "The service $service has correctly executed the action ${action}."
break
fi
fi
if [ $i -eq 30 ]; then
echo "(this may take some time)" >&2
fi
# Also check the timeout using actual timestamp, because sometimes for some reason,
# journalctl may take a huge time to run, and we end up waiting literally an entire hour
# instead of 5 min ...
if [[ "$(( $(date +%s) - $starttime))" -gt "$timeout" ]]
then
i=$timeout
break
fi
sleep 1
done
set -o xtrace # set -x
if [ $i -ge 3 ]; then
echo "" >&2
fi
if [ $i -eq $timeout ]; then
ynh_print_warn "The service $service didn't fully executed the action ${action} before the timeout."
ynh_print_warn "Please find here an extract of the end of the log of the service $service:"
journalctl --quiet --no-hostname --no-pager --lines=$length --unit=$service >&2
if [ -e "$log_path" ]; then
ynh_print_warn "==="
tail --lines=$length "$log_path" >&2
fi
# If we tried to reload/start/restart the service but systemctl consider it to be still inactive/broken, then handle it as a failure
if ([ "$action" == "reload" ] || [ "$action" == "start" ] || [ "$action" == "restart" ]) && ! systemctl --quiet is-active $service
then
_ynh_clean_check_starting
return 1
fi
fi
_ynh_clean_check_starting
fi
}
_ynh_clean_check_starting() {
if [ -n "${pid_tail:-}" ]; then
# Stop the execution of tail.
kill -SIGTERM $pid_tail 2>&1
fi
if [ -n "${templog:-}" ]; then
ynh_safe_rm "$templog" 2>&1
fi
}

105
helpers/helpers.v2.1.d/systemuser Normal file
View file

@ -0,0 +1,105 @@
#!/bin/bash
# Check if a user exists on the system
#
# usage: ynh_system_user_exists --username=username
# | arg: --username= - the username to check
# | ret: 0 if the user exists, 1 otherwise.
ynh_system_user_exists() {
# ============ Argument parsing =============
local -A args_array=([u]=username=)
local username
ynh_handle_getopts_args "$@"
# ===========================================
getent passwd "$username" &>/dev/null
}
# Check if a group exists on the system
#
# usage: ynh_system_group_exists --group=group
# | arg: --group= - the group to check
# | ret: 0 if the group exists, 1 otherwise.
ynh_system_group_exists() {
# ============ Argument parsing =============
local -A args_array=([g]=group=)
local group
ynh_handle_getopts_args "$@"
# ===========================================
getent group "$group" &>/dev/null
}
# Create a system user
#
# usage: ynh_system_user_create --username=user_name [--home_dir=home_dir] [--use_shell] [--groups="group1 group2"]
# | arg: --username= - Name of the system user that will be create
# | arg: --home_dir= - Path of the home dir for the user. Usually the final path of the app. If this argument is omitted, the user will be created without home
# | arg: --use_shell - Create a user using the default login shell if present. If this argument is omitted, the user will be created with /usr/sbin/nologin shell
# | arg: --groups - Add the user to system groups. Typically meant to add the user to the ssh.app / sftp.app group (e.g. for borgserver, my_webapp)
#
# Create a nextcloud user with no home directory and /usr/sbin/nologin login shell (hence no login capability) :
# ```
# ynh_system_user_create --username=nextcloud
# ```
# Create a discourse user using /var/www/discourse as home directory and the default login shell :
# ```
# ynh_system_user_create --username=discourse --home_dir=/var/www/discourse --use_shell
# ```
ynh_system_user_create() {
# ============ Argument parsing =============
local -A args_array=([u]=username= [h]=home_dir= [s]=use_shell [g]=groups=)
local username
local home_dir
local use_shell
local groups
ynh_handle_getopts_args "$@"
use_shell="${use_shell:-0}"
home_dir="${home_dir:-}"
groups="${groups:-}"
# ===========================================
if ! ynh_system_user_exists --username="$username"; then # Check if the user exists on the system
# If the user doesn't exist
if [ -n "$home_dir" ]; then # If a home dir is mentioned
local user_home_dir="--home-dir $home_dir"
else
local user_home_dir="--no-create-home"
fi
if [ $use_shell -eq 1 ]; then # If we want a shell for the user
local shell="" # Use default shell
else
local shell="--shell /usr/sbin/nologin"
fi
useradd $user_home_dir --system --user-group $username $shell || ynh_die "Unable to create $username system account"
fi
local group
for group in $groups; do
usermod -a -G "$group" "$username"
done
}
# Delete a system user
#
# usage: ynh_system_user_delete --username=user_name
# | arg: --username= - Name of the system user that will be create
ynh_system_user_delete() {
# ============ Argument parsing =============
local -A args_array=([u]=username=)
local username
ynh_handle_getopts_args "$@"
# ===========================================
# Check if the user exists on the system
if ynh_system_user_exists --username="$username"; then
deluser $username
else
ynh_print_warn "The user $username was not found"
fi
# Check if the group exists on the system
if ynh_system_group_exists --group="$username"; then
delgroup $username
fi
}

333
helpers/helpers.v2.1.d/templating Normal file
View file

@ -0,0 +1,333 @@
#!/bin/bash
# Create a dedicated config file from a template
#
# usage: ynh_config_add --template="template" --destination="destination"
# | arg: --template= - Template config file to use
# | arg: --destination= - Destination of the config file
# | arg: --jinja - Use jinja template instead of the simple `__MY_VAR__` templating format
#
# examples:
# ynh_add_config --template=".env" --destination="$install_dir/.env" # (use the template file "conf/.env" from the app's package)
# ynh_add_config --jinja --template="config.j2" --destination="$install_dir/config" # (use the template file "conf/config.j2" from the app's package)
#
# The template can be 1) the name of a file in the `conf` directory of
# the app, 2) a relative path or 3) an absolute path.
#
# This applies a simple templating format which covers a good 95% of cases,
# where patterns like `__FOO__` are replaced by the bash variable `$foo`, for example:
# `__DOMAIN__` by `$domain`
# `__PATH__` by `$path`
# `__APP__` by `$app`
# `__VAR_1__` by `$var_1`
# `__VAR_2__` by `$var_2`
#
# Special case for `__PATH__/` which is replaced by `/` instead of `//` if `$path` is `/`
#
# ##### When --jinja is enabled
#
# This option is meant for advanced use-cases where the "simple" templating
# mode ain't enough because you need conditional blocks or loops.
#
# For a full documentation of jinja's syntax you can refer to:
# https://jinja.palletsprojects.com/en/3.1.x/templates/
#
# Note that in YunoHost context, all variables are from shell variables and therefore are strings
#
# ##### Keeping track of manual changes by the admin
#
# The helper will verify the checksum and backup the destination file
# if it's different before applying the new template.
#
# And it will calculate and store the destination file checksum
# into the app settings when configuration is done.
ynh_config_add() {
# ============ Argument parsing =============
local -A args_array=([t]=template= [d]=destination= [j]=jinja)
local template
local destination
local jinja
ynh_handle_getopts_args "$@"
jinja="${jinja:-0}"
# ===========================================
local template_path
if [ -f "$YNH_APP_BASEDIR/conf/$template" ]; then
template_path="$YNH_APP_BASEDIR/conf/$template"
elif [ -f "$template" ]; then
template_path=$template
else
ynh_die "The provided template $template doesn't exist"
fi
ynh_backup_if_checksum_is_different "$destination"
# Make sure to set the permissions before we copy the file
# This is to cover a case where an attacker could have
# created a file beforehand to have control over it
# (cp won't overwrite ownership / modes by default...)
touch $destination
chmod 640 $destination
_ynh_apply_default_permissions $destination
if [[ "$jinja" == 1 ]]
then
# This is ran in a subshell such that the "export" does not "contaminate" the main process
(
export $(compgen -v)
j2 "$template_path" -f env -o $destination
)
else
cp -f "$template_path" "$destination"
_ynh_replace_vars "$destination"
fi
ynh_store_file_checksum "$destination"
}
# Replace `__FOO__` patterns in file with bash variable `$foo`
#
# [internal]
#
# usage: ynh_replace_vars "/path/to/file"
# | arg: /path/to/file - File where to replace variables
#
# This applies a simple templating format which covers a good 95% of cases,
# where patterns like `__FOO__` are replaced by the bash variable `$foo`, for example:
# `__DOMAIN__` by `$domain`
# `__PATH__` by `$path`
# `__APP__` by `$app`
# `__VAR_1__` by `$var_1`
# `__VAR_2__` by `$var_2`
#
# Special case for `__PATH__/` which is replaced by `/` instead of `//` if `$path` is `/`
_ynh_replace_vars() {
local file=$1
# List unique (__ __) variables in $file
local uniques_vars=($(grep -oP '__[A-Z0-9]+?[A-Z0-9_]*?[A-Z0-9]*?__' $file | sort --unique | sed "s@__\([^.]*\)__@\L\1@g"))
set +o xtrace # set +x
# Specific trick to make sure that __PATH__/ doesn't end up in "//" if $path=/
if [[ "${path:-}" == "/" ]] && grep -q '__PATH__/' $file; then
sed --in-place "s@__PATH__/@/@g" "$file"
fi
# Do the replacement
local delimit=@
for one_var in "${uniques_vars[@]}"; do
# Validate that one_var is indeed defined
# -v checks if the variable is defined, for example:
# -v FOO tests if $FOO is defined
# -v $FOO tests if ${!FOO} is defined
# More info: https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash/17538964#comment96392525_17538964
[[ -v "${one_var:-}" ]] || ynh_die "Variable \$$one_var wasn't initialized when trying to replace __${one_var^^}__ in $file"
# Escape delimiter in match/replace string
match_string="__${one_var^^}__"
match_string=${match_string//${delimit}/"\\${delimit}"}
replace_string="${!one_var}"
replace_string=${replace_string//\\/\\\\}
replace_string=${replace_string//${delimit}/"\\${delimit}"}
# Actually replace (sed is used instead of ynh_replace_string to avoid triggering an epic amount of debug logs)
sed --in-place "s${delimit}${match_string}${delimit}${replace_string}${delimit}g" "$file"
done
set -o xtrace # set -x
}
# Get a value from heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_read_var_in_file --file=PATH --key=KEY
# | arg: --file= - the path to the file
# | arg: --key= - the key to get
# | arg: --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
#
# This helpers match several var affectation use case in several languages
# We don't use jq or equivalent to keep comments and blank space in files
# This helpers work line by line, it is not able to work correctly
# if you have several identical keys in your files
#
# Example of line this helpers can managed correctly
# .yml
# title: YunoHost documentation
# email: 'yunohost@yunohost.org'
# .json
# "theme": "colib'ris",
# "port": 8102
# "some_boolean": false,
# "user": null
# .ini
# some_boolean = On
# action = "Clear"
# port = 20
# .php
# $user=
# user => 20
# .py
# USER = 8102
# user = 'https://donate.local'
# CUSTOM['user'] = 'YunoHost'
ynh_read_var_in_file() {
# ============ Argument parsing =============
local -A args_array=([f]=file= [k]=key= [a]=after=)
local file
local key
local after
ynh_handle_getopts_args "$@"
after="${after:-}"
# ===========================================
[[ -f $file ]] || ynh_die "File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local line_number=1
if [[ -n "$after" ]]; then
line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
echo YNH_NULL
return 0
fi
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
local first_char="${expression:0:1}"
if [[ "$first_char" == '"' ]]; then
echo "$expression" | grep -m1 -o -P '"\K([^"](\\")?)*[^\\](?=")' | head -n1 | sed 's/\\"/"/g'
elif [[ "$first_char" == "'" ]]; then
echo "$expression" | grep -m1 -o -P "'\K([^'](\\\\')?)*[^\\\\](?=')" | head -n1 | sed "s/\\\\'/'/g"
else
echo "$expression"
fi
set -o xtrace # set -x
}
# Set a value into heterogeneous file (yaml, json, php, python...)
#
# usage: ynh_write_var_in_file --file=PATH --key=KEY --value=VALUE
# | arg: --file= - the path to the file
# | arg: --key= - the key to set
# | arg: --value= - the value to set
# | arg: --after= - the line just before the key (in case of multiple lines with the name of the key in the file)
ynh_write_var_in_file() {
# ============ Argument parsing =============
local -A args_array=([f]=file= [k]=key= [v]=value= [a]=after=)
local file
local key
local value
local after
ynh_handle_getopts_args "$@"
after="${after:-}"
# ===========================================
[[ -f $file ]] || ynh_die "File $file does not exists"
set +o xtrace # set +x
# Get the line number after which we search for the variable
local after_line_number=1
if [[ -n "$after" ]]; then
after_line_number=$(grep -m1 -n $after $file | cut -d: -f1)
if [[ -z "$after_line_number" ]]; then
set -o xtrace # set -x
return 1
fi
fi
local filename="$(basename -- "$file")"
local ext="${filename##*.}"
local endline=',;'
local assign="=>|:|="
local comments="#"
local string="\"'"
if [[ "$ext" =~ ^ini|env|toml|yml|yaml$ ]]; then
endline='#'
fi
if [[ "$ext" =~ ^ini|env$ ]]; then
comments="[;#]"
fi
if [[ "php" == "$ext" ]] || [[ "$ext" == "js" ]]; then
comments="//"
fi
local list='\[\s*['$string']?\w+['$string']?\]'
local var_part='^\s*((const|var|let)\s+)?\$?(\w+('$list')*(->|\.|\[))*\s*'
var_part+="[$string]?${key}[$string]?"
var_part+='\s*\]?\s*'
var_part+="($assign)"
var_part+='\s*'
# Extract the part after assignation sign
local expression_with_comment="$((tail +$after_line_number ${file} | grep -i -o -P $var_part'\K.*$' || echo YNH_NULL) | head -n1)"
if [[ "$expression_with_comment" == "YNH_NULL" ]]; then
set -o xtrace # set -x
return 1
fi
local value_line_number="$(tail +$after_line_number ${file} | grep -m1 -n -i -P $var_part'\K.*$' | cut -d: -f1)"
value_line_number=$((after_line_number + value_line_number))
local range="${after_line_number},${value_line_number} "
# Remove comments if needed
local expression="$(echo "$expression_with_comment" | sed "s@${comments}[^$string]*\$@@g" | sed "s@\s*[$endline]*\s*]*\$@@")"
endline=${expression_with_comment#"$expression"}
endline="$(echo "$endline" | sed 's/\\/\\\\/g')"
value="$(echo "$value" | sed 's/\\/\\\\/g')"
value=${value//&/"\&"}
local first_char="${expression:0:1}"
delimiter=$'\001'
if [[ "$first_char" == '"' ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# So we need \\\\ to go through 2 sed
value="$(echo "$value" | sed 's/"/\\\\"/g')"
sed -ri "${range}s$delimiter"'(^'"${var_part}"'")([^"]|\\")*("[\s;,]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}"'"'"${endline}${delimiter}i" ${file}
elif [[ "$first_char" == "'" ]]; then
# \ and sed is quite complex you need 2 \\ to get one in a sed
# However double quotes implies to double \\ to
# So we need \\\\\\\\ to go through 2 sed and 1 double quotes str
value="$(echo "$value" | sed "s/'/\\\\\\\\'/g")"
sed -ri "${range}s$delimiter(^${var_part}')([^']|\\')*('"'[\s,;]*)(\s*'$comments'.*)?$'$delimiter'\1'"${value}'${endline}${delimiter}i" ${file}
else
if [[ "$value" == *"'"* ]] || [[ "$value" == *'"'* ]] || [[ "$ext" =~ ^php|py|json|js$ ]]; then
value='\"'"$(echo "$value" | sed 's/"/\\\\"/g')"'\"'
fi
if [[ "$ext" =~ ^yaml|yml$ ]]; then
value=" $value"
fi
sed -ri "${range}s$delimiter(^${var_part}).*\$$delimiter\1${value}${endline}${delimiter}i" ${file}
fi
set -o xtrace # set -x
}

395
helpers/helpers.v2.1.d/utils Normal file
View file

@ -0,0 +1,395 @@
#!/bin/bash
YNH_APP_BASEDIR=${YNH_APP_BASEDIR:-$(realpath ..)}
# Handle script crashes / failures
#
# [internal]
#
# usage:
# ynh_exit_properly is used only by the helper ynh_abort_if_errors.
# You should not use it directly.
# Instead, add to your script:
# ynh_clean_setup () {
# instructions...
# }
#
# This function provide a way to clean some residual of installation that not managed by remove script.
#
# It prints a warning to inform that the script was failed, and execute the ynh_clean_setup function if used in the app script
ynh_exit_properly() {
local exit_code=$?
if [[ "${YNH_APP_ACTION:-}" =~ ^install$|^upgrade$|^restore$ ]]
then
rm -rf "/var/cache/yunohost/download/"
fi
if [ "$exit_code" -eq 0 ]; then
exit 0 # Exit without error if the script ended correctly
fi
trap '' EXIT # Ignore new exit signals
# Do not exit anymore if a command fail or if a variable is empty
set +o errexit # set +e
set +o nounset # set +u
# Small tempo to avoid the next message being mixed up with other DEBUG messages
sleep 0.5
if type -t ynh_clean_setup >/dev/null; then # Check if the function exist in the app script.
ynh_clean_setup # Call the function to do specific cleaning for the app.
fi
# Exit with error status
# We don't call ynh_die basically to avoid unecessary 10-ish
# debug lines about parsing args and stuff just to exit 1..
exit 1
}
# Exits if an error occurs during the execution of the script.
#
# [packagingv1]
#
# usage: ynh_abort_if_errors
#
# This configure the rest of the script execution such that, if an error occurs
# or if an empty variable is used, the execution of the script stops immediately
# and a call to `ynh_clean_setup` is triggered if it has been defined by your script.
ynh_abort_if_errors() {
set -o errexit # set -e; Exit if a command fail
set -o nounset # set -u; And if a variable is used unset
trap ynh_exit_properly EXIT # Capturing exit signals on shell script
}
# When running an app script, auto-enable ynh_abort_if_errors except for remove script
if [[ "${YNH_CONTEXT:-}" != "regenconf" ]] && [[ "${YNH_APP_ACTION}" != "remove" ]]
then
ynh_abort_if_errors
fi
# Execute a command after sudoing as $app
#
# Note that the $PATH variable is preserved (using --preserve-env=PATH)
#
# usage: ynh_exec_as_app COMMAND [ARG ...]
ynh_exec_as_app() {
sudo --preserve-env=PATH -u "$app" "$@"
}
# Curl abstraction to help with POST requests to local pages (such as installation forms)
#
# usage: ynh_local_curl "page_uri" "key1=value1" "key2=value2" ...
# | arg: page_uri - Path (relative to `$path`) of the page where POST data will be sent
# | arg: key1=value1 - (Optionnal) POST key and corresponding value
# | arg: key2=value2 - (Optionnal) Another POST key and corresponding value
# | arg: ... - (Optionnal) More POST keys and values
#
# example: ynh_local_curl "/install.php?installButton" "foo=$var1" "bar=$var2"
#
# For multiple calls, cookies are persisted between each call for the same app
#
# `$domain` and `$path` should be defined externally (and correspond to the domain.tld and the /path (of the app?))
ynh_local_curl() {
# Define url of page to curl
local local_page=$(ynh_normalize_url_path $1)
local full_path=$path$local_page
if [ "${path}" == "/" ]; then
full_path=$local_page
fi
local full_page_url=https://localhost$full_path
# Concatenate all other arguments with '&' to prepare POST data
local POST_data=""
local arg=""
for arg in "${@:2}"; do
POST_data="${POST_data}${arg}&"
done
if [ -n "$POST_data" ]; then
# Add --data arg and remove the last character, which is an unecessary '&'
POST_data="--data ${POST_data::-1}"
fi
# Wait untils nginx has fully reloaded (avoid curl fail with http2)
sleep 2
local cookiefile=/tmp/ynh-$app-cookie.txt
touch $cookiefile
chown root $cookiefile
chmod 700 $cookiefile
# Temporarily enable visitors if needed...
local visitors_enabled=$(ynh_permission_has_user --permission="main" --user="visitors" && echo yes || echo no)
if [[ $visitors_enabled == "no" ]]; then
ynh_permission_update --permission="main" --add="visitors"
fi
# Curl the URL
curl --silent --show-error --insecure --location --header "Host: $domain" --resolve $domain:443:127.0.0.1 $POST_data "$full_page_url" --cookie-jar $cookiefile --cookie $cookiefile
if [[ $visitors_enabled == "no" ]]; then
ynh_permission_update --permission="main" --remove="visitors"
fi
}
_acceptable_path_to_delete() {
local file=$1
local forbidden_paths=$(ls -d / /* /{var,home,usr}/* /etc/{default,sudoers.d,yunohost,cron*} /etc/yunohost/{apps,domains,hooks.d} /opt/yunohost 2> /dev/null)
# Legacy : A couple apps still have data in /home/$app ...
if [[ -n "${app:-}" ]]
then
forbidden_paths=$(echo "$forbidden_paths" | grep -v "/home/$app")
fi
# Use realpath to normalize the path ..
# i.e convert ///foo//bar//..///baz//// to /foo/baz
file=$(realpath --no-symlinks "$file")
if [ -z "$file" ] || grep -q -x -F "$file" <<< "$forbidden_paths"; then
return 1
else
return 0
fi
}
# Remove a file or a directory, checking beforehand that it's not a disastrous location to rm such as entire /var or /home
#
# usage: ynh_safe_rm path_to_remove
ynh_safe_rm() {
local target="$1"
set +o xtrace # set +x
if [ $# -ge 2 ]; then
ynh_print_warn "/!\ Packager ! You provided more than one argument to ynh_safe_rm but it will be ignored... Use this helper with one argument at time."
fi
if [[ -z "$target" ]]; then
ynh_print_warn "ynh_safe_rm called with empty argument, ignoring."
elif [[ ! -e "$target" ]] && [[ ! -L "$target" ]]; then
ynh_print_info "'$target' wasn't deleted because it doesn't exist."
elif ! _acceptable_path_to_delete "$target"; then
ynh_print_warn "Not deleting '$target' because it is not an acceptable path to delete."
else
rm --recursive "$target"
fi
set -o xtrace # set -x
}
# Read the value of a key in the app's manifest
#
# usage: ynh_read_manifest "key"
# | arg: key - Name of the key to find
# | ret: the value associate to that key
ynh_read_manifest() {
cat $YNH_APP_BASEDIR/manifest.toml | toml_to_json | jq ".$1" --raw-output
}
# Return the app upstream version, deduced from `$YNH_APP_MANIFEST_VERSION` and strippig the `~ynhX` part
#
# usage: ynh_app_upstream_version
# | ret: the version number of the upstream app
#
# For example, if the manifest contains `4.3-2~ynh3` the function will return `4.3-2`
ynh_app_upstream_version() {
echo "${YNH_APP_MANIFEST_VERSION/~ynh*/}"
}
# Return 0 if the "upstream" part of the version changed, or 1 otherwise (ie only the ~ynh suffix changed)
#
# usage: if ynh_app_upstream_version_changed; then ...
ynh_app_upstream_version_changed() {
# "UPGRADE_PACKAGE" means only the ~ynh prefix changed
[[ "$YNH_APP_UPGRADE_TYPE" == "UPGRADE_PACKAGE" ]] && return 1 || return 0
}
# Compare the current package version is strictly lower than another version given as an argument
#
# example: if ynh_app_upgrading_from_version_before 2.3.2~ynh1; then ...
ynh_app_upgrading_from_version_before() {
local version=$1
[[ $version =~ '~ynh' ]] || ynh_die "Invalid argument for version, should include the ~ynhX prefix"
dpkg --compare-versions $YNH_APP_CURRENT_VERSION lt $version
}
# Compare the current package version is lower or equal to another version given as an argument
#
# example: if ynh_app_upgrading_from_version_before_or_equal_to 2.3.2~ynh1; then ...
ynh_app_upgrading_from_version_before_or_equal_to() {
local version=$1
[[ $version =~ '~ynh' ]] || ynh_die "Invalid argument for version, should include the ~ynhX prefix"
dpkg --compare-versions $YNH_APP_CURRENT_VERSION le $version
}
# Apply sane permissions for files installed by ynh_setup_source and ynh_config_add.
#
# [internal]
#
# * Anything below $install_dir is chown $app:$app and chmod o-rwx,g-w
# * The rest is considered as system configuration and chown root, chmod 400
#
_ynh_apply_default_permissions() {
local target=$1
is_in_dir() {
# Returns false if parent is empty
[ -n "$2" ] || return 1
local child=$(realpath "$1" 2>/dev/null)
local parent=$(realpath "$2" 2>/dev/null)
[[ "${child}" =~ ^$parent ]]
}
# App files can have files of their own
if ynh_system_user_exists --username="$app"; then
# If this is a file in $install_dir or $data_dir : it should be owned and read+writable by $app only
if [ -f "$target" ] && (is_in_dir "$target" "${install_dir:-}" || is_in_dir "$target" "${data_dir:-}" || is_in_dir "$target" "/etc/$app")
then
chmod 600 "$target"
chown "$app:$app" "$target"
return
fi
# If this is the install dir (so far this is the only way this helper is called with a directory)
if [ "$target" == "${install_dir:-}" ]
then
# Read the group from the install_dir manifest resource
local group="$(ynh_read_manifest 'resources.install_dir.group' | sed 's/null//g' | sed "s/__APP__/$app/g" | cut -f1 -d:)"
if [[ -z "$group" ]]
then
# We set the group to www-data for webapps that do serve static assets, which therefore need to be readable by nginx ...
# The fact that the app needs this is infered by the existence of an nginx.conf and the presence of "alias" or "root" directive
if grep -q '^\s*alias\s\|^\s*root\s' "$YNH_APP_BASEDIR/conf/nginx.conf" 2>/dev/null;
then
group="www-data"
# Or default to "$app"
else
group="$app"
fi
fi
# Files inside should be owned by $app with rw-r----- (+x for folders or files that already have +x)
# The group needs read/dirtraversal (in particular if it's www-data)
chmod -R u=rwX,g=r-X,o=--- "$target"
chown -R "$app:$group" "$target"
return
fi
fi
# Other files are considered system
chmod 400 "$target"
chown root:root "$target"
}
int_to_bool() {
sed -e 's/^1$/True/g' -e 's/^0$/False/g' -e 's/^true$/True/g' -e 's/^false$/False/g'
}
toml_to_json() {
python3 -c 'import toml, json, sys; print(json.dumps(toml.load(sys.stdin)))'
}
# Validate an IP address
#
# usage: ynh_validate_ip --family=family --ip_address=ip_address
# | ret: 0 for valid ip addresses, 1 otherwise
#
# example: ynh_validate_ip 4 111.222.333.444
ynh_validate_ip() {
# ============ Argument parsing =============
local -A args_array=([f]=family= [i]=ip_address=)
local family
local ip_address
ynh_handle_getopts_args "$@"
# ===========================================
[ "$family" == "4" ] || [ "$family" == "6" ] || return 1
# http://stackoverflow.com/questions/319279/how-to-validate-ip-address-in-python#319298
python3 /dev/stdin <<EOF
import socket
import sys
family = { "4" : socket.AF_INET, "6" : socket.AF_INET6 }
try:
socket.inet_pton(family["$family"], "$ip_address")
except socket.error:
sys.exit(1)
sys.exit(0)
EOF
}
# Get the total or free amount of RAM+swap on the system
#
# [packagingv1]
#
# usage: ynh_get_ram [--free|--total]
# | arg: --free - Count free RAM+swap
# | arg: --total - Count total RAM+swap
# | ret: the amount of free ram, in MB (MegaBytes)
ynh_get_ram() {
# ============ Argument parsing =============
local -A args_array=([f]=free [t]=total)
local free
local total
ynh_handle_getopts_args "$@"
free=${free:-0}
total=${total:-0}
# ===========================================
if [ $free -eq $total ]; then
ynh_print_warn "You have to choose --free or --total when using ynh_get_ram"
ram=0
elif [ $free -eq 1 ]; then
local free_ram=$(LC_ALL=C vmstat --stats --unit M | grep "free memory" | awk '{print $1}')
local free_swap=$(LC_ALL=C vmstat --stats --unit M | grep "free swap" | awk '{print $1}')
local free_ram_swap=$((free_ram + free_swap))
local ram=$free_ram_swap
elif [ $total -eq 1 ]; then
local total_ram=$(LC_ALL=C vmstat --stats --unit M | grep "total memory" | awk '{print $1}')
local total_swap=$(LC_ALL=C vmstat --stats --unit M | grep "total swap" | awk '{print $1}')
local total_ram_swap=$((total_ram + total_swap))
local ram=$total_ram_swap
fi
echo $ram
}
# Check if the scripts are being run by the package_check in CI
#
# usage: ynh_in_ci_tests
#
# Return 0 if in CI, 1 otherwise
ynh_in_ci_tests() {
[ "${PACKAGE_CHECK_EXEC:-0}" -eq 1 ]
}
# Retrieve a YunoHost user information
#
# usage: ynh_user_get_info --username=username --key=key
# | arg: --username= - the username to retrieve info from
# | arg: --key= - the key to retrieve
# | ret: the value associate to that key
#
# example: mail=$(ynh_user_get_info --username="toto" --key=mail)
ynh_user_get_info() {
# ============ Argument parsing =============
local -A args_array=([u]=username= [k]=key=)
local username
local key
ynh_handle_getopts_args "$@"
# ===========================================
yunohost user info "$username" --output-as json --quiet | jq -r ".$key"
}
# Get the list of YunoHost users
#
# usage: ynh_user_list
# | ret: one username per line as strings
#
# example: for u in $(ynh_user_list); do ... ; done
ynh_user_list() {
yunohost user list --output-as json --quiet | jq -r ".users | keys[]"
}

1
helpers/helpers.v2.1.d/vendor Symbolic link
View file

@ -0,0 +1 @@
../vendor

2
hooks/conf_regen/01-yunohost
View file

@ -289,6 +289,8 @@ EOF
mkdir -p ${pending_dir}/etc/dpkg/origins/
cp dpkg-origins ${pending_dir}/etc/dpkg/origins/yunohost
# Remove legacy hackish/clumsy nodejs autoupdate which ends up filling up space with ambiguous upgrades >_>
touch "/etc/cron.daily/node_update"
}
do_post_regen() {

4
locales/eu.json
View file

@ -601,8 +601,8 @@
"restore_hook_unavailable": "'{part}'-(e)rako lehengoratze agindua ez dago erabilgarri ez sisteman ezta fitxategian ere",
"restore_cleaning_failed": "Ezin izan dira lehengoratzeko behin-behineko fitxategiak ezabatu",
"restore_confirm_yunohost_installed": "Ziur al zaude dagoeneko instalatuta dagoen sistema lehengoratu nahi duzula? [{answers}]",
"restore_may_be_not_enough_disk_space": "Badirudi zure sistemak ez duela nahikoa espazio (erabilgarri: {free_space} B, beharrezkoa {needed_space} B, segurtasuneko tartea: {margin} B)",
"restore_not_enough_disk_space": "Ez dago nahikoa espazio (erabilgarri: {free_space} B, beharrezkoa {needed_space} B, segurtasuneko tartea: {margin} B)",
"restore_may_be_not_enough_disk_space": "Badirudi zure sistemak ez duela nahikoa espazio (erabilgarri: {free_space} B, beharrezkoa {needed_space} B, segurtasun-tartea: {margin} B)",
"restore_not_enough_disk_space": "Ez dago nahikoa espazio (erabilgarri: {free_space} B, beharrezkoa {needed_space} B, segurtasun-tartea: {margin} B)",
"restore_running_hooks": "Lehengoratzeko 'hook'ak exekutatzen…",
"restore_system_part_failed": "Ezinezkoa izan da sistemaren '{part}' atala lehengoratzea",
"server_reboot": "Zerbitzaria berrabiaraziko da",

15
locales/sk.json
View file

@ -265,5 +265,18 @@
"service_description_rspamd": "Filtruje spam a iné funkcie týkajúce sa e-mailu",
"log_letsencrypt_cert_renew": "Obnoviť '{}' certifikát Let's Encrypt",
"domain_config_cert_summary_selfsigned": "UPOZORNENIE: Aktuálny certifikát je vlastnoručne podpísaný. Prehliadače budú návštevníkom zobrazovať strašidelné varovanie!",
"global_settings_setting_ssowat_panel_overlay_enabled": "Povoliť malú štvorcovú ikonu portálu „YunoHost“ na aplikáciach"
"global_settings_setting_ssowat_panel_overlay_enabled": "Povoliť malú štvorcovú ikonu portálu „YunoHost“ na aplikáciach",
"domain_config_mail_out": "Odchádzajúce e-maily",
"domain_config_default_app": "Predvolená aplikácia",
"domain_config_xmpp_help": "Pozor: niektoré funkcie XMPP vyžadujú aktualizáciu vašich DNS záznamov a obnovenie Lets Encrypt certifikátu pred tým, ako je ich možné zapnúť",
"domain_config_default_app_help": "Návštevníci budú pri návšteve tejto domény automaticky presmerovaní na túto doménu. Ak nenastavíte žiadnu aplikáciu, zobrazí sa stránka s prihlasovacím formulárom na portál.",
"registrar_infos": "Informácie o registrátorovi",
"domain_dns_registrar_managed_in_parent_domain": "Táto doména je subdoména {parent_domain_link}. Nastavenie DNS registrátora je spravovaná v konfiguračnom paneli {parent_domain}.",
"log_letsencrypt_cert_install": "Inštalovať certifikát Let's Encrypt na doménu '{}'",
"domain_config_cert_no_checks": "Ignorovať kontroly diagnostiky",
"domain_config_cert_install": "Nainštalovať certifikát Let's Encrypt",
"domain_config_mail_in": "Prichádzajúce e-maily",
"domain_config_cert_summary": "Stav certifikátu",
"domain_config_xmpp": "Krátke správy (XMPP)",
"log_app_makedefault": "Nastaviť '{}' ako predvolenú aplikáciu"
}

99
src/app.py
View file

@ -754,7 +754,10 @@ def app_upgrade(
from yunohost.utils.resources import AppResourceManager
AppResourceManager(
app_instance_name, wanted=manifest, current=app_dict["manifest"]
app_instance_name,
wanted=manifest,
current=app_dict["manifest"],
workdir=extracted_app_folder,
).apply(
rollback_and_raise_exception_if_failure=True,
operation_logger=operation_logger,
@ -830,6 +833,41 @@ def app_upgrade(
+ "\n -".join(manually_modified_files_by_app)
)
# If the upgrade didnt fail, update the revision and app files (even if it broke the system, otherwise we end up in a funky intermediate state where the app files don't match the installed version or settings, for example for v1->v2 upgrade marked as "broke the system" for some reason)
if not upgrade_failed:
now = int(time.time())
app_setting(app_instance_name, "update_time", now)
app_setting(
app_instance_name,
"current_revision",
manifest.get("remote", {}).get("revision", "?"),
)
# Clean hooks and add new ones
hook_remove(app_instance_name)
if "hooks" in os.listdir(extracted_app_folder):
for hook in os.listdir(extracted_app_folder + "/hooks"):
hook_add(
app_instance_name, extracted_app_folder + "/hooks/" + hook
)
# Replace scripts and manifest and conf (if exists)
# Move scripts and manifest to the right place
for file_to_copy in APP_FILES_TO_COPY:
rm(f"{app_setting_path}/{file_to_copy}", recursive=True, force=True)
if os.path.exists(os.path.join(extracted_app_folder, file_to_copy)):
cp(
f"{extracted_app_folder}/{file_to_copy}",
f"{app_setting_path}/{file_to_copy}",
recursive=True,
)
# Clean and set permissions
shutil.rmtree(extracted_app_folder)
chmod(app_setting_path, 0o600)
chmod(f"{app_setting_path}/settings.yml", 0o400)
chown(app_setting_path, "root", recursive=True)
# If upgrade failed or broke the system,
# raise an error and interrupt all other pending upgrades
if upgrade_failed or broke_the_system:
@ -880,36 +918,6 @@ def app_upgrade(
)
# Otherwise we're good and keep going !
now = int(time.time())
app_setting(app_instance_name, "update_time", now)
app_setting(
app_instance_name,
"current_revision",
manifest.get("remote", {}).get("revision", "?"),
)
# Clean hooks and add new ones
hook_remove(app_instance_name)
if "hooks" in os.listdir(extracted_app_folder):
for hook in os.listdir(extracted_app_folder + "/hooks"):
hook_add(app_instance_name, extracted_app_folder + "/hooks/" + hook)
# Replace scripts and manifest and conf (if exists)
# Move scripts and manifest to the right place
for file_to_copy in APP_FILES_TO_COPY:
rm(f"{app_setting_path}/{file_to_copy}", recursive=True, force=True)
if os.path.exists(os.path.join(extracted_app_folder, file_to_copy)):
cp(
f"{extracted_app_folder}/{file_to_copy}",
f"{app_setting_path}/{file_to_copy}",
recursive=True,
)
# Clean and set permissions
shutil.rmtree(extracted_app_folder)
chmod(app_setting_path, 0o600)
chmod(f"{app_setting_path}/settings.yml", 0o400)
chown(app_setting_path, "root", recursive=True)
# So much win
logger.success(m18n.n("app_upgraded", app=app_instance_name))
@ -1461,6 +1469,9 @@ def app_remove(operation_logger, app, purge=False, force_workdir=None):
for permission_name in user_permission_list(apps=[app])["permissions"].keys():
permission_delete(permission_name, force=True, sync_perm=False)
if purge and os.path.exists(f"/var/log/{app}"):
shutil.rmtree(f"/var/log/{app}")
if os.path.exists(app_setting_path):
shutil.rmtree(app_setting_path)
@ -2891,19 +2902,43 @@ def _make_environment_for_app_script(
# If packaging format v2, load all settings
if manifest["packaging_format"] >= 2 or force_include_app_settings:
env_dict["app"] = app
data_to_redact = []
prefixes_or_suffixes_to_redact = [
"pwd",
"pass",
"passwd",
"password",
"passphrase",
"secret",
"key",
"token",
]
for setting_name, setting_value in _get_app_settings(app).items():
# Ignore special internal settings like checksum__
# (not a huge deal to load them but idk...)
if setting_name.startswith("checksum__"):
continue
env_dict[setting_name] = str(setting_value)
setting_value = str(setting_value)
env_dict[setting_name] = setting_value
# Check if we should redact this setting value
# (the check on the setting length exists to prevent stupid stuff like redacting empty string or something which is actually just 0/1, true/false, ...
if len(setting_value) > 6 and any(
setting_name.startswith(p) or setting_name.endswith(p)
for p in prefixes_or_suffixes_to_redact
):
data_to_redact.append(setting_value)
# Special weird case for backward compatibility...
# 'path' was loaded into 'path_url' .....
if "path" in env_dict:
env_dict["path_url"] = env_dict["path"]
for operation_logger in OperationLogger._instances:
operation_logger.data_to_redact.extend(data_to_redact)
return env_dict

15
src/log.py
View file

@ -42,12 +42,18 @@ LOG_FILE_EXT = ".log"
BORING_LOG_LINES = [
r"set [+-]x$",
r"set [+-]o xtrace$",
r"\+ set \+o$",
r"\+ grep xtrace$",
r"local 'xtrace_enable=",
r"set [+-]o errexit$",
r"set [+-]o nounset$",
r"trap '' EXIT",
r"local \w+$",
r"local exit_code=(1|0)$",
r"local legacy_args=.*$",
r"local _globalapp=.*$",
r"local checksum_setting_name=.*$",
r"ynh_app_setting ", # (note the trailing space to match the "low level" one called by other setting helpers)
r"local -A args_array$",
r"args_array=.*$",
r"ret_code=1",
@ -59,8 +65,17 @@ BORING_LOG_LINES = [
r"\[?\['? -n '' '?\]\]?$",
r"rm -rf /var/cache/yunohost/download/$",
r"type -t ynh_clean_setup$",
r"DEBUG - \+ unset \S+$",
r"DEBUG - \+ echo '",
r"DEBUG - \+ LC_ALL=C$",
r"DEBUG - \+ DEBIAN_FRONTEND=noninteractive$",
r"DEBUG - \+ exit (1|0)$",
r"DEBUG - \+ app=\S+$",
r"DEBUG - \+\+ app=\S+$",
r"DEBUG - \+\+ jq -r .\S+$",
r"DEBUG - \+\+ sed 's/\^null\$//'$",
"DEBUG - \\+ sed --in-place \\$'s\\\\001",
"DEBUG - \\+ sed --in-place 's\u0001.*$",
]

11
src/service.py
View file

@ -687,13 +687,20 @@ def _get_services():
]
for name in services_with_package_condition:
package = services[name]["ignore_if_package_is_not_installed"]
if os.system(f"dpkg --list | grep -q 'ii *{package}'") != 0:
if (
check_output(
f"dpkg-query --show --showformat='${{db:Status-Status}}' '{package}' 2>/dev/null || true"
)
!= "installed"
):
del services[name]
php_fpm_versions = check_output(
r"dpkg --list | grep -P 'ii php\d.\d-fpm' | awk '{print $2}' | grep -o -P '\d.\d' || true"
r"dpkg --list | grep -P 'ii php\d.\d-fpm' | awk '{print $2}' | grep -o -P '\d.\d' || true",
cwd="/tmp",
)
php_fpm_versions = [v for v in php_fpm_versions.split("\n") if v.strip()]
for version in php_fpm_versions:
# Skip php 7.3 which is most likely dead after buster->bullseye migration
# because users get spooked

79
src/utils/resources.py
View file

@ -39,10 +39,11 @@ logger = getLogger("yunohost.utils.resources")
class AppResourceManager:
def __init__(self, app: str, current: Dict, wanted: Dict):
def __init__(self, app: str, current: Dict, wanted: Dict, workdir=None):
self.app = app
self.current = current
self.wanted = wanted
self.workdir = workdir
if "resources" not in self.current:
self.current["resources"] = {}
@ -155,6 +156,31 @@ class AppResource:
elif manager and manager.current and "version" in manager.current:
app_upstream_version = manager.current["version"].split("~")[0]
# FIXME : should use packaging.version to properly parse / compare versions >_>
self.helpers_version: float = 0
if (
manager
and manager.wanted
and manager.wanted.get("integration", {}).get("helpers_version")
):
self.helpers_version = float(
manager.wanted.get("integration", {}).get("helpers_version")
)
elif (
manager
and manager.current
and manager.current.get("integration", {}).get("helpers_version")
):
self.helpers_version = float(
manager.current.get("integration", {}).get("helpers_version")
)
elif manager and manager.wanted and manager.wanted.get("packaging_format"):
self.helpers_version = float(manager.wanted.get("packaging_format"))
elif manager and manager.current and manager.current.get("packaging_format"):
self.helpers_version = float(manager.current.get("packaging_format"))
if not self.helpers_version:
self.helpers_version = 1.0
replacements: dict[str, str] = {
"__APP__": self.app,
"__YNH_ARCH__": system_arch(),
@ -231,17 +257,21 @@ class AppResource:
)
from yunohost.hook import hook_exec_with_script_debug_if_failure
tmpdir = _make_tmp_workdir_for_app(app=self.app)
workdir = (
self.manager.workdir
if self.manager and self.manager.workdir
else _make_tmp_workdir_for_app(app=self.app)
)
env_ = _make_environment_for_app_script(
self.app,
workdir=tmpdir,
workdir=workdir,
action=f"{action}_{self.type}",
force_include_app_settings=True,
)
env_.update(env)
script_path = f"{tmpdir}/{action}_{self.type}"
script_path = f"{workdir}/{action}_{self.type}"
script = f"""
source /usr/share/yunohost/helpers
ynh_abort_if_errors
@ -1182,11 +1212,23 @@ class AptDependenciesAppResource(AppResource):
}
def provision_or_update(self, context: Dict = {}):
script = " ".join(["ynh_install_app_dependencies", *self.packages])
if self.helpers_version >= 2.1:
ynh_apt_install_dependencies = "ynh_apt_install_dependencies"
ynh_apt_install_dependencies_from_extra_repository = (
"ynh_apt_install_dependencies_from_extra_repository"
)
else:
ynh_apt_install_dependencies = "ynh_install_app_dependencies"
ynh_apt_install_dependencies_from_extra_repository = (
"ynh_install_extra_app_dependencies"
)
script = " ".join([ynh_apt_install_dependencies, *self.packages])
for repo, values in self.extras.items():
script += "\n" + " ".join(
[
"ynh_install_extra_app_dependencies",
ynh_apt_install_dependencies_from_extra_repository,
f"--repo='{values['repo']}'",
f"--key='{values['key']}'",
f"--package='{' '.join(values['packages'])}'",
@ -1197,7 +1239,12 @@ class AptDependenciesAppResource(AppResource):
self._run_script("provision_or_update", script)
def deprovision(self, context: Dict = {}):
self._run_script("deprovision", "ynh_remove_app_dependencies")
if self.helpers_version >= 2.1:
ynh_apt_remove_dependencies = "ynh_apt_remove_dependencies"
else:
ynh_apt_remove_dependencies = "ynh_remove_app_dependencies"
self._run_script("deprovision", ynh_apt_remove_dependencies)
class PortsResource(AppResource):
@ -1455,13 +1502,19 @@ class DatabaseAppResource(AppResource):
db_name = self.get_setting("db_name") or db_user
if self.dbtype == "mysql":
self._run_script(
"deprovision", f"ynh_mysql_remove_db '{db_name}' '{db_user}'"
)
db_helper_name = "mysql"
elif self.dbtype == "postgresql":
self._run_script(
"deprovision", f"ynh_psql_remove_db '{db_name}' '{db_user}'"
)
db_helper_name = "psql"
else:
raise RuntimeError(f"Invalid dbtype {self.dbtype}")
self._run_script(
"deprovision",
f"""
ynh_{db_helper_name}_database_exists "{db_name}" && ynh_{db_helper_name}_drop_db "{db_name}" || true
ynh_{db_helper_name}_user_exists "{db_user}" && ynh_{db_helper_name}_drop_user "{db_user}" || true
""",
)
self.delete_setting("db_name")
self.delete_setting("db_user")

2
src/utils/system.py
View file

@ -159,7 +159,7 @@ def ynh_packages_version(*args, **kwargs):
def dpkg_is_broken():
if check_output("dpkg --audit") != "":
if check_output("dpkg --audit", cwd="/tmp/") != "":
return True
# If dpkg is broken, /var/lib/dpkg/updates
# will contains files like 0001, 0002, ...