Merge branch 'testing' into v2

.github/ISSUE_TEMPLATE.md

@@ -5,12 +5,14 @@ about: When creating a bug report, please use the following template to provide
 ---
 
 **How to post a meaningful bug report**
+
 1. *Read this whole template first.*
 2. *Determine if you are on the right place:*
    - *If you were performing an action on the app from the webadmin or the CLI (install, update, backup, restore, change_url...), you are on the right place!*
    - *Otherwise, the issue may be due to the app itself. Refer to its documentation or repository for help.*
    - *When in doubt, post here and we will figure it out together.*
 3. *Delete the italic comments as you write over them below, and remove this guide.*
+
 ---
 
 ### Describe the bug
@@ -30,9 +32,11 @@ about: When creating a bug report, please use the following template to provide
 ### Steps to reproduce
 
 - *If you performed a command from the CLI, the command itself is enough. For example:*
+
     ```sh
     sudo yunohost app install the_app
     ```
+
 - *If you used the webadmin, please perform the equivalent command from the CLI first.*
 - *If the error occurs in your browser, explain what you did:*
    1. *Go to '...'*
@@ -47,6 +51,7 @@ about: When creating a bug report, please use the following template to provide
 ### Logs
 
 *When an operation fails, YunoHost provides a simple way to share the logs.*
+
 - *In the webadmin, the error message contains a link to the relevant log page. On that page, you will be able to 'Share with Yunopaste'. If you missed it, the logs of previous operations are also available under Tools > Logs.*
 - *In command line, the command to share the logs is displayed at the end of the operation and looks like `yunohost log display [log name] --share`. If you missed it, you can find the log ID of a previous operation using `yunohost log list`.*
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -13,4 +13,4 @@
 
 ## Automatic tests
 
-Automatic tests can be triggered on https://ci-apps-dev.yunohost.org/ *after creating the PR*, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)
+Automatic tests can be triggered on <https://ci-apps-dev.yunohost.org/> *after creating the PR*, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)

README.md

@@ -19,7 +19,8 @@ If you don't have YunoHost, please consult [the guide](https://yunohost.org/#/in
 GoToSocial is an [ActivityPub](https://activitypub.rocks/) social network server, written in Golang. With GoToSocial, you can keep in touch with your friends, post, read, and share images and articles. All without being tracked or advertised to!
 
 
-**Shipped version:** 0.11.0~ynh1
+**Shipped version:** 0.11.0~ynh4
+
 
 ## Screenshots
 

README_fr.md

@@ -18,8 +18,7 @@ Si vous n’avez pas YunoHost, regardez [ici](https://yunohost.org/#/install) po
 
 Un serveur de réseau social basé sur [ActivityPub](https://activitypub.rocks/) écrit en Golang. Avec GoToSocial, vous pouvez rester en contact avec vos amis, publier, lire et partager des images et des articles. Tout cela sans être pisté ni subir de publicité !
 
-
+**Version incluse :** 0.11.1~ynh4
-**Version incluse :** 0.11.0~ynh1
 
 ## Captures d’écran
 

conf/config.yaml

@@ -238,7 +238,7 @@ cache:
   # in-memory objects, and so NOT AT ALL EXACT.
   # Examples: ["100MiB", "200MiB", "500MiB", "1GiB"]
   # Default: "100MiB"
-  memory-target: "100MiB"
+  memory-target: __CACHE_MEMORY_TARGET__
 
 ######################
 ##### WEB CONFIG #####
@@ -310,7 +310,7 @@ instance-deliver-to-shared-inboxes: __INSTANCE_DELIVER_TO_SHARED_INBOXES__
 #
 # Options: [true, false]
 # Default: false
-instance-inject-mastodon-version: false
+instance-inject-mastodon-version: __INSTANCE_INJECT_MASTODON_VERSION__
 
 ###########################
 ##### ACCOUNTS CONFIG #####
@@ -389,7 +389,7 @@ media-description-max-chars: __MEDIA_DESCRIPTION_MAX_CHARS__
 # are kept so that it can be fetched again if requested by a user.
 #
 # If this is set to 0, then media from remote instances will be cached indefinitely.
-# Examples: 7
+# Examples: [30, 60, 7, 0]
 # Default: 7
 media-remote-cache-days: __MEDIA_REMOTE_CACHE_DAYS__
 
@@ -904,3 +904,21 @@ advanced-throttling-retry-after: "30s"
 # 2 cpu = 1 concurrent sender
 # 4 cpu = 1 concurrent sender
 advanced-sender-multiplier: 2
+
+# Array of string. Extra URIs to add to 'img-src' and 'media-src'
+# when building the Content-Security-Policy header for your instance.
+#
+# This can be used to allow the browser to load resources from additional
+# sources like S3 buckets and so on when viewing your instance's pages
+# and profiles in the browser.
+#
+# Since non-proxying S3 storage will be probed on instance launch to
+# generate a correct Content-Security-Policy, you probably won't need
+# to ever touch this setting, but it's included in the 'spirit of more
+# configurable (usually) means more good'.
+# 
+# See: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
+#
+# Example: ["s3.example.org", "some-bucket-name.s3.example.org"]
+# Default: []
+advanced-csp-extra-uris: []

conf/nginx.conf

@@ -3,13 +3,43 @@ location __PATH__/ {
 
   proxy_pass http://127.0.0.1:__PORT__;
   proxy_http_version 1.1;
+  proxy_set_header Host $host;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection "upgrade";
+  proxy_set_header X-Forwarded-For $remote_addr;
+  proxy_set_header X-Forwarded-Proto $scheme;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-  proxy_set_header Host $host;
 
   client_max_body_size __CLIENT_MAX_BODY_SIZE__;
 
   # Include SSOWAT user panel.
   include conf.d/yunohost_panel.conf.inc;
+
+
+  # media caching stuff
+  # https://docs.gotosocial.org/en/latest/advanced/caching/assets-media/#nginx
+  location /assets/ {
+    alias __FINAL_PATH__/web/assets/;
+    autoindex off;
+    expires 5m;
+    add_header Cache-Control "public";
+  }
+
+  location /fileserver/ {
+    alias __DATADIR__;
+    autoindex off;
+    expires 1w;
+    add_header Cache-Control "private, immutable";
+    try_files $uri @fileserver;
+  }
+
+}
+
+location @fileserver {
+  proxy_pass http://127.0.0.1:__PORT__;
+  proxy_set_header Host $host;
+  proxy_set_header Upgrade $http_upgrade;
+  proxy_set_header Connection "upgrade";
+  proxy_set_header X-Forwarded-For $remote_addr;
+  proxy_set_header X-Forwarded-Proto $scheme;
 }

config_panel.toml

@@ -288,6 +288,22 @@ La livraison dans la boîte de réception partagée peut réduire de manière si
 Voir : https://www.w3.org/TR/activitypub/#shared-inbox-delivery"""
 type = "select"
 
+[main.instance.instance_inject_mastodon_version]
+ask.en = "Inject Mastodon version?"
+ask.fr = "Injecter une version Mastodon ?"
+bind = "instance-inject-mastodon-version:__FINALPATH__/config.yaml"
+choices = ["true", "false"]
+default = "false"
+help.en = """This flag will inject a Mastodon version into the version field that is included in /api/v1/instance.\
+This version is often used by Mastodon clients to do API feature detection.\
+By injecting a Mastodon compatible version, it is possible to cajole those clients to behave correctly with GoToSocial.\
+Default: false"""
+help.fr = """Ce paramètre injecte une version de Mastodon dans le champ version qui est inclus dans /api/v1/instance.\
+Cette version est souvent utilisée par les clients Mastodon pour détecter les caractéristiques de l'API.\
+En injectant une version compatible avec Mastodon, il est possible d'inciter ces clients à se comporter correctement avec GoToSocial.
+Par défautl : false"""
+type = "select"
+
 ################
 #### SMTP CONFIG
 ################
@@ -355,6 +371,29 @@ help.fr = """true : divulguer tous les destinataires dans le champ À\
 false : l'e-mail sera envoyé sans divulguer les destinataires"""
 type = "select"
 
+####################
+#### CACHE SETTINGS
+####################
+
+[main.cache]
+
+name = "Cache settings"
+
+help = "Settings pertaining to... the cache"
+
+[main.cache.cache_memory_target]
+ask.en = "Value of the cache target"
+ask.fr = "Valeur du niveau de cache"
+bind = "memory-target:__FINALPATH__/config.yaml"
+default = "100MiB"
+help.en = """Sets a target limit that the application will try to keep it's caches within.\
+This is based on estimated sizes of in-memory objects, and so NOT AT ALL EXACT.
+Examples: 100MiB, 200MiB, 500MiB, 1GiB; Default: 100MiB"""
+help.fr = """Définit une limite cible que l'application essaiera de ne pas dépasser pour ses caches.\
+Cette limite est basée sur des estimations de la taille des objets en mémoire et N'EST DONC PAS DU TOUT EXACTE.
+Exemples : 100MiB, 200MiB, 500MiB, 1GiB; Par défaut : 100MiB"""
+type = "string"
+
 ####################
 #### ADVANCED SETTINGS
 ####################

scripts/backup

@@ -41,6 +41,13 @@ ynh_backup --src_path="/etc/nginx/conf.d/$domain.d/$app.conf"
 
 ynh_backup --src_path="/etc/logrotate.d/$app"
 
+#=================================================
+# BACKUP FAIL2BAN CONFIGURATION
+#=================================================
+
+ynh_backup --src_path="/etc/fail2ban/jail.d/$app.conf"
+ynh_backup --src_path="/etc/fail2ban/filter.d/$app.conf"
+
 #=================================================
 # BACKUP SYSTEMD
 #=================================================

scripts/install

@@ -25,12 +25,13 @@ instance_expose_suspended="false"
 instance_expose_suspended_web="false"
 instance_expose_public_timeline="false"
 instance_deliver_to_shared_inboxes="true"
+instance_inject_mastodon_version="false"
 
 media_image_max_size="10485760"
 media_video_max_size="41943040"
 media_description_min_chars="0"
 media_description_max_chars="500"
-media_remote_cache_days="30"
+media_remote_cache_days="7"
 media_emoji_local_max_size="51200"
 media_emoji_remote_max_size="102400"
 
@@ -65,6 +66,8 @@ ynh_app_setting_set --app="$app" --key=landing_page_user --value="$landing_page_
 ynh_app_setting_set --app="$app" --key=client_max_body_size --value="$client_max_body_size"
 ynh_app_setting_set --app="$app" --key=email --value="$email"
 
+ynh_app_setting_set --app="$app" --key=cache_memory_target --value="$cache_memory_target"
+
 ynh_app_setting_set --app="$app" --key=accounts_registration_open --value="$accounts_registration_open"
 ynh_app_setting_set --app="$app" --key=accounts_approval_required --value="$accounts_approval_required"
 ynh_app_setting_set --app="$app" --key=accounts_reason_required --value="$accounts_reason_required"
@@ -76,6 +79,7 @@ ynh_app_setting_set --app="$app" --key=instance_expose_suspended --value="$insta
 ynh_app_setting_set --app="$app" --key=instance_expose_suspended_web --value="$instance_expose_suspended_web"
 ynh_app_setting_set --app="$app" --key=instance_expose_public_timeline --value="$instance_expose_public_timeline"
 ynh_app_setting_set --app="$app" --key=instance_deliver_to_shared_inboxes --value="$instance_deliver_to_shared_inboxes"
+ynh_app_setting_set --app="$app" --key=instance_inject_mastodon_version --value="$instance_inject_mastodon_version"
 
 ynh_app_setting_set --app="$app" --key=media_image_max_size --value="$media_image_max_size"
 ynh_app_setting_set --app="$app" --key=media_video_max_size --value="$media_video_max_size"
@@ -155,6 +159,17 @@ ynh_script_progression --message="Configuring log rotation..." --weight=1
 # Use logrotate to manage application logfile(s)
 ynh_use_logrotate
 
+#=================================================
+# SETUP FAIL2BAN
+#=================================================
+ynh_script_progression --message="Configuring fail2ban..." --weight=1
+
+# Create the logfile, required before configuring fail2ban
+touch "/var/log/${app}/${app}.log"
+
+# Create a dedicated Fail2Ban config
+ynh_add_fail2ban_config --logpath="/var/log/${app}/${app}.log" --failregex="statusCode=401 path=/auth/sign_in clientIP=<HOST> .* msg=\"Unauthorized:" --max_retry=5
+
 #=================================================
 # INTEGRATE SERVICE IN YUNOHOST
 #=================================================

scripts/restore

@@ -49,6 +49,38 @@ ynh_psql_execute_as_root --sql="CREATE EXTENSION IF NOT EXISTS citext;" --databa
 ynh_psql_execute_as_root --sql="CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\";" --database="$db_name"
 ynh_psql_execute_file_as_root --file="./db.sql" --database="$db_name"
 
+#=================================================
+# DOWNLOAD, CHECK AND UNPACK SOURCE
+#=================================================
+ynh_script_progression --message="Setting up source files..." --weight=1
+
+### `ynh_setup_source` is used to install an app from a zip or tar.gz file,
+### downloaded from an upstream source, like a git repository.
+### `ynh_setup_source` use the file conf/app.src
+
+# detect_arch comes from _common.sh / personnal helpers
+architecture="$(detect_arch)"
+
+# compare if the system arch is different from the binary arch
+# if so, download the correct binary
+if [ "$architecture" != "$(file "$final_path"/gotosocial | cut -d ',' -f 2 | tr -d ' ')" ]
+then
+	ynh_script_progression --message="Migrating binary architecture..."
+
+	# Download, check integrity, uncompress and patch the source from app.src
+	ynh_setup_source --dest_dir="$final_path" --source_id="$architecture" --keep="config.yaml"
+fi
+
+# FIXME: this should be managed by the core in the future
+# Here, as a packager, you may have to tweak the ownerhsip/permissions
+# such that the appropriate users (e.g. maybe www-data) can access
+# files in some cases.
+# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder _
+# this will be treated as a security issue.
+chmod 750 "$final_path"
+chmod -R o-rwx "$final_path"
+chown -R "$app:www-data" "$final_path"
+
 #=================================================
 # RESTORE VARIOUS FILES
 #=================================================
@@ -84,6 +116,14 @@ ynh_script_progression --message="Restoring the logrotate configuration..." --we
 
 ynh_restore_file --origin_path="/etc/logrotate.d/$app"
 
+#=================================================
+# RESTORE THE FAIL2BAN CONFIGURATION
+#=================================================
+
+ynh_restore_file --origin_path="/etc/fail2ban/jail.d/$app.conf"
+ynh_restore_file --origin_path="/etc/fail2ban/filter.d/$app.conf"
+ynh_systemd_action --action=restart --service_name=fail2ban
+
 #=================================================
 # GENERIC FINALIZATION
 #=================================================

scripts/upgrade

@@ -217,6 +217,24 @@ then
 	ynh_app_setting_set --app="$app" --key=accounts_custom_css_length --value="$accounts_custom_css_length"
 fi
 
+# Upgrade from <0.11.0~ynh1:
+if ynh_compare_current_package_version --comparison lt --version 0.11.0~ynh1 || [ -z "$cache_memory_target" ]
+then
+	# declaration of new parameter
+	cache_memory_target="100MiB"
+	instance_inject_mastodon_version="false"
+	# update default config
+	if [ "$media_remote_cache_days" == "30" ]; then
+		# "30" is the old default value, "7" the new one
+		# updating to the new default only if the old default value is used (to not overwrite an user modified value) 
+		media_remote_cache_days="7"
+		ynh_app_setting_set --app="$app" --key=media_remote_cache_days --value="$media_remote_cache_days"
+	fi
+	# registration of parameter
+	ynh_app_setting_set --app="$app" --key=cache_memory_target --value="$cache_memory_target"
+	ynh_app_setting_set --app="$app" --key=instance_inject_mastodon_version --value="$instance_inject_mastodon_version"
+fi
+
 #=================================================
 # DOWNLOAD, CHECK AND UNPACK SOURCE
 #=================================================
@@ -268,6 +286,14 @@ ynh_script_progression --message="Upgrading logrotate configuration..."
 # Use logrotate to manage app-specific logfile(s)
 ynh_use_logrotate --non-append
 
+#=================================================
+# SETUP FAIL2BAN
+#=================================================
+ynh_script_progression --message="Upgrading fail2ban configuration..."
+
+# Create a dedicated Fail2Ban config
+ynh_add_fail2ban_config --logpath="/var/log/${app}/${app}.log" --failregex="statusCode=401 path=/auth/sign_in clientIP=<HOST> .* msg=\"Unauthorized:" --max_retry=5
+
 #=================================================
 # INTEGRATE SERVICE IN YUNOHOST
 #=================================================