diff --git a/LICENSE b/LICENSE index 7d1e40b..fc6d9ac 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,8 @@ -File containing the license of your package. +Paperless-ngx Pacakge for YunoHost +Copyright (C) 2022 YunoHost Apps Team -More information here: -https://yunohost.org/packaging_apps_guidelines#yep-1-3 +This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with this program. If not, see . diff --git a/README.md b/README.md index e30bd1b..5f3182b 100644 --- a/README.md +++ b/README.md @@ -15,39 +15,46 @@ If you don't have YunoHost, please consult [the guide](https://yunohost.org/#/in ## Overview -Some long and extensive description of what the app is and does, lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. +Paperless-ngx is a document management system that transforms your physical documents into a searchable online archive so you can keep, well, *less paper*. ### Features -- Ut enim ad minim veniam, quis nostrud exercitation ullamco ; -- Laboris nisi ut aliquip ex ea commodo consequat ; -- Duis aute irure dolor in reprehenderit in voluptate ; -- Velit esse cillum dolore eu fugiat nulla pariatur ; -- Excepteur sint occaecat cupidatat non proident, sunt in culpa." +* Organize and index your scanned documents with tags, correspondents, types, and more. +* Performs OCR on your documents, adds selectable text to image only documents and adds tags, correspondents and document types to your documents. +* Supports PDF documents, images, plain text files, and Office documents (Word, Excel, Powerpoint, and LibreOffice equivalents). +* Paperless stores your documents plain on disk. Filenames and folders are managed by paperless and their format can be configured freely. +* Single page application front end. +* Full text search helps you find what you need. +* Email processing: Paperless adds documents from your email accounts. +* Machine learning powered document matching. +* Optimized for multi core systems: Paperless-ngx consumes multiple documents in parallel. +* The integrated sanity checker makes sure that your document archive is in good health. +* [More screenshots are available in the documentation](https://paperless-ngx.readthedocs.io/en/latest/screenshots.html). -**Shipped version:** 1.6~ynh1 +**Shipped version:** 1.11.0~ynh1 **Demo:** https://demo.paperless-ngx.com/ ## Screenshots -![Screenshot of Paperless-ngx](./doc/screenshots/example.jpg) +![Screenshot of Paperless-ngx](./doc/screenshots/documents-wchrome-dark.png) ## Disclaimers / important information -* Any known limitations, constrains or stuff not working, such as (but not limited to): - * requiring a full dedicated domain ? - * architectures not supported ? - * not-working single-sign on or LDAP integration ? - * the app requires an important amount of RAM / disk / .. to install or to work properly - * etc... +* This app require a dedicated domain. +* There is no LDAP or SSO support. -* Other infos that people should be aware of, such as: - * any specific step to perform after installing (such as manually finishing the install, specific admin credentials, ...) - * how to configure / administrate the application if it ain't obvious - * upgrade process / specificities / things to be aware of ? - * security considerations ? +* Paperless is performing OCR on documents and images. English is installed by default. More languages can be installed: + * Display a list of all Tesseract language packs `apt-cache search tesseract-ocr` + * Install additional language packs + * Example for french `sudo apt-get install tesseract-ocr-fra` + * Example for german `sudo apt-get install tesseract-ocr-deu` + * Modify config to add new languages + * Open config-panel: https://my-domain.tld/yunohost/admin/#/apps/$app_id/config-panel + * You can combine multiple languages like this: + * One language: eng + * Two languages: eng+fra ## Documentation and resources diff --git a/README_fr.md b/README_fr.md index 490d18d..665bc4f 100644 --- a/README_fr.md +++ b/README_fr.md @@ -15,39 +15,46 @@ Si vous n'avez pas YunoHost, regardez [ici](https://yunohost.org/#/install) pour ## Vue d'ensemble -Some long and extensive description of what the app is and does, lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. +Paperless-ngx is a document management system that transforms your physical documents into a searchable online archive so you can keep, well, *less paper*. ### Features -- Ut enim ad minim veniam, quis nostrud exercitation ullamco ; -- Laboris nisi ut aliquip ex ea commodo consequat ; -- Duis aute irure dolor in reprehenderit in voluptate ; -- Velit esse cillum dolore eu fugiat nulla pariatur ; -- Excepteur sint occaecat cupidatat non proident, sunt in culpa." +* Organize and index your scanned documents with tags, correspondents, types, and more. +* Performs OCR on your documents, adds selectable text to image only documents and adds tags, correspondents and document types to your documents. +* Supports PDF documents, images, plain text files, and Office documents (Word, Excel, Powerpoint, and LibreOffice equivalents). +* Paperless stores your documents plain on disk. Filenames and folders are managed by paperless and their format can be configured freely. +* Single page application front end. +* Full text search helps you find what you need. +* Email processing: Paperless adds documents from your email accounts. +* Machine learning powered document matching. +* Optimized for multi core systems: Paperless-ngx consumes multiple documents in parallel. +* The integrated sanity checker makes sure that your document archive is in good health. +* [More screenshots are available in the documentation](https://paperless-ngx.readthedocs.io/en/latest/screenshots.html). -**Version incluse :** 1.6~ynh1 +**Version incluse :** 1.11.0~ynh1 **Démo :** https://demo.paperless-ngx.com/ ## Captures d'écran -![Capture d'écran de Paperless-ngx](./doc/screenshots/example.jpg) +![Capture d'écran de Paperless-ngx](./doc/screenshots/documents-wchrome-dark.png) ## Avertissements / informations importantes -* Any known limitations, constrains or stuff not working, such as (but not limited to): - * requiring a full dedicated domain ? - * architectures not supported ? - * not-working single-sign on or LDAP integration ? - * the app requires an important amount of RAM / disk / .. to install or to work properly - * etc... +* This app require a dedicated domain. +* There is no LDAP or SSO support. -* Other infos that people should be aware of, such as: - * any specific step to perform after installing (such as manually finishing the install, specific admin credentials, ...) - * how to configure / administrate the application if it ain't obvious - * upgrade process / specificities / things to be aware of ? - * security considerations ? +* Paperless is performing OCR on documents and images. English is installed by default. More languages can be installed: + * Display a list of all Tesseract language packs `apt-cache search tesseract-ocr` + * Install additional language packs + * Example for french `sudo apt-get install tesseract-ocr-fra` + * Example for german `sudo apt-get install tesseract-ocr-deu` + * Modify config to add new languages + * Open config-panel: https://my-domain.tld/yunohost/admin/#/apps/$app_id/config-panel + * You can combine multiple languages like this: + * One language: eng + * Two languages: eng+fra ## Documentations et ressources diff --git a/check_process b/check_process index f41c4c9..954e403 100644 --- a/check_process +++ b/check_process @@ -6,21 +6,16 @@ ;; Test complet ; Manifest domain="domain.tld" - path="/path" - is_public=1 - language="fr" admin="john" - password="1Strong-Password" - port="666" ; Checks pkg_linter=1 - setup_sub_dir=1 + setup_sub_dir=0 setup_root=1 setup_nourl=0 setup_private=1 setup_public=1 upgrade=1 - upgrade=1 from_commit=CommitHash + upgrade=0 from_commit=CommitHash backup_restore=1 multi_instance=1 port_already_use=0 diff --git a/conf/app.src b/conf/app.src index 17489bf..eacaec9 100644 --- a/conf/app.src +++ b/conf/app.src @@ -1,5 +1,5 @@ -SOURCE_URL=url of app's source -SOURCE_SUM=sha256 checksum +SOURCE_URL=https://github.com/paperless-ngx/paperless-ngx/releases/download/v1.11.0/paperless-ngx-v1.11.0.tar.xz +SOURCE_SUM=6b175daf2b9c7411b9a63f747ad62661c9733d87072c16a494394a189d2e240f SOURCE_SUM_PRG=sha256sum SOURCE_FORMAT=tar.gz SOURCE_IN_SUBDIR=true diff --git a/conf/nginx.conf b/conf/nginx.conf index 1f1945c..fe3f78e 100644 --- a/conf/nginx.conf +++ b/conf/nginx.conf @@ -1,27 +1,19 @@ -#sub_path_only rewrite ^__PATH__$ __PATH__/ permanent; location __PATH__/ { - # Path to source - alias __FINALPATH__/ ; + proxy_pass http://localhost:__PORT__/; -### Example PHP configuration (remove it if not used) - index index.php; + # These configuration options are required for WebSockets to work. + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; - # Common parameter to increase upload size limit in conjunction with dedicated php-fpm file - #client_max_body_size 50M; + proxy_redirect off; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Host $server_name; - try_files $uri $uri/ index.php; - location ~ [^/]\.php(/|$) { - fastcgi_split_path_info ^(.+?\.php)(/.*)$; - fastcgi_pass unix:/var/run/php/php__PHPVERSION__-fpm-__NAME__.sock; - - fastcgi_index index.php; - include fastcgi_params; - fastcgi_param REMOTE_USER $remote_user; - fastcgi_param PATH_INFO $fastcgi_path_info; - fastcgi_param SCRIPT_FILENAME $request_filename; - } -### End of PHP configuration part + client_max_body_size 25M; # Include SSOWAT user panel. include conf.d/yunohost_panel.conf.inc; diff --git a/conf/paperless.conf.example b/conf/paperless.conf.example new file mode 100644 index 0000000..282872d --- /dev/null +++ b/conf/paperless.conf.example @@ -0,0 +1,88 @@ +# Have a look at the docs for documentation. +# https://paperless-ngx.readthedocs.io/en/latest/configuration.html + +# Debug. Only enable this for development. + +PAPERLESS_DEBUG=false + +# Required services + +PAPERLESS_REDIS=redis://localhost:6379/__REDIS_DB__ + +PAPERLESS_DBHOST=localhost +PAPERLESS_DBPORT=5432 +PAPERLESS_DBNAME=__DB_NAME__ +PAPERLESS_DBUSER=__DB_USER__ +PAPERLESS_DBPASS=__DB_PWD__ +#PAPERLESS_DBSSLMODE=prefer + +# Paths and folders + +PAPERLESS_SCRATCH_DIR=/tmp/__APP__ +PAPERLESS_CONSUMPTION_DIR=__DATADIR__/consume +PAPERLESS_DATA_DIR=__DATADIR__/data +#PAPERLESS_TRASH_DIR= +PAPERLESS_MEDIA_ROOT=__DATADIR__/media +#PAPERLESS_STATICDIR=../static +#PAPERLESS_FILENAME_FORMAT= + +# Security and hosting + +PAPERLESS_URL=https://__DOMAIN__ +PAPERLESS_SECRET_KEY=__PAPERLESS_SECRET_KEY__ +PAPERLESS_ALLOWED_HOSTS=__DOMAIN__ +PAPERLESS_CORS_ALLOWED_HOSTS=https://__DOMAIN__ +#PAPERLESS_FORCE_SCRIPT_NAME= +#PAPERLESS_STATIC_URL=/static/ +#PAPERLESS_AUTO_LOGIN_USERNAME= +#PAPERLESS_COOKIE_PREFIX= +#PAPERLESS_ENABLE_HTTP_REMOTE_USER=false + +# OCR settings + +PAPERLESS_OCR_LANGUAGE=eng +#PAPERLESS_OCR_MODE=skip +#PAPERLESS_OCR_OUTPUT_TYPE=pdfa +#PAPERLESS_OCR_PAGES=1 +#PAPERLESS_OCR_IMAGE_DPI=300 +#PAPERLESS_OCR_CLEAN=clean +#PAPERLESS_OCR_DESKEW=true +#PAPERLESS_OCR_ROTATE_PAGES=true +#PAPERLESS_OCR_ROTATE_PAGES_THRESHOLD=12.0 +#PAPERLESS_OCR_USER_ARGS={} +#PAPERLESS_CONVERT_MEMORY_LIMIT=0 +#PAPERLESS_CONVERT_TMPDIR=/var/tmp/paperless + +# Software tweaks + +#PAPERLESS_TASK_WORKERS=1 +#PAPERLESS_THREADS_PER_WORKER=1 +PAPERLESS_TIME_ZONE=__TIMEZONE__ +#PAPERLESS_CONSUMER_POLLING=10 +#PAPERLESS_CONSUMER_DELETE_DUPLICATES=false +#PAPERLESS_CONSUMER_RECURSIVE=false +PAPERLESS_CONSUMER_IGNORE_PATTERNS=[".DS_STORE/*", "._*", ".stfolder/*", ".*"] +#PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS=false +#PAPERLESS_OPTIMIZE_THUMBNAILS=true +#PAPERLESS_PRE_CONSUME_SCRIPT=/path/to/an/arbitrary/script.sh +#PAPERLESS_POST_CONSUME_SCRIPT=/path/to/an/arbitrary/script.sh +#PAPERLESS_FILENAME_DATE_ORDER=YMD +#PAPERLESS_FILENAME_PARSE_TRANSFORMS=[] +#PAPERLESS_THUMBNAIL_FONT_NAME= +#PAPERLESS_IGNORE_DATES= + +# Tika settings + +#PAPERLESS_TIKA_ENABLED=false +#PAPERLESS_TIKA_ENDPOINT=http://localhost:9998 +#PAPERLESS_TIKA_GOTENBERG_ENDPOINT=http://localhost:3000 + +# Binaries + +#PAPERLESS_CONVERT_BINARY=/usr/bin/convert +#PAPERLESS_GS_BINARY=/usr/bin/gs +#PAPERLESS_OPTIPNG_BINARY=/usr/bin/optipng + +# YunoHost tweaks + +PAPERLESS_LOGOUT_REDIRECT_URL=https://__MAIN_DOMAIN__/yunohost/sso/?action=logout diff --git a/conf/php-fpm.conf b/conf/php-fpm.conf deleted file mode 100644 index ab1a471..0000000 --- a/conf/php-fpm.conf +++ /dev/null @@ -1,430 +0,0 @@ -; Start a new pool named 'www'. -; the variable $pool can be used in any directive and will be replaced by the -; pool name ('www' here) -[__NAMETOCHANGE__] - -; Per pool prefix -; It only applies on the following directives: -; - 'access.log' -; - 'slowlog' -; - 'listen' (unixsocket) -; - 'chroot' -; - 'chdir' -; - 'php_values' -; - 'php_admin_values' -; When not set, the global prefix (or /usr) applies instead. -; Note: This directive can also be relative to the global prefix. -; Default Value: none -;prefix = /path/to/pools/$pool - -; Unix user/group of processes -; Note: The user is mandatory. If the group is not set, the default user's group -; will be used. -user = __USER__ -group = __USER__ - -; The address on which to accept FastCGI requests. -; Valid syntaxes are: -; 'ip.add.re.ss:port' - to listen on a TCP socket to a specific IPv4 address on -; a specific port; -; '[ip:6:addr:ess]:port' - to listen on a TCP socket to a specific IPv6 address on -; a specific port; -; 'port' - to listen on a TCP socket to all addresses -; (IPv6 and IPv4-mapped) on a specific port; -; '/path/to/unix/socket' - to listen on a unix socket. -; Note: This value is mandatory. -listen = /var/run/php/php__PHPVERSION__-fpm-__NAMETOCHANGE__.sock - -; Set listen(2) backlog. -; Default Value: 511 (-1 on FreeBSD and OpenBSD) -;listen.backlog = 511 - -; Set permissions for unix socket, if one is used. In Linux, read/write -; permissions must be set in order to allow connections from a web server. Many -; BSD-derived systems allow connections regardless of permissions. -; Default Values: user and group are set as the running user -; mode is set to 0660 -listen.owner = www-data -listen.group = www-data -;listen.mode = 0660 -; When POSIX Access Control Lists are supported you can set them using -; these options, value is a comma separated list of user/group names. -; When set, listen.owner and listen.group are ignored -;listen.acl_users = -;listen.acl_groups = - -; List of addresses (IPv4/IPv6) of FastCGI clients which are allowed to connect. -; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original -; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address -; must be separated by a comma. If this value is left blank, connections will be -; accepted from any ip address. -; Default Value: any -;listen.allowed_clients = 127.0.0.1 - -; Specify the nice(2) priority to apply to the pool processes (only if set) -; The value can vary from -19 (highest priority) to 20 (lower priority) -; Note: - It will only work if the FPM master process is launched as root -; - The pool processes will inherit the master process priority -; unless it specified otherwise -; Default Value: no set -; process.priority = -19 - -; Set the process dumpable flag (PR_SET_DUMPABLE prctl) even if the process user -; or group is differrent than the master process user. It allows to create process -; core dump and ptrace the process for the pool user. -; Default Value: no -; process.dumpable = yes - -; Choose how the process manager will control the number of child processes. -; Possible Values: -; static - a fixed number (pm.max_children) of child processes; -; dynamic - the number of child processes are set dynamically based on the -; following directives. With this process management, there will be -; always at least 1 children. -; pm.max_children - the maximum number of children that can -; be alive at the same time. -; pm.start_servers - the number of children created on startup. -; pm.min_spare_servers - the minimum number of children in 'idle' -; state (waiting to process). If the number -; of 'idle' processes is less than this -; number then some children will be created. -; pm.max_spare_servers - the maximum number of children in 'idle' -; state (waiting to process). If the number -; of 'idle' processes is greater than this -; number then some children will be killed. -; ondemand - no children are created at startup. Children will be forked when -; new requests will connect. The following parameter are used: -; pm.max_children - the maximum number of children that -; can be alive at the same time. -; pm.process_idle_timeout - The number of seconds after which -; an idle process will be killed. -; Note: This value is mandatory. -pm = dynamic - -; The number of child processes to be created when pm is set to 'static' and the -; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'. -; This value sets the limit on the number of simultaneous requests that will be -; served. Equivalent to the ApacheMaxClients directive with mpm_prefork. -; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP -; CGI. The below defaults are based on a server without much resources. Don't -; forget to tweak pm.* to fit your needs. -; Note: Used when pm is set to 'static', 'dynamic' or 'ondemand' -; Note: This value is mandatory. -pm.max_children = 5 - -; The number of child processes created on startup. -; Note: Used only when pm is set to 'dynamic' -; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2 -pm.start_servers = 2 - -; The desired minimum number of idle server processes. -; Note: Used only when pm is set to 'dynamic' -; Note: Mandatory when pm is set to 'dynamic' -pm.min_spare_servers = 1 - -; The desired maximum number of idle server processes. -; Note: Used only when pm is set to 'dynamic' -; Note: Mandatory when pm is set to 'dynamic' -pm.max_spare_servers = 3 - -; The number of seconds after which an idle process will be killed. -; Note: Used only when pm is set to 'ondemand' -; Default Value: 10s -;pm.process_idle_timeout = 10s; - -; The number of requests each child process should execute before respawning. -; This can be useful to work around memory leaks in 3rd party libraries. For -; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS. -; Default Value: 0 -;pm.max_requests = 500 - -; The URI to view the FPM status page. If this value is not set, no URI will be -; recognized as a status page. It shows the following informations: -; pool - the name of the pool; -; process manager - static, dynamic or ondemand; -; start time - the date and time FPM has started; -; start since - number of seconds since FPM has started; -; accepted conn - the number of request accepted by the pool; -; listen queue - the number of request in the queue of pending -; connections (see backlog in listen(2)); -; max listen queue - the maximum number of requests in the queue -; of pending connections since FPM has started; -; listen queue len - the size of the socket queue of pending connections; -; idle processes - the number of idle processes; -; active processes - the number of active processes; -; total processes - the number of idle + active processes; -; max active processes - the maximum number of active processes since FPM -; has started; -; max children reached - number of times, the process limit has been reached, -; when pm tries to start more children (works only for -; pm 'dynamic' and 'ondemand'); -; Value are updated in real time. -; Example output: -; pool: www -; process manager: static -; start time: 01/Jul/2011:17:53:49 +0200 -; start since: 62636 -; accepted conn: 190460 -; listen queue: 0 -; max listen queue: 1 -; listen queue len: 42 -; idle processes: 4 -; active processes: 11 -; total processes: 15 -; max active processes: 12 -; max children reached: 0 -; -; By default the status page output is formatted as text/plain. Passing either -; 'html', 'xml' or 'json' in the query string will return the corresponding -; output syntax. Example: -; http://www.foo.bar/status -; http://www.foo.bar/status?json -; http://www.foo.bar/status?html -; http://www.foo.bar/status?xml -; -; By default the status page only outputs short status. Passing 'full' in the -; query string will also return status for each pool process. -; Example: -; http://www.foo.bar/status?full -; http://www.foo.bar/status?json&full -; http://www.foo.bar/status?html&full -; http://www.foo.bar/status?xml&full -; The Full status returns for each process: -; pid - the PID of the process; -; state - the state of the process (Idle, Running, ...); -; start time - the date and time the process has started; -; start since - the number of seconds since the process has started; -; requests - the number of requests the process has served; -; request duration - the duration in µs of the requests; -; request method - the request method (GET, POST, ...); -; request URI - the request URI with the query string; -; content length - the content length of the request (only with POST); -; user - the user (PHP_AUTH_USER) (or '-' if not set); -; script - the main script called (or '-' if not set); -; last request cpu - the %cpu the last request consumed -; it's always 0 if the process is not in Idle state -; because CPU calculation is done when the request -; processing has terminated; -; last request memory - the max amount of memory the last request consumed -; it's always 0 if the process is not in Idle state -; because memory calculation is done when the request -; processing has terminated; -; If the process is in Idle state, then informations are related to the -; last request the process has served. Otherwise informations are related to -; the current request being served. -; Example output: -; ************************ -; pid: 31330 -; state: Running -; start time: 01/Jul/2011:17:53:49 +0200 -; start since: 63087 -; requests: 12808 -; request duration: 1250261 -; request method: GET -; request URI: /test_mem.php?N=10000 -; content length: 0 -; user: - -; script: /home/fat/web/docs/php/test_mem.php -; last request cpu: 0.00 -; last request memory: 0 -; -; Note: There is a real-time FPM status monitoring sample web page available -; It's available in: /usr/share/php/7.0/fpm/status.html -; -; Note: The value must start with a leading slash (/). The value can be -; anything, but it may not be a good idea to use the .php extension or it -; may conflict with a real PHP file. -; Default Value: not set -;pm.status_path = /status - -; The ping URI to call the monitoring page of FPM. If this value is not set, no -; URI will be recognized as a ping page. This could be used to test from outside -; that FPM is alive and responding, or to -; - create a graph of FPM availability (rrd or such); -; - remove a server from a group if it is not responding (load balancing); -; - trigger alerts for the operating team (24/7). -; Note: The value must start with a leading slash (/). The value can be -; anything, but it may not be a good idea to use the .php extension or it -; may conflict with a real PHP file. -; Default Value: not set -;ping.path = /ping - -; This directive may be used to customize the response of a ping request. The -; response is formatted as text/plain with a 200 response code. -; Default Value: pong -;ping.response = pong - -; The access log file -; Default: not set -;access.log = log/$pool.access.log - -; The access log format. -; The following syntax is allowed -; %%: the '%' character -; %C: %CPU used by the request -; it can accept the following format: -; - %{user}C for user CPU only -; - %{system}C for system CPU only -; - %{total}C for user + system CPU (default) -; %d: time taken to serve the request -; it can accept the following format: -; - %{seconds}d (default) -; - %{miliseconds}d -; - %{mili}d -; - %{microseconds}d -; - %{micro}d -; %e: an environment variable (same as $_ENV or $_SERVER) -; it must be associated with embraces to specify the name of the env -; variable. Some exemples: -; - server specifics like: %{REQUEST_METHOD}e or %{SERVER_PROTOCOL}e -; - HTTP headers like: %{HTTP_HOST}e or %{HTTP_USER_AGENT}e -; %f: script filename -; %l: content-length of the request (for POST request only) -; %m: request method -; %M: peak of memory allocated by PHP -; it can accept the following format: -; - %{bytes}M (default) -; - %{kilobytes}M -; - %{kilo}M -; - %{megabytes}M -; - %{mega}M -; %n: pool name -; %o: output header -; it must be associated with embraces to specify the name of the header: -; - %{Content-Type}o -; - %{X-Powered-By}o -; - %{Transfert-Encoding}o -; - .... -; %p: PID of the child that serviced the request -; %P: PID of the parent of the child that serviced the request -; %q: the query string -; %Q: the '?' character if query string exists -; %r: the request URI (without the query string, see %q and %Q) -; %R: remote IP address -; %s: status (response code) -; %t: server time the request was received -; it can accept a strftime(3) format: -; %d/%b/%Y:%H:%M:%S %z (default) -; The strftime(3) format must be encapsuled in a %{}t tag -; e.g. for a ISO8601 formatted timestring, use: %{%Y-%m-%dT%H:%M:%S%z}t -; %T: time the log has been written (the request has finished) -; it can accept a strftime(3) format: -; %d/%b/%Y:%H:%M:%S %z (default) -; The strftime(3) format must be encapsuled in a %{}t tag -; e.g. for a ISO8601 formatted timestring, use: %{%Y-%m-%dT%H:%M:%S%z}t -; %u: remote user -; -; Default: "%R - %u %t \"%m %r\" %s" -;access.format = "%R - %u %t \"%m %r%Q%q\" %s %f %{mili}d %{kilo}M %C%%" - -; The log file for slow requests -; Default Value: not set -; Note: slowlog is mandatory if request_slowlog_timeout is set -;slowlog = log/$pool.log.slow - -; The timeout for serving a single request after which a PHP backtrace will be -; dumped to the 'slowlog' file. A value of '0s' means 'off'. -; Available units: s(econds)(default), m(inutes), h(ours), or d(ays) -; Default Value: 0 -;request_slowlog_timeout = 0 - -; The timeout for serving a single request after which the worker process will -; be killed. This option should be used when the 'max_execution_time' ini option -; does not stop script execution for some reason. A value of '0' means 'off'. -; Available units: s(econds)(default), m(inutes), h(ours), or d(ays) -; Default Value: 0 -request_terminate_timeout = 1d - -; Set open file descriptor rlimit. -; Default Value: system defined value -;rlimit_files = 1024 - -; Set max core size rlimit. -; Possible Values: 'unlimited' or an integer greater or equal to 0 -; Default Value: system defined value -;rlimit_core = 0 - -; Chroot to this directory at the start. This value must be defined as an -; absolute path. When this value is not set, chroot is not used. -; Note: you can prefix with '$prefix' to chroot to the pool prefix or one -; of its subdirectories. If the pool prefix is not set, the global prefix -; will be used instead. -; Note: chrooting is a great security feature and should be used whenever -; possible. However, all PHP paths will be relative to the chroot -; (error_log, sessions.save_path, ...). -; Default Value: not set -;chroot = - -; Chdir to this directory at the start. -; Note: relative path can be used. -; Default Value: current directory or / when chroot -chdir = __FINALPATH__ - -; Redirect worker stdout and stderr into main error log. If not set, stdout and -; stderr will be redirected to /dev/null according to FastCGI specs. -; Note: on highloaded environement, this can cause some delay in the page -; process time (several ms). -; Default Value: no -;catch_workers_output = yes - -; Clear environment in FPM workers -; Prevents arbitrary environment variables from reaching FPM worker processes -; by clearing the environment in workers before env vars specified in this -; pool configuration are added. -; Setting to "no" will make all environment variables available to PHP code -; via getenv(), $_ENV and $_SERVER. -; Default Value: yes -;clear_env = no - -; Limits the extensions of the main script FPM will allow to parse. This can -; prevent configuration mistakes on the web server side. You should only limit -; FPM to .php extensions to prevent malicious users to use other extensions to -; execute php code. -; Note: set an empty value to allow all extensions. -; Default Value: .php -;security.limit_extensions = .php .php3 .php4 .php5 .php7 - -; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from -; the current environment. -; Default Value: clean env -;env[HOSTNAME] = $HOSTNAME -;env[PATH] = /usr/local/bin:/usr/bin:/bin -;env[TMP] = /tmp -;env[TMPDIR] = /tmp -;env[TEMP] = /tmp - -; Additional php.ini defines, specific to this pool of workers. These settings -; overwrite the values previously defined in the php.ini. The directives are the -; same as the PHP SAPI: -; php_value/php_flag - you can set classic ini defines which can -; be overwritten from PHP call 'ini_set'. -; php_admin_value/php_admin_flag - these directives won't be overwritten by -; PHP call 'ini_set' -; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no. - -; Defining 'extension' will load the corresponding shared extension from -; extension_dir. Defining 'disable_functions' or 'disable_classes' will not -; overwrite previously defined php.ini values, but will append the new value -; instead. - -; Note: path INI options can be relative and will be expanded with the prefix -; (pool, global or /usr) - -; Default Value: nothing is defined by default except the values in php.ini and -; specified at startup with the -d argument -;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com -;php_flag[display_errors] = off -;php_admin_value[error_log] = /var/log/fpm-php.www.log -;php_admin_flag[log_errors] = on -;php_admin_value[memory_limit] = 32M - -; Common values to change to increase file upload limit -; php_admin_value[upload_max_filesize] = 50M -; php_admin_value[post_max_size] = 50M -; php_admin_flag[mail.add_x_header] = Off - -; Other common parameters -; php_admin_value[max_execution_time] = 600 -; php_admin_value[max_input_time] = 300 -; php_admin_value[memory_limit] = 256M -; php_admin_flag[short_open_tag] = On diff --git a/conf/systemd-consumer.service b/conf/systemd-consumer.service new file mode 100644 index 0000000..f3f4d06 --- /dev/null +++ b/conf/systemd-consumer.service @@ -0,0 +1,43 @@ +[Unit] +Description=Paperless consumer +Requires=redis.service + +[Service] +Type=simple +User=__APP__ +Group=__APP__ +WorkingDirectory=__FINALPATH__/src/ +ExecStart=__FINALPATH__/venv/bin/python3 manage.py document_consumer +StandardOutput=append:/var/log/__APP__/__APP__-consumer.log +StandardError=inherit + +# Sandboxing options to harden security +# Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html +NoNewPrivileges=yes +PrivateTmp=yes +PrivateDevices=yes +RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 +RestrictNamespaces=yes +RestrictRealtime=yes +DevicePolicy=closed +ProtectSystem=full +ProtectControlGroups=yes +ProtectKernelModules=yes +ProtectKernelTunables=yes +LockPersonality=yes +SystemCallFilter=~@clock @debug @module @mount @obsolete @reboot @setuid @swap + +# Denying access to capabilities that should not be relevant for webapps +# Doc: https://man7.org/linux/man-pages/man7/capabilities.7.html +CapabilityBoundingSet=~CAP_RAWIO CAP_MKNOD +CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE +CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_TIME CAP_SYS_MODULE CAP_SYS_PACCT +CapabilityBoundingSet=~CAP_LEASE CAP_LINUX_IMMUTABLE CAP_IPC_LOCK +CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_WAKE_ALARM +CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG +CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE +CapabilityBoundingSet=~CAP_NET_ADMIN CAP_NET_BROADCAST CAP_NET_RAW +CapabilityBoundingSet=~CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_SYSLOG + +[Install] +WantedBy=multi-user.target diff --git a/conf/systemd-scheduler.service b/conf/systemd-scheduler.service new file mode 100644 index 0000000..1bc4d39 --- /dev/null +++ b/conf/systemd-scheduler.service @@ -0,0 +1,43 @@ +[Unit] +Description=Paperless scheduler +Requires=redis.service + +[Service] +Type=simple +User=__APP__ +Group=__APP__ +WorkingDirectory=__FINALPATH__/src/ +ExecStart=__FINALPATH__/venv/bin/celery --app paperless beat --loglevel INFO +StandardOutput=append:/var/log/__APP__/__APP__-scheduler.log +StandardError=inherit + +# Sandboxing options to harden security +# Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html +NoNewPrivileges=yes +PrivateTmp=yes +PrivateDevices=yes +RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 +RestrictNamespaces=yes +RestrictRealtime=yes +DevicePolicy=closed +ProtectSystem=full +ProtectControlGroups=yes +ProtectKernelModules=yes +ProtectKernelTunables=yes +LockPersonality=yes +SystemCallFilter=~@clock @debug @module @mount @obsolete @reboot @setuid @swap + +# Denying access to capabilities that should not be relevant for webapps +# Doc: https://man7.org/linux/man-pages/man7/capabilities.7.html +CapabilityBoundingSet=~CAP_RAWIO CAP_MKNOD +CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE +CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_TIME CAP_SYS_MODULE CAP_SYS_PACCT +CapabilityBoundingSet=~CAP_LEASE CAP_LINUX_IMMUTABLE CAP_IPC_LOCK +CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_WAKE_ALARM +CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG +CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE +CapabilityBoundingSet=~CAP_NET_ADMIN CAP_NET_BROADCAST CAP_NET_RAW +CapabilityBoundingSet=~CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_SYSLOG + +[Install] +WantedBy=multi-user.target diff --git a/conf/systemd-task-queue.service b/conf/systemd-task-queue.service new file mode 100644 index 0000000..ea6d556 --- /dev/null +++ b/conf/systemd-task-queue.service @@ -0,0 +1,44 @@ +[Unit] +Description=Paperless task-queue +Requires=redis.service + +[Service] +Type=simple +User=__APP__ +Group=__APP__ +WorkingDirectory=__FINALPATH__/src/ +ExecStart=__FINALPATH__/venv/bin/celery --app paperless worker --loglevel INFO +StandardOutput=append:/var/log/__APP__/__APP__-task-queue.log +StandardError=inherit + +# Sandboxing options to harden security +# Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html +NoNewPrivileges=yes +#PrivateTmp - Must be disabled for full functionality +#PrivateTmp=yes +PrivateDevices=yes +RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 +RestrictNamespaces=yes +RestrictRealtime=yes +DevicePolicy=closed +ProtectSystem=full +ProtectControlGroups=yes +ProtectKernelModules=yes +ProtectKernelTunables=yes +LockPersonality=yes +SystemCallFilter=~@clock @debug @module @mount @obsolete @reboot @setuid @swap + +# Denying access to capabilities that should not be relevant for webapps +# Doc: https://man7.org/linux/man-pages/man7/capabilities.7.html +CapabilityBoundingSet=~CAP_RAWIO CAP_MKNOD +CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE +CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_TIME CAP_SYS_MODULE CAP_SYS_PACCT +CapabilityBoundingSet=~CAP_LEASE CAP_LINUX_IMMUTABLE CAP_IPC_LOCK +CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_WAKE_ALARM +CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG +CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE +CapabilityBoundingSet=~CAP_NET_ADMIN CAP_NET_BROADCAST CAP_NET_RAW +CapabilityBoundingSet=~CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_SYSLOG + +[Install] +WantedBy=multi-user.target diff --git a/conf/systemd.service b/conf/systemd.service index 31e9da3..636126f 100644 --- a/conf/systemd.service +++ b/conf/systemd.service @@ -1,22 +1,23 @@ [Unit] -Description=Small description of the service +Description=Paperless webserver After=network.target +Wants=network.target +Requires=redis.service [Service] Type=simple User=__APP__ Group=__APP__ -WorkingDirectory=__FINALPATH__/ -ExecStart=__FINALPATH__/script +WorkingDirectory=__FINALPATH__/src/ +ExecStart=__FINALPATH__/venv/bin/uvicorn --port=__PORT__ --log-level=warning paperless.asgi:application StandardOutput=append:/var/log/__APP__/__APP__.log StandardError=inherit # Sandboxing options to harden security -# Depending on specificities of your service/app, you may need to tweak these -# .. but this should be a good baseline # Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html NoNewPrivileges=yes -PrivateTmp=yes +#PrivateTmp - Must be disabled for full functionality +#PrivateTmp=yes PrivateDevices=yes RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 RestrictNamespaces=yes diff --git a/doc/DESCRIPTION.md b/doc/DESCRIPTION.md index 0685205..340e90e 100644 --- a/doc/DESCRIPTION.md +++ b/doc/DESCRIPTION.md @@ -1,9 +1,15 @@ -Some long and extensive description of what the app is and does, lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. +Paperless-ngx is a document management system that transforms your physical documents into a searchable online archive so you can keep, well, *less paper*. ### Features -- Ut enim ad minim veniam, quis nostrud exercitation ullamco ; -- Laboris nisi ut aliquip ex ea commodo consequat ; -- Duis aute irure dolor in reprehenderit in voluptate ; -- Velit esse cillum dolore eu fugiat nulla pariatur ; -- Excepteur sint occaecat cupidatat non proident, sunt in culpa." +* Organize and index your scanned documents with tags, correspondents, types, and more. +* Performs OCR on your documents, adds selectable text to image only documents and adds tags, correspondents and document types to your documents. +* Supports PDF documents, images, plain text files, and Office documents (Word, Excel, Powerpoint, and LibreOffice equivalents). +* Paperless stores your documents plain on disk. Filenames and folders are managed by paperless and their format can be configured freely. +* Single page application front end. +* Full text search helps you find what you need. +* Email processing: Paperless adds documents from your email accounts. +* Machine learning powered document matching. +* Optimized for multi core systems: Paperless-ngx consumes multiple documents in parallel. +* The integrated sanity checker makes sure that your document archive is in good health. +* [More screenshots are available in the documentation](https://paperless-ngx.readthedocs.io/en/latest/screenshots.html). diff --git a/doc/DISCLAIMER.md b/doc/DISCLAIMER.md index aded581..45eb9f5 100644 --- a/doc/DISCLAIMER.md +++ b/doc/DISCLAIMER.md @@ -1,12 +1,13 @@ -* Any known limitations, constrains or stuff not working, such as (but not limited to): - * requiring a full dedicated domain ? - * architectures not supported ? - * not-working single-sign on or LDAP integration ? - * the app requires an important amount of RAM / disk / .. to install or to work properly - * etc... +* This app require a dedicated domain. +* There is no LDAP or SSO support. -* Other infos that people should be aware of, such as: - * any specific step to perform after installing (such as manually finishing the install, specific admin credentials, ...) - * how to configure / administrate the application if it ain't obvious - * upgrade process / specificities / things to be aware of ? - * security considerations ? +* Paperless is performing OCR on documents and images. English is installed by default. More languages can be installed: + * Display a list of all Tesseract language packs `apt-cache search tesseract-ocr` + * Install additional language packs + * Example for french `sudo apt-get install tesseract-ocr-fra` + * Example for german `sudo apt-get install tesseract-ocr-deu` + * Modify config to add new languages + * Open config-panel: https://my-domain.tld/yunohost/admin/#/apps/$app_id/config-panel + * You can combine multiple languages like this: + * One language: eng + * Two languages: eng+fra diff --git a/doc/screenshots/documents-wchrome-dark.png b/doc/screenshots/documents-wchrome-dark.png new file mode 100644 index 0000000..51bd617 Binary files /dev/null and b/doc/screenshots/documents-wchrome-dark.png differ diff --git a/doc/screenshots/example.jpg b/doc/screenshots/example.jpg deleted file mode 100644 index a1efa1a..0000000 Binary files a/doc/screenshots/example.jpg and /dev/null differ diff --git a/manifest.json b/manifest.json index c729a06..00c3583 100644 --- a/manifest.json +++ b/manifest.json @@ -6,7 +6,7 @@ "en": "Scan, index and archive all your physical documents", "fr": "Scannez, triez et archivez tous vos documents papiers" }, - "version": "1.6~ynh1", + "version": "1.11.0~ynh1", "url": "https://paperless-ngx.com", "upstream": { "license": "GPL-3.0-or-later", @@ -21,7 +21,7 @@ "name": "Tagada" }, "requirements": { - "yunohost": ">= 4.3.0" + "yunohost": ">= 11.0.0" }, "multi_instance": true, "services": [ @@ -38,6 +38,26 @@ { "name": "admin", "type": "user" + }, + { + "name": "admin_pw", + "type": "password", + "ask": { + "en": "Please chose a password for admin user." + } + }, + { + "name": "is_public", + "type": "boolean", + "default": false + }, + { + "name": "is_public_api", + "type": "boolean", + "default": false, + "ask": { + "en": "Should Paperless-ngx be accessible for apps." + } } ] } diff --git a/scripts/_common.sh b/scripts/_common.sh index 7e55ac0..ff70f8e 100644 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -5,7 +5,12 @@ #================================================= # dependencies used by the app -pkg_dependencies="deb1 deb2 php$YNH_DEFAULT_PHP_VERSION-deb1 php$YNH_DEFAULT_PHP_VERSION-deb2" +pkg_dependencies="python3 python3-pip python3-dev python3-venv default-libmysqlclient-dev fonts-liberation imagemagick gnupg libpq-dev libmagic-dev mime-support libzbar0 poppler-utils postgresql postgresql-contrib " +ocr_pkg_dependencies="unpaper ghostscript icc-profiles-free qpdf liblept5 libxml2 pngquant zlib1g tesseract-ocr" +raspberry_pkg_dependencies="libatlas-base-dev libxslt1-dev" + +timezone=$(cat /etc/timezone) +main_domain=$(cat /etc/yunohost/current_host) #================================================= # PERSONAL HELPERS diff --git a/scripts/backup b/scripts/backup index f99225d..f2ae68d 100755 --- a/scripts/backup +++ b/scripts/backup @@ -31,7 +31,6 @@ app=$YNH_APP_INSTANCE_NAME final_path=$(ynh_app_setting_get --app=$app --key=final_path) domain=$(ynh_app_setting_get --app=$app --key=domain) db_name=$(ynh_app_setting_get --app=$app --key=db_name) -phpversion=$(ynh_app_setting_get --app=$app --key=phpversion) datadir=$(ynh_app_setting_get --app=$app --key=datadir) #================================================= @@ -62,18 +61,12 @@ ynh_backup --src_path="$datadir" --is_big ynh_backup --src_path="/etc/nginx/conf.d/$domain.d/$app.conf" -#================================================= -# BACKUP THE PHP-FPM CONFIGURATION -#================================================= - -ynh_backup --src_path="/etc/php/$phpversion/fpm/pool.d/$app.conf" - #================================================= # BACKUP FAIL2BAN CONFIGURATION #================================================= -ynh_backup --src_path="/etc/fail2ban/jail.d/$app.conf" -ynh_backup --src_path="/etc/fail2ban/filter.d/$app.conf" +#ynh_backup --src_path="/etc/fail2ban/jail.d/$app.conf" +#ynh_backup --src_path="/etc/fail2ban/filter.d/$app.conf" #================================================= # SPECIFIC BACKUP @@ -88,14 +81,15 @@ ynh_backup --src_path="/etc/logrotate.d/$app" #================================================= ynh_backup --src_path="/etc/systemd/system/$app.service" +ynh_backup --src_path="/etc/systemd/system/$app-consumer.service" +ynh_backup --src_path="/etc/systemd/system/$app-scheduler.service" +ynh_backup --src_path="/etc/systemd/system/$app-task-queue.service" #================================================= # BACKUP VARIOUS FILES #================================================= -ynh_backup --src_path="/etc/cron.d/$app" - -ynh_backup --src_path="/etc/$app/" +#ynh_backup --src_path="/etc/$app/" #================================================= # BACKUP THE MYSQL DATABASE @@ -105,7 +99,7 @@ ynh_print_info --message="Backing up the MySQL database..." ### (However, things like MySQL dumps *do* take some time to run, though the ### copy of the generated dump to the archive still happens later) -ynh_mysql_dump_db --database="$db_name" > db.sql +ynh_psql_dump_db --database="$db_name" > db.sql #================================================= # END OF SCRIPT diff --git a/scripts/change_url b/scripts/change_url index 495b7eb..18bf00c 100644 --- a/scripts/change_url +++ b/scripts/change_url @@ -17,27 +17,29 @@ old_domain=$YNH_APP_OLD_DOMAIN old_path=$YNH_APP_OLD_PATH new_domain=$YNH_APP_NEW_DOMAIN -new_path=$YNH_APP_NEW_PATH +new_path="/" app=$YNH_APP_INSTANCE_NAME #================================================= # LOAD SETTINGS #================================================= -ynh_script_progression --message="Loading installation settings..." --time --weight=1 +ynh_script_progression --message="Loading installation settings..." --weight=1 # Needed for helper "ynh_add_nginx_config" final_path=$(ynh_app_setting_get --app=$app --key=final_path) - -# Add settings here as needed by your application -#db_name=$(ynh_app_setting_get --app=$app --key=db_name) -#db_user=$db_name -#db_pwd=$(ynh_app_setting_get --app=$app --key=db_pwd) +datadir=$(ynh_app_setting_get --app=$app --key=datadir) +port=$(ynh_app_setting_get --app=$app --key=port) +db_name=$(ynh_app_setting_get --app=$app --key=db_name) +db_user=$db_name +db_pwd=$(ynh_app_setting_get --app=$app --key=psqlpwd) +redis_db=$(ynh_app_setting_get --app=$app --key=redis_db) +paperless_secret_key=$(ynh_app_setting_get --app=$app --key=paperless_secret_key) #================================================= # BACKUP BEFORE CHANGE URL THEN ACTIVE TRAP #================================================= -ynh_script_progression --message="Backing up the app before changing its URL (may take a while)..." --time --weight=1 +ynh_script_progression --message="Backing up the app before changing its URL (may take a while)..." --weight=1 # Backup the current version of the app ynh_backup_before_upgrade @@ -72,14 +74,17 @@ fi #================================================= # STOP SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Stopping a systemd service..." --time --weight=1 +ynh_script_progression --message="Stopping a systemd service..." --weight=1 -ynh_systemd_action --service_name=$app --action="stop" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="stop" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="stop" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="stop" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="stop" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # MODIFY URL IN NGINX CONF #================================================= -ynh_script_progression --message="Updating NGINX web server configuration..." --time --weight=1 +ynh_script_progression --message="Updating NGINX web server configuration..." --weight=1 nginx_conf_path=/etc/nginx/conf.d/$old_domain.d/$app.conf @@ -108,22 +113,34 @@ fi #================================================= # SPECIFIC MODIFICATIONS #================================================= -# ... +# MODIFY A CONFIGURATION #================================================= +ynh_script_progression --message="Modifying a configuration file..." --weight=1 + +domain="$new_domain" +path_url="$new_path" + +ynh_add_config --template="paperless.conf.example" --destination="$final_path/paperless.conf" + +chmod 400 "$final_path/paperless.conf" +chown $app:$app "$final_path/paperless.conf" #================================================= # GENERIC FINALISATION #================================================= # START SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Starting a systemd service..." --time --weight=1 +ynh_script_progression --message="Starting a systemd service..." --weight=1 -ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="start" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="start" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="start" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # RELOAD NGINX #================================================= -ynh_script_progression --message="Reloading NGINX web server..." --time --weight=1 +ynh_script_progression --message="Reloading NGINX web server..." --weight=1 ynh_systemd_action --service_name=nginx --action=reload @@ -131,4 +148,4 @@ ynh_systemd_action --service_name=nginx --action=reload # END OF SCRIPT #================================================= -ynh_script_progression --message="Change of URL completed for $app" --time --last +ynh_script_progression --message="Change of URL completed for $app" --last diff --git a/scripts/install b/scripts/install index 97c0584..05a1509 100755 --- a/scripts/install +++ b/scripts/install @@ -7,6 +7,7 @@ #================================================= source _common.sh +source ynh_redis source /usr/share/yunohost/helpers #================================================= @@ -25,40 +26,20 @@ ynh_abort_if_errors #================================================= domain=$YNH_APP_ARG_DOMAIN -path_url=$YNH_APP_ARG_PATH +path_url="/" is_public=$YNH_APP_ARG_IS_PUBLIC -language=$YNH_APP_ARG_LANGUAGE +is_public_api=$YNH_APP_ARG_IS_PUBLIC_API admin=$YNH_APP_ARG_ADMIN -password=$YNH_APP_ARG_PASSWORD +admin_pw=$YNH_APP_ARG_ADMIN_PW -### If it's a multi-instance app, meaning it can be installed several times independently -### The id of the app as stated in the manifest is available as $YNH_APP_ID -### The instance number is available as $YNH_APP_INSTANCE_NUMBER (equals "1", "2"...) -### The app instance name is available as $YNH_APP_INSTANCE_NAME -### - the first time the app is installed, YNH_APP_INSTANCE_NAME = ynhexample -### - the second time the app is installed, YNH_APP_INSTANCE_NAME = ynhexample__2 -### - ynhexample__{N} for the subsequent installations, with N=3,4... -### The app instance name is probably what interests you most, since this is -### guaranteed to be unique. This is a good unique identifier to define installation path, -### db names... app=$YNH_APP_INSTANCE_NAME #================================================= # CHECK IF THE APP CAN BE INSTALLED WITH THESE ARGS #================================================= -### About --weight and --time -### ynh_script_progression will show to your final users the progression of each scripts. -### In order to do that, --weight will represent the relative time of execution compared to the other steps in the script. -### --time is a packager option, it will show you the execution time since the previous call. -### This option should be removed before releasing your app. -### Use the execution time, given by --time, to estimate the weight of a step. -### A common way to do it is to set a weight equal to the execution time in second +1. -### The execution time is given for the duration since the previous call. So the weight should be applied to this previous call. -ynh_script_progression --message="Validating installation parameters..." --time --weight=1 +ynh_script_progression --message="Validating installation parameters..." --weight=1 -### If the app uses NGINX as web server (written in HTML/PHP in most cases), the final path should be "/var/www/$app". -### If the app provides an internal web server (or uses another application server such as uWSGI), the final path should be "/opt/yunohost/$app" -final_path=/var/www/$app +final_path=/opt/yunohost/$app test ! -e "$final_path" || ynh_die --message="This path already contains a folder" # Register (book) web path @@ -67,11 +48,10 @@ ynh_webpath_register --app=$app --domain=$domain --path_url=$path_url #================================================= # STORE SETTINGS FROM MANIFEST #================================================= -ynh_script_progression --message="Storing installation settings..." --time --weight=1 +ynh_script_progression --message="Storing installation settings..." --weight=1 ynh_app_setting_set --app=$app --key=domain --value=$domain ynh_app_setting_set --app=$app --key=path --value=$path_url -ynh_app_setting_set --app=$app --key=language --value=$language ynh_app_setting_set --app=$app --key=admin --value=$admin #================================================= @@ -79,149 +59,91 @@ ynh_app_setting_set --app=$app --key=admin --value=$admin #================================================= # FIND AND OPEN A PORT #================================================= -ynh_script_progression --message="Finding an available port..." --time --weight=1 - -### Use these lines if you have to open a port for the application -### `ynh_find_port` will find the first available port starting from the given port. -### If you're not using these lines: -### - Remove the section "CLOSE A PORT" in the remove script +ynh_script_progression --message="Finding an available port..." --weight=1 # Find an available port port=$(ynh_find_port --port=8095) ynh_app_setting_set --app=$app --key=port --value=$port -# Optional: Expose this port publicly -# (N.B.: you only need to do this if the app actually needs to expose the port publicly. -# If you do this and the app doesn't actually need you are CREATING SECURITY HOLES IN THE SERVER !) - -# Open the port -# ynh_script_progression --message="Configuring firewall..." --time --weight=1 -# ynh_exec_warn_less yunohost firewall allow --no-upnp TCP $port - #================================================= # INSTALL DEPENDENCIES #================================================= -ynh_script_progression --message="Installing dependencies..." --time --weight=1 +ynh_script_progression --message="Installing dependencies..." --weight=1 -### `ynh_install_app_dependencies` allows you to add any "apt" dependencies to the package. -### Those deb packages will be installed as dependencies of this package. -### If you're not using this helper: -### - Remove the section "REMOVE DEPENDENCIES" in the remove script -### - Remove the variable "pkg_dependencies" in _common.sh -### - As well as the section "REINSTALL DEPENDENCIES" in the restore script -### - And the section "UPGRADE DEPENDENCIES" in the upgrade script - -ynh_install_app_dependencies $pkg_dependencies +# FIXME: Only on a Raspberry Pi (armv6 v7?) +# ynh_add_app_dependencies $raspberry_pkg_dependencies +ynh_exec_warn_less ynh_install_app_dependencies $pkg_dependencies $ocr_pkg_dependencies #================================================= # CREATE DEDICATED USER #================================================= -ynh_script_progression --message="Configuring system user..." --time --weight=1 +ynh_script_progression --message="Configuring system user..." --weight=1 # Create a system user ynh_system_user_create --username=$app --home_dir="$final_path" #================================================= -# CREATE A MYSQL DATABASE +# CREATE A POSTGRESQL DATABASE #================================================= -ynh_script_progression --message="Creating a MySQL database..." --time --weight=1 - -### Use these lines if you need a database for the application. -### `ynh_mysql_setup_db` will create a database, an associated user and a ramdom password. -### The password will be stored as 'mysqlpwd' into the app settings, -### and will be available as $db_pwd -### If you're not using these lines: -### - Remove the section "BACKUP THE MYSQL DATABASE" in the backup script -### - Remove also the section "REMOVE THE MYSQL DATABASE" in the remove script -### - As well as the section "RESTORE THE MYSQL DATABASE" in the restore script +ynh_script_progression --message="Creating a PostgreSQL database..." --weight=1 db_name=$(ynh_sanitize_dbid --db_name=$app) db_user=$db_name ynh_app_setting_set --app=$app --key=db_name --value=$db_name -ynh_mysql_setup_db --db_user=$db_user --db_name=$db_name + +ynh_psql_setup_db --db_user=$db_user --db_name=$db_name #================================================= # DOWNLOAD, CHECK AND UNPACK SOURCE #================================================= -ynh_script_progression --message="Setting up source files..." --time --weight=1 - -### `ynh_setup_source` is used to install an app from a zip or tar.gz file, -### downloaded from an upstream source, like a git repository. -### `ynh_setup_source` use the file conf/app.src +ynh_script_progression --message="Setting up source files..." --weight=1 ynh_app_setting_set --app=$app --key=final_path --value=$final_path -# Download, check integrity, uncompress and patch the source from app.src + ynh_setup_source --dest_dir="$final_path" -# FIXME: this should be managed by the core in the future -# Here, as a packager, you may have to tweak the ownerhsip/permissions -# such that the appropriate users (e.g. maybe www-data) can access -# files in some cases. -# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder - -# this will be treated as a security issue. chmod 750 "$final_path" chmod -R o-rwx "$final_path" -chown -R $app:www-data "$final_path" +chown -R $app:$app "$final_path" #================================================= # NGINX CONFIGURATION #================================================= -ynh_script_progression --message="Configuring NGINX web server..." --time --weight=1 - -### `ynh_add_nginx_config` will use the file conf/nginx.conf +ynh_script_progression --message="Configuring NGINX web server..." --weight=1 # Create a dedicated NGINX config ynh_add_nginx_config -#================================================= -# PHP-FPM CONFIGURATION -#================================================= -ynh_script_progression --message="Configuring PHP-FPM..." --time --weight=1 - -### `ynh_add_fpm_config` is used to set up a PHP config. -### You can remove it if your app doesn't use PHP. -### `ynh_add_fpm_config` will use the files conf/php-fpm.conf -### If you're not using these lines: -### - You can remove these files in conf/. -### - Remove the section "BACKUP THE PHP-FPM CONFIGURATION" in the backup script -### - Remove also the section "REMOVE PHP-FPM CONFIGURATION" in the remove script -### - As well as the section "RESTORE THE PHP-FPM CONFIGURATION" in the restore script -### with the reload at the end of the script. -### - And the section "PHP-FPM CONFIGURATION" in the upgrade script - -# Create a dedicated PHP-FPM config -ynh_add_fpm_config - #================================================= # SPECIFIC SETUP #================================================= -# ... +# INSTALL PYTHON DEPENDENCIES #================================================= +ynh_script_progression --message="Installing Python dependencies..." + +pushd $final_path + python3 -m venv venv + chown -R "$app:" "$final_path" +( + source "$final_path/venv/bin/activate" + ynh_exec_as $app $final_path/venv/bin/pip3 install --upgrade pip setuptools wheel + ynh_exec_as $app $final_path/venv/bin/pip3 install -r "$final_path/requirements.txt" + deactivate +) +popd #================================================= # CREATE DATA DIRECTORY #================================================= -ynh_script_progression --message="Creating a data directory..." --time --weight=1 - -### Use these lines if you need to create a directory to store "persistent files" for the application. -### Usually this directory is used to store uploaded files or any file that won't be updated during -### an upgrade and that won't be deleted during app removal unless "--purge" option is used. -### If you're not using these lines: -### - Remove the section "BACKUP THE DATA DIR" in the backup script -### - Remove the section "RESTORE THE DATA DIRECTORY" in the restore script -### - As well as the section "REMOVE DATA DIR" in the remove script +ynh_script_progression --message="Creating a data directory..." --weight=1 datadir=/home/yunohost.app/$app ynh_app_setting_set --app=$app --key=datadir --value=$datadir -mkdir -p $datadir +mkdir -p "$datadir/consume" +mkdir -p "$datadir/data" +mkdir -p "$datadir/media" -# FIXME: this should be managed by the core in the future -# Here, as a packager, you may have to tweak the ownerhsip/permissions -# such that the appropriate users (e.g. maybe www-data) can access -# files in some cases. -# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder - -# this will be treated as a security issue. chmod 750 "$datadir" chmod -R o-rwx "$datadir" chown -R $app:www-data "$datadir" @@ -229,89 +151,63 @@ chown -R $app:www-data "$datadir" #================================================= # ADD A CONFIGURATION #================================================= -ynh_script_progression --message="Adding a configuration file..." --time --weight=1 +ynh_script_progression --message="Adding a configuration file..." --weight=1 -### You can add specific configuration files. -### -### Typically, put your template conf file in ../conf/your_config_file -### The template may contain strings such as __FOO__ or __FOO_BAR__, -### which will automatically be replaced by the values of $foo and $foo_bar -### -### ynh_add_config will also keep track of the config file's checksum, -### which later during upgrade may allow to automatically backup the config file -### if it's found that the file was manually modified -### -### Check the documentation of `ynh_add_config` for more info. +redis_db=$(ynh_redis_get_free_db) +ynh_app_setting_set --app=$app --key=redis_db --value=$redis_db -ynh_add_config --template="some_config_file" --destination="$final_path/some_config_file" +paperless_secret_key=$(ynh_string_random) +ynh_app_setting_set --app=$app --key=paperless_secret_key --value=$paperless_secret_key -# FIXME: this should be handled by the core in the future -# You may need to use chmod 600 instead of 400, -# for example if the app is expected to be able to modify its own config -chmod 400 "$final_path/some_config_file" -chown $app:$app "$final_path/some_config_file" +ynh_add_config --template="paperless.conf.example" --destination="$final_path/paperless.conf" -### For more complex cases where you want to replace stuff using regexes, -### you shoud rely on ynh_replace_string (which is basically a wrapper for sed) -### When doing so, you also need to manually call ynh_store_file_checksum -### -### ynh_replace_string --match_string="match_string" --replace_string="replace_string" --target_file="$final_path/some_config_file" -### ynh_store_file_checksum --file="$final_path/some_config_file" +chmod 400 "$final_path/paperless.conf" +chown $app:$app "$final_path/paperless.conf" + +#================================================= +# SETUP THE DATABASE +#================================================= +ynh_script_progression --message="Setting up the database..." --weight=1 + +pushd "$final_path/src" +( + source "$final_path/venv/bin/activate" + ynh_exec_as $app $final_path/venv/bin/python manage.py migrate + deactivate +) +popd + +#================================================= +# CREATE THE ADMIN USER +#================================================= +ynh_script_progression --message="Creating the admin user..." --weight=1 + +pushd "$final_path/src" +( + source "$final_path/venv/bin/activate" + email=$(ynh_user_get_info $admin 'mail') + ynh_exec_as $app env "DJANGO_SUPERUSER_PASSWORD=$admin_pw" $final_path/venv/bin/python3 manage.py createsuperuser --noinput --username "$admin" --email "$email" + deactivate +) +popd #================================================= # SETUP SYSTEMD #================================================= -ynh_script_progression --message="Configuring a systemd service..." --time --weight=1 - -### `ynh_systemd_config` is used to configure a systemd script for an app. -### It can be used for apps that use sysvinit (with adaptation) or systemd. -### Have a look at the app to be sure this app needs a systemd script. -### `ynh_systemd_config` will use the file conf/systemd.service -### If you're not using these lines: -### - You can remove those files in conf/. -### - Remove the section "BACKUP SYSTEMD" in the backup script -### - Remove also the section "STOP AND REMOVE SERVICE" in the remove script -### - As well as the section "RESTORE SYSTEMD" in the restore script -### - And the section "SETUP SYSTEMD" in the upgrade script +ynh_script_progression --message="Configuring a systemd service..." --weight=1 # Create a dedicated systemd config -ynh_add_systemd_config - -#================================================= -# SETUP APPLICATION WITH CURL -#================================================= - -### Use these lines only if the app installation needs to be finalized through -### web forms. We generally don't want to ask the final user, -### so we're going to use curl to automatically fill the fields and submit the -### forms. - -# Set the app as temporarily public for curl call -ynh_script_progression --message="Configuring SSOwat..." --time --weight=1 -# Making the app public for curl -ynh_permission_update --permission="main" --add="visitors" - -# Installation with curl -ynh_script_progression --message="Finalizing installation..." --time --weight=1 -ynh_local_curl "/INSTALL_PATH" "key1=value1" "key2=value2" "key3=value3" - -# Remove the public access -ynh_permission_update --permission="main" --remove="visitors" +ynh_add_systemd_config --service="$app" --template="systemd.service" +ynh_add_systemd_config --service="$app-consumer" --template="systemd-consumer.service" +ynh_add_systemd_config --service="$app-scheduler" --template="systemd-scheduler.service" +ynh_add_systemd_config --service="$app-task-queue" --template="systemd-task-queue.service" #================================================= # GENERIC FINALIZATION #================================================= # SETUP LOGROTATE #================================================= -ynh_script_progression --message="Configuring log rotation..." --time --weight=1 - -### `ynh_use_logrotate` is used to configure a logrotate configuration for the logs of this app. -### Use this helper only if there is effectively a log file for this app. -### If you're not using this helper: -### - Remove the section "BACKUP LOGROTATE" in the backup script -### - Remove also the section "REMOVE LOGROTATE CONFIGURATION" in the remove script -### - As well as the section "RESTORE THE LOGROTATE CONFIGURATION" in the restore script -### - And the section "SETUP LOGROTATE" in the upgrade script +ynh_script_progression --message="Configuring log rotation..." --weight=1 # Use logrotate to manage application logfile(s) ynh_use_logrotate @@ -319,87 +215,54 @@ ynh_use_logrotate #================================================= # INTEGRATE SERVICE IN YUNOHOST #================================================= -ynh_script_progression --message="Integrating service in YunoHost..." --time --weight=1 +ynh_script_progression --message="Integrating service in YunoHost..." --weight=1 -### `yunohost service add` integrates a service in YunoHost. It then gets -### displayed in the admin interface and through the others `yunohost service` commands. -### (N.B.: this line only makes sense if the app adds a service to the system!) -### If you're not using these lines: -### - You can remove these files in conf/. -### - Remove the section "REMOVE SERVICE INTEGRATION IN YUNOHOST" in the remove script -### - As well as the section "INTEGRATE SERVICE IN YUNOHOST" in the restore script -### - And the section "INTEGRATE SERVICE IN YUNOHOST" in the upgrade script - -yunohost service add $app --description="A short description of the app" --log="/var/log/$app/$app.log" - -### Additional options starting with 3.8: -### -### --needs_exposed_ports "$port" a list of ports that needs to be publicly exposed -### which will then be checked by YunoHost's diagnosis system -### (N.B. DO NOT USE THIS is the port is only internal!!!) -### -### --test_status "some command" a custom command to check the status of the service -### (only relevant if 'systemctl status' doesn't do a good job) -### -### --test_conf "some command" some command similar to "nginx -t" that validates the conf of the service -### -### Re-calling 'yunohost service add' during the upgrade script is the right way -### to proceed if you later realize that you need to enable some flags that -### weren't enabled on old installs (be careful it'll override the existing -### service though so you should re-provide all relevant flags when doing so) +yunohost service add "$app" --log="/var/log/$app/$app.log" +yunohost service add "$app-consumer" --log="/var/log/$app/$app-consumer.log" +yunohost service add "$app-scheduler" --log="/var/log/$app/$app-scheduler.log" +yunohost service add "$app-task-queue" --log="/var/log/$app/$app-task-queue.log" #================================================= # START SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Starting a systemd service..." --time --weight=1 +ynh_script_progression --message="Starting a systemd service..." --weight=1 -### `ynh_systemd_action` is used to start a systemd service for an app. -### Only needed if you have configure a systemd service -### If you're not using these lines: -### - Remove the section "STOP SYSTEMD SERVICE" and "START SYSTEMD SERVICE" in the backup script -### - As well as the section "START SYSTEMD SERVICE" in the restore script -### - As well as the section"STOP SYSTEMD SERVICE" and "START SYSTEMD SERVICE" in the upgrade script -### - And the section "STOP SYSTEMD SERVICE" and "START SYSTEMD SERVICE" in the change_url script - -# Start a systemd service -ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="start" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="start" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="start" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # SETUP FAIL2BAN #================================================= -ynh_script_progression --message="Configuring Fail2Ban..." --time --weight=1 +#ynh_script_progression --message="Configuring Fail2Ban..." --weight=1 # Create a dedicated Fail2Ban config -ynh_add_fail2ban_config --logpath="/var/log/nginx/${domain}-error.log" --failregex="Regex to match into the log for a failed login" +#ynh_add_fail2ban_config --logpath="/var/log/nginx/${domain}-error.log" --failregex="Regex to match into the log for a failed login" +# FIXME fail2ban +# ynh_add_fail2ban_config --logpath="/var/log/nginx/${domain}-access.log" --failregex=".* \"POST /api/v1/token/ HTTP/1.1\" 400 68.*$" --max_retry=5 #================================================= # SETUP SSOWAT #================================================= -ynh_script_progression --message="Configuring permissions..." --time --weight=1 +ynh_script_progression --message="Configuring permissions..." --weight=1 # Make app public if necessary if [ $is_public -eq 1 ] then - # Everyone can access the app. - # The "main" permission is automatically created before the install script. ynh_permission_update --permission="main" --add="visitors" fi -### N.B. : the following extra permissions only make sense if your app -### does have for example an admin interface or an API. - -# Only the admin can access the admin panel of the app (if the app has an admin panel) -ynh_permission_create --permission="admin" --url="/admin" --allowed=$admin - -# Everyone can access the API part -# We don't want to display the tile in the SSO so we put --show_tile="false" -# And we don't want the YunoHost admin to be able to remove visitors group to this permission, so we put --protected="true" -ynh_permission_create --permission="api" --url="/api" --allowed="visitors" --show_tile="false" --protected="true" +ynh_permission_create --permission="api" --url="/api" --allowed="all_users" --auth_header="false" --label="$app API" --show_tile="false" --protected="false" +if [ $is_public_api -eq 1 ] +then + ynh_permission_update --permission="api" --add="visitors" +fi #================================================= # RELOAD NGINX #================================================= -ynh_script_progression --message="Reloading NGINX web server..." --time --weight=1 +ynh_script_progression --message="Reloading NGINX web server..." --weight=1 ynh_systemd_action --service_name=nginx --action=reload @@ -407,4 +270,4 @@ ynh_systemd_action --service_name=nginx --action=reload # END OF SCRIPT #================================================= -ynh_script_progression --message="Installation of $app completed" --time --last +ynh_script_progression --message="Installation of $app completed" --last diff --git a/scripts/remove b/scripts/remove index 29da1fd..0730658 100755 --- a/scripts/remove +++ b/scripts/remove @@ -7,12 +7,13 @@ #================================================= source _common.sh +source ynh_redis source /usr/share/yunohost/helpers #================================================= # LOAD SETTINGS #================================================= -ynh_script_progression --message="Loading installation settings..." --time --weight=1 +ynh_script_progression --message="Loading installation settings..." --weight=1 app=$YNH_APP_INSTANCE_NAME @@ -22,6 +23,7 @@ db_name=$(ynh_app_setting_get --app=$app --key=db_name) db_user=$db_name final_path=$(ynh_app_setting_get --app=$app --key=final_path) datadir=$(ynh_app_setting_get --app=$app --key=datadir) +redis_db=$(ynh_app_setting_get --app=$app --key=redis_db) #================================================= # STANDARD REMOVE @@ -32,38 +34,44 @@ datadir=$(ynh_app_setting_get --app=$app --key=datadir) # Remove the service from the list of services known by YunoHost (added from `yunohost service add`) if ynh_exec_warn_less yunohost service status $app >/dev/null then - ynh_script_progression --message="Removing $app service integration..." --time --weight=1 - yunohost service remove $app + ynh_script_progression --message="Removing $app service integration..." --weight=1 + yunohost service remove "$app" + yunohost service remove "$app-consumer" + yunohost service remove "$app-scheduler" + yunohost service remove "$app-task-queue" fi #================================================= # STOP AND REMOVE SERVICE #================================================= -ynh_script_progression --message="Stopping and removing the systemd service..." --time --weight=1 +ynh_script_progression --message="Stopping and removing the systemd service..." --weight=1 # Remove the dedicated systemd config -ynh_remove_systemd_config +ynh_remove_systemd_config --service="$app" +ynh_remove_systemd_config --service="$app-consumer" +ynh_remove_systemd_config --service="$app-scheduler" +ynh_remove_systemd_config --service="$app-task-queue" #================================================= # REMOVE LOGROTATE CONFIGURATION #================================================= -ynh_script_progression --message="Removing logrotate configuration..." --time --weight=1 +ynh_script_progression --message="Removing logrotate configuration..." --weight=1 # Remove the app-specific logrotate config ynh_remove_logrotate #================================================= -# REMOVE THE MYSQL DATABASE +# REMOVE THE POSTGRESQL DATABASE #================================================= -ynh_script_progression --message="Removing the MySQL database..." --time --weight=1 +ynh_script_progression --message="Removing the PostgreSQL database..." --weight=1 # Remove a database if it exists, along with the associated user -ynh_mysql_remove_db --db_user=$db_user --db_name=$db_name +ynh_psql_remove_db --db_user=$db_user --db_name=$db_name #================================================= # REMOVE APP MAIN DIR #================================================= -ynh_script_progression --message="Removing app main directory..." --time --weight=1 +ynh_script_progression --message="Removing app main directory..." --weight=1 # Remove the app directory securely ynh_secure_remove --file="$final_path" @@ -75,64 +83,44 @@ ynh_secure_remove --file="$final_path" # Remove the data directory if --purge option is used if [ "${YNH_APP_PURGE:-0}" -eq 1 ] then - ynh_script_progression --message="Removing app data directory..." --time --weight=1 + ynh_script_progression --message="Removing app data directory..." --weight=1 ynh_secure_remove --file="$datadir" fi #================================================= # REMOVE NGINX CONFIGURATION #================================================= -ynh_script_progression --message="Removing NGINX web server configuration..." --time --weight=1 +ynh_script_progression --message="Removing NGINX web server configuration..." --weight=1 # Remove the dedicated NGINX config ynh_remove_nginx_config -#================================================= -# REMOVE PHP-FPM CONFIGURATION -#================================================= -ynh_script_progression --message="Removing PHP-FPM configuration..." --time --weight=1 - -# Remove the dedicated PHP-FPM config -ynh_remove_fpm_config - #================================================= # REMOVE DEPENDENCIES #================================================= -ynh_script_progression --message="Removing dependencies..." --time --weight=1 +ynh_script_progression --message="Removing dependencies..." --weight=1 # Remove metapackage and its dependencies +ynh_redis_remove_db "$redis_db" ynh_remove_app_dependencies -#================================================= -# CLOSE A PORT -#================================================= - -if yunohost firewall list | grep -q "\- $port$" -then - ynh_script_progression --message="Closing port $port..." --time --weight=1 - ynh_exec_warn_less yunohost firewall disallow TCP $port -fi - #================================================= # REMOVE FAIL2BAN CONFIGURATION #================================================= -ynh_script_progression --message="Removing Fail2Ban configuration..." --time --weight=1 +#ynh_script_progression --message="Removing Fail2Ban configuration..." --weight=1 # Remove the dedicated Fail2Ban config -ynh_remove_fail2ban_config +#ynh_remove_fail2ban_config #================================================= # SPECIFIC REMOVE #================================================= # REMOVE VARIOUS FILES #================================================= -ynh_script_progression --message="Removing various files..." --time --weight=1 - -# Remove a cron file -ynh_secure_remove --file="/etc/cron.d/$app" +ynh_script_progression --message="Removing various files..." --weight=1 # Remove a directory securely -ynh_secure_remove --file="/etc/$app" +#ynh_secure_remove --file="/etc/$app" # Remove the log files ynh_secure_remove --file="/var/log/$app" @@ -142,7 +130,7 @@ ynh_secure_remove --file="/var/log/$app" #================================================= # REMOVE DEDICATED USER #================================================= -ynh_script_progression --message="Removing the dedicated system user..." --time --weight=1 +ynh_script_progression --message="Removing the dedicated system user..." --weight=1 # Delete a system user ynh_system_user_delete --username=$app @@ -151,4 +139,4 @@ ynh_system_user_delete --username=$app # END OF SCRIPT #================================================= -ynh_script_progression --message="Removal of $app completed" --time --last +ynh_script_progression --message="Removal of $app completed" --last diff --git a/scripts/restore b/scripts/restore index 03432ab..a6933ea 100755 --- a/scripts/restore +++ b/scripts/restore @@ -8,6 +8,7 @@ # Keep this path for calling _common.sh inside the execution's context of backup and restore scripts source ../settings/scripts/_common.sh +source ../settings/scripts/ynh_redis source /usr/share/yunohost/helpers #================================================= @@ -24,39 +25,37 @@ ynh_abort_if_errors #================================================= # LOAD SETTINGS #================================================= -ynh_script_progression --message="Loading installation settings..." --time --weight=1 +ynh_script_progression --message="Loading installation settings..." --weight=1 app=$YNH_APP_INSTANCE_NAME domain=$(ynh_app_setting_get --app=$app --key=domain) path_url=$(ynh_app_setting_get --app=$app --key=path) +admin=$(ynh_app_setting_get --app=$app --key=admin) final_path=$(ynh_app_setting_get --app=$app --key=final_path) db_name=$(ynh_app_setting_get --app=$app --key=db_name) db_user=$db_name -phpversion=$(ynh_app_setting_get --app=$app --key=phpversion) +db_pwd=$(ynh_app_setting_get --app=$app --key=psqlpwd) datadir=$(ynh_app_setting_get --app=$app --key=datadir) +paperless_secret_key=$(ynh_app_setting_get --app=$app --key=paperless_secret_key) + +redis_db=$(ynh_redis_get_free_db) +ynh_app_setting_set --app=$app --key=redis_db --value=$redis_db #================================================= # CHECK IF THE APP CAN BE RESTORED #================================================= -ynh_script_progression --message="Validating restoration parameters..." --time --weight=1 +ynh_script_progression --message="Validating restoration parameters..." --weight=1 test ! -d $final_path \ || ynh_die --message="There is already a directory: $final_path " #================================================= # STANDARD RESTORATION STEPS -#================================================= -# RESTORE THE NGINX CONFIGURATION -#================================================= -ynh_script_progression --message="Restoring the NGINX web server configuration..." --time --weight=1 - -ynh_restore_file --origin_path="/etc/nginx/conf.d/$domain.d/$app.conf" - #================================================= # RECREATE THE DEDICATED USER #================================================= -ynh_script_progression --message="Recreating the dedicated system user..." --time --weight=1 +ynh_script_progression --message="Recreating the dedicated system user..." --weight=1 # Create the dedicated user (if not existing) ynh_system_user_create --username=$app --home_dir="$final_path" @@ -64,124 +63,153 @@ ynh_system_user_create --username=$app --home_dir="$final_path" #================================================= # RESTORE THE APP MAIN DIR #================================================= -ynh_script_progression --message="Restoring the app main directory..." --time --weight=1 +ynh_script_progression --message="Restoring the app main directory..." --weight=1 ynh_restore_file --origin_path="$final_path" -# FIXME: this should be managed by the core in the future -# Here, as a packager, you may have to tweak the ownerhsip/permissions -# such that the appropriate users (e.g. maybe www-data) can access -# files in some cases. -# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder - -# this will be treated as a security issue. chmod 750 "$final_path" chmod -R o-rwx "$final_path" -chown -R $app:www-data "$final_path" +chown -R $app:$app "$final_path" #================================================= # RESTORE THE DATA DIRECTORY #================================================= -ynh_script_progression --message="Restoring the data directory..." --time --weight=1 +ynh_script_progression --message="Restoring the data directory..." --weight=1 ynh_restore_file --origin_path="$datadir" --not_mandatory -mkdir -p $datadir +mkdir -p "$datadir/consume" +mkdir -p "$datadir/data" +mkdir -p "$datadir/media" -# FIXME: this should be managed by the core in the future -# Here, as a packager, you may have to tweak the ownerhsip/permissions -# such that the appropriate users (e.g. maybe www-data) can access -# files in some cases. -# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder - -# this will be treated as a security issue. chmod 750 "$datadir" chmod -R o-rwx "$datadir" -chown -R $app:www-data "$datadir" +chown -R $app:$app "$datadir" #================================================= -# RESTORE THE PHP-FPM CONFIGURATION +# ADD A CONFIGURATION #================================================= -ynh_script_progression --message="Restoring the PHP-FPM configuration..." --time --weight=1 +ynh_script_progression --message="Adding a configuration file..." --weight=1 -ynh_restore_file --origin_path="/etc/php/$phpversion/fpm/pool.d/$app.conf" +ynh_add_config --template="../settings/conf/paperless.conf.example" --destination="$final_path/paperless.conf" + +chmod 400 "$final_path/paperless.conf" +chown $app:$app "$final_path/paperless.conf" #================================================= # RESTORE FAIL2BAN CONFIGURATION #================================================= -ynh_script_progression --message="Restoring the Fail2Ban configuration..." --time --weight=1 +#ynh_script_progression --message="Restoring the Fail2Ban configuration..." --weight=1 -ynh_restore_file --origin_path="/etc/fail2ban/jail.d/$app.conf" -ynh_restore_file --origin_path="/etc/fail2ban/filter.d/$app.conf" -ynh_systemd_action --action=restart --service_name=fail2ban +#ynh_restore_file --origin_path="/etc/fail2ban/jail.d/$app.conf" +#ynh_restore_file --origin_path="/etc/fail2ban/filter.d/$app.conf" +#ynh_systemd_action --action=restart --service_name=fail2ban #================================================= # SPECIFIC RESTORATION #================================================= # REINSTALL DEPENDENCIES #================================================= -ynh_script_progression --message="Reinstalling dependencies..." --time --weight=1 +ynh_script_progression --message="Reinstalling dependencies..." --weight=1 # Define and install dependencies -ynh_install_app_dependencies $pkg_dependencies +ynh_exec_warn_less ynh_install_app_dependencies $pkg_dependencies $ocr_pkg_dependencies #================================================= -# RESTORE THE MYSQL DATABASE +# INSTALL PYTHON DEPENDENCIES #================================================= -ynh_script_progression --message="Restoring the MySQL database..." --time --weight=1 +ynh_script_progression --message="Installing Python dependencies..." -db_pwd=$(ynh_app_setting_get --app=$app --key=mysqlpwd) -ynh_mysql_setup_db --db_user=$db_user --db_name=$db_name --db_pwd=$db_pwd -ynh_mysql_connect_as --user=$db_user --password=$db_pwd --database=$db_name < ./db.sql +pushd $final_path + ynh_secure_remove --file="$final_path/venv" + python3 -m venv venv + chown -R "$app:" "$final_path" +( + source "$final_path/venv/bin/activate" + ynh_exec_as $app $final_path/venv/bin/pip3 install --upgrade pip setuptools wheel + ynh_exec_as $app $final_path/venv/bin/pip3 install -r "$final_path/requirements.txt" + deactivate +) +popd + +#================================================= +# RESTORE THE NGINX CONFIGURATION +#================================================= +ynh_script_progression --message="Restoring the NGINX web server configuration..." --weight=1 + +ynh_restore_file --origin_path="/etc/nginx/conf.d/$domain.d/$app.conf" + +#================================================= +# RESTORE THE POSTGRESQL DATABASE +#================================================= +ynh_script_progression --message="Restoring the PostgresSQL database..." --weight=1 + +db_pwd=$(ynh_app_setting_get --app=$app --key=psqlpwd) +ynh_psql_setup_db --db_user=$db_user --db_name=$db_name --db_pwd=$db_pwd +ynh_psql_connect_as --user=$db_user --password=$db_pwd --database=$db_name < ./db.sql #================================================= # RESTORE VARIOUS FILES #================================================= -ynh_script_progression --message="Restoring various files..." --time --weight=1 +#ynh_script_progression --message="Restoring various files..." --weight=1 -ynh_restore_file --origin_path="/etc/cron.d/$app" - -ynh_restore_file --origin_path="/etc/$app/" +#ynh_restore_file --origin_path="/etc/$app/" #================================================= # RESTORE SYSTEMD #================================================= -ynh_script_progression --message="Restoring the systemd configuration..." --time --weight=1 +ynh_script_progression --message="Restoring the systemd configuration..." --weight=1 ynh_restore_file --origin_path="/etc/systemd/system/$app.service" -systemctl enable $app.service --quiet +ynh_restore_file --origin_path="/etc/systemd/system/$app-consumer.service" +ynh_restore_file --origin_path="/etc/systemd/system/$app-scheduler.service" +ynh_restore_file --origin_path="/etc/systemd/system/$app-task-queue.service" +systemctl enable "$app.service" --quiet +systemctl enable "$app-consumer.service" --quiet +systemctl enable "$app-scheduler.service" --quiet +systemctl enable "$app-task-queue.service" --quiet #================================================= # RESTORE THE LOGROTATE CONFIGURATION #================================================= -ynh_script_progression --message="Restoring the logrotate configuration..." --time --weight=1 +ynh_script_progression --message="Restoring the logrotate configuration..." --weight=1 + +mkdir -p "/var/log/$app" +chown -R $app: "/var/log/$app" ynh_restore_file --origin_path="/etc/logrotate.d/$app" #================================================= # INTEGRATE SERVICE IN YUNOHOST #================================================= -ynh_script_progression --message="Integrating service in YunoHost..." --time --weight=1 +ynh_script_progression --message="Integrating service in YunoHost..." --weight=1 -yunohost service add $app --description="A short description of the app" --log="/var/log/$app/$app.log" +yunohost service add "$app" --log="/var/log/$app/$app.log" +yunohost service add "$app-consumer" --log="/var/log/$app/$app-consumer.log" +yunohost service add "$app-scheduler" --log="/var/log/$app/$app-scheduler.log" +yunohost service add "$app-task-queue" --log="/var/log/$app/$app-task-queue.log" #================================================= # START SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Starting a systemd service..." --time --weight=1 +ynh_script_progression --message="Starting a systemd service..." --weight=1 -ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="start" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="start" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="start" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # GENERIC FINALIZATION #================================================= # RELOAD NGINX AND PHP-FPM #================================================= -ynh_script_progression --message="Reloading NGINX web server and PHP-FPM..." --time --weight=1 +ynh_script_progression --message="Reloading NGINX web server..." --weight=1 -ynh_systemd_action --service_name=php$phpversion-fpm --action=reload ynh_systemd_action --service_name=nginx --action=reload #================================================= # END OF SCRIPT #================================================= -ynh_script_progression --message="Restoration completed for $app" --time --last +ynh_script_progression --message="Restoration completed for $app" --last diff --git a/scripts/upgrade b/scripts/upgrade index f52658e..16a65f5 100644 --- a/scripts/upgrade +++ b/scripts/upgrade @@ -7,38 +7,37 @@ #================================================= source _common.sh +source ynh_redis source /usr/share/yunohost/helpers #================================================= # LOAD SETTINGS #================================================= -ynh_script_progression --message="Loading installation settings..." --time --weight=1 +ynh_script_progression --message="Loading installation settings..." --weight=1 app=$YNH_APP_INSTANCE_NAME domain=$(ynh_app_setting_get --app=$app --key=domain) path_url=$(ynh_app_setting_get --app=$app --key=path) -language=$(ynh_app_setting_get --app=$app --key=language) +port=$(ynh_app_setting_get --app=$app --key=port) admin=$(ynh_app_setting_get --app=$app --key=admin) final_path=$(ynh_app_setting_get --app=$app --key=final_path) +datadir=$(ynh_app_setting_get --app=$app --key=datadir) db_name=$(ynh_app_setting_get --app=$app --key=db_name) +db_user=$db_name +db_pwd=$(ynh_app_setting_get --app=$app --key=psqlpwd) +redis_db=$(ynh_app_setting_get --app=$app --key=redis_db) +paperless_secret_key=$(ynh_app_setting_get --app=$app --key=paperless_secret_key) #================================================= # CHECK VERSION #================================================= - -### This helper will compare the version of the currently installed app and the version of the upstream package. -### $upgrade_type can have 2 different values -### - UPGRADE_APP if the upstream app version has changed -### - UPGRADE_PACKAGE if only the YunoHost package has changed -### ynh_check_app_version_changed will stop the upgrade if the app is up to date. -### UPGRADE_APP should be used to upgrade the core app only if there's an upgrade to do. upgrade_type=$(ynh_check_app_version_changed) #================================================= # BACKUP BEFORE UPGRADE THEN ACTIVE TRAP #================================================= -ynh_script_progression --message="Backing up the app before upgrading (may take a while)..." --time --weight=1 +ynh_script_progression --message="Backing up the app before upgrading (may take a while)..." --weight=1 # Backup the current version of the app ynh_backup_before_upgrade @@ -54,14 +53,17 @@ ynh_abort_if_errors #================================================= # STOP SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Stopping a systemd service..." --time --weight=1 +ynh_script_progression --message="Stopping a systemd service..." --weight=1 -ynh_systemd_action --service_name=$app --action="stop" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="stop" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="stop" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="stop" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="stop" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # ENSURE DOWNWARD COMPATIBILITY #================================================= -ynh_script_progression --message="Ensuring downward compatibility..." --time --weight=1 +ynh_script_progression --message="Ensuring downward compatibility..." --weight=1 # # N.B. : the followings setting migrations snippets are provided as *EXAMPLES* @@ -85,26 +87,26 @@ ynh_script_progression --message="Ensuring downward compatibility..." --time --w ### then you may safely remove these lines # Cleaning legacy permissions -if ynh_legacy_permissions_exists; then - ynh_legacy_permissions_delete_all - - ynh_app_setting_delete --app=$app --key=is_public -fi - -if ! ynh_permission_exists --permission="admin"; then - # Create the required permissions - ynh_permission_create --permission="admin" --url="/admin" --allowed=$admin -fi - -# Create a permission if needed -if ! ynh_permission_exists --permission="api"; then - ynh_permission_create --permission="api" --url="/api" --allowed="visitors" --show_tile="false" --protected="true" -fi +#if ynh_legacy_permissions_exists; then +# ynh_legacy_permissions_delete_all +# +# ynh_app_setting_delete --app=$app --key=is_public +#fi +# +#if ! ynh_permission_exists --permission="admin"; then +# # Create the required permissions +# ynh_permission_create --permission="admin" --url="/admin" --allowed=$admin +#fi +# +## Create a permission if needed +#if ! ynh_permission_exists --permission="api"; then +# ynh_permission_create --permission="api" --url="/api" --allowed="visitors" --show_tile="false" --protected="true" +#fi #================================================= # CREATE DEDICATED USER #================================================= -ynh_script_progression --message="Making sure dedicated system user exists..." --time --weight=1 +ynh_script_progression --message="Making sure dedicated system user exists..." --weight=1 # Create a dedicated user (if not existing) ynh_system_user_create --username=$app --home_dir="$final_path" @@ -115,120 +117,127 @@ ynh_system_user_create --username=$app --home_dir="$final_path" if [ "$upgrade_type" == "UPGRADE_APP" ] then - ynh_script_progression --message="Upgrading source files..." --time --weight=1 + ynh_script_progression --message="Upgrading source files..." --weight=1 - # Download, check integrity, uncompress and patch the source from app.src - ynh_setup_source --dest_dir="$final_path" + ynh_setup_source --dest_dir="$final_path" --keep="paperless.conf" fi -# FIXME: this should be managed by the core in the future -# Here, as a packager, you may have to tweak the ownerhsip/permissions -# such that the appropriate users (e.g. maybe www-data) can access -# files in some cases. -# But FOR THE LOVE OF GOD, do not allow r/x for "others" on the entire folder - -# this will be treated as a security issue. chmod 750 "$final_path" chmod -R o-rwx "$final_path" -chown -R $app:www-data "$final_path" +chown -R $app:$app "$final_path" + +#================================================= +# UPGRADE DEPENDENCIES +#================================================= +ynh_script_progression --message="Upgrading dependencies..." --weight=1 + +ynh_exec_warn_less ynh_install_app_dependencies $pkg_dependencies $ocr_pkg_dependencies #================================================= # NGINX CONFIGURATION #================================================= -ynh_script_progression --message="Upgrading NGINX web server configuration..." --time --weight=1 +ynh_script_progression --message="Upgrading NGINX web server configuration..." --weight=1 # Create a dedicated NGINX config ynh_add_nginx_config -#================================================= -# UPGRADE DEPENDENCIES -#================================================= -ynh_script_progression --message="Upgrading dependencies..." --time --weight=1 - -ynh_install_app_dependencies $pkg_dependencies - -#================================================= -# PHP-FPM CONFIGURATION -#================================================= -ynh_script_progression --message="Upgrading PHP-FPM configuration..." --time --weight=1 - -# Create a dedicated PHP-FPM config -ynh_add_fpm_config - #================================================= # SPECIFIC UPGRADE #================================================= -# ... +# INSTALL PYTHON DEPENDENCIES #================================================= +ynh_script_progression --message="Installing Python dependencies..." + +pushd $final_path + python3 -m venv venv + chown -R "$app:" "$final_path" +( + source "$final_path/venv/bin/activate" + ynh_exec_as $app $final_path/venv/bin/pip3 install --upgrade pip setuptools wheel + ynh_exec_as $app $final_path/venv/bin/pip3 install -r "$final_path/requirements.txt" + deactivate +) +popd #================================================= # UPDATE A CONFIG FILE #================================================= -ynh_script_progression --message="Updating a configuration file..." --time --weight=1 +ynh_script_progression --message="Updating a configuration file..." --weight=1 -### Same as during install -### -### The file will automatically be backed-up if it's found to be manually modified (because -### ynh_add_config keeps track of the file's checksum) +ynh_add_config --template="paperless.conf.example" --destination="$final_path/paperless.conf" -ynh_add_config --template="some_config_file" --destination="$final_path/some_config_file" +chmod 400 "$final_path/paperless.conf" +chown $app:$app "$final_path/paperless.conf" -# FIXME: this should be handled by the core in the future -# You may need to use chmod 600 instead of 400, -# for example if the app is expected to be able to modify its own config -chmod 400 "$final_path/some_config_file" -chown $app:$app "$final_path/some_config_file" +#================================================= +# SETUP THE DATABASE +#================================================= +ynh_script_progression --message="Setting up the database..." --weight=1 -### For more complex cases where you want to replace stuff using regexes, -### you shoud rely on ynh_replace_string (which is basically a wrapper for sed) -### When doing so, you also need to manually call ynh_store_file_checksum -### -### ynh_replace_string --match_string="match_string" --replace_string="replace_string" --target_file="$final_path/some_config_file" -### ynh_store_file_checksum --file="$final_path/some_config_file" +pushd "$final_path/src" +( + source "$final_path/venv/bin/activate" + ynh_exec_as $app $final_path/venv/bin/python manage.py migrate + deactivate +) +popd #================================================= # SETUP SYSTEMD #================================================= -ynh_script_progression --message="Upgrading systemd configuration..." --time --weight=1 +ynh_script_progression --message="Upgrading systemd configuration..." --weight=1 # Create a dedicated systemd config -ynh_add_systemd_config +ynh_add_systemd_config --service="$app" --template="systemd.service" +ynh_add_systemd_config --service="$app-consumer" --template="systemd-consumer.service" +ynh_add_systemd_config --service="$app-scheduler" --template="systemd-scheduler.service" +ynh_add_systemd_config --service="$app-task-queue" --template="systemd-task-queue.service" #================================================= # GENERIC FINALIZATION #================================================= # SETUP LOGROTATE #================================================= -ynh_script_progression --message="Upgrading logrotate configuration..." --time --weight=1 +ynh_script_progression --message="Upgrading logrotate configuration..." --weight=1 # Use logrotate to manage app-specific logfile(s) -ynh_use_logrotate --non-append +ynh_use_logrotate --logfile="/var/log/$app/$app.log" +ynh_use_logrotate --logfile="/var/log/$app/$app-consumer.log" +ynh_use_logrotate --logfile="/var/log/$app/$app-scheduler.log" +ynh_use_logrotate --logfile="/var/log/$app/$app-task-queue.log" #================================================= # INTEGRATE SERVICE IN YUNOHOST #================================================= -ynh_script_progression --message="Integrating service in YunoHost..." --time --weight=1 +ynh_script_progression --message="Integrating service in YunoHost..." --weight=1 -yunohost service add $app --description="A short description of the app" --log="/var/log/$app/$app.log" +yunohost service add "$app" --log="/var/log/$app/$app.log" +yunohost service add "$app-consumer" --log="/var/log/$app/$app-consumer.log" +yunohost service add "$app-scheduler" --log="/var/log/$app/$app-scheduler.log" +yunohost service add "$app-task-queue" --log="/var/log/$app/$app-task-queue.log" #================================================= # START SYSTEMD SERVICE #================================================= -ynh_script_progression --message="Starting a systemd service..." --time --weight=1 +ynh_script_progression --message="Starting a systemd service..." --weight=1 -ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app" --action="start" --log_path="/var/log/$app/$app.log" +ynh_systemd_action --service_name="$app-consumer" --action="start" --log_path="/var/log/$app/$app-consumer.log" +ynh_systemd_action --service_name="$app-scheduler" --action="start" --log_path="/var/log/$app/$app-scheduler.log" +ynh_systemd_action --service_name="$app-task-queue" --action="start" --log_path="/var/log/$app/$app-task-queue.log" #================================================= # UPGRADE FAIL2BAN #================================================= -ynh_script_progression --message="Reconfiguring Fail2Ban..." --time --weight=1 +#ynh_script_progression --message="Reconfiguring Fail2Ban..." --weight=1 # Create a dedicated Fail2Ban config -ynh_add_fail2ban_config --logpath="/var/log/nginx/${domain}-error.log" --failregex="Regex to match into the log for a failed login" +#ynh_add_fail2ban_config --logpath="/var/log/nginx/${domain}-error.log" --failregex="Regex to match into the log for a failed login" #================================================= # RELOAD NGINX #================================================= -ynh_script_progression --message="Reloading NGINX web server..." --time --weight=1 +ynh_script_progression --message="Reloading NGINX web server..." --weight=1 ynh_systemd_action --service_name=nginx --action=reload @@ -236,4 +245,4 @@ ynh_systemd_action --service_name=nginx --action=reload # END OF SCRIPT #================================================= -ynh_script_progression --message="Upgrade of $app completed" --time --last +ynh_script_progression --message="Upgrade of $app completed" --last diff --git a/scripts/ynh_redis b/scripts/ynh_redis new file mode 100644 index 0000000..0071247 --- /dev/null +++ b/scripts/ynh_redis @@ -0,0 +1,39 @@ +#!/bin/bash + +# get the first available redis database +# +# usage: ynh_redis_get_free_db +# | returns: the database number to use +ynh_redis_get_free_db() { + local result max db + result=$(redis-cli INFO keyspace) + + # get the num + max=$(cat /etc/redis/redis.conf | grep ^databases | grep -Eow "[0-9]+") + + db=0 + # default Debian setting is 15 databases + for i in $(seq 0 "$max") + do + if ! echo "$result" | grep -q "db$i" + then + db=$i + break 1 + fi + db=-1 + done + + test "$db" -eq -1 && ynh_die --message="No available Redis databases..." + + echo "$db" +} + +# Create a master password and set up global settings +# Please always call this script in install and restore scripts +# +# usage: ynh_redis_remove_db database +# | arg: database - the database to erase +ynh_redis_remove_db() { + local db=$1 + redis-cli -n "$db" flushall +}