1
0
Fork 0
mirror of https://github.com/YunoHost-Apps/loki_ynh.git synced 2024-09-03 19:36:16 +02:00
Add promtail and update links
This commit is contained in:
Florent 2022-12-31 11:31:16 +01:00 committed by GitHub
parent 9d2dd4a9ae
commit 4b63266ca7
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
21 changed files with 212 additions and 43 deletions

View file

@ -64,11 +64,17 @@ echo "Handling asset at $asset_url"
# Here we base the source file name upon a unique keyword in the assets url (admin vs. update)
# Leave $src empty to ignore the asset
case $asset_url in
*"promtail-linux-amd64.zip")
src="promtail-amd64"
;;
*"promtail-linux-arm64.zip")
src="promtail-arm64"
;;
*"loki-linux-amd64.zip")
src="amd64"
src="loki-amd64"
;;
*"loki-linux-arm64.zip")
src="arm64"
src="loki-arm64"
;;
*)
src=""

View file

@ -3,14 +3,14 @@ N.B.: This README was automatically generated by https://github.com/YunoHost/app
It shall NOT be edited by hand.
-->
# Loki for YunoHost
# Loki + Promtail for YunoHost
[![Integration level](https://dash.yunohost.org/integration/loki.svg)](https://dash.yunohost.org/appci/app/loki) ![Working status](https://ci-apps.yunohost.org/ci/badges/loki.status.svg) ![Maintenance status](https://ci-apps.yunohost.org/ci/badges/loki.maintain.svg)
[![Install Loki with YunoHost](https://install-app.yunohost.org/install-with-yunohost.svg)](https://install-app.yunohost.org/?app=loki)
[![Install Loki + Promtail with YunoHost](https://install-app.yunohost.org/install-with-yunohost.svg)](https://install-app.yunohost.org/?app=loki)
*[Lire ce readme en français.](./README_fr.md)*
> *This package allows you to install Loki quickly and simply on a YunoHost server.
> *This package allows you to install Loki + Promtail quickly and simply on a YunoHost server.
If you don't have YunoHost, please consult [the guide](https://yunohost.org/#/install) to learn how to install it.*
## Overview
@ -25,17 +25,17 @@ Compared to other log aggregation systems, Loki:
- has native support in Grafana (needs Grafana v6.0).
A Loki-based logging stack consists of 3 components:
- [promtail](/Yunohost-Apps/promtail_ynh) is the agent, responsible for gathering logs and sending them to Loki.
- [loki](/Yunohost-Apps/loki_ynh) is the main server, responsible for storing logs and processing queries.
- [Grafana](/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.
- Promtail (included in this package) is the agent, responsible for gathering logs and sending them to Loki.
- Loki is the main server, responsible for storing logs and processing queries.
- [Grafana](https://github.com/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.
**Shipped version:** 2.7.1~ynh1
**Shipped version:** 2.7.1~ynh2
## Disclaimers / important information
- Loki is not exposed to the web;
- The configuration file is overwritten across updates;
- If you want to add pieces of configuration or overwrite the configuration, add a new file in `/etc/loki/loki.d/` for Loki or in `/etc/loki/promtail.d/` for promtail
## Documentation and resources

View file

@ -3,14 +3,14 @@ N.B.: This README was automatically generated by https://github.com/YunoHost/app
It shall NOT be edited by hand.
-->
# Loki pour YunoHost
# Loki + Promtail pour YunoHost
[![Niveau d'intégration](https://dash.yunohost.org/integration/loki.svg)](https://dash.yunohost.org/appci/app/loki) ![Statut du fonctionnement](https://ci-apps.yunohost.org/ci/badges/loki.status.svg) ![Statut de maintenance](https://ci-apps.yunohost.org/ci/badges/loki.maintain.svg)
[![Installer Loki avec YunoHost](https://install-app.yunohost.org/install-with-yunohost.svg)](https://install-app.yunohost.org/?app=loki)
[![Installer Loki + Promtail avec YunoHost](https://install-app.yunohost.org/install-with-yunohost.svg)](https://install-app.yunohost.org/?app=loki)
*[Read this readme in english.](./README.md)*
> *Ce package vous permet d'installer Loki rapidement et simplement sur un serveur YunoHost.
> *Ce package vous permet d'installer Loki + Promtail rapidement et simplement sur un serveur YunoHost.
Si vous n'avez pas YunoHost, regardez [ici](https://yunohost.org/#/install) pour savoir comment l'installer et en profiter.*
## Vue d'ensemble
@ -25,17 +25,17 @@ Compared to other log aggregation systems, Loki:
- has native support in Grafana (needs Grafana v6.0).
A Loki-based logging stack consists of 3 components:
- [promtail](/Yunohost-Apps/promtail_ynh) is the agent, responsible for gathering logs and sending them to Loki.
- [loki](/Yunohost-Apps/loki_ynh) is the main server, responsible for storing logs and processing queries.
- [Grafana](/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.
- Promtail (included in this package) is the agent, responsible for gathering logs and sending them to Loki.
- Loki is the main server, responsible for storing logs and processing queries.
- [Grafana](https://github.com/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.
**Version incluse :** 2.7.1~ynh1
**Version incluse :** 2.7.1~ynh2
## Avertissements / informations importantes
- Loki is not exposed to the web;
- The configuration file is overwritten across updates;
- If you want to add pieces of configuration or overwrite the configuration, add a new file in `/etc/loki/loki.d/` for Loki or in `/etc/loki/promtail.d/` for promtail
## Documentations et ressources

View file

@ -7,8 +7,8 @@ Type=simple
User=__APP__
Group=__APP__
WorkingDirectory=__FINALPATH__/
ExecStart=__FINALPATH__/loki-linux-__ARCH__ --config.file __FINALPATH__/local-config.yaml
StandardOutput=append:/var/log/__APP__/__APP__.log
ExecStart=/bin/bash -c '__FINALPATH__/loki-linux-__ARCH__ --config.file <(/bin/bash __FINALPATH__/merge_yaml.sh /etc/__APP__/loki-default.yaml /etc/__APP__/loki.d/*.y{a,}ml)'
StandardOutput=append:/var/log/__APP__/loki.log
StandardError=inherit
# Sandboxing options to harden security

6
conf/promtail-amd64.src Normal file
View file

@ -0,0 +1,6 @@
SOURCE_URL=https://github.com/grafana/loki/releases/download/v2.7.1/promtail-linux-amd64.zip
SOURCE_SUM=aeccf5c2ac067210e7737ab408f1e36cd55be4dfc521a56f0cc5ee01582e71d6
SOURCE_SUM_PRG=sha256sum
SOURCE_FORMAT=zip
SOURCE_IN_SUBDIR=false
SOURCE_FILENAME=

6
conf/promtail-arm64.src Normal file
View file

@ -0,0 +1,6 @@
SOURCE_URL=https://github.com/grafana/loki/releases/download/v2.7.1/promtail-linux-arm64.zip
SOURCE_SUM=770283cabc274cbbd969e12d0f58c75205394ee23fd99ec262c943532d64a12e
SOURCE_SUM_PRG=sha256sum
SOURCE_FORMAT=zip
SOURCE_IN_SUBDIR=false
SOURCE_FILENAME=

View file

@ -0,0 +1,11 @@
server:
http_listen_port: __PROMTAIL_PORT__
clients:
- url: http://localhost:__HTTP_PORT__/loki/api/v1/push
positions:
filename: /tmp/promtail_positions.yaml
target_config:
sync_period: 10s

49
conf/promtail.service Normal file
View file

@ -0,0 +1,49 @@
[Unit]
Description=Promtail daemon
After=network.target
[Service]
Type=simple
User=__APP__
Group=__APP__
WorkingDirectory=__FINALPATH__/
ExecStart=/bin/bash -c '__FINALPATH__/promtail-linux-__ARCH__ --config.file <(/bin/bash __FINALPATH__/merge_yaml.sh /etc/__APP__/promtail-default.yaml /etc/__APP__/promtail.d/*.y{a,}ml)'
StandardOutput=append:/var/log/__APP__/promtail.log
StandardError=inherit
# Sandboxing options to harden security
# Depending on specificities of your service/app, you may need to tweak these
# .. but this should be a good baseline
# Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
DevicePolicy=closed
ProtectClock=yes
ProtectHostname=yes
ProtectProc=invisible
ProtectSystem=full
ProtectControlGroups=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
LockPersonality=yes
SystemCallArchitectures=native
SystemCallFilter=~@clock @debug @module @mount @obsolete @reboot @setuid @swap @cpu-emulation @privileged
# Denying access to capabilities that should not be relevant for webapps
# Doc: https://man7.org/linux/man-pages/man7/capabilities.7.html
CapabilityBoundingSet=~CAP_RAWIO CAP_MKNOD
CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE
CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_TIME CAP_SYS_MODULE CAP_SYS_PACCT
CapabilityBoundingSet=~CAP_LEASE CAP_LINUX_IMMUTABLE CAP_IPC_LOCK
CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_WAKE_ALARM
CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG
CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE
CapabilityBoundingSet=~CAP_NET_ADMIN CAP_NET_BROADCAST CAP_NET_RAW
CapabilityBoundingSet=~CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_SYSLOG
[Install]
WantedBy=multi-user.target

View file

@ -8,7 +8,7 @@ Compared to other log aggregation systems, Loki:
- has native support in Grafana (needs Grafana v6.0).
A Loki-based logging stack consists of 3 components:
- [promtail](/Yunohost-Apps/promtail_ynh) is the agent, responsible for gathering logs and sending them to Loki.
- [loki](/Yunohost-Apps/loki_ynh) is the main server, responsible for storing logs and processing queries.
- [Grafana](/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.
- Promtail (included in this package) is the agent, responsible for gathering logs and sending them to Loki.
- Loki is the main server, responsible for storing logs and processing queries.
- [Grafana](https://github.com/Yunohost-Apps/grafana_ynh) for querying and displaying the logs.

View file

@ -1,2 +1,2 @@
- Loki is not exposed to the web;
- The configuration file is overwritten across updates;
- If you want to add pieces of configuration or overwrite the configuration, add a new file in `/etc/loki/loki.d/` for Loki or in `/etc/loki/promtail.d/` for promtail

View file

@ -1,12 +1,12 @@
{
"name": "Loki",
"name": "Loki + Promtail",
"id": "loki",
"packaging_format": 1,
"description": {
"en": "Grafana Loki is a set of components that can be composed into a fully featured logging stack.",
"fr": "Grafana Loki est un ensemble de composants qui peuvent être composés pour former une pile de journalisation complète."
},
"version": "2.7.1~ynh1",
"version": "2.7.1~ynh2",
"url": "https://grafana.com/docs/loki/latest/",
"upstream": {
"license": "AGPL-3.0",

View file

@ -43,6 +43,12 @@ ynh_backup --src_path="$final_path"
#=================================================
# SPECIFIC BACKUP
#=================================================
# BACKUP CONFIGURATION
#=================================================
ynh_backup --src_path="/etc/$app"
#=================================================
# BACKUP LOGROTATE
#=================================================
@ -54,6 +60,7 @@ ynh_backup --src_path="/etc/logrotate.d/$app"
#=================================================
ynh_backup --src_path="/etc/systemd/system/$app.service"
ynh_backup --src_path="/etc/systemd/system/$app-promtail.service"
#=================================================
# END OF SCRIPT

View file

@ -44,6 +44,9 @@ ynh_app_setting_set --app=$app --key=http_port --value=$http_port
grpc_port=$(ynh_find_port --port=9096)
ynh_app_setting_set --app=$app --key=grpc_port --value=$grpc_port
promtail_port=$(ynh_find_port --port=9080)
ynh_app_setting_set --app=$app --key=promtail_port --value=$promtail_port
#=================================================
# INSTALL DEPENDENCIES
#=================================================
@ -65,32 +68,49 @@ ynh_system_user_create --username=$app --home_dir="$final_path"
ynh_script_progression --message="Setting up source files..." --weight=1
ynh_app_setting_set --app=$app --key=final_path --value=$final_path
ynh_setup_source --dest_dir="$final_path" --source_id="$YNH_ARCH"
ynh_setup_source --dest_dir="$final_path" --source_id="loki-$YNH_ARCH"
ynh_setup_source --dest_dir="$final_path" --source_id="promtail-$YNH_ARCH"
chmod 750 "$final_path"
chmod -R o-rwx "$final_path"
chown -R $app:www-data "$final_path"
mkdir -p "/etc/$app/loki.d"
chmod 700 "/etc/$app/loki.d"
chown $app:www-data "/etc/$app/loki.d"
mkdir -p "/etc/$app/promtail.d"
chmod 700 "/etc/$app/promtail.d"
chown $app:www-data "/etc/$app/promtail.d"
#=================================================
# ADD A CONFIGURATION
#=================================================
ynh_script_progression --message="Adding a configuration file..." --weight=1
ynh_add_config --template="local-config.yaml" --destination="$final_path/local-config.yaml"
ynh_add_config --template="loki-default.yaml" --destination="/etc/$app/loki-default.yaml"
# FIXME: this should be handled by the core in the future
# You may need to use chmod 600 instead of 400,
# for example if the app is expected to be able to modify its own config
chmod 400 "$final_path/local-config.yaml"
chown $app:www-data "$final_path/local-config.yaml"
chmod 400 "/etc/$app/loki-default.yaml"
chown $app:www-data "/etc/$app/loki-default.yaml"
ynh_add_config --template="promtail-default.yaml" --destination="/etc/$app/promtail-default.yaml"
# FIXME: this should be handled by the core in the future
# You may need to use chmod 600 instead of 400,
# for example if the app is expected to be able to modify its own config
chmod 400 "/etc/$app/promtail-default.yaml"
chown $app:www-data "/etc/$app/promtail-default.yaml"
#=================================================
# SETUP SYSTEMD
#=================================================
ynh_script_progression --message="Configuring a systemd service..." --weight=1
# Faire 2 applications différentes
ynh_add_systemd_config
ynh_add_systemd_config --template="loki.service"
ynh_add_systemd_config --template="promtail.service" --service="$app-promtail"
#=================================================
# GENERIC FINALIZATION
@ -107,7 +127,8 @@ ynh_use_logrotate
#=================================================
ynh_script_progression --message="Integrating service in YunoHost..." --weight=1
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/$app.log"
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/loki.log"
yunohost service add $app-promtail --description="Promtail daemon" --log="/var/log/$app/promtail.log"
#=================================================
# START SYSTEMD SERVICE
@ -115,7 +136,8 @@ yunohost service add $app --description="Loki daemon" --log="/var/log/$app/$app.
ynh_script_progression --message="Starting a systemd service..." --weight=1
# Start a systemd service
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log"
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/loki.log"
ynh_systemd_action --service_name=$app-promtail --action="start" --log_path="/var/log/$app/promtail.log"
#=================================================
# END OF SCRIPT

View file

@ -32,6 +32,7 @@ if ynh_exec_warn_less yunohost service status $app >/dev/null
then
ynh_script_progression --message="Removing $app service integration..." --weight=1
yunohost service remove $app
yunohost service remove $app-promtail
fi
#=================================================
@ -41,6 +42,7 @@ ynh_script_progression --message="Stopping and removing the systemd service..."
# Remove the dedicated systemd config
ynh_remove_systemd_config
ynh_remove_systemd_config --service="$app-promtail"
#=================================================
# REMOVE LOGROTATE CONFIGURATION
@ -58,6 +60,17 @@ ynh_script_progression --message="Removing app main directory..." --weight=1
# Remove the app directory securely
ynh_secure_remove --file="$final_path"
#=================================================
# REMOVE CONFIGURATION
#=================================================
# Remove the data directory if --purge option is used
if [ "${YNH_APP_PURGE:-0}" -eq 1 ]
then
ynh_script_progression --message="Removing app data directory..." --weight=1
ynh_secure_remove --file="/etc/$app"
fi
#=================================================
# REMOVE DEPENDENCIES
#=================================================

View file

@ -70,6 +70,14 @@ ynh_script_progression --message="Reinstalling dependencies..." --weight=1
# Define and install dependencies
ynh_install_app_dependencies $pkg_dependencies
#=================================================
# RESTORE CONFIGURATION
#=================================================
ynh_script_progression --message="Restoring the configuration files..." --weight=1
ynh_restore_file --origin_path="/etc/$app"
#=================================================
# RESTORE SYSTEMD
#=================================================
@ -78,6 +86,9 @@ ynh_script_progression --message="Restoring the systemd configuration..." --weig
ynh_restore_file --origin_path="/etc/systemd/system/$app.service"
systemctl enable $app.service --quiet
ynh_restore_file --origin_path="/etc/systemd/system/$app-promtail.service"
systemctl enable $app-promtail.service --quiet
#=================================================
# RESTORE THE LOGROTATE CONFIGURATION
#=================================================
@ -90,14 +101,16 @@ ynh_restore_file --origin_path="/etc/logrotate.d/$app"
#=================================================
ynh_script_progression --message="Integrating service in YunoHost..." --weight=1
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/$app.log"
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/loki.log"
yunohost service add $app-promtail --description="Promtail daemon" --log="/var/log/$app/promtail.log"
#=================================================
# START SYSTEMD SERVICE
#=================================================
ynh_script_progression --message="Starting a systemd service..." --weight=1
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log"
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/loki.log"
ynh_systemd_action --service_name=$app-promtail --action="start" --log_path="/var/log/$app/promtail.log"
#=================================================
# END OF SCRIPT

View file

@ -20,6 +20,7 @@ arch=$YNH_ARCH
final_path=$(ynh_app_setting_get --app=$app --key=final_path)
http_port=$(ynh_app_setting_get --app=$app --key=http_port)
grpc_port=$(ynh_app_setting_get --app=$app --key=grpc_port)
promtail_port=$(ynh_app_setting_get --app=$app --key=promtail_port)
#=================================================
# CHECK VERSION
@ -67,13 +68,23 @@ then
ynh_script_progression --message="Upgrading source files..." --weight=1
# Download, check integrity, uncompress and patch the source from app.src
ynh_setup_source --dest_dir="$final_path" --source_id="$YNH_ARCH"
ynh_setup_source --dest_dir="$final_path" --source_id="loki-$YNH_ARCH"
ynh_setup_source --dest_dir="$final_path" --source_id="promtail-$YNH_ARCH"
fi
chmod 750 "$final_path"
chmod -R o-rwx "$final_path"
chown -R $app:www-data "$final_path"
mkdir -p "/etc/$app/loki.d"
chmod 700 "/etc/$app/loki.d"
chown $app:www-data "/etc/$app/loki.d"
mkdir -p "/etc/$app/promtail.d"
chmod 700 "/etc/$app/promtail.d"
chown $app:www-data "/etc/$app/promtail.d"
#=================================================
# UPGRADE DEPENDENCIES
#=================================================
@ -92,13 +103,21 @@ ynh_install_app_dependencies $pkg_dependencies
#=================================================
ynh_script_progression --message="Updating a configuration file..." --weight=1
ynh_add_config --template="local-config.yaml" --destination="$final_path/local-config.yaml"
ynh_add_config --template="loki-default.yaml" --destination="/etc/$app/loki-default.yaml"
# FIXME: this should be handled by the core in the future
# You may need to use chmod 600 instead of 400,
# for example if the app is expected to be able to modify its own config
chmod 400 "$final_path/local-config.yaml"
chown $app:www-data "$final_path/local-config.yaml"
chmod 400 "/etc/$app/loki-default.yaml"
chown $app:www-data "/etc/$app/loki-default.yaml"
ynh_add_config --template="promtail-default.yaml" --destination="/etc/$app/promtail-default.yaml"
# FIXME: this should be handled by the core in the future
# You may need to use chmod 600 instead of 400,
# for example if the app is expected to be able to modify its own config
chmod 400 "/etc/$app/promtail-default.yaml"
chown $app:www-data "/etc/$app/promtail-default.yaml"
#=================================================
# SETUP SYSTEMD
@ -106,7 +125,8 @@ chown $app:www-data "$final_path/local-config.yaml"
ynh_script_progression --message="Upgrading systemd configuration..." --weight=1
# Create a dedicated systemd config
ynh_add_systemd_config
ynh_add_systemd_config --template="loki.service"
ynh_add_systemd_config --template="promtail.service" --service="$app-promtail"
#=================================================
# GENERIC FINALIZATION
@ -123,14 +143,16 @@ ynh_use_logrotate --non-append
#=================================================
ynh_script_progression --message="Integrating service in YunoHost..." --weight=1
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/$app.log"
yunohost service add $app --description="Loki daemon" --log="/var/log/$app/loki.log"
yunohost service add $app-promtail --description="Promtail daemon" --log="/var/log/$app/promtail.log"
#=================================================
# START SYSTEMD SERVICE
#=================================================
ynh_script_progression --message="Starting a systemd service..." --weight=1
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/$app.log"
ynh_systemd_action --service_name=$app --action="start" --log_path="/var/log/$app/loki.log"
ynh_systemd_action --service_name=$app-promtail --action="start" --log_path="/var/log/$app/promtail.log"
#=================================================
# END OF SCRIPT

View file

@ -0,0 +1,14 @@
#!/bin/bash
shopt -s nullglob
file_list=($@)
{
for file in "${file_list[@]}"
do
/usr/bin/python3 -c "import yaml, json; print(json.dumps(yaml.safe_load(open('$file')), default=str))"
done
} \
| /usr/bin/jq --slurp 'reduce .[] as $item ({}; . * $item)' \
| /usr/bin/python3 -c "import yaml, json; print(yaml.dump(json.load(open('/dev/stdin'))))"