From 439d39cd6ca465f39a57ee895a87e6bd0976fa6a Mon Sep 17 00:00:00 2001 From: yalh76 Date: Fri, 17 Apr 2020 23:45:42 +0200 Subject: [PATCH 1/9] First commit --- contributordoc.md | 1 + packaging_apps_advanced.md | 109 +++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+) create mode 100644 packaging_apps_advanced.md diff --git a/contributordoc.md b/contributordoc.md index 4570723d..3983e24d 100644 --- a/contributordoc.md +++ b/contributordoc.md @@ -25,6 +25,7 @@ * [App CI dashboard](https://dash.yunohost.org/appci/branch/stable) * [App Continuous Integration for packagers](/packaging_apps_ci) * [YEP - YunoHost Enhancement Proposals](/packaging_apps_guidelines) + * [Advanced packaging](/packaging_apps_advanced) * [Contributing to the YunoHost core](/dev) * [Setting up a dev environment with ynh-dev](https://github.com/YunoHost/ynh-dev/blob/master/README.md) * [Deb build chain](https://github.com/YunoHost/vinaigrette/blob/master/README.md) diff --git a/packaging_apps_advanced.md b/packaging_apps_advanced.md new file mode 100644 index 00000000..74210ffe --- /dev/null +++ b/packaging_apps_advanced.md @@ -0,0 +1,109 @@ +# Packing Applications : Advanced + +Here come the time: +- you know all the [YEPs](/packaging_apps_guidelines) +- you masterise [apps packaging](/packaging_apps), [package_check](https://github.com/YunoHost/package_check), [example_ynh](https://github.com/YunoHost/example_ynh) and [experimnetal helpers](https://github.com/YunoHost-Apps/Experimental_helpers) +- you have integrated the [YunoHost Apps Group](https://yunohost.org/#/project_organization) +- you know what means `¯\_(ツ)_/¯` +- you use to say `Don't ask to ask, just ask` in the [YunoHost support room](https://yunohost.org/#/help) + +And now you want to improve your packaging speed before climbing into [Yunohost Core developpement](/dev). + +Here are few ideas that will help you. + +There are things that slow you down: +- Installing your freshly done package because of dependencies installation or npm/rust/elixir/composer builds +- Waiting a package_check to see the level of your package +- etc... + +The idea is to use YunoHost stuff to help you improve this lost time. + +## Building your infrastructure +First idea is to build several VMs or LXCs with YunoHost to let you parallelize actions, like having the first one installing an apps, the second installing another apps, the third one debugging an apps upgrade, etc... + +After first installation of those VMs/LXCs, each one has to be reacheable with its domain name, like for example: +- ynh1.mydomain.org +- ynh2.mydomain.org +- ynh3.mydomain.org + +For that you will need a public IP for each one or reverse proxy... +If your are lucky enough to have IPV6 and a /64 IPV6 scope, it won't be a problem to assign a public IPV6 to each YunoHost. + +In addition, it's helpfull to have several domains names for each YunoHost, for example: +- ynh1.mydomain.org + - test11.mydomain.org + - test12.mydomain.org + - test13.mydomain.org +- ynh2.mydomain.org + - test21.mydomain.org + - test22.mydomain.org + - test23.mydomain.org +- ynh3.mydomain.org + - test31.mydomain.org + - test32.mydomain.org + - test33.mydomain.org + +With that you will be able to install two apps on the same YunoHost first one at https://test11.mydomain.org and a second app at https://test12.mydomain.org (let say the first app is nextcloud_ynh the second one is collabora_ynh or onlyoffice_ynh) + +When your infrastructure is up and ready, you will have to do snapshots of each VMs/LXCs. Doing that, you will be able to revert back to an happy and healthy YunoHost after doing install/backup/restore/upgrade of and app... + +Don't forget to regularly upgrade your YunoHosts VMs/LXCs to the last release doing: revert back to last happy/healthy YunoHost snapshot and doing `yunohost domain cert-renew --no-check && yunohost tools update && yunohost tools upgrade --apps && yunohost tools upgrade --system && apt autoremove`. Take a snapshot after that successfull full upgrade. And after some testings or one week later, remove old snapshot. PS: that can be automated... + +Doing that you will have all the time a full/ready to go YunoHosts infrastructure for advanced packagers. And if require, you can have some VMs/LXCs runnning the YunoHost testing or YunoHost next Debian main version. + +## Having your packages/commits available in each VMs/LXCs +There are several ways to have your freshly created package available from each VMs/LXCs +1- Do an ssh from the YunoHost VM/LXC to a central repository on your computer +2- rsync you central repository to each VM/LXC +3- use [syncthing](https://github.com/YunoHost-Apps/syncthing_ynh) to syncrhonise you main repository on each VM/LXC + +## Shortcuts +Use and abuse of `yunohost app install` `--args` argument + +You can do ugly thing considering mynewapp is the name/REPLACEBYYOURAPP of your app + +To install your mynewapp app: +``` +YNHAPP=mynewapp +yunohost app install /mycentralrepo/${YNHAPP}_ynh/ --debug --force --args domain=test11.mydomain.org&path=/myapp&admin=alice&is_public=true&language=en&password=awesomepassword +``` + +To remove your mynewapp app and reinstall it +``` +YNHAPP=mynewapp +yunohost app remove ${YNHAPP} --debug && yunohost app install /mycentralrepo/${YNHAPP}_ynh/ --debug --force --args domain=test11.mydomain.org&path=/myapp&admin=alice&is_public=true&language=en&password=awesomepassword +``` + +To backup your mynewapp app +``` +YNHAPP=mynewapp +yunohost backup create --apps ${YNHAPP} --debug +``` + +To restore your mynewapp app from the backup just made +``` +YNHAPP=mynewapp +yunohost backup restore XXXXXXXX-XXXXXX --apps ${YNHAPP} --debug +``` + +To upgrade your mynewapp +``` +YNHAPP=mynewapp +yunohost app upgrade ${YNHAPP} -f /mycentralrepo/${YNHAPP}_ynh/ --debug + + +``` + +With that, you will be able to launch different actions on differents YunoHost VMs/LXCs + +## Package_check + +With advanced packaging comes [CI_package_check](https://github.com/YunoHost/CI_package_check), it lets you serialize and automate package_check. + + +## Managing all the parallel packaging you are doing + +When you do several things at the same time, sometimes you don't remember what is the next step for this or that app. +A good tool to keep your TODO list organized is to use a kaban like: +- [kanboard](https://github.com/YunoHost-Apps/kanboard_ynh) +- [wekan](https://github.com/YunoHost-Apps/wekan_ynh) From 3c1e2fa074b07d81f5a3fe44d14cfb61aa313239 Mon Sep 17 00:00:00 2001 From: yalh76 Date: Fri, 17 Apr 2020 23:55:15 +0200 Subject: [PATCH 2/9] Adding spacing --- packaging_apps_advanced.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/packaging_apps_advanced.md b/packaging_apps_advanced.md index 74210ffe..fc43eab4 100644 --- a/packaging_apps_advanced.md +++ b/packaging_apps_advanced.md @@ -63,31 +63,36 @@ Use and abuse of `yunohost app install` `--args` argument You can do ugly thing considering mynewapp is the name/REPLACEBYYOURAPP of your app To install your mynewapp app: -``` + +```bash YNHAPP=mynewapp yunohost app install /mycentralrepo/${YNHAPP}_ynh/ --debug --force --args domain=test11.mydomain.org&path=/myapp&admin=alice&is_public=true&language=en&password=awesomepassword ``` To remove your mynewapp app and reinstall it -``` + +```bash YNHAPP=mynewapp yunohost app remove ${YNHAPP} --debug && yunohost app install /mycentralrepo/${YNHAPP}_ynh/ --debug --force --args domain=test11.mydomain.org&path=/myapp&admin=alice&is_public=true&language=en&password=awesomepassword ``` To backup your mynewapp app -``` + +```bash YNHAPP=mynewapp yunohost backup create --apps ${YNHAPP} --debug ``` To restore your mynewapp app from the backup just made -``` + +```bash YNHAPP=mynewapp yunohost backup restore XXXXXXXX-XXXXXX --apps ${YNHAPP} --debug ``` To upgrade your mynewapp -``` + +```bash YNHAPP=mynewapp yunohost app upgrade ${YNHAPP} -f /mycentralrepo/${YNHAPP}_ynh/ --debug From 870d7dc675f226a740e344d55b9372d6c2641702 Mon Sep 17 00:00:00 2001 From: yalh76 Date: Fri, 15 May 2020 01:00:50 +0200 Subject: [PATCH 3/9] switching to packaging_apps_final_boss --- contributordoc.md | 2 +- packaging_apps_advanced.md => packaging_apps_final_boss.md | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename packaging_apps_advanced.md => packaging_apps_final_boss.md (100%) diff --git a/contributordoc.md b/contributordoc.md index 3983e24d..0ff0b100 100644 --- a/contributordoc.md +++ b/contributordoc.md @@ -25,7 +25,7 @@ * [App CI dashboard](https://dash.yunohost.org/appci/branch/stable) * [App Continuous Integration for packagers](/packaging_apps_ci) * [YEP - YunoHost Enhancement Proposals](/packaging_apps_guidelines) - * [Advanced packaging](/packaging_apps_advanced) + * [Advanced packaging](/packaging_apps_final_boss) * [Contributing to the YunoHost core](/dev) * [Setting up a dev environment with ynh-dev](https://github.com/YunoHost/ynh-dev/blob/master/README.md) * [Deb build chain](https://github.com/YunoHost/vinaigrette/blob/master/README.md) diff --git a/packaging_apps_advanced.md b/packaging_apps_final_boss.md similarity index 100% rename from packaging_apps_advanced.md rename to packaging_apps_final_boss.md From 4995449d0c6ad9c0b402870416e7e0dada6f07a2 Mon Sep 17 00:00:00 2001 From: yalh76 Date: Fri, 15 May 2020 01:09:13 +0200 Subject: [PATCH 4/9] Typo --- packaging_apps_final_boss.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packaging_apps_final_boss.md b/packaging_apps_final_boss.md index fc43eab4..aaa2e5f0 100644 --- a/packaging_apps_final_boss.md +++ b/packaging_apps_final_boss.md @@ -2,7 +2,7 @@ Here come the time: - you know all the [YEPs](/packaging_apps_guidelines) -- you masterise [apps packaging](/packaging_apps), [package_check](https://github.com/YunoHost/package_check), [example_ynh](https://github.com/YunoHost/example_ynh) and [experimnetal helpers](https://github.com/YunoHost-Apps/Experimental_helpers) +- you masterise [apps packaging](/packaging_apps), [package_check](https://github.com/YunoHost/package_check), [example_ynh](https://github.com/YunoHost/example_ynh) and [experimental helpers](https://github.com/YunoHost-Apps/Experimental_helpers) - you have integrated the [YunoHost Apps Group](https://yunohost.org/#/project_organization) - you know what means `¯\_(ツ)_/¯` - you use to say `Don't ask to ask, just ask` in the [YunoHost support room](https://yunohost.org/#/help) From 7b73edecada70c010d4ebf834da2965284f5c1cb Mon Sep 17 00:00:00 2001 From: yalh76 Date: Fri, 15 May 2020 01:10:31 +0200 Subject: [PATCH 5/9] removing not a goal to be core dev --- packaging_apps_final_boss.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/packaging_apps_final_boss.md b/packaging_apps_final_boss.md index aaa2e5f0..1b7ecef5 100644 --- a/packaging_apps_final_boss.md +++ b/packaging_apps_final_boss.md @@ -7,8 +7,6 @@ Here come the time: - you know what means `¯\_(ツ)_/¯` - you use to say `Don't ask to ask, just ask` in the [YunoHost support room](https://yunohost.org/#/help) -And now you want to improve your packaging speed before climbing into [Yunohost Core developpement](/dev). - Here are few ideas that will help you. There are things that slow you down: From 0940db009ae6d580c7b27a494c872048273241dd Mon Sep 17 00:00:00 2001 From: yalh76 Date: Sat, 13 Mar 2021 16:05:54 +0100 Subject: [PATCH 6/9] Update packaging_apps_final_boss.md --- packaging_apps_final_boss.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/packaging_apps_final_boss.md b/packaging_apps_final_boss.md index 1b7ecef5..b51a43cb 100644 --- a/packaging_apps_final_boss.md +++ b/packaging_apps_final_boss.md @@ -1,20 +1,20 @@ # Packing Applications : Advanced -Here come the time: +Here comes the time: - you know all the [YEPs](/packaging_apps_guidelines) - you masterise [apps packaging](/packaging_apps), [package_check](https://github.com/YunoHost/package_check), [example_ynh](https://github.com/YunoHost/example_ynh) and [experimental helpers](https://github.com/YunoHost-Apps/Experimental_helpers) - you have integrated the [YunoHost Apps Group](https://yunohost.org/#/project_organization) - you know what means `¯\_(ツ)_/¯` - you use to say `Don't ask to ask, just ask` in the [YunoHost support room](https://yunohost.org/#/help) -Here are few ideas that will help you. +Here are few ideas that could help you to improve YunoHost apps packaging. -There are things that slow you down: +Usually, there are things that slow you down: - Installing your freshly done package because of dependencies installation or npm/rust/elixir/composer builds - Waiting a package_check to see the level of your package - etc... -The idea is to use YunoHost stuff to help you improve this lost time. +The idea is to use YunoHost stuffs to help you improve this lost time. ## Building your infrastructure First idea is to build several VMs or LXCs with YunoHost to let you parallelize actions, like having the first one installing an apps, the second installing another apps, the third one debugging an apps upgrade, etc... From 12c19576422a6bbd862554892f54b48bee80a29b Mon Sep 17 00:00:00 2001 From: yalh76 Date: Sat, 13 Mar 2021 16:26:29 +0100 Subject: [PATCH 7/9] packaging_apps_infrastructure --- contributordoc.md | 1 - packaging_apps_final_boss.md => packaging_apps_infrastructure.md | 0 2 files changed, 1 deletion(-) rename packaging_apps_final_boss.md => packaging_apps_infrastructure.md (100%) diff --git a/contributordoc.md b/contributordoc.md index 0ff0b100..4570723d 100644 --- a/contributordoc.md +++ b/contributordoc.md @@ -25,7 +25,6 @@ * [App CI dashboard](https://dash.yunohost.org/appci/branch/stable) * [App Continuous Integration for packagers](/packaging_apps_ci) * [YEP - YunoHost Enhancement Proposals](/packaging_apps_guidelines) - * [Advanced packaging](/packaging_apps_final_boss) * [Contributing to the YunoHost core](/dev) * [Setting up a dev environment with ynh-dev](https://github.com/YunoHost/ynh-dev/blob/master/README.md) * [Deb build chain](https://github.com/YunoHost/vinaigrette/blob/master/README.md) diff --git a/packaging_apps_final_boss.md b/packaging_apps_infrastructure.md similarity index 100% rename from packaging_apps_final_boss.md rename to packaging_apps_infrastructure.md From 4d5b4e6c9956ac348ba2ac53e8b018f7be2152a8 Mon Sep 17 00:00:00 2001 From: yalh76 Date: Sat, 13 Mar 2021 16:59:37 +0100 Subject: [PATCH 8/9] packaging_apps_advanced_packagers --- ...md => packaging_apps_advanced_packagers.md | 100 ++++++++++++++++-- 1 file changed, 93 insertions(+), 7 deletions(-) rename packaging_apps_infrastructure.md => packaging_apps_advanced_packagers.md (65%) diff --git a/packaging_apps_infrastructure.md b/packaging_apps_advanced_packagers.md similarity index 65% rename from packaging_apps_infrastructure.md rename to packaging_apps_advanced_packagers.md index b51a43cb..a75ee2f1 100644 --- a/packaging_apps_infrastructure.md +++ b/packaging_apps_advanced_packagers.md @@ -1,4 +1,4 @@ -# Packing Applications : Advanced +# Packing Applications : Advanced Packagers Here comes the time: - you know all the [YEPs](/packaging_apps_guidelines) @@ -17,7 +17,7 @@ Usually, there are things that slow you down: The idea is to use YunoHost stuffs to help you improve this lost time. ## Building your infrastructure -First idea is to build several VMs or LXCs with YunoHost to let you parallelize actions, like having the first one installing an apps, the second installing another apps, the third one debugging an apps upgrade, etc... +First idea is to build several VMs, LXCs or VirtualBoxs with YunoHost to let you parallelize actions, like having the first one installing an app, the second installing another app, the third one debugging an app upgrade, etc... After first installation of those VMs/LXCs, each one has to be reacheable with its domain name, like for example: - ynh1.mydomain.org @@ -27,6 +27,8 @@ After first installation of those VMs/LXCs, each one has to be reacheable with i For that you will need a public IP for each one or reverse proxy... If your are lucky enough to have IPV6 and a /64 IPV6 scope, it won't be a problem to assign a public IPV6 to each YunoHost. +Another way the DNS stuff is to simply use an hosts file. + In addition, it's helpfull to have several domains names for each YunoHost, for example: - ynh1.mydomain.org - test11.mydomain.org @@ -47,13 +49,98 @@ When your infrastructure is up and ready, you will have to do snapshots of each Don't forget to regularly upgrade your YunoHosts VMs/LXCs to the last release doing: revert back to last happy/healthy YunoHost snapshot and doing `yunohost domain cert-renew --no-check && yunohost tools update && yunohost tools upgrade --apps && yunohost tools upgrade --system && apt autoremove`. Take a snapshot after that successfull full upgrade. And after some testings or one week later, remove old snapshot. PS: that can be automated... +Example of a script to upgrade a master VirtualBox VM + +``` +#!/bin/bash + +# printers +normal="\e[0m" +bold="\e[1m" +white="\e[97m" +blue="\e[34m" +echo_info() +{ + echo -e "[${bold}${blue}INFO${normal}] ${bold}${white}$1${normal}" +} + +echo_info "### Update host" +sudo apt update +sudo apt dist-upgrade + +sudo yunohost settings set security.password.user.strength -v -1 +sudo yunohost settings set security.password.admin.strength -v -1 + +echo_info "### Update package_check" +( cd package_check; git pull) +echo_info "### Update LXC container" +package_check/sub_scripts/lxc_upgrade.sh + +echo_info "### Clean hosts" +sudo apt autoclean +sudo apt autoremove + +echo_info "### Fill free space with zero" +cat /dev/zero > zeros ; sync ; rm zeros + +echo_info "### Run 'VBoxManage modifyhd --compact file.vdi' to reduce the size of this VM" +``` + +Example of a script to customize a cloned VM + +``` +#!/bin/bash + +master_domain=domain1.fr +domainX=domain2.fr +master_ip=70 +ipX=68 + + +sudo yunohost --admin-password admin domain add $domainX +sudo yunohost --admin-password admin domain add sous.$domainX +sudo yunohost --admin-password admin tools maindomain -n $domainX +sudo yunohost --admin-password admin domain remove $master_domain +sudo yunohost --admin-password admin domain remove sous.$master_domain + +sudo sed -i "s/address 192.168.1.$master_ip/address 192.168.1.${ipX}/" /etc/network/interfaces + +sudo reboot +``` + Doing that you will have all the time a full/ready to go YunoHosts infrastructure for advanced packagers. And if require, you can have some VMs/LXCs runnning the YunoHost testing or YunoHost next Debian main version. ## Having your packages/commits available in each VMs/LXCs There are several ways to have your freshly created package available from each VMs/LXCs 1- Do an ssh from the YunoHost VM/LXC to a central repository on your computer -2- rsync you central repository to each VM/LXC -3- use [syncthing](https://github.com/YunoHost-Apps/syncthing_ynh) to syncrhonise you main repository on each VM/LXC +2- Having a shared directory available from each VMs, LXCs VirtualBoxs +3- Rsync you central repository to each VM/LXC +4- Use [syncthing](https://github.com/YunoHost-Apps/syncthing_ynh) to syncrhonise you main repository on each VM/LXC + +Example of a script to mount as shared directory a packaging directory into a VirtualBox VM + +``` +#!/bin/bash + +shared_folder=my_directory + +# printers +normal="\e[0m" +bold="\e[1m" +white="\e[97m" +blue="\e[34m" +echo_info() +{ + echo -e "[${bold}${blue}INFO${normal}] ${bold}${white}$1${normal}" +} + +# Create the main directory for the mount +sudo mkdir -p "./$shared_folder" + +# Mount ynh-dev directory from the host +echo_info "Mount shared directory $shared_folder" +sudo mount -o defaults -t vboxsf $shared_folder "./$shared_folder" +``` ## Shortcuts Use and abuse of `yunohost app install` `--args` argument @@ -103,10 +190,9 @@ With that, you will be able to launch different actions on differents YunoHost V With advanced packaging comes [CI_package_check](https://github.com/YunoHost/CI_package_check), it lets you serialize and automate package_check. - -## Managing all the parallel packaging you are doing +## Managing all the parallel packaging you are working on When you do several things at the same time, sometimes you don't remember what is the next step for this or that app. -A good tool to keep your TODO list organized is to use a kaban like: +A good tool to keep your TODO list organized is to use a kaban like apps: - [kanboard](https://github.com/YunoHost-Apps/kanboard_ynh) - [wekan](https://github.com/YunoHost-Apps/wekan_ynh) From 116fde13b7093986a4e789cd813ad72617931242 Mon Sep 17 00:00:00 2001 From: yalh76 Date: Sat, 13 Mar 2021 17:22:14 +0100 Subject: [PATCH 9/9] Advanced Packagers --- .../04.packaging_apps/16.advanced_packagers/advanced_packagers.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename packaging_apps_advanced_packagers.md => pages/04.contribute/04.packaging_apps/16.advanced_packagers/advanced_packagers.md (100%) diff --git a/packaging_apps_advanced_packagers.md b/pages/04.contribute/04.packaging_apps/16.advanced_packagers/advanced_packagers.md similarity index 100% rename from packaging_apps_advanced_packagers.md rename to pages/04.contribute/04.packaging_apps/16.advanced_packagers/advanced_packagers.md