Merge pull request #102 from jeromelebleu/test

Refactor the moulinette core

.gitignore

@@ -10,6 +10,7 @@ build
 eggs
 parts
 bin
+cache
 var
 sdist
 develop-eggs
@@ -22,10 +23,12 @@ pip-log.txt
 .coverage
 .tox
 
-#Translations
+# Translations
 *.mo
 
-#Mr Developer
+# Mr Developer
 .mr.developer.cfg
 
+# Moulinette
 doc/*.json
+src/moulinette/package.py

README.md

@@ -1,172 +1,48 @@
-YunoHost CLI
+Moulinette
-============
+==========
+
+The *moulinette* is a Python package that allows to quickly and easily
+deploy an application for different interfaces.
 
 
-Specifications
+Overview
---------------
+--------
+
+Initially, the moulinette was an application made for the
+[YunoHost](https://yunohost.org/) project in order to regroup all its
+related operations into a single program called *moulinette*. Those
+operations were available from a command-line interface and a Web server
+providing an API. Moreover, the usage of these operations (e.g.
+required/optional arguments) was defined into a simple yaml file -
+called *actionsmap*. This file was parsed in order to construct an
+*ArgumentParser* object and to parse the command arguments to process
+the proper operation.
+
+During a long refactoring with the goal of unify both interfaces, the
+idea to separate the core of the YunoHost operations has emerged.
+The core kept the same name *moulinette* and try to follow the same
+initial principle. An [Actions Map](#actions-map) - which defines
+available operations and their usage - is parsed and it's used to
+process an operation from several unified [Interfaces](#interfaces). It
+also supports a configuration mechanism - which allows to restrict an
+operation on an interface for example (see
+[Authenticators](#authenticators)).
 
 
-### User
+### Actions Map
+...
 
-    yunohost user list [-h] [--fields FIELDS [FIELDS ...]] [-o OFFSET]
+### Interfaces
-                            [-f FILTER] [-l LIMIT]
+...
-    yunohost user create [-h] [-u USERNAME] [-l LASTNAME] [-f FIRSTNAME]
+
-                              [-p PASSWORD] [-m MAIL]
+### Authenticators
-    yunohost user delete [-h] [--purge] users [users ...]
+...
-    yunohost user update [-h] [-f FIRSTNAME]
-                              [--add-mailalias MAIL [MAIL ...]] [-m MAIL]
-                              [-l LASTNAME]
-                              [--remove-mailforward MAIL [MAIL ...]]
-                              [--remove-mailalias MAIL [MAIL ...]]
-                              [--add-mailforward MAIL [MAIL ...]]
-                              [-p PASSWORD]
-                              user
-    yunohost user info [-h] [-u USER] [-m MAIL]
-
-
-### Domain
-
-    yunohost domain list [-h] [-l LIMIT] [-o OFFSET] [-f FILTER]
-    yunohost domain add [-h] [domain [domain ...]]
-    yunohost domain remove [-h] [domain [domain ...]]
-    yunohost domain info [-h] domain
-    yunohost domain renewcert [-h] domain
-
-
-### App
-
-    yunohost app updatelist [-h] [-u URL]
-    yunohost app list [-h] [--fields FIELDS [FIELDS ...]] [-o OFFSET]
-                           [-f FILTER] [-l LIMIT]
-    yunohost app install [-h] [-d DOMAIN] [--public] [-l LABEL] [-p PATH]
-                              [--protected]
-                              app
-    yunohost app remove [-h] app [app ...]
-    yunohost app upgrade [-h] [app [app ...]]
-    yunohost app info [-h] app
-    yunohost app addaccess [-h] [-u USER [USER ...]] app [app ...]
-    yunohost app removeaccess [-h] [-u USER [USER ...]] app [app ...]
-
-
-### Firewall
-
-    yunohost firewall list [-h]
-    yunohost firewall allow [-h] {UDP,TCP,Both} port name
-    yunohost firewall disallow [-h] name
-
-
-### Monitoring
-
-    yunohost monitor disk [-h] [-m MOUNTPOINT] [-t] [-f] [-H]
-    yunohost monitor network [-h] [-u] [-i] [-H]
-    yunohost monitor system [-h] [-m] [-u] [-i] [-p] [-c] [-H]
-
-
-### Services
-
-    yunohost service status [-h] [NAME [NAME ...]]
-    yunohost service start [-h] NAME [NAME ...]
-    yunohost service stop [-h] NAME [NAME ...]
-    yunohost service enable [-h] NAME [NAME ...]
-    yunohost service disable [-h] NAME [NAME ...]
-
-
-### Tools
-
-    yunohost tools postinstall [-h] [-d DOMAIN] [-p PASSWORD]
-    yunohost tools maindomain [-h] [-o OLD_DOMAIN] [-n NEW_DOMAIN]
-    yunohost tools adminpw [-h] [-o OLD_PASSWORD] [-n NEW_PASSWORD]
-    yunohost tools ldapinit [-h] [-d DOMAIN]
-
-
-How to use "as is" ?
---------------------
-
-The executable file is yunohost, for example:
-
-    ./yunohost user create
-
-
-Contribute / FAQ
-----------------
-
-*What a lovely idea !* :)
-
-### Dafuq is dat moulinette ?
-We decided to regroup all YunoHost related operations into a single program called "moulinette". This will allow us to entirely manipulate our YunoHost instances through a wonderful CLI. Additionally the web interface will just have to call the same "moulinette" functions. Magic power inside :p
-
-### Important files
-* `` yunohost `` File executed on function calling - i.e `` ./yunohost user create ``.
-* `` action_map.yml `` Defines all CLI actions and links arguments.
-* `` yunohost.py `` Contains all YunoHost functions likely to be shared between moulinette files. Also contains service connections classes (erk).
-* `` yunohost_*.py `` Files containing action functions. `` * `` is the category: user, domain, firewall, etc.
-
-### How to add a function ?
-1. Check if the action is already in the `` action_map.yml `` file. If not, follow the file documentation to add it.
-2. Also check if the file `` yunohost_category.py `` is created in the working tree. If not, just create it (you may take example of `` yunohost_user.py `` file).
-3. Add your function `` category_action() `` in this file - i.e `` user_create() ``
-
-**Note:** `` category_action() `` takes one parameter,`` args `` which contains the arguments passed to the command. Refers to `` action_map.yml `` documentation for more informations.
-
-### Error handling
-Moulinette has a unified way to handle errors. First, you need to import the ``YunoHostError`` exception:
-`` from yunohost import YunoHostError ``
-
-Then you need to raise errors like this:
-`` raise YunoHostError(<ERROR CODE>, <MESSAGE>) ``
-
-For example:
-`` raise YunoHostError(125, _("Interrupted, user not created")) ``
-
-**Note:** Standard error codes can be found in the ``YunoHostError`` class in `` yunohost.py `` file.
-
-### Print results
-Moulinette also have a unified way to print results. In fact we don't only print result for the CLI, but we also have to export the result in a JSON way.
-Results are automatically printed OR exported, you don't have to print it yourself in the action's functions. Your function just need is to return results as a dictionary, for example:
-`` return { 'Fullname' : 'Homer Simpson', 'Mail' : 'homer@simpson.org', 'Username' : 'hsimpson' } ``
-
-### i18n
-We will have to translate YunoHost, and we have already initialized i18n module in the moulinette. As a result, do not forget to put potentially translated strings into `` _() `` function. For example:
-`` raise YunoHostError(125, _("Interrupted, user not created")) ``
-
-### Git is pissing me off !
-OK, this is the workflow !
-
-**For gitlab:**
-Development is handle with git branches and you have your own (i.e dev_beudbeud).
-```
-git clone git@dev.yunohost.org:moulinette.git
-git checkout -b dev_beudbeud ``
-git rebase origin/dev
-```
-
-
-Do your modifications, then :
-```
-git commit -am 'My commit message'
-git pull origin dev (merge manually if conflicts)
-git push origin dev_beudbeud
-```
-
-Then you could ask for a 'merge request' in gitlab.
-
-**For github (here):**
-Development is handle with forked repos and you have your own (i.e beudbeud/moulinette).
-```
-git clone https://github.com/beudbeud/moulinette.git ``
-git checkout -b dev
-git rebase origin/dev
-```
-
-Do your modifications, then:
-```
-git commit -am 'My commit message'
-git remote add vanilla https://github.com/YunoHost/moulinette.git
-git pull vanilla dev (merge manually if conflicts)
-git push origin dev
-```
-
-Then you could ask for a 'pull request' in github.
 
 
+Requirements
+------------
 
+* Python 2.7
+* python-bottle (>= 0.10)
+* python-gnupg (>= 0.3)
+* python-ldap (>= 2.4)

bin/yunohost

@@ -0,0 +1,46 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import sys
+import os.path
+
+from_source = False
+
+# Run from source
+basedir = os.path.abspath('%s/../' % os.path.dirname(__file__))
+if os.path.isdir('%s/moulinette' % basedir):
+    sys.path.insert(0, basedir)
+    from_source = True
+
+from moulinette import init, cli, MoulinetteError
+from moulinette.helpers import YunoHostError, colorize
+
+
+## Main action
+
+if __name__ == '__main__':
+    # Run from source
+    init(_from_source=from_source)
+
+    # Additional arguments
+    use_cache = True
+    if '--no-cache' in sys.argv:
+        use_cache = False
+        sys.argv.remove('--no-cache')
+
+    try:
+        args = list(sys.argv)
+        args.pop(0)
+
+        # Check that YunoHost is installed
+        if not os.path.isfile('/etc/yunohost/installed') \
+           and (len(args) < 2 or args[1] != 'tools' or args[2] != 'postinstall'):
+            raise YunoHostError(17, _("YunoHost is not correctly installed, please execute 'yunohost tools postinstall'"))
+
+        # Execute the action
+        ret = cli(['yunohost'], args, use_cache)
+    except YunoHostError as e:
+        # TODO: Remove this and associated import when yunohost package has been revisited
+        print(colorize(_("Error: "), 'red') + e.message)
+        sys.exit(e.code)
+    sys.exit(ret)

bin/yunohost-api

@@ -0,0 +1,52 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import sys
+import os.path
+
+from_source = False
+
+# Run from source
+basedir = os.path.abspath('%s/../' % os.path.dirname(__file__))
+if os.path.isdir('%s/moulinette' % basedir):
+    sys.path.insert(0, basedir)
+    from_source = True
+
+from moulinette import init, api, MoulinetteError
+
+
+## Callbacks for additional routes
+
+def is_installed():
+    """
+    Check whether YunoHost is installed or not
+
+    """
+    installed = False
+    if os.path.isfile('/etc/yunohost/installed'):
+        installed = True
+    return { 'installed': installed }
+
+
+## Main action
+
+if __name__ == '__main__':
+    # Run from source
+    init(_from_source=from_source)
+
+    # Additional arguments
+    use_cache = True
+    if '--no-cache' in sys.argv:
+        use_cache = False
+        sys.argv.remove('--no-cache')
+    # TODO: Add log argument
+
+    try:
+        # Run the server
+        api(['yunohost'], 6787,
+            {('GET', '/installed'): is_installed}, use_cache)
+    except MoulinetteError as e:
+        from moulinette.interfaces.cli import colorize
+        print(_('%s: %s' % (colorize(_('Error'), 'red'), e.strerror)))
+        sys.exit(e.errno)
+    sys.exit(0)

data/actionsmap/test.yml

@@ -0,0 +1,59 @@
+
+#############################
+#     Global parameters     #
+#############################
+_global:
+    configuration:
+        authenticate:
+            - api
+        authenticator:
+            default:
+                vendor: ldap
+                help: Admin Password
+                parameters:
+                    uri: ldap://localhost:389
+                    base_dn: dc=yunohost,dc=org
+                    user_rdn: cn=admin
+            ldap-anonymous:
+                vendor: ldap
+                parameters:
+                    uri: ldap://localhost:389
+                    base_dn: dc=yunohost,dc=org
+            test-profile:
+                vendor: ldap
+                help: Admin Password (profile)
+                parameters:
+                    uri: ldap://localhost:389
+                    base_dn: dc=yunohost,dc=org
+                    user_rdn: cn=admin
+        argument_auth: true
+
+#############################
+#       Test Actions        #
+#############################
+test:
+    actions:
+        non-auth:
+            api: GET /test/non-auth
+            configuration:
+                authenticate: false
+        auth:
+            api: GET /test/auth
+            configuration:
+                authenticate: all
+        auth-profile:
+            api: GET /test/auth-profile
+            configuration:
+                authenticate: all
+                authenticator: test-profile
+        auth-cli:
+            api: GET /test/auth-cli
+            configuration:
+                authenticate:
+                    - cli
+        anonymous:
+            api: GET /test/anon
+            configuration:
+                authenticate: all
+                authenticator: ldap-anonymous
+                argument_auth: false

data/actionsmap/yunohost.yml

@@ -29,12 +29,27 @@
 #
 ##########################################################################
 
-# TODO: Add patern for all this
-
 #############################
-#       General args        #
+#     Global parameters     #
 #############################
-general_arguments:
+_global:
+    configuration:
+        authenticate:
+            - api
+        authenticator:
+            default:
+                vendor: ldap
+                help: Admin Password
+                parameters:
+                    uri: ldap://localhost:389
+                    base_dn: dc=yunohost,dc=org
+                    user_rdn: cn=admin
+            ldap-anonymous:
+                vendor: ldap
+                parameters:
+                    uri: ldap://localhost:389
+                    base_dn: dc=yunohost,dc=org
+    arguments:
         -v:
             full: --version
             help: Display moulinette version
@@ -74,42 +89,53 @@ user:
                 -u:
                     full: --username
                     help: Must be unique
-                    ask: "Username"
+                    extra:
-                    pattern: '^[a-z0-9_]+$'
+                        ask: "Username"
+                        pattern:
+                            - '^[a-z0-9_]+$'
+                            - "Must be alphanumeric and underscore characters only"
                 -f:
                     full: --firstname
-                    ask: "Firstname"
+                    extra:
+                        ask: "Firstname"
                 -l:
                     full: --lastname
-                    ask: "Lastname"
+                    extra:
+                        ask: "Lastname"
                 -m:
                     full: --mail
                     help: Main mail address must be unique
-                    ask: "Mail address"
+                    extra:
-                    pattern: '^[\w.-]+@[\w.-]+\.[a-zA-Z]{2,6}$'
+                        ask: "Mail address"
+                        pattern:
+                            - '^[\w.-]+@[\w.-]+\.[a-zA-Z]{2,6}$'
+                            - "Must be a valid email address (e.g. someone@domain.org)"
                 -p:
                     full: --password
-                    ask: "User password"
+                    extra:
-                    password: yes
+                        password: "User password"
 
         ### user_delete()
         delete:
             action_help: Delete user
-            api: 'DELETE /users/{users}'
+            api: 'DELETE /users/<users>'
             arguments:
                 -u:
                     full: --users
                     help: Username of users to delete
-                    ask: "Users to delete"
-                    pattern: '^[a-z0-9_]+$'
                     nargs: "*"
+                    extra:
+                        ask: "Users to delete"
+                        pattern:
+                            - '^[a-z0-9_]+$'
+                            - "Must be alphanumeric and underscore characters only"
                 --purge:
                     action: store_true
 
         ### user_update()
         update:
             action_help: Update user informations
-            api: 'PUT /users/{username}'
+            api: 'PUT /users/<username>'
             arguments:
                 username:
                     help: Username of user to update
@@ -143,7 +169,7 @@ user:
         ### user_info()
         info:
             action_help: Get user informations
-            api: 'GET /users/{username}'
+            api: 'GET /users/<username>'
             arguments:
                 username:
                     help: Username or mail to get informations
@@ -179,7 +205,10 @@ domain:
                 domains:
                     help: Domain name to add
                     nargs: '+'
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                    extra:
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
                 -m:
                     full: --main
                     help: Is the main domain
@@ -197,16 +226,22 @@ domain:
                 domains:
                     help: Domain(s) to delete
                     nargs: "+"
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                    extra:
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
 
         ### domain_info()
         info:
             action_help: Get domain informations
-            api: 'GET /domains/{domain}'
+            api: 'GET /domains/<domain>'
             arguments:
                 domain:
                     help: ""
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                    extra:
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
 
 
 #############################
@@ -241,8 +276,11 @@ app:
                 -n:
                     full: --name
                     help: Name of the list to remove
-                    ask: "List to remove"
+                    extra:
-                    pattern: '^[a-z0-9_]+$'
+                        ask: "List to remove"
+                        pattern:
+                            - '^[a-z0-9_]+$'
+                            - "Must be alphanumeric and underscore characters only"
 
         ### app_list()
         list:
@@ -266,7 +304,7 @@ app:
         ### app_info()
         info:
             action_help: Get app info
-            api: GET /app/{app}
+            api: GET /app/<app>
             arguments:
                 app:
                     help: Specific app ID
@@ -290,7 +328,10 @@ app:
                 -u:
                     full: --user
                     help: Allowed app map for a user
-                    pattern: '^[a-z0-9_]+$'
+                    extra:
+                        pattern:
+                            - '^[a-z0-9_]+$'
+                            - "Must be alphanumeric and underscore characters only"
 
 
         ### app_install() TODO: Write help
@@ -333,7 +374,7 @@ app:
         ### app_setting()
         setting:
             action_help: Set ou get an app setting value
-            api: GET /app/{app}/setting
+            api: GET /app/<app>/setting
             arguments:
                 app:
                     help: App ID
@@ -350,7 +391,7 @@ app:
         ### app_service()
         service:
             action_help: Add or remove a YunoHost monitored service
-            api: POST /app/service/{service}
+            api: POST /app/service/<service>
             arguments:
                 service:
                     help: Service to add/remove
@@ -375,7 +416,10 @@ app:
             arguments:
                 port:
                     help: Port to check
-                    pattern: '^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$'
+                    extra:
+                        pattern:
+                            - '^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$'
+                            - "Must be a valid port number (i.e. 0-65535)"
 
         ### app_checkurl()
         checkurl:
@@ -651,8 +695,11 @@ service:
                 -n:
                     full: --number
                     help: Number of lines to display
-                    pattern: '^[0-9]+$'
                     default: "50"
+                    extra:
+                        pattern:
+                            - '^[0-9]+$'
+                            - "Must be a valid number"
 
 
 #############################
@@ -683,7 +730,10 @@ firewall:
             arguments:
                 port:
                     help: Port to open
-                    pattern: '^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$'
+                    extra:
+                        pattern:
+                            - '^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$'
+                            - "Must be a valid port number (i.e. 0-65535)"
                 protocol:
                     help: Protocol associated with port
                     choices:
@@ -817,12 +867,12 @@ tools:
             arguments:
                 -o:
                     full: --old-password
-                    ask: "Current admin password"
+                    extra:
-                    password: yes
+                        password: "Current admin password"
                 -n:
                     full: --new-password
-                    ask: "New admin password"
+                    extra:
-                    password: yes
+                        password: "New admin password"
 
         ### tools_maindomain()
         maindomain:
@@ -831,10 +881,17 @@ tools:
             arguments:
                 -o:
                     full: --old-domain
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                    extra:
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
                 -n:
                     full: --new-domain
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                    extra:
+                        ask: "New main domain"
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
 
         ### tools_postinstall()
         postinstall:
@@ -844,13 +901,16 @@ tools:
                 -d:
                     full: --domain
                     help: YunoHost main domain
-                    ask: "Main domain"
+                    extra:
-                    pattern: '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                        ask: "Main domain"
+                        pattern:
+                            - '^([a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)(\.[a-zA-Z0-9]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)*(\.[a-zA-Z]{1}([a-zA-Z0-9\-]*[a-zA-Z0-9])*)$'
+                            - "Must be a valid domain name (e.g. my-domain.org)"
                 -p:
                     full: --password
                     help: YunoHost admin password
-                    ask: "New admin password"
+                    extra:
-                    password: yes
+                        password: "New admin password"
                 --dyndns:
                     help: Subscribe domain to a DynDNS service
                     action: store_true
@@ -869,7 +929,7 @@ tools:
         upgrade:
             action_help: YunoHost upgrade
             api: POST /upgrade
-            
+
         ### tools_upgradelog()
         upgradelog:
             action_help: Show dpkg log

data/moulinette_cli

@@ -7,14 +7,14 @@ COMPREPLY=()
 
 argc=${COMP_CWORD}
 cur="${COMP_WORDS[argc]}"
-prev="${COMP_WORDS[argc-1]}" 
+prev="${COMP_WORDS[argc-1]}"
 opts=$(yunohost -h | sed -n "/usage/,/}/p" | awk -F"{" '{print $2}' | awk -F"}" '{print $1}' | tr ',' ' ')
 
 if [[ $argc = 1 ]];
 then
 	COMPREPLY=( $(compgen -W "$opts --help" -- $cur ) )
 fi
-	
+
 if [[ "$prev" != "--help" ]];
 then
 	if [[ $argc = 2 ]];
@@ -23,9 +23,9 @@ then
 		COMPREPLY=( $(compgen -W "$opts2 --help" -- $cur ) )
 	elif [[ $argc = 3 ]];
 	then
-        	COMPREPLY=( $(compgen -W "--help" $cur ) )
+		COMPREPLY=( $(compgen -W "--help" $cur ) )
 	fi
-else 
+else
 	COMPREPLY=()
 fi
 

lib/test/test.py

@@ -0,0 +1,19 @@
+
+def test_non_auth():
+    return {'action': 'non-auth'}
+
+def test_auth(auth):
+    return {'action': 'auth',
+            'authenticator': 'default', 'authenticate': 'all'}
+
+def test_auth_profile(auth):
+    return {'action': 'auth-profile',
+            'authenticator': 'test-profile', 'authenticate': 'all'}
+
+def test_auth_cli():
+    return {'action': 'auth-cli',
+            'authenticator': 'default', 'authenticate': ['cli']}
+
+def test_anonymous():
+    return {'action': 'anonymous',
+            'authenticator': 'ldap-anonymous', 'authenticate': 'all'}

lib/yunohost/app.py

@@ -23,6 +23,9 @@
 
     Manage apps
 """
+import logging
+logging.warning('the module yunohost.app has not been revisited and updated yet')
+
 import os
 import sys
 import json
@@ -33,10 +36,11 @@ import time
 import re
 import socket
 import urlparse
-from yunohost import YunoHostError, YunoHostLDAP, win_msg, random_password, is_true, validate
+from domain import domain_list, domain_add
-from yunohost_domain import domain_list, domain_add
+from user import user_info, user_list
-from yunohost_user import user_info, user_list
+from hook import hook_exec, hook_add, hook_remove
-from yunohost_hook import hook_exec, hook_add, hook_remove
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, win_msg, random_password, is_true, validate
 
 repo_path        = '/var/cache/yunohost/repo'
 apps_path        = '/usr/share/yunohost/apps'
@@ -836,7 +840,7 @@ def app_ssowatconf():
     unprotected_regex = []
     protected_urls = []
     protected_regex = []
-    
+
     apps = {}
     for app in app_list()['Apps']:
         if _is_installed(app['ID']):
@@ -896,7 +900,7 @@ def app_ssowatconf():
     conf_dict['unprotected_regex'] = unprotected_regex
     conf_dict['protected_regex'] = protected_regex
     conf_dict['users'] = users
-        
+
     with open('/etc/ssowat/conf.json', 'wb') as f:
         json.dump(conf_dict, f)
 

lib/yunohost/backup.py

@@ -23,12 +23,16 @@
 
     Manage backups
 """
+import logging
+logging.warning('the module yunohost.backup has not been revisited and updated yet')
+
 import os
 import sys
 import json
 import yaml
 import glob
-from yunohost import YunoHostError, YunoHostLDAP, validate, colorize, win_msg
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, validate, colorize, win_msg
 
 def backup_init(helper=False):
     """

lib/yunohost/data/checkupdate

@@ -62,6 +62,6 @@ then
 	exit 3
 else
 	if [ -x /usr/bin/apt-listchanges ] ; then
-        	/usr/bin/apt-listchanges --which=both -f text $DEBS > /tmp/yunohost/changelog 2>/dev/null
+                /usr/bin/apt-listchanges --which=both -f text $DEBS > /tmp/yunohost/changelog 2>/dev/null
 	fi
 fi

lib/yunohost/data/firewall.yml

@@ -1,6 +1,6 @@
-UPNP: 
+UPNP:
     cron: false
-    ports: 
+    ports:
         TCP: [22, 25, 53, 80, 443, 465, 993, 5222, 5269, 5290]
         UDP: [53, 137, 138]
 ipv4:
@@ -9,4 +9,3 @@ ipv4:
 ipv6:
     TCP: [22]
     UDP: [53]
-

lib/yunohost/domain.py

@@ -23,6 +23,9 @@
 
     Manage domains
 """
+import logging
+logging.warning('the module yunohost.backup has not been revisited and updated yet')
+
 import os
 import sys
 import datetime
@@ -32,8 +35,9 @@ import json
 import yaml
 import requests
 from urllib import urlopen
-from yunohost import YunoHostError, YunoHostLDAP, win_msg, colorize, validate, get_required_args
+from dyndns import dyndns_subscribe
-from yunohost_dyndns import dyndns_subscribe
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, win_msg, colorize, validate, get_required_args
 
 
 def domain_list(filter=None, limit=None, offset=None):

lib/yunohost/dyndns.py

@@ -23,13 +23,17 @@
 
     Subscribe and Update DynDNS Hosts
 """
+import logging
+logging.warning('the module yunohost.dyndns has not been revisited and updated yet')
+
 import os
 import sys
 import requests
 import json
 import glob
 import base64
-from yunohost import YunoHostError, YunoHostLDAP, validate, colorize, win_msg
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, validate, colorize, win_msg
 
 def dyndns_subscribe(subscribe_host="dyndns.yunohost.org", domain=None, key=None):
     """

lib/yunohost/firewall.py

@@ -23,6 +23,9 @@
 
     Manage firewall rules
 """
+import logging
+logging.warning('the module yunohost.firewall has not been revisited and updated yet')
+
 import os
 import sys
 try:
@@ -36,8 +39,9 @@ except ImportError:
     sys.stderr.write('Error: Yunohost CLI Require yaml lib\n')
     sys.stderr.write('apt-get install python-yaml\n')
     sys.exit(1)
-from yunohost import YunoHostError, win_msg
+from hook import hook_callback
-from yunohost_hook import hook_callback
+
+from moulinette.helpers import YunoHostError, win_msg
 
 
 def firewall_allow(protocol=None, port=None, ipv6=None, upnp=False):

lib/yunohost/hook.py

@@ -23,11 +23,15 @@
 
     Manage hooks
 """
+import logging
+logging.warning('the module yunohost.hook has not been revisited and updated yet')
+
 import os
 import sys
 import re
 import json
-from yunohost import YunoHostError, YunoHostLDAP, win_msg, colorize
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, win_msg, colorize
 
 hook_folder = '/usr/share/yunohost/hooks/'
 

lib/yunohost/monitor.py

@@ -23,6 +23,9 @@
 
     Monitoring functions
 """
+import logging
+logging.warning('the module yunohost.monitor has not been revisited and updated yet')
+
 import re
 import json
 import time
@@ -34,10 +37,11 @@ import os.path
 import cPickle as pickle
 from urllib import urlopen
 from datetime import datetime, timedelta
-from yunohost import YunoHostError, win_msg
+from service import (service_enable, service_disable,
-from yunohost_service import (service_enable, service_disable,
     service_start, service_stop, service_status)
 
+from moulinette.helpers import YunoHostError, win_msg
+
 glances_uri  = 'http://127.0.0.1:61209'
 stats_path   = '/var/lib/yunohost/stats'
 crontab_path = '/etc/cron.d/yunohost-monitor'

lib/yunohost/service.py

@@ -23,11 +23,15 @@
 
     Manage services
 """
+import logging
+logging.warning('the module yunohost.service has not been revisited and updated yet')
+
 import yaml
 import glob
 import subprocess
 import os.path
-from yunohost import YunoHostError, win_msg
+
+from moulinette.helpers import YunoHostError, win_msg
 
 
 def service_start(names):
@@ -169,7 +173,7 @@ def service_log(name, number=50):
 
     if name not in services.keys():
         raise YunoHostError(1, _("Unknown service '%s'") % service)
-    
+
     if 'log' in services[name]:
         log_list = services[name]['log']
         result = {}
@@ -253,7 +257,7 @@ def _tail(file, n, offset=None):
                 pos = f.tell()
                 lines = f.read().splitlines()
                 if len(lines) >= to_read or pos == 0:
-                    return lines[-to_read:offset and -offset or None] 
+                    return lines[-to_read:offset and -offset or None]
                 avg_line_length *= 1.3
 
     except IOError: return []

lib/yunohost/tools.py

@@ -23,6 +23,9 @@
 
     Specific tools
 """
+import logging
+logging.warning('the module yunohost.tools has not been revisited and updated yet')
+
 import os
 import sys
 import yaml
@@ -32,11 +35,12 @@ import subprocess
 import requests
 import json
 from subprocess import Popen, PIPE
-from yunohost import YunoHostError, YunoHostLDAP, validate, colorize, get_required_args, win_msg
+from domain import domain_add, domain_list
-from yunohost_domain import domain_add, domain_list
+from dyndns import dyndns_subscribe
-from yunohost_dyndns import dyndns_subscribe
+from backup import backup_init
-from yunohost_backup import backup_init
+from app import app_ssowatconf
-from yunohost_app import app_ssowatconf
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, validate, colorize, get_required_args, win_msg
 
 
 def tools_ldapinit(password=None):
@@ -276,7 +280,7 @@ def tools_update():
         raise YunoHostError(17, _("Error during update"))
     else:
         return { "Update" : stdout.splitlines() }
-        
+
 
 def tools_changelog():
     """
@@ -289,7 +293,7 @@ def tools_changelog():
             return { "Changelog" : read_data.splitlines() }
     else:
         raise YunoHostError(17, _("Launch update before upgrade"))
-        
+
 
 def tools_upgrade():
     """
@@ -312,7 +316,7 @@ def tools_upgrade():
             win_msg( _("Upgrade in progress"))
         else:
             raise YunoHostError(17, _("Launch update before upgrade"))
-            
+
 
 def tools_upgradelog():
     """

lib/yunohost/user.py

@@ -23,6 +23,9 @@
 
     Manage users
 """
+import logging
+logging.warning('the module yunohost.user has not been revisited and updated yet')
+
 import os
 import sys
 import ldap
@@ -30,9 +33,10 @@ import crypt
 import random
 import string
 import getpass
-from yunohost import YunoHostError, YunoHostLDAP, win_msg, colorize, validate, get_required_args
+from domain import domain_list
-from yunohost_domain import domain_list
+from hook import hook_callback
-from yunohost_hook import hook_callback
+
+from moulinette.helpers import YunoHostError, YunoHostLDAP, win_msg, colorize, validate, get_required_args
 
 def user_list(fields=None, filter=None, limit=None, offset=None):
     """
@@ -117,7 +121,7 @@ def user_create(username, firstname, lastname, mail, password):
             uid = str(random.randint(200, 99999))
             uid_check = os.system("getent passwd " + uid)
             gid_check = os.system("getent group " + uid)
- 
+
         # Adapt values for LDAP
         fullname = firstname + ' ' + lastname
         rdn = 'uid=' + username + ',ou=users'
@@ -139,7 +143,7 @@ def user_create(username, firstname, lastname, mail, password):
             'uidNumber'     : uid,
             'homeDirectory' : '/home/' + username,
             'loginShell'    : '/bin/false'
- 
+
         }
 
         if yldap.add(rdn, attr_dict):

moulinette/__init__.py

@@ -0,0 +1,108 @@
+# -*- coding: utf-8 -*-
+
+__title__ = 'moulinette'
+__version__ = '0.1'
+__author__ = ['Kload',
+              'jlebleu',
+              'titoko',
+              'beudbeud',
+              'npze']
+__license__ = 'AGPL 3.0'
+__credits__ = """
+    Copyright (C) 2014 YUNOHOST.ORG
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published
+    by the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program; if not, see http://www.gnu.org/licenses
+    """
+__all__ = [
+    'init', 'api', 'cli',
+    'init_interface', 'MoulinetteError',
+]
+
+from moulinette.core import init_interface, MoulinetteError
+
+
+## Package functions
+
+def init(**kwargs):
+    """Package initialization
+
+    Initialize directories and global variables. It must be called
+    before any of package method is used - even the easy access
+    functions.
+
+    Keyword arguments:
+        - **kwargs -- See core.Package
+
+    At the end, the global variable 'pkg' will contain a Package
+    instance. See core.Package for available methods and variables.
+
+    """
+    import sys
+    import __builtin__
+    from moulinette.core import Package, install_i18n
+    __builtin__.__dict__['pkg'] = Package(**kwargs)
+
+    # Initialize internationalization
+    install_i18n()
+
+    # Add library directory to python path
+    sys.path.insert(0, pkg.libdir)
+
+
+## Easy access to interfaces
+
+def api(namespaces, port, routes={}, use_cache=True):
+    """Web server (API) interface
+
+    Run a HTTP server with the moulinette for an API usage.
+
+    Keyword arguments:
+        - namespaces -- The list of namespaces to use
+        - port -- Port number to run on
+        - routes -- A dict of additional routes to add in the form of
+            {(method, uri): callback}
+        - use_cache -- False if it should parse the actions map file
+            instead of using the cached one
+
+    """
+    moulinette = init_interface('api',
+                                kwargs={'routes': routes},
+                                actionsmap={'namespaces': namespaces,
+                                            'use_cache': use_cache})
+    moulinette.run(port)
+
+def cli(namespaces, args, use_cache=True):
+    """Command line interface
+
+    Execute an action with the moulinette from the CLI and print its
+    result in a readable format.
+
+    Keyword arguments:
+        - namespaces -- The list of namespaces to use
+        - args -- A list of argument strings
+        - use_cache -- False if it should parse the actions map file
+            instead of using the cached one
+
+    """
+    from moulinette.interfaces.cli import colorize
+
+    try:
+        moulinette = init_interface('cli',
+                                    actionsmap={'namespaces': namespaces,
+                                                'use_cache': use_cache})
+        moulinette.run(args)
+    except MoulinetteError as e:
+        print(_('%s: %s' % (colorize(_('Error'), 'red'), e.strerror)))
+        return e.errno
+    return 0

moulinette/actionsmap.py

@@ -0,0 +1,595 @@
+# -*- coding: utf-8 -*-
+
+import os
+import re
+import errno
+import logging
+import yaml
+import cPickle as pickle
+from collections import OrderedDict
+
+from moulinette.core import (MoulinetteError, MoulinetteLock)
+from moulinette.interfaces import BaseActionsMapParser
+
+## Actions map Signals -------------------------------------------------
+
+class ActionsMapSignals(object):
+    """Actions map's Signals interface
+
+    Allow to easily connect signals of the actions map to handlers. They
+    can be given as arguments in the form of { signal: handler }.
+
+    """
+    def __init__(self, **kwargs):
+        # Initialize handlers
+        for s in self.signals:
+            self.clear_handler(s)
+
+        # Iterate over signals to connect
+        for s, h in kwargs.items():
+            self.set_handler(s, h)
+
+    def set_handler(self, signal, handler):
+        """Set the handler for a signal"""
+        if signal not in self.signals:
+            raise ValueError("unknown signal '%s'" % signal)
+        setattr(self, '_%s' % signal, handler)
+
+    def clear_handler(self, signal):
+        """Clear the handler of a signal"""
+        if signal not in self.signals:
+            raise ValueError("unknown signal '%s'" % signal)
+        setattr(self, '_%s' % signal, self._notimplemented)
+
+
+    ## Signals definitions
+
+    """The list of available signals"""
+    signals = { 'authenticate', 'prompt' }
+
+    def authenticate(self, authenticator, help):
+        """Process the authentication
+
+        Attempt to authenticate to the given authenticator and return
+        it.
+        It is called when authentication is needed (e.g. to process an
+        action).
+
+        Keyword arguments:
+            - authenticator -- The authenticator object to use
+            - help -- A help message for the authenticator
+
+        Returns:
+            The authenticator object
+
+        """
+        if authenticator.is_authenticated:
+            return authenticator
+        return self._authenticate(authenticator, help)
+
+    def prompt(self, message, is_password=False, confirm=False):
+        """Prompt for a value
+
+        Prompt the interface for a parameter value which is a password
+        if 'is_password' and must be confirmed if 'confirm'.
+        Is is called when a parameter value is needed and when the
+        current interface should allow user interaction (e.g. to parse
+        extra parameter 'ask' in the cli).
+
+        Keyword arguments:
+            - message -- The message to display
+            - is_password -- True if the parameter is a password
+            - confirm -- True if the value must be confirmed
+
+        Returns:
+            The collected value
+
+        """
+        return self._prompt(message, is_password, confirm)
+
+    @staticmethod
+    def _notimplemented(**kwargs):
+        raise NotImplementedError("this signal is not handled")
+
+shandler = ActionsMapSignals()
+
+
+## Extra parameters ----------------------------------------------------
+
+# Extra parameters definition
+
+class _ExtraParameter(object):
+    """
+    Argument parser for an extra parameter.
+
+    It is a pure virtual class that each extra parameter classes must
+    implement.
+
+    """
+    def __init__(self, iface):
+        # TODO: Add conn argument which contains authentification object
+        self.iface = iface
+
+
+    ## Required variables
+    # Each extra parameters classes must overwrite these variables.
+
+    """The extra parameter name"""
+    name = None
+
+
+    ## Optional variables
+    # Each extra parameters classes can overwrite these variables.
+
+    """A list of interface for which the parameter doesn't apply"""
+    skipped_iface = {}
+
+
+    ## Virtual methods
+    # Each extra parameters classes can implement these methods.
+
+    def __call__(self, parameter, arg_name, arg_value):
+        """
+        Parse the argument
+
+        Keyword arguments:
+            - parameter -- The value of this parameter for the action
+            - arg_name -- The argument name
+            - arg_value -- The argument value
+
+        Returns:
+            The new argument value
+
+        """
+        return arg_value
+
+    @staticmethod
+    def validate(value, arg_name):
+        """
+        Validate the parameter value for an argument
+
+        Keyword arguments:
+            - value -- The parameter value
+            - arg_name -- The argument name
+
+        Returns:
+            The validated parameter value
+
+        """
+        return value
+
+class AskParameter(_ExtraParameter):
+    """
+    Ask for the argument value if possible and needed.
+
+    The value of this parameter corresponds to the message to display
+    when asking the argument value.
+
+    """
+    name = 'ask'
+    skipped_iface = { 'api' }
+
+    def __call__(self, message, arg_name, arg_value):
+        # TODO: Fix asked arguments ordering
+        if arg_value:
+            return arg_value
+
+        try:
+            # Ask for the argument value
+            return shandler.prompt(message)
+        except NotImplementedError:
+            return arg_value
+
+    @classmethod
+    def validate(klass, value, arg_name):
+        # Allow boolean or empty string
+        if isinstance(value, bool) or (isinstance(value, str) and not value):
+            logging.debug("value of '%s' extra parameter for '%s' argument should be a string" \
+                % (klass.name, arg_name))
+            value = arg_name
+        elif not isinstance(value, str):
+            raise TypeError("Invalid type of '%s' extra parameter for '%s' argument" \
+                % (klass.name, arg_name))
+        return value
+
+class PasswordParameter(AskParameter):
+    """
+    Ask for the password argument value if possible and needed.
+
+    The value of this parameter corresponds to the message to display
+    when asking the password.
+
+    """
+    name = 'password'
+
+    def __call__(self, message, arg_name, arg_value):
+        if arg_value:
+            return arg_value
+
+        try:
+            # Ask for the password
+            return shandler.prompt(message, True, True)
+        except NotImplementedError:
+            return arg_value
+
+class PatternParameter(_ExtraParameter):
+    """
+    Check if the argument value match a pattern.
+
+    The value of this parameter corresponds to a list of the pattern and
+    the message to display if it doesn't match.
+
+    """
+    name = 'pattern'
+
+    def __call__(self, arguments, arg_name, arg_value):
+        pattern, message = (arguments[0], arguments[1])
+
+        if not re.match(pattern, arg_value or ''):
+            raise MoulinetteError(errno.EINVAL, message)
+        return arg_value
+
+    @staticmethod
+    def validate(value, arg_name):
+        # Tolerate string type
+        if isinstance(value, str):
+            logging.warning("value of 'pattern' extra parameter for '%s' argument should be a list" % arg_name)
+            value = [value, _("'%s' argument is not matching the pattern") % arg_name]
+        elif not isinstance(value, list) or len(value) != 2:
+            raise TypeError("Invalid type of 'pattern' extra parameter for '%s' argument" % arg_name)
+        return value
+
+"""
+The list of available extra parameters classes. It will keep to this list
+order on argument parsing.
+
+"""
+extraparameters_list = [AskParameter, PasswordParameter, PatternParameter]
+
+# Extra parameters argument Parser
+
+class ExtraArgumentParser(object):
+    """
+    Argument validator and parser for the extra parameters.
+
+    Keyword arguments:
+        - iface -- The running interface
+
+    """
+    def __init__(self, iface):
+        self.iface = iface
+        self.extra = OrderedDict()
+
+        # Append available extra parameters for the current interface
+        for klass in extraparameters_list:
+            if iface in klass.skipped_iface:
+                continue
+            self.extra[klass.name] = klass
+
+    def validate(self, arg_name, parameters):
+        """
+        Validate values of extra parameters for an argument
+
+        Keyword arguments:
+            - arg_name -- The argument name
+            - parameters -- A dict of extra parameters with their values
+
+        """
+        # Iterate over parameters to validate
+        for p, v in parameters.items():
+            klass = self.extra.get(p, None)
+            if not klass:
+                # Remove unknown parameters
+                del parameters[p]
+            else:
+                # Validate parameter value
+                parameters[p] = klass.validate(v, arg_name)
+
+        return parameters
+
+    def parse(self, arg_name, arg_value, parameters):
+        """
+        Parse argument with extra parameters
+
+        Keyword arguments:
+            - arg_name -- The argument name
+            - arg_value -- The argument value
+            - parameters -- A dict of extra parameters with their values
+
+        """
+        # Iterate over available parameters
+        for p, klass in self.extra.items():
+            if p not in parameters.keys():
+                continue
+
+            # Initialize the extra parser
+            parser = klass(self.iface)
+
+            # Parse the argument
+            if isinstance(arg_value, list):
+                for v in arg_value:
+                    r = parser(parameters[p], arg_name, v)
+                    if r not in arg_value:
+                        arg_value.append(r)
+            else:
+                arg_value = parser(parameters[p], arg_name, arg_value)
+
+        return arg_value
+
+
+## Main class ----------------------------------------------------------
+
+class ActionsMap(object):
+    """Validate and process actions defined into an actions map
+
+    The actions map defines the features - and their usage - of an
+    application which will be available through the moulinette.
+    It is composed by categories which contain one or more action(s).
+    Moreover, the action can have specific argument(s).
+
+    This class allows to manipulate one or several actions maps
+    associated to a namespace. If no namespace is given, it will load
+    all available namespaces.
+
+    Keyword arguments:
+        - parser -- The BaseActionsMapParser derived class to use for
+            parsing the actions map
+        - namespaces -- The list of namespaces to use
+        - use_cache -- False if it should parse the actions map file
+            instead of using the cached one.
+
+    """
+    def __init__(self, parser, namespaces=[], use_cache=True):
+        self.use_cache = use_cache
+        if not issubclass(parser, BaseActionsMapParser):
+            raise MoulinetteError(errno.EINVAL, _("Invalid parser class '%s'" % parser.__name__))
+        self._parser_class = parser
+
+        logging.debug("initializing ActionsMap for the interface '%s'" % parser.interface)
+
+        if len(namespaces) == 0:
+            namespaces = self.get_namespaces()
+        actionsmaps = OrderedDict()
+
+        # Iterate over actions map namespaces
+        for n in namespaces:
+            logging.debug("loading '%s' actions map namespace" % n)
+
+            if use_cache:
+                try:
+                    # Attempt to load cache
+                    with open('%s/actionsmap/%s.pkl' % (pkg.cachedir, n)) as f:
+                        actionsmaps[n] = pickle.load(f)
+                # TODO: Switch to python3 and catch proper exception
+                except IOError:
+                    self.use_cache = False
+                    actionsmaps = self.generate_cache(namespaces)
+                    break
+            else:
+                with open('%s/actionsmap/%s.yml' % (pkg.datadir, n)) as f:
+                    actionsmaps[n] = yaml.load(f)
+
+        # Generate parsers
+        self.extraparser = ExtraArgumentParser(parser.interface)
+        self._parser = self._construct_parser(actionsmaps)
+
+    @property
+    def parser(self):
+        """Return the instance of the interface's actions map parser"""
+        return self._parser
+
+    def get_authenticator(self, profile='default'):
+        """Get an authenticator instance
+
+        Retrieve the authenticator for the given profile and return a
+        new instance.
+
+        Keyword arguments:
+            - profile -- An authenticator profile name
+
+        Returns:
+            A new _BaseAuthenticator derived instance
+
+        """
+        try:
+            auth = self.parser.get_global_conf('authenticator', profile)[1]
+        except KeyError:
+            raise MoulinetteError(errno.EINVAL, _("Unknown authenticator profile '%s'") % profile)
+        else:
+            return auth()
+
+    def connect(self, signal, handler):
+        """Connect a signal to a handler
+
+        Connect a signal emitted by actions map while processing to a
+        handler. Note that some signals need a return value.
+
+        Keyword arguments:
+            - signal -- The name of the signal
+            - handler -- The method to handle the signal
+
+        """
+        global shandler
+        shandler.set_handler(signal, handler)
+
+    def process(self, args, timeout=0, **kwargs):
+        """
+        Parse arguments and process the proper action
+
+        Keyword arguments:
+            - args -- The arguments to parse
+            - timeout -- The time period before failing if the lock
+                cannot be acquired for the action
+            - **kwargs -- Additional interface arguments
+
+        """
+        # Parse arguments
+        arguments = vars(self.parser.parse_args(args, **kwargs))
+        for an, parameters in (arguments.pop('_extra', {})).items():
+            arguments[an] = self.extraparser.parse(an, arguments[an], parameters)
+
+        # Retrieve action information
+        namespace, category, action = arguments.pop('_tid')
+        func_name = '%s_%s' % (category, action.replace('-', '_'))
+
+        # Lock the moulinette for the namespace
+        with MoulinetteLock(namespace, timeout):
+            try:
+                mod = __import__('%s.%s' % (namespace, category),
+                                 globals=globals(), level=0,
+                                 fromlist=[func_name])
+                func = getattr(mod, func_name)
+            except (AttributeError, ImportError):
+                raise MoulinetteError(errno.ENOSYS, _('Function is not defined'))
+            else:
+                # Process the action
+                return func(**arguments)
+
+    @staticmethod
+    def get_namespaces():
+        """
+        Retrieve available actions map namespaces
+
+        Returns:
+            A list of available namespaces
+
+        """
+        namespaces = []
+
+        for f in os.listdir('%s/actionsmap' % pkg.datadir):
+            if f.endswith('.yml'):
+                namespaces.append(f[:-4])
+        return namespaces
+
+    @classmethod
+    def generate_cache(klass, namespaces=None):
+        """
+        Generate cache for the actions map's file(s)
+
+        Keyword arguments:
+            - namespaces -- A list of namespaces to generate cache for
+
+        Returns:
+            A dict of actions map for each namespaces
+
+        """
+        actionsmaps = {}
+        if not namespaces:
+            namespaces = klass.get_namespaces()
+
+        # Iterate over actions map namespaces
+        for n in namespaces:
+            logging.debug("generating cache for '%s' actions map namespace" % n)
+
+            # Read actions map from yaml file
+            am_file = '%s/actionsmap/%s.yml' % (pkg.datadir, n)
+            with open(am_file, 'r') as f:
+                actionsmaps[n] = yaml.load(f)
+
+            # Cache actions map into pickle file
+            with pkg.open_cachefile('%s.pkl' % n, 'w', subdir='actionsmap') as f:
+                pickle.dump(actionsmaps[n], f)
+
+        return actionsmaps
+
+
+    ## Private methods
+
+    def _construct_parser(self, actionsmaps):
+        """
+        Construct the parser with the actions map
+
+        Keyword arguments:
+            - actionsmaps -- A dict of multi-level dictionnary of
+                categories/actions/arguments list for each namespaces
+
+        Returns:
+            An interface relevant's parser object
+
+        """
+        ## Get extra parameters
+        if not self.use_cache:
+            _get_extra = lambda an, e: self.extraparser.validate(an, e)
+        else:
+            _get_extra = lambda an, e: e
+
+        ## Add arguments to the parser
+        def _add_arguments(parser, arguments):
+            extras = {}
+            for argn, argp in arguments.items():
+                names = top_parser.format_arg_names(argn,
+                                                    argp.pop('full', None))
+                try:
+                    extra = argp.pop('extra')
+                    arg_dest = (parser.add_argument(*names, **argp)).dest
+                    extras[arg_dest] = _get_extra(arg_dest, extra)
+                except KeyError:
+                    # No extra parameters
+                    parser.add_argument(*names, **argp)
+            if extras:
+                parser.set_defaults(_extra=extras)
+
+        # Instantiate parser
+        top_parser = self._parser_class(shandler)
+
+        # Iterate over actions map namespaces
+        for n, actionsmap in actionsmaps.items():
+            # Retrieve global parameters
+            _global = actionsmap.pop('_global', {})
+
+            # -- Parse global configuration
+            if 'configuration' in _global:
+                # Set global configuration
+                top_parser.set_global_conf(_global['configuration'])
+
+            # -- Parse global arguments
+            if 'arguments' in _global:
+                try:
+                    # Get global arguments parser
+                    parser = top_parser.add_global_parser()
+                except AttributeError:
+                    # No parser for global arguments
+                    pass
+                else:
+                    # Add arguments
+                    _add_arguments(parser, _global['arguments'])
+
+            # -- Parse categories
+            for cn, cp in actionsmap.items():
+                try:
+                    actions = cp.pop('actions')
+                except KeyError:
+                    # Invalid category without actions
+                    logging.warning("no actions found in category '%s'" % cn)
+                    continue
+
+                # Get category parser
+                cat_parser = top_parser.add_category_parser(cn, **cp)
+
+                # -- Parse actions
+                for an, ap in actions.items():
+                    args = ap.pop('arguments', {})
+                    tid = (n, cn, an)
+                    try:
+                        conf = ap.pop('configuration')
+                        _set_conf = lambda p: p.set_conf(tid, conf)
+                    except KeyError:
+                        # No action configuration
+                        _set_conf = lambda p: False
+
+                    try:
+                        # Get action parser
+                        parser = cat_parser.add_action_parser(an, tid, **ap)
+                    except AttributeError:
+                        # No parser for the action
+                        continue
+                    except ValueError as e:
+                        logging.warning("cannot add action (%s, %s, %s): %s" % (n, cn, an, e))
+                        continue
+                    else:
+                        # Store action identifier and add arguments
+                        parser.set_defaults(_tid=tid)
+                        _add_arguments(parser, args)
+                        _set_conf(cat_parser)
+
+        return top_parser

moulinette/authenticators/__init__.py

@@ -0,0 +1,148 @@
+# -*- coding: utf-8 -*-
+
+import errno
+import gnupg
+import logging
+
+from moulinette.core import MoulinetteError
+
+# Base Class -----------------------------------------------------------
+
+class BaseAuthenticator(object):
+    """Authenticator base representation
+
+    Each authenticators must implement an Authenticator class derived
+    from this class which must overrides virtual properties and methods.
+    It is used to authenticate and manage session. It implements base
+    methods to authenticate with a password or a session token.
+
+    Authenticators configurations are identified by a profile name which
+    must be given on instantiation - with the corresponding vendor
+    configuration of the authenticator.
+
+    Keyword arguments:
+        - name -- The authenticator profile name
+
+    """
+    def __init__(self, name):
+        self._name = name
+
+    @property
+    def name(self):
+        """Return the name of the authenticator instance"""
+        return self._name
+
+
+    ## Virtual properties
+    # Each authenticator classes must implement these properties.
+
+    """The vendor name of the authenticator"""
+    vendor = None
+
+    @property
+    def is_authenticated(self):
+        """Either the instance is authenticated or not"""
+        raise NotImplementedError("derived class '%s' must override this property" % \
+                                      self.__class__.__name__)
+
+
+    ## Virtual methods
+    # Each authenticator classes must implement these methods.
+
+    def authenticate(password=None):
+        """Attempt to authenticate
+
+        Attempt to authenticate with given password. It should raise an
+        AuthenticationError exception if authentication fails.
+
+        Keyword arguments:
+            - password -- A clear text password
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+
+    ## Authentication methods
+
+    def __call__(self, password=None, token=None):
+        """Attempt to authenticate
+
+        Attempt to authenticate either with password or with session
+        token if 'password' is None. If the authentication succeed, the
+        instance is returned and the session is registered for the token
+        if 'token' and 'password' are given.
+        The token is composed by the session identifier and a session
+        hash - to use for encryption - as a 2-tuple.
+
+        Keyword arguments:
+            - password -- A clear text password
+            - token -- The session token in the form of (id, hash)
+
+        Returns:
+            The authenticated instance
+
+        """
+        if self.is_authenticated:
+            return self
+        store_session = True if password and token else False
+
+        if token:
+            try:
+                # Extract id and hash from token
+                s_id, s_hash = token
+            except TypeError:
+                if not password:
+                    raise MoulinetteError(errno.EINVAL, _("Invalid format for token"))
+                else:
+                    # TODO: Log error
+                    store_session = False
+            else:
+                if password is None:
+                    # Retrieve session
+                    password = self._retrieve_session(s_id, s_hash)
+
+        try:
+            # Attempt to authenticate
+            self.authenticate(password)
+        except MoulinetteError:
+            raise
+        except Exception as e:
+            logging.error("authentication (name: '%s', type: '%s') fails: %s" % \
+                              (self.name, self.vendor, e))
+            raise MoulinetteError(errno.EACCES, _("Unable to authenticate"))
+
+        # Store session
+        if store_session:
+            self._store_session(s_id, s_hash, password)
+
+        return self
+
+
+    ## Private methods
+
+    def _open_sessionfile(self, session_id, mode='r'):
+        """Open a session file for this instance in given mode"""
+        return pkg.open_cachefile('%s.asc' % session_id, mode,
+                                  subdir='session/%s' % self.name)
+
+    def _store_session(self, session_id, session_hash, password):
+        """Store a session and its associated password"""
+        gpg = gnupg.GPG()
+        gpg.encoding = 'utf-8'
+        with self._open_sessionfile(session_id, 'w') as f:
+            f.write(str(gpg.encrypt(password, None, symmetric=True,
+                                    passphrase=session_hash)))
+
+    def _retrieve_session(self, session_id, session_hash):
+        """Retrieve a session and return its associated password"""
+        try:
+            with self._open_sessionfile(session_id, 'r') as f:
+                enc_pwd = f.read()
+        except IOError:
+            # TODO: Set proper error code
+            raise MoulinetteError(167, _("Unable to retrieve session"))
+        else:
+            gpg = gnupg.GPG()
+            gpg.encoding = 'utf-8'
+            return str(gpg.decrypt(enc_pwd, passphrase=session_hash))

moulinette/authenticators/ldap.py

@@ -0,0 +1,192 @@
+# -*- coding: utf-8 -*-
+
+# TODO: Use Python3 to remove this fix!
+from __future__ import absolute_import
+import ldap
+import ldap.modlist as modlist
+
+from moulinette.core import MoulinetteError
+from moulinette.authenticators import BaseAuthenticator
+
+# LDAP Class Implementation --------------------------------------------
+
+class Authenticator(BaseAuthenticator):
+    """LDAP Authenticator
+
+    Initialize a LDAP connexion for the given arguments. It attempts to
+    authenticate a user if 'user_rdn' is given - by associating user_rdn
+    and base_dn - and provides extra methods to manage opened connexion.
+
+    Keyword arguments:
+        - uri -- The LDAP server URI
+        - base_dn -- The base dn
+        - user_rdn -- The user rdn to authenticate
+
+    """
+    def __init__(self, name, uri, base_dn, user_rdn=None):
+        super(Authenticator, self).__init__(name)
+
+        self.uri = uri
+        self.basedn = base_dn
+        if user_rdn:
+            self.userdn = '%s,%s' % (user_rdn, base_dn)
+            self.con = None
+        else:
+            # Initialize anonymous usage
+            self.userdn = ''
+            self.authenticate(None)
+
+
+    ## Implement virtual properties
+
+    vendor = 'ldap'
+
+    @property
+    def is_authenticated(self):
+        try:
+            # Retrieve identity
+            who = self.con.whoami_s()
+        except:
+            return False
+        else:
+            if who[3:] == self.userdn:
+                return True
+        return False
+
+
+    ## Implement virtual methods
+
+    def authenticate(self, password):
+        try:
+            con = ldap.initialize(self.uri)
+            if self.userdn:
+                con.simple_bind_s(self.userdn, password)
+            else:
+                con.simple_bind_s()
+        except ldap.INVALID_CREDENTIALS:
+            raise MoulinetteError(errno.EACCES, _("Invalid password"))
+        else:
+            self.con = con
+
+
+    ## Additional LDAP methods
+    # TODO: Review these methods
+
+    def search(self, base=None, filter='(objectClass=*)', attrs=['dn']):
+        """
+        Search in LDAP base
+
+        Keyword arguments:
+            base    -- Base to search into
+            filter  -- LDAP filter
+            attrs   -- Array of attributes to fetch
+
+        Returns:
+            Boolean | Dict
+
+        """
+        if not base:
+            base = self.basedn
+
+        try:
+            result = self.con.search_s(base, ldap.SCOPE_SUBTREE, filter, attrs)
+        except:
+            raise MoulinetteError(169, _('An error occured during LDAP search'))
+
+        if result:
+            result_list = []
+            for dn, entry in result:
+                if attrs != None:
+                    if 'dn' in attrs:
+                        entry['dn'] = [dn]
+                result_list.append(entry)
+            return result_list
+        else:
+            return False
+
+    def add(self, rdn, attr_dict):
+        """
+        Add LDAP entry
+
+        Keyword arguments:
+            rdn         -- DN without domain
+            attr_dict   -- Dictionnary of attributes/values to add
+
+        Returns:
+            Boolean | MoulinetteError
+
+        """
+        dn = rdn + ',' + self.basedn
+        ldif = modlist.addModlist(attr_dict)
+
+        try:
+            self.con.add_s(dn, ldif)
+        except:
+            raise MoulinetteError(169, _('An error occured during LDAP entry creation'))
+        else:
+            return True
+
+    def remove(self, rdn):
+        """
+        Remove LDAP entry
+
+        Keyword arguments:
+            rdn         -- DN without domain
+
+        Returns:
+            Boolean | MoulinetteError
+
+        """
+        dn = rdn + ',' + self.basedn
+        try:
+            self.con.delete_s(dn)
+        except:
+            raise MoulinetteError(169, _('An error occured during LDAP entry deletion'))
+        else:
+            return True
+
+    def update(self, rdn, attr_dict, new_rdn=False):
+        """
+        Modify LDAP entry
+
+        Keyword arguments:
+            rdn         -- DN without domain
+            attr_dict   -- Dictionnary of attributes/values to add
+            new_rdn     -- New RDN for modification
+
+        Returns:
+            Boolean | MoulinetteError
+
+        """
+        dn = rdn + ',' + self.basedn
+        actual_entry = self.search(base=dn, attrs=None)
+        ldif = modlist.modifyModlist(actual_entry[0], attr_dict, ignore_oldexistent=1)
+
+        try:
+            if new_rdn:
+                self.con.rename_s(dn, new_rdn)
+                dn = new_rdn + ',' + self.basedn
+
+            self.con.modify_ext_s(dn, ldif)
+        except:
+            raise MoulinetteError(169, _('An error occured during LDAP entry update'))
+        else:
+            return True
+
+    def validate_uniqueness(self, value_dict):
+        """
+        Check uniqueness of values
+
+        Keyword arguments:
+            value_dict -- Dictionnary of attributes/values to check
+
+        Returns:
+            Boolean | MoulinetteError
+
+        """
+        for attr, value in value_dict.items():
+            if not self.search(filter=attr + '=' + value):
+                continue
+            else:
+                raise MoulinetteError(17, _('Attribute already exists') + ' "' + attr + '=' + value + '"')
+        return True

moulinette/core.py

@@ -0,0 +1,290 @@
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+import time
+import errno
+import gettext
+import logging
+
+from importlib import import_module
+
+# Package manipulation -------------------------------------------------
+
+def install_i18n(namespace=None):
+    """Install internationalization
+
+    Install translation based on the package's default gettext domain or
+    on 'namespace' if provided.
+
+    Keyword arguments:
+        - namespace -- The namespace to initialize i18n for
+
+    """
+    if namespace:
+        try:
+            t = gettext.translation(namespace, pkg.localedir)
+        except IOError:
+            # TODO: Log error
+            return
+        else:
+            t.install()
+    else:
+        gettext.install('moulinette', pkg.localedir)
+
+class Package(object):
+    """Package representation and easy access methods
+
+    Initialize directories and variables for the package and give them
+    easy access.
+
+    Keyword arguments:
+        - _from_source -- Either the package is running from source or
+            not (only for debugging)
+
+    """
+    def __init__(self, _from_source=False):
+        if _from_source:
+            import sys
+            basedir = os.path.abspath(os.path.dirname(sys.argv[0]) +'/../')
+
+            # Set local directories
+            self._datadir = '%s/data' % basedir
+            self._libdir = '%s/lib' % basedir
+            self._localedir = '%s/po' % basedir
+            self._cachedir = '%s/cache' % basedir
+        else:
+            import package
+
+            # Set system directories
+            self._datadir = package.datadir
+            self._libdir = package.libdir
+            self._localedir = package.localedir
+            self._cachedir = package.cachedir
+
+    def __setattr__(self, name, value):
+        if name[0] == '_' and self.__dict__.has_key(name):
+            # Deny reassignation of package directories
+            raise TypeError("cannot reassign constant '%s'")
+        self.__dict__[name] = value
+
+
+    ## Easy access to package directories
+
+    @property
+    def datadir(self):
+        """Return the data directory of the package"""
+        return self._datadir
+
+    @property
+    def libdir(self):
+        """Return the lib directory of the package"""
+        return self._libdir
+
+    @property
+    def localedir(self):
+        """Return the locale directory of the package"""
+        return self._localedir
+
+    @property
+    def cachedir(self):
+        """Return the cache directory of the package"""
+        return self._cachedir
+
+
+    ## Additional methods
+
+    def get_cachedir(self, subdir='', make_dir=True):
+        """Get the path to a cache directory
+
+        Return the path to the cache directory from an optional
+        subdirectory and create it if needed.
+
+        Keyword arguments:
+            - subdir -- A cache subdirectory
+            - make_dir -- False to not make directory if it not exists
+
+        """
+        path = os.path.join(self.cachedir, subdir)
+
+        if make_dir and not os.path.isdir(path):
+            os.makedirs(path)
+        return path
+
+    def open_cachefile(self, filename, mode='r', **kwargs):
+        """Open a cache file and return a stream
+
+        Attempt to open in 'mode' the cache file 'filename' from the
+        default cache directory and in the subdirectory 'subdir' if
+        given. Directories are created if needed and a stream is
+        returned if the file can be written.
+
+        Keyword arguments:
+            - filename -- The cache filename
+            - mode -- The mode in which the file is opened
+            - **kwargs -- Optional arguments for get_cachedir
+
+        """
+        # Set make_dir if not given
+        kwargs['make_dir'] = kwargs.get('make_dir',
+                                        True if mode[0] == 'w' else False)
+        return open('%s/%s' % (self.get_cachedir(**kwargs), filename), mode)
+
+
+# Interfaces & Authenticators management -------------------------------
+
+def init_interface(name, kwargs={}, actionsmap={}):
+    """Return a new interface instance
+
+    Retrieve the given interface module and return a new instance of its
+    Interface class. It is initialized with arguments 'kwargs' and
+    connected to 'actionsmap' if it's an ActionsMap object, otherwise
+    a new ActionsMap instance will be initialized with arguments
+    'actionsmap'.
+
+    Keyword arguments:
+        - name -- The interface name
+        - kwargs -- A dict of arguments to pass to Interface
+        - actionsmap -- Either an ActionsMap instance or a dict of
+            arguments to pass to ActionsMap
+
+    """
+    from moulinette.actionsmap import ActionsMap
+
+    try:
+        mod = import_module('moulinette.interfaces.%s' % name)
+    except ImportError:
+        # TODO: List available interfaces
+        raise MoulinetteError(errno.EINVAL, _("Unknown interface '%s'" % name))
+    else:
+        try:
+            # Retrieve interface classes
+            parser = mod.ActionsMapParser
+            interface = mod.Interface
+        except AttributeError as e:
+            raise MoulinetteError(errno.EFAULT, _("Invalid interface '%s': %s") % (name, e))
+
+    # Instantiate or retrieve ActionsMap
+    if isinstance(actionsmap, dict):
+        amap = ActionsMap(actionsmap.pop('parser', parser), **actionsmap)
+    elif isinstance(actionsmap, ActionsMap):
+        amap = actionsmap
+    else:
+        raise MoulinetteError(errno.EINVAL, _("Invalid actions map '%r'" % actionsmap))
+
+    return interface(amap, **kwargs)
+
+def init_authenticator((vendor, name), kwargs={}):
+    """Return a new authenticator instance
+
+    Retrieve the given authenticator vendor and return a new instance of
+    its Authenticator class for the given profile.
+
+    Keyword arguments:
+        - vendor -- The authenticator vendor name
+        - name -- The authenticator profile name
+        - kwargs -- A dict of arguments for the authenticator profile
+
+    """
+    try:
+        mod = import_module('moulinette.authenticators.%s' % vendor)
+    except ImportError:
+        # TODO: List available authenticators vendors
+        raise MoulinetteError(errno.EINVAL, _("Unknown authenticator vendor '%s'" % vendor))
+    else:
+        return mod.Authenticator(name, **kwargs)
+
+def clean_session(session_id, profiles=[]):
+    """Clean a session cache
+
+    Remove cache for the session 'session_id' and for profiles in
+    'profiles' or for all of them if the list is empty.
+
+    Keyword arguments:
+        - session_id -- The session id to clean
+        - profiles -- A list of profiles to clean
+
+    """
+    sessiondir = pkg.get_cachedir('session')
+    if not profiles:
+        profiles = os.listdir(sessiondir)
+
+    for p in profiles:
+        try:
+            os.unlink(os.path.join(sessiondir, p, '%s.asc' % session_id))
+        except OSError:
+            pass
+
+
+# Moulinette core classes ----------------------------------------------
+
+class MoulinetteError(OSError):
+    """Moulinette base exception"""
+    pass
+
+
+class MoulinetteLock(object):
+    """Locker for a moulinette instance
+
+    It provides a lock mechanism for a given moulinette instance. It can
+    be used in a with statement as it has a context-manager support.
+
+    Keyword arguments:
+        - namespace -- The namespace to lock
+        - timeout -- The time period before failing if the lock cannot
+            be acquired
+        - interval -- The time period before trying again to acquire the
+            lock
+
+    """
+    def __init__(self, namespace, timeout=0, interval=.5):
+        self.namespace = namespace
+        self.timeout = timeout
+        self.interval = interval
+
+        self._lockfile = '/var/run/moulinette_%s.lock' % namespace
+        self._locked = False
+
+    def acquire(self):
+        """Attempt to acquire the lock for the moulinette instance
+
+        It will try to write to the lock file only if it doesn't exist.
+        Otherwise, it will wait and try again until the timeout expires
+        or the lock file doesn't exist.
+
+        """
+        start_time = time.time()
+
+        while True:
+            if not os.path.isfile(self._lockfile):
+                # Create the lock file
+                (open(self._lockfile, 'w')).close()
+                break
+
+            if (time.time() - start_time) > self.timeout:
+                raise MoulinetteError(errno.EBUSY, _("An instance is already running for '%s'") \
+                                          % self.namespace)
+            # Wait before checking again
+            time.sleep(self.interval)
+        self._locked = True
+
+    def release(self):
+        """Release the lock of the moulinette instance
+
+        It will delete the lock file if the lock has been acquired.
+
+        """
+        if self._locked:
+            os.unlink(self._lockfile)
+            self._locked = False
+
+    def __enter__(self):
+        if not self._locked:
+            self.acquire()
+        return self
+
+    def __exit__(self, *args):
+        self.release()
+
+    def __del__(self):
+        self.release()

moulinette/helpers.py

@@ -1,67 +1,15 @@
 # -*- coding: utf-8 -*-
 
-""" License
-
-    Copyright (C) 2013 YunoHost
-
-    This program is free software; you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published
-    by the Free Software Foundation, either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
-
-    You should have received a copy of the GNU Affero General Public License
-    along with this program; if not, see http://www.gnu.org/licenses
-
-"""
-
-"""
-    YunoHost core classes & functions
-"""
-
-__credits__ = """
-    Copyright (C) 2012 YunoHost
-
-    This program is free software; you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published
-    by the Free Software Foundation, either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
-
-    You should have received a copy of the GNU Affero General Public License
-    along with this program; if not, see http://www.gnu.org/licenses
-    """
-__author__  = 'Kload <kload@kload.fr>'
-__version__ = '695'
-
 import os
-import sys
+import ldap
-try:
-    import ldap
-except ImportError:
-    sys.stderr.write('Error: Yunohost CLI Require LDAP lib\n')
-    sys.stderr.write('apt-get install python-ldap\n')
-    sys.exit(1)
 import ldap.modlist as modlist
-import yaml
 import json
 import re
 import getpass
 import random
 import string
-import argparse
 import gettext
 import getpass
-if not __debug__:
-    import traceback
 
 win = []
 
@@ -132,33 +80,6 @@ def win_msg(astr):
     win.append(astr)
 
 
-def str_to_func(astr):
-    """
-    Call a function from a string name
-
-    Keyword arguments:
-        astr -- Name of function to call
-
-    Returns:
-        Function
-
-    """
-    try:
-        module, _, function = astr.rpartition('.')
-        if module:
-            __import__(module)
-            mod = sys.modules[module]
-        else:
-            mod = sys.modules['__main__']  # default module
-
-        func = getattr(mod, function)
-    except (AttributeError, ImportError):
-        #raise YunoHostError(168, _('Function is not defined'))
-        return None
-    else:
-        return func
-
-
 def validate(pattern, array):
     """
     Validate attributes with a pattern
@@ -468,115 +389,3 @@ class YunoHostLDAP(Singleton):
             else:
                 raise YunoHostError(17, _('Attribute already exists') + ' "' + attr + '=' + value + '"')
         return True
-
-
-def parse_dict(action_map):
-    """
-    Turn action dictionnary to parser, subparsers and arguments
-
-    Keyword arguments:
-        action_map -- Multi-level dictionnary of categories/actions/arguments list
-
-    Returns:
-        Namespace of args
-
-    """
-    # Intialize parsers
-    parsers = subparsers_category = subparsers_action = {}
-    parsers['general'] = argparse.ArgumentParser()
-    subparsers = parsers['general'].add_subparsers()
-    new_args = []
-    patterns = {}
-
-    # Add general arguments
-    for arg_name, arg_params in action_map['general_arguments'].items():
-        if 'version' in arg_params:
-            v = arg_params['version']
-            arg_params['version'] = v.replace('%version%', __version__)
-        if 'full' in arg_params:
-            arg_names = [arg_name, arg_params['full']]
-            arg_fullname = arg_params['full']
-            del arg_params['full']
-        else: arg_names = [arg_name]
-        parsers['general'].add_argument(*arg_names, **arg_params)
-
-    del action_map['general_arguments']
-
-    # Split categories into subparsers
-    for category, category_params in action_map.items():
-        if 'category_help' not in category_params: category_params['category_help'] = ''
-        subparsers_category[category] = subparsers.add_parser(category, help=category_params['category_help'])
-        subparsers_action[category] = subparsers_category[category].add_subparsers()
-        # Split actions
-        if 'actions' in category_params:
-            for action, action_params in category_params['actions'].items():
-                if 'action_help' not in action_params: action_params['action_help'] = ''
-                parsers[category + '_' + action] = subparsers_action[category].add_parser(action, help=action_params['action_help'])
-                # Set the action s related function
-                parsers[category + '_' + action].set_defaults(
-                    func=str_to_func('yunohost_' + category + '.'
-                                     + category + '_' + action.replace('-', '_')))
-                # Add arguments
-                if 'arguments' in action_params:
-                    for arg_name, arg_params in action_params['arguments'].items():
-                        arg_fullname = False
-
-                        if 'password' in arg_params:
-                            if arg_params['password']: is_password = True
-                            del arg_params['password']
-                        else: is_password = False
-
-                        if 'full' in arg_params:
-                            arg_names = [arg_name, arg_params['full']]
-                            arg_fullname = arg_params['full']
-                            del arg_params['full']
-                        else: arg_names = [arg_name]
-
-                        if 'ask' in arg_params:
-                            require_input = True
-                            if '-h' in sys.argv or '--help' in sys.argv:
-                                require_input = False
-                            if (category != sys.argv[1]) or (action != sys.argv[2]):
-                                require_input = False
-                            for name in arg_names:
-                                if name in sys.argv[2:]: require_input = False
-
-                            if require_input:
-                                if is_password:
-                                    if os.isatty(1):
-                                        pwd1 = getpass.getpass(colorize(arg_params['ask'] + ': ', 'cyan'))
-                                        pwd2 = getpass.getpass(colorize('Retype ' + arg_params['ask'][0].lower() + arg_params['ask'][1:] + ': ', 'cyan'))
-                                        if pwd1 != pwd2:
-                                            raise YunoHostError(22, _("Passwords don't match"))
-                                            sys.exit(1)
-                                    else:
-                                        raise YunoHostError(22, _("Missing arguments") + ': ' + arg_name)
-                                    if arg_name[0] == '-': arg_extend = [arg_name, pwd1]
-                                    else: arg_extend = [pwd1]
-                                else:
-                                    if os.isatty(1):
-                                        arg_value = raw_input(colorize(arg_params['ask'] + ': ', 'cyan'))
-                                    else:
-                                        raise YunoHostError(22, _("Missing arguments") + ': ' + arg_name)
-                                    if arg_name[0] == '-': arg_extend = [arg_name, arg_value]
-                                    else: arg_extend = [arg_value]
-                                new_args.extend(arg_extend)
-                            del arg_params['ask']
-
-                        if 'pattern' in arg_params:
-                            if (category == sys.argv[1]) and (action == sys.argv[2]):
-                                if 'dest' in arg_params: name = arg_params['dest']
-                                elif arg_fullname: name = arg_fullname[2:]
-                                else: name = arg_name
-                                name = name.replace('-', '_')
-                                patterns[name] = arg_params['pattern']
-                            del arg_params['pattern']
-
-                        parsers[category + '_' + action].add_argument(*arg_names, **arg_params)
-
-    args = parsers['general'].parse_args(sys.argv.extend(new_args))
-    args_dict = vars(args)
-    for key, value in patterns.items():
-        validate(value, args_dict[key])
-
-    return args

moulinette/interfaces/__init__.py

@@ -0,0 +1,307 @@
+# -*- coding: utf-8 -*-
+
+import errno
+import logging
+
+from moulinette.core import (init_authenticator, MoulinetteError)
+
+# Base Class -----------------------------------------------------------
+
+class BaseActionsMapParser(object):
+    """Actions map's base Parser
+
+    Each interfaces must implement an ActionsMapParser class derived
+    from this class which must overrides virtual properties and methods.
+    It is used to parse the main parts of the actions map (i.e. global
+    arguments, categories and actions). It implements methods to set/get
+    the global and actions configuration.
+
+    Keyword arguments:
+        - shandler -- A actionsmap.ActionsMapSignals instance
+        - parent -- A parent BaseActionsMapParser derived object
+
+    """
+    def __init__(self, shandler, parent=None):
+        if parent:
+            self.shandler = parent.shandler
+            self._o = parent
+        else:
+            self.shandler = shandler
+            self._o = self
+            self._global_conf = {}
+            self._conf = {}
+
+
+    ## Virtual properties
+    # Each parser classes must implement these properties.
+
+    """The name of the interface for which it is the parser"""
+    interface = None
+
+
+    ## Virtual methods
+    # Each parser classes must implement these methods.
+
+    @staticmethod
+    def format_arg_names(name, full):
+        """Format argument name
+
+        Format agument name depending on its 'full' parameter and return
+        a list of strings which will be used as name or option strings
+        for the argument parser.
+
+        Keyword arguments:
+            - name -- The argument name
+            - full -- The argument's 'full' parameter
+
+        Returns:
+            A list of option strings
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+    def add_global_parser(self, **kwargs):
+        """Add a parser for global arguments
+
+        Create and return an argument parser for global arguments.
+
+        Returns:
+            An ArgumentParser based object
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+    def add_category_parser(self, name, **kwargs):
+        """Add a parser for a category
+
+        Create a new category and return a parser for it.
+
+        Keyword arguments:
+            - name -- The category name
+
+        Returns:
+            A BaseParser based object
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+    def add_action_parser(self, name, tid, **kwargs):
+        """Add a parser for an action
+
+        Create a new action and return an argument parser for it.
+
+        Keyword arguments:
+            - name -- The action name
+            - tid -- The tuple identifier of the action
+
+        Returns:
+            An ArgumentParser based object
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+    def parse_args(self, args, **kwargs):
+        """Parse arguments
+
+        Convert argument variables to objects and assign them as
+        attributes of the namespace.
+
+        Keyword arguments:
+            - args -- Arguments string or dict (TODO)
+
+        Returns:
+            The populated namespace
+
+        """
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)
+
+
+    ## Configuration access
+
+    @property
+    def global_conf(self):
+        """Return the global configuration of the parser"""
+        return self._o._global_conf
+
+    def get_global_conf(self, name, profile='default'):
+        """Get the global value of a configuration
+
+        Return the formated global value of the configuration 'name' for
+        the given profile. If the configuration doesn't provide profile,
+        the formated default value is returned.
+
+        Keyword arguments:
+            - name -- The configuration name
+            - profile -- The profile of the configuration
+
+        """
+        if name == 'authenticator':
+            value = self.global_conf[name][profile]
+        else:
+            value = self.global_conf[name]
+        return self._format_conf(name, value)
+
+    def set_global_conf(self, configuration):
+        """Set global configuration
+
+        Set the global configuration to use for the parser.
+
+        Keyword arguments:
+            - configuration -- The global configuration
+
+        """
+        self._o._global_conf.update(self._validate_conf(configuration, True))
+
+    def get_conf(self, action, name):
+        """Get the value of an action configuration
+
+        Return the formated value of configuration 'name' for the action
+        identified by 'action'. If the configuration for the action is
+        not set, the default one is returned.
+
+        Keyword arguments:
+            - action -- An action identifier
+            - name -- The configuration name
+
+        """
+        try:
+            value = self._o._conf[action][name]
+        except KeyError:
+            return self.get_global_conf(name)
+        else:
+            return self._format_conf(name, value)
+
+    def set_conf(self, action, configuration):
+        """Set configuration for an action
+
+        Set the configuration to use for a given action identified by
+        'action' which is specific to the parser.
+
+        Keyword arguments:
+            - action -- The action identifier
+            - configuration -- The configuration for the action
+
+        """
+        self._o._conf[action] = self._validate_conf(configuration)
+
+
+    def _validate_conf(self, configuration, is_global=False):
+        """Validate configuration for the parser
+
+        Return the validated configuration for the interface's actions
+        map parser.
+
+        Keyword arguments:
+            - configuration -- The configuration to pre-format
+
+        """
+        # TODO: Create a class with a validator method for each configuration
+        conf = {}
+
+        # -- 'authenficate'
+        try:
+            ifaces = configuration['authenticate']
+        except KeyError:
+            pass
+        else:
+            if ifaces == 'all':
+                conf['authenticate'] = ifaces
+            elif ifaces == False:
+                conf['authenticate'] = False
+            elif isinstance(ifaces, list):
+                # Store only if authentication is needed
+                conf['authenticate'] = True if self.interface in ifaces else False
+            else:
+                # TODO: Log error instead and tell valid values
+                raise MoulinetteError(errno.EINVAL, "Invalid value '%r' for configuration 'authenticate'" % ifaces)
+
+        # -- 'authenticator'
+        try:
+            auth = configuration['authenticator']
+        except KeyError:
+            pass
+        else:
+            if not is_global and isinstance(auth, str):
+                try:
+                    # Store needed authenticator profile
+                    conf['authenticator'] = self.global_conf['authenticator'][auth]
+                except KeyError:
+                    raise MoulinetteError(errno.EINVAL, "Undefined authenticator '%s' in global configuration" % auth)
+            elif is_global and isinstance(auth, dict):
+                if len(auth) == 0:
+                    logging.warning('no authenticator defined in global configuration')
+                else:
+                    auths = {}
+                    for auth_name, auth_conf in auth.items():
+                        # Add authenticator profile as a 3-tuple
+                        # (identifier, configuration, parameters) with
+                        # - identifier: the authenticator vendor and its
+                        #     profile name as a 2-tuple
+                        # - configuration: a dict of additional global
+                        #     configuration (i.e. 'help')
+                        # - parameters: a dict of arguments for the
+                        #     authenticator profile
+                        auths[auth_name] = ((auth_conf.get('vendor'), auth_name),
+                                            { 'help': auth_conf.get('help', None) },
+                                            auth_conf.get('parameters', {}))
+                    conf['authenticator'] = auths
+            else:
+                # TODO: Log error instead and tell valid values
+                raise MoulinetteError(errno.EINVAL, "Invalid value '%r' for configuration 'authenticator'" % auth)
+
+        # -- 'argument_auth'
+        try:
+            arg_auth = configuration['argument_auth']
+        except KeyError:
+            pass
+        else:
+            if isinstance(arg_auth, bool):
+                conf['argument_auth'] = arg_auth
+            else:
+                # TODO: Log error instead and tell valid values
+                raise MoulinetteError(errno.EINVAL, "Invalid value '%r' for configuration 'argument_auth'" % arg_auth)
+
+        return conf
+
+    def _format_conf(self, name, value):
+        """Format a configuration value
+
+        Return the formated value of the configuration 'name' from its
+        given value.
+
+        Keyword arguments:
+            - name -- The name of the configuration
+            - value -- The value to format
+
+        """
+        if name == 'authenticator' and value:
+            (identifier, configuration, parameters) = value
+
+            # Return global configuration and an authenticator
+            # instanciator as a 2-tuple
+            return (configuration,
+                    lambda: init_authenticator(identifier, parameters))
+
+        return value
+
+
+class BaseInterface(object):
+    """Moulinette's base Interface
+
+    Each interfaces must implement an Interface class derived from this
+    class which must overrides virtual properties and methods.
+    It is used to provide a user interface for an actions map.
+
+    Keyword arguments:
+        - actionsmap -- The ActionsMap instance to connect to
+
+    """
+    # TODO: Add common interface methods and try to standardize default ones
+    def __init__(self, actionsmap):
+        raise NotImplementedError("derived class '%s' must override this method" % \
+                                      self.__class__.__name__)

moulinette/interfaces/api.py

@@ -0,0 +1,487 @@
+# -*- coding: utf-8 -*-
+
+import os
+import re
+import errno
+import binascii
+import argparse
+from json import dumps as json_encode
+from bottle import run, request, response, Bottle, HTTPResponse
+
+from moulinette.core import MoulinetteError, clean_session
+from moulinette.interfaces import (BaseActionsMapParser, BaseInterface)
+
+# API helpers ----------------------------------------------------------
+
+def random_ascii(length=20):
+    """Return a random ascii string"""
+    return binascii.hexlify(os.urandom(length)).decode('ascii')
+
+class _HTTPArgumentParser(object):
+    """Argument parser for HTTP requests
+
+    Object for parsing HTTP requests into Python objects. It is based
+    on argparse.ArgumentParser class and implements some of its methods.
+
+    """
+    def __init__(self):
+        # Initialize the ArgumentParser object
+        self._parser = argparse.ArgumentParser(usage='',
+                                               prefix_chars='@',
+                                               add_help=False)
+        self._parser.error = self._error
+
+        self._positional = []   # list(arg_name)
+        self._optional = {}     # dict({arg_name: option_strings})
+
+    def set_defaults(self, **kwargs):
+        return self._parser.set_defaults(**kwargs)
+
+    def get_default(self, dest):
+        return self._parser.get_default(dest)
+
+    def add_argument(self, *args, **kwargs):
+        action = self._parser.add_argument(*args, **kwargs)
+
+        # Append newly created action
+        if len(action.option_strings) == 0:
+            self._positional.append(action.dest)
+        else:
+            self._optional[action.dest] = action.option_strings
+
+        return action
+
+    def parse_args(self, args={}, namespace=None):
+        arg_strings = []
+
+        ## Append an argument to the current one
+        def append(arg_strings, value, option_string=None):
+            # TODO: Process list arguments
+            if isinstance(value, bool):
+                # Append the option string only
+                if option_string is not None:
+                    arg_strings.append(option_string)
+            elif isinstance(value, str):
+                if option_string is not None:
+                    arg_strings.append(option_string)
+                    arg_strings.append(value)
+                else:
+                    arg_strings.append(value)
+
+            return arg_strings
+
+        # Iterate over positional arguments
+        for dest in self._positional:
+            if dest in args:
+                arg_strings = append(arg_strings, args[dest])
+
+        # Iterate over optional arguments
+        for dest, opt in self._optional.items():
+            if dest in args:
+                arg_strings = append(arg_strings, args[dest], opt[0])
+        return self._parser.parse_args(arg_strings, namespace)
+
+    def _error(self, message):
+        # TODO: Raise a proper exception
+        raise MoulinetteError(1, message)
+
+class _ActionsMapPlugin(object):
+    """Actions map Bottle Plugin
+
+    Process relevant action for the request using the actions map and
+    manage authentication.
+
+    Keyword arguments:
+        - actionsmap -- An ActionsMap instance
+
+    """
+    name = 'actionsmap'
+    api = 2
+
+    def __init__(self, actionsmap):
+        # Connect signals to handlers
+        actionsmap.connect('authenticate', self._do_authenticate)
+
+        self.actionsmap = actionsmap
+        # TODO: Save and load secrets?
+        self.secrets = {}
+
+    def setup(self, app):
+        """Setup plugin on the application
+
+        Add routes according to the actions map to the application.
+
+        Keyword arguments:
+            - app -- The application instance
+
+        """
+        ## Login wrapper
+        def _login(callback):
+            def wrapper():
+                kwargs = {}
+                try:
+                    kwargs['password'] = request.POST['password']
+                except KeyError:
+                    raise HTTPBadRequestResponse(_("Missing password parameter"))
+                try:
+                    kwargs['profile'] = request.POST['profile']
+                except KeyError:
+                    pass
+                return callback(**kwargs)
+            return wrapper
+
+        ## Logout wrapper
+        def _logout(callback):
+            def wrapper():
+                kwargs = {}
+                try:
+                    kwargs['profile'] = request.POST.get('profile')
+                except KeyError:
+                    pass
+                return callback(**kwargs)
+            return wrapper
+
+        # Append authentication routes
+        app.route('/login', name='login', method='POST',
+                  callback=self.login, skip=['actionsmap'], apply=_login)
+        app.route('/logout', name='logout', method='GET',
+                  callback=self.logout, skip=['actionsmap'], apply=_logout)
+
+        # Append routes from the actions map
+        for (m, p) in self.actionsmap.parser.routes:
+            app.route(p, method=m, callback=self.process)
+
+    def apply(self, callback, context):
+        """Apply plugin to the route callback
+
+        Install a wrapper which replace callback and process the
+        relevant action for the route.
+
+        Keyword arguments:
+            callback -- The route callback
+            context -- An instance of Route
+
+        """
+        def wrapper(*args, **kwargs):
+            # Bring arguments together
+            params = kwargs
+            for a in args:
+                params[a] = True
+            for k, v in request.params.items():
+                params[k] = v
+
+            # Process the action
+            return callback((request.method, context.rule), params)
+        return wrapper
+
+
+    ## Routes callbacks
+
+    def login(self, password, profile='default'):
+        """Log in to an authenticator profile
+
+        Attempt to authenticate to a given authenticator profile and
+        register it with the current session - a new one will be created
+        if needed.
+
+        Keyword arguments:
+            - password -- A clear text password
+            - profile -- The authenticator profile name to log in
+
+        """
+        # Retrieve session values
+        s_id = request.get_cookie('session.id') or random_ascii()
+        try:
+            s_secret = self.secrets[s_id]
+        except KeyError:
+            s_hashes = {}
+        else:
+            s_hashes = request.get_cookie('session.hashes',
+                                          secret=s_secret) or {}
+        s_hash = random_ascii()
+
+        try:
+            # Attempt to authenticate
+            auth = self.actionsmap.get_authenticator(profile)
+            auth(password, token=(s_id, s_hash))
+        except MoulinetteError as e:
+            if len(s_hashes) > 0:
+                try: self.logout(profile)
+                except: pass
+            if e.errno == errno.EACCES:
+                raise HTTPUnauthorizedResponse(e.strerror)
+            raise HTTPErrorResponse(e.strerror)
+        else:
+            # Update dicts with new values
+            s_hashes[profile] = s_hash
+            self.secrets[s_id] = s_secret = random_ascii()
+
+            response.set_cookie('session.id', s_id, secure=True)
+            response.set_cookie('session.hashes', s_hashes, secure=True,
+                                secret=s_secret)
+            raise HTTPOKResponse()
+
+    def logout(self, profile=None):
+        """Log out from an authenticator profile
+
+        Attempt to unregister a given profile - or all by default - from
+        the current session.
+
+        Keyword arguments:
+            - profile -- The authenticator profile name to log out
+
+        """
+        s_id = request.get_cookie('session.id')
+        try:
+            del self.secrets[s_id]
+        except KeyError:
+            raise HTTPUnauthorizedResponse(_("You are not logged in"))
+        else:
+            # TODO: Clean the session for profile only
+            # Delete cookie and clean the session
+            response.set_cookie('session.hashes', '', max_age=-1)
+            clean_session(s_id)
+        raise HTTPOKResponse()
+
+    def process(self, _route, arguments={}):
+        """Process the relevant action for the route
+
+        Call the actions map in order to process the relevant action for
+        the route with the given arguments and process the returned
+        value.
+
+        Keyword arguments:
+            - _route -- The action route as a 2-tuple (method, path)
+            - arguments -- A dict of arguments for the route
+
+        """
+        try:
+            ret = self.actionsmap.process(arguments, route=_route)
+        except MoulinetteError as e:
+            raise HTTPErrorResponse(e.strerror)
+        else:
+            return ret
+
+
+    ## Signals handlers
+
+    def _do_authenticate(self, authenticator, help):
+        """Process the authentication
+
+        Handle the actionsmap._AMapSignals.authenticate signal.
+
+        """
+        s_id = request.get_cookie('session.id')
+        try:
+            s_secret = self.secrets[s_id]
+            s_hash = request.get_cookie('session.hashes',
+                                        secret=s_secret)[authenticator.name]
+        except KeyError:
+            if authenticator.name == 'default':
+                msg = _("Needing authentication")
+            else:
+                msg = _("Needing authentication to profile '%s'") % authenticator.name
+            raise HTTPUnauthorizedResponse(msg)
+        else:
+            return authenticator(token=(s_id, s_hash))
+
+
+# HTTP Responses -------------------------------------------------------
+
+class HTTPOKResponse(HTTPResponse):
+    def __init__(self, output=''):
+        super(HTTPOKResponse, self).__init__(output, 200)
+
+class HTTPBadRequestResponse(HTTPResponse):
+    def __init__(self, output=''):
+        super(HTTPBadRequestResponse, self).__init__(output, 400)
+
+class HTTPUnauthorizedResponse(HTTPResponse):
+    def __init__(self, output=''):
+        super(HTTPUnauthorizedResponse, self).__init__(output, 401)
+
+class HTTPErrorResponse(HTTPResponse):
+    def __init__(self, output=''):
+        super(HTTPErrorResponse, self).__init__(output, 500)
+
+
+# API Classes Implementation -------------------------------------------
+
+class ActionsMapParser(BaseActionsMapParser):
+    """Actions map's Parser for the API
+
+    Provide actions map parsing methods for a CLI usage. The parser for
+    the arguments is represented by a argparse.ArgumentParser object.
+
+    """
+    def __init__(self, shandler, parent=None):
+        super(ActionsMapParser, self).__init__(shandler, parent)
+
+        self._parsers = {} # dict({(method, path): _HTTPArgumentParser})
+
+    @property
+    def routes(self):
+        """Get current routes"""
+        return self._parsers.keys()
+
+
+    ## Implement virtual properties
+
+    name = 'api'
+
+
+    ## Implement virtual methods
+
+    @staticmethod
+    def format_arg_names(name, full):
+        if name[0] != '-':
+            return [name]
+        if full:
+            return [full.replace('--', '@', 1)]
+        if name.startswith('--'):
+            return [name.replace('--', '@', 1)]
+        return [name.replace('-', '@', 1)]
+
+    def add_global_parser(self, **kwargs):
+        raise AttributeError("global arguments are not managed")
+
+    def add_category_parser(self, name, **kwargs):
+        return self
+
+    def add_action_parser(self, name, tid, api=None, **kwargs):
+        """Add a parser for an action
+
+        Keyword arguments:
+            - api -- The action route (e.g. 'GET /' )
+
+        Returns:
+            A new _HTTPArgumentParser object for the route
+
+        """
+        try:
+            # Validate action route
+            m = re.match('(GET|POST|PUT|DELETE) (/\S+)', api)
+        except TypeError:
+            raise AttributeError("the action '%s' doesn't provide api access" % name)
+        if not m:
+            # TODO: Log error
+            raise ValueError("the action '%s' doesn't provide api access" % name)
+
+        # Check if a parser already exists for the route
+        key = (m.group(1), m.group(2))
+        if key in self.routes:
+            raise AttributeError("a parser for '%s' already exists" % key)
+
+        # Create and append parser
+        parser = _HTTPArgumentParser()
+        self._parsers[key] = (tid, parser)
+
+        # Return the created parser
+        return parser
+
+    def parse_args(self, args, route, **kwargs):
+        """Parse arguments
+
+        Keyword arguments:
+            - route -- The action route as a 2-tuple (method, path)
+
+        """
+        try:
+            # Retrieve the tid and the parser for the route
+            tid, parser = self._parsers[route]
+        except KeyError:
+            raise MoulinetteError(errno.EINVAL, "No parser found for route '%s'" % route)
+        ret = argparse.Namespace()
+
+        # Perform authentication if needed
+        if self.get_conf(tid, 'authenticate'):
+            auth_conf, klass = self.get_conf(tid, 'authenticator')
+
+            # TODO: Catch errors
+            auth = self.shandler.authenticate(klass(), **auth_conf)
+            if not auth.is_authenticated:
+                # TODO: Set proper error code
+                raise MoulinetteError(errno.EACCES, _("This action need authentication"))
+            if self.get_conf(tid, 'argument_auth') and \
+               self.get_conf(tid, 'authenticate') == 'all':
+                ret.auth = auth
+
+        return parser.parse_args(args, ret)
+
+
+class Interface(BaseInterface):
+    """Application Programming Interface for the moulinette
+
+    Initialize a HTTP server which serves the API connected to a given
+    actions map.
+
+    Keyword arguments:
+        - actionsmap -- The ActionsMap instance to connect to
+        - routes -- A dict of additional routes to add in the form of
+            {(method, path): callback}
+
+    """
+    def __init__(self, actionsmap, routes={}):
+        # TODO: Return OK to 'OPTIONS' xhr requests (l173)
+        app = Bottle(autojson=False)
+
+        ## Wrapper which sets proper header
+        def apiheader(callback):
+            def wrapper(*args, **kwargs):
+                response.content_type = 'application/json'
+                response.set_header('Access-Control-Allow-Origin', '*')
+                return json_encode(callback(*args, **kwargs))
+            return wrapper
+
+        # Install plugins
+        app.install(apiheader)
+        app.install(_ActionsMapPlugin(actionsmap))
+
+        # Append default routes
+        app.route(['/api', '/api/<category:re:[a-z]+>'], method='GET',
+                  callback=self.doc, skip=['actionsmap'])
+
+        # Append additional routes
+        # TODO: Add optional authentication to those routes?
+        for (m, p), c in routes.items():
+            app.route(p, method=m, callback=c, skip=['actionsmap'])
+
+        self._app = app
+
+    def run(self, _port):
+        """Run the moulinette
+
+        Start a server instance on the given port to serve moulinette
+        actions.
+
+        Keyword arguments:
+            - _port -- Port number to run on
+
+        """
+        try:
+            run(self._app, port=_port)
+        except IOError as e:
+            if e.args[0] == errno.EADDRINUSE:
+                raise MoulinetteError(errno.EADDRINUSE, _("A server is already running"))
+            raise
+
+
+    ## Routes handlers
+
+    def doc(self, category=None):
+        """
+        Get API documentation for a category (all by default)
+
+        Keyword argument:
+            category -- Name of the category
+
+        """
+        if category is None:
+            with open('%s/../doc/resources.json' % pkg.datadir) as f:
+                return f.read()
+
+        try:
+            with open('%s/../doc/%s.json' % (pkg.datadir, category)) as f:
+                return f.read()
+        except IOError:
+            return None

moulinette/interfaces/cli.py

@@ -0,0 +1,209 @@
+# -*- coding: utf-8 -*-
+
+import errno
+import getpass
+import argparse
+
+from moulinette.core import MoulinetteError
+from moulinette.interfaces import (BaseActionsMapParser, BaseInterface)
+
+# CLI helpers ----------------------------------------------------------
+
+colors_codes = {
+    'red'   : 31,
+    'green' : 32,
+    'yellow': 33,
+    'cyan'  : 34,
+    'purple': 35
+}
+
+def colorize(astr, color):
+    """Colorize a string
+
+    Return a colorized string for printing in shell with style ;)
+
+    Keyword arguments:
+        - astr -- String to colorize
+        - color -- Name of the color
+
+    """
+    return '\033[{:d}m\033[1m{:s}\033[m'.format(colors_codes[color], astr)
+
+def pretty_print_dict(d, depth=0):
+    """Print a dictionary recursively
+
+    Print a dictionary recursively with colors to the standard output.
+
+    Keyword arguments:
+        - d -- The dictionary to print
+        - depth -- The recursive depth of the dictionary
+
+    """
+    for k,v in sorted(d.items(), key=lambda x: x[0]):
+        k = colorize(str(k), 'purple')
+        if isinstance(v, list) and len(v) == 1:
+            v = v[0]
+        if isinstance(v, dict):
+            print(("  ") * depth + ("%s: " % str(k)))
+            pretty_print_dict(v, depth+1)
+        elif isinstance(v, list):
+            print(("  ") * depth + ("%s: " % str(k)))
+            for key, value in enumerate(v):
+                if isinstance(value, tuple):
+                    pretty_print_dict({value[0]: value[1]}, depth+1)
+                elif isinstance(value, dict):
+                    pretty_print_dict({key: value}, depth+1)
+                else:
+                    print(("  ") * (depth+1) + "- " +str(value))
+        else:
+            if not isinstance(v, basestring):
+                v = str(v)
+            print(("  ") * depth + "%s: %s" % (str(k), v))
+
+
+# CLI Classes Implementation -------------------------------------------
+
+class ActionsMapParser(BaseActionsMapParser):
+    """Actions map's Parser for the CLI
+
+    Provide actions map parsing methods for a CLI usage. The parser for
+    the arguments is represented by a argparse.ArgumentParser object.
+
+    Keyword arguments:
+        - parser -- The argparse.ArgumentParser object to use
+
+    """
+    def __init__(self, shandler, parent=None, parser=None):
+        super(ActionsMapParser, self).__init__(shandler, parent)
+
+        self._parser = parser or argparse.ArgumentParser()
+        self._subparsers = self._parser.add_subparsers()
+
+
+    ## Implement virtual properties
+
+    interface = 'cli'
+
+
+    ## Implement virtual methods
+
+    @staticmethod
+    def format_arg_names(name, full):
+        if name[0] == '-' and full:
+            return [name, full]
+        return [name]
+
+    def add_global_parser(self, **kwargs):
+        return self._parser
+
+    def add_category_parser(self, name, category_help=None, **kwargs):
+        """Add a parser for a category
+
+        Keyword arguments:
+            - category_help -- A brief description for the category
+
+        Returns:
+            A new ActionsMapParser object for the category
+
+        """
+        parser = self._subparsers.add_parser(name, help=category_help)
+        return self.__class__(None, self, parser)
+
+    def add_action_parser(self, name, tid, action_help=None, **kwargs):
+        """Add a parser for an action
+
+        Keyword arguments:
+            - action_help -- A brief description for the action
+
+        Returns:
+            A new argparse.ArgumentParser object for the action
+
+        """
+        return self._subparsers.add_parser(name, help=action_help)
+
+    def parse_args(self, args, **kwargs):
+        ret = self._parser.parse_args(args)
+
+        # Perform authentication if needed
+        if self.get_conf(ret._tid, 'authenticate'):
+            auth_conf, klass = self.get_conf(ret._tid, 'authenticator')
+
+            # TODO: Catch errors
+            auth = self.shandler.authenticate(klass(), **auth_conf)
+            if not auth.is_authenticated:
+                # TODO: Set proper error code
+                raise MoulinetteError(errno.EACCES, _("This action need authentication"))
+            if self.get_conf(ret._tid, 'argument_auth') and \
+               self.get_conf(ret._tid, 'authenticate') == 'all':
+                ret.auth = auth
+
+        return ret
+
+
+class Interface(BaseInterface):
+    """Command-line Interface for the moulinette
+
+    Initialize an interface connected to the standard input/output
+    stream and to a given actions map.
+
+    Keyword arguments:
+        - actionsmap -- The ActionsMap instance to connect to
+
+    """
+    def __init__(self, actionsmap):
+        # Connect signals to handlers
+        actionsmap.connect('authenticate', self._do_authenticate)
+        actionsmap.connect('prompt', self._do_prompt)
+
+        self.actionsmap = actionsmap
+
+    def run(self, args):
+        """Run the moulinette
+
+        Process the action corresponding to the given arguments 'args'
+        and print the result.
+
+        Keyword arguments:
+            - args -- A list of argument strings
+
+        """
+        try:
+            ret = self.actionsmap.process(args, timeout=5)
+        except KeyboardInterrupt, EOFError:
+            raise MoulinetteError(errno.EINTR, _("Interrupted"))
+
+        if isinstance(ret, dict):
+            pretty_print_dict(ret)
+        elif ret:
+            print(ret)
+
+
+    ## Signals handlers
+
+    def _do_authenticate(self, authenticator, help):
+        """Process the authentication
+
+        Handle the actionsmap._AMapSignals.authenticate signal.
+
+        """
+        # TODO: Allow token authentication?
+        msg = help or _("Password")
+        return authenticator(password=self._do_prompt(msg, True, False))
+
+    def _do_prompt(self, message, is_password, confirm):
+        """Prompt for a value
+
+        Handle the actionsmap._AMapSignals.prompt signal.
+
+        """
+        if is_password:
+            prompt = lambda m: getpass.getpass(colorize(_('%s: ') % m, 'cyan'))
+        else:
+            prompt = lambda m: raw_input(colorize(_('%s: ') % m, 'cyan'))
+        value = prompt(message)
+
+        if confirm:
+            if prompt(_('Retype %s: ') % message) != value:
+                raise MoulinetteError(errno.EINVAL, _("Values don't match"))
+
+        return value

moulinette/package.py

@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+
+# Public constants defined during build
+
+"""Package's data directory (e.g. /usr/share/moulinette)"""
+datadir = '/usr/share/moulinette'
+
+"""Package's library directory (e.g. /usr/lib/moulinette)"""
+libdir = '/usr/lib/moulinette'
+
+"""Locale directory for the package (e.g. /usr/lib/moulinette/locale)"""
+localedir = '/usr/lib/moulinette/locale'
+
+"""Cache directory for the package (e.g. /var/cache/moulinette)"""
+cachedir = '/var/cache/moulinette'

moulinette/package.py.in

@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+
+# Public constants defined during build
+
+"""Package's data directory (e.g. /usr/share/moulinette)"""
+datadir = %PKGDATADIR%
+
+"""Package's library directory (e.g. /usr/lib/moulinette)"""
+libdir = %PKGLIBDIR%
+
+"""Locale directory for the package (e.g. /usr/lib/moulinette/locale)"""
+localedir = %PKGLOCALEDIR%
+
+"""Cache directory for the package (e.g. /var/cache/moulinette)"""
+cachedir = %PKGCACHEDIR%

txrestapi/.gitignore

@@ -1,3 +0,0 @@
-_trial_temp
-txrestapi.egg-info
-txrestapi/_trial_temp

txrestapi/README.rst

@@ -1,146 +0,0 @@
-============
-Introduction
-============
-
-``txrestapi`` makes it easier to create Twisted REST API services. Normally, one
-would create ``Resource`` subclasses defining each segment of a path; this is
-cubersome to implement and results in output that isn't very readable.
-``txrestapi`` provides an ``APIResource`` class allowing complex mapping of path to
-callback (a la Django) with a readable decorator.
-
-===============================
-Basic URL callback registration
-===============================
-
-First, let's create a bare API service::
-
-    >>> from txrestapi.resource import APIResource
-    >>> api = APIResource()
-
-and a web server to serve it::
-
-    >>> from twisted.web.server import Site
-    >>> from twisted.internet import reactor
-    >>> site = Site(api, timeout=None)
-
-and a function to make it easy for us to make requests (only for doctest
-purposes; normally you would of course use ``reactor.listenTCP(8080, site)``)::
-
-    >>> from twisted.web.server import Request
-    >>> class FakeChannel(object):
-    ...     transport = None
-    >>> def makeRequest(method, path):
-    ...     req = Request(FakeChannel(), None)
-    ...     req.prepath = req.postpath = None
-    ...     req.method = method; req.path = path
-    ...     resource = site.getChildWithDefault(path, req)
-    ...     return resource.render(req)
-
-We can now register callbacks for paths we care about. We can provide different
-callbacks for different methods; they must accept ``request`` as the first
-argument::
-
-    >>> def get_callback(request): return 'GET callback'
-    >>> api.register('GET', '^/path/to/method', get_callback)
-    >>> def post_callback(request): return 'POST callback'
-    >>> api.register('POST', '^/path/to/method', post_callback)
-
-Then, when we make a call, the request is routed to the proper callback::
-
-    >>> print makeRequest('GET', '/path/to/method')
-    GET callback
-    >>> print makeRequest('POST', '/path/to/method')
-    POST callback
-
-We can register multiple callbacks for different requests; the first one that
-matches wins::
-
-    >>> def default_callback(request):
-    ...     return 'Default callback'
-    >>> api.register('GET', '^/.*$', default_callback) # Matches everything
-    >>> print makeRequest('GET', '/path/to/method')
-    GET callback
-    >>> print makeRequest('GET', '/path/to/different/method')
-    Default callback
-
-Our default callback, however, will only match GET requests. For a true default
-callback, we can either register callbacks for each method individually, or we
-can use ALL::
-
-    >>> api.register('ALL', '^/.*$', default_callback)
-    >>> print makeRequest('PUT', '/path/to/method')
-    Default callback
-    >>> print makeRequest('DELETE', '/path/to/method')
-    Default callback
-    >>> print makeRequest('GET', '/path/to/method')
-    GET callback
-
-Let's unregister all references to the default callback so it doesn't interfere
-with later tests (default callbacks should, of course, always be registered
-last, so they don't get called before other callbacks)::
-
-    >>> api.unregister(callback=default_callback)
-
-=============
-URL Arguments
-=============
-
-Since callbacks accept ``request``, they have access to POST data or query
-arguments, but we can also pull arguments out of the URL by using named groups
-in the regular expression (similar to Django). These will be passed into the
-callback as keyword arguments::
-
-    >>> def get_info(request, id):
-    ...     return 'Information for id %s' % id
-    >>> api.register('GET', '/(?P<id>[^/]+)/info$', get_info)
-    >>> print makeRequest('GET', '/someid/info')
-    Information for id someid
-
-Bear in mind all arguments will come in as strings, so code should be
-accordingly defensive.
-
-================
-Decorator syntax
-================
-
-Registration via the ``register()`` method is somewhat awkward, so decorators
-are provided making it much more straightforward. ::
-
-    >>> from txrestapi.methods import GET, POST, PUT, ALL
-    >>> class MyResource(APIResource):
-    ...
-    ...     @GET('^/(?P<id>[^/]+)/info')
-    ...     def get_info(self, request, id):
-    ...         return 'Info for id %s' % id
-    ...
-    ...     @PUT('^/(?P<id>[^/]+)/update')
-    ...     @POST('^/(?P<id>[^/]+)/update')
-    ...     def set_info(self, request, id):
-    ...         return "Setting info for id %s" % id
-    ...
-    ...     @ALL('^/')
-    ...     def default_view(self, request):
-    ...         return "I match any URL"
-
-Again, registrations occur top to bottom, so methods should be written from
-most specific to least. Also notice that one can use the decorator syntax as
-one would expect to register a method as the target for two URLs ::
-
-    >>> site = Site(MyResource(), timeout=None)
-    >>> print makeRequest('GET', '/anid/info')
-    Info for id anid
-    >>> print makeRequest('PUT', '/anid/update')
-    Setting info for id anid
-    >>> print makeRequest('POST', '/anid/update')
-    Setting info for id anid
-    >>> print makeRequest('DELETE', '/anid/delete')
-    I match any URL
-
-======================
-Callback return values
-======================
-
-You can return Resource objects from a callback if you wish, allowing you to
-have APIs that send you to other kinds of resources, or even other APIs.
-Normally, however, you'll most likely want to return strings, which will be
-wrapped in a Resource object for convenience.

txrestapi/__init__.py

@@ -1 +0,0 @@
-#

txrestapi/methods.py

@@ -1,29 +0,0 @@
-from zope.interface.advice import addClassAdvisor
-
-def method_factory_factory(method):
-    def factory(regex):
-        _f = {}
-        def decorator(f):
-            _f[f.__name__] = f
-            return f
-        def advisor(cls):
-            def wrapped(f):
-                def __init__(self, *args, **kwargs):
-                    f(self, *args, **kwargs)
-                    for func_name in _f:
-                        orig = _f[func_name]
-                        func = getattr(self, func_name)
-                    if func.im_func==orig:
-                        self.register(method, regex, func)
-                return __init__
-            cls.__init__ = wrapped(cls.__init__)
-            return cls
-        addClassAdvisor(advisor)
-        return decorator
-    return factory
-
-ALL    = method_factory_factory('ALL')
-GET    = method_factory_factory('GET')
-POST   = method_factory_factory('POST')
-PUT    = method_factory_factory('PUT')
-DELETE = method_factory_factory('DELETE')

txrestapi/resource.py

@@ -1,65 +0,0 @@
-import re
-from itertools import ifilter
-from functools import wraps
-from twisted.web.resource import Resource, NoResource
-
-class _FakeResource(Resource):
-    _result = ''
-    isLeaf = True
-    def __init__(self, result):
-        Resource.__init__(self)
-        self._result = result
-    def render(self, request):
-        return self._result
-
-
-def maybeResource(f):
-    @wraps(f)
-    def inner(*args, **kwargs):
-        result = f(*args, **kwargs)
-        if not isinstance(result, Resource):
-            result = _FakeResource(result)
-        return result
-    return inner
-
-
-class APIResource(Resource):
-
-    _registry = None
-
-    def __init__(self, *args, **kwargs):
-        Resource.__init__(self, *args, **kwargs)
-        self._registry = []
-
-    def _get_callback(self, request):
-        filterf = lambda t:t[0] in (request.method, 'ALL')
-        path_to_check = getattr(request, '_remaining_path', request.path)
-        for m, r, cb in ifilter(filterf, self._registry):
-            result = r.search(path_to_check)
-            if result:
-                request._remaining_path = path_to_check[result.span()[1]:]
-                return cb, result.groupdict()
-        return None, None
-
-    def register(self, method, regex, callback):
-        self._registry.append((method, re.compile(regex), callback))
-
-    def unregister(self, method=None, regex=None, callback=None):
-        if regex is not None: regex = re.compile(regex)
-        for m, r, cb in self._registry[:]:
-            if not method or (method and m==method):
-                if not regex or (regex and r==regex):
-                    if not callback or (callback and cb==callback):
-                        self._registry.remove((m, r, cb))
-
-    def getChild(self, name, request):
-        r = self.children.get(name, None)
-        if r is None:
-            # Go into the thing
-            callback, args = self._get_callback(request)
-            if callback is None:
-                return NoResource()
-            else:
-                return maybeResource(callback)(request, **args)
-        else:
-            return r

txrestapi/service.py

@@ -1,7 +0,0 @@
-from twisted.web.server import Site
-from .resource import APIResource
-
-
-class RESTfulService(Site):
-    def __init__(self, port=8080):
-        self.root = APIResource()

txrestapi/tests.py

@@ -1,194 +0,0 @@
-import txrestapi
-__package__="txrestapi"
-import re
-import os.path
-from twisted.internet import reactor
-from twisted.internet.defer import inlineCallbacks
-from twisted.web.resource import Resource, NoResource
-from twisted.web.server import Request, Site
-from twisted.web.client import getPage
-from twisted.trial import unittest
-from .resource import APIResource
-from .methods import GET, PUT
-
-class FakeChannel(object):
-    transport = None
-
-def getRequest(method, url):
-    req = Request(FakeChannel(), None)
-    req.method = method
-    req.path = url
-    return req
-
-class APIResourceTest(unittest.TestCase):
-
-    def test_returns_normal_resources(self):
-        r = APIResource()
-        a = Resource()
-        r.putChild('a', a)
-        req = Request(FakeChannel(), None)
-        a_ = r.getChild('a', req)
-        self.assertEqual(a, a_)
-
-    def test_registry(self):
-        compiled = re.compile('regex')
-        r = APIResource()
-        r.register('GET', 'regex', None)
-        self.assertEqual([x[0] for x in r._registry], ['GET'])
-        self.assertEqual(r._registry[0], ('GET', compiled, None))
-
-    def test_method_matching(self):
-        r = APIResource()
-        r.register('GET', 'regex', 1)
-        r.register('PUT', 'regex', 2)
-        r.register('GET', 'another', 3)
-
-        req = getRequest('GET', 'regex')
-        result = r._get_callback(req)
-        self.assert_(result)
-        self.assertEqual(result[0], 1)
-
-        req = getRequest('PUT', 'regex')
-        result = r._get_callback(req)
-        self.assert_(result)
-        self.assertEqual(result[0], 2)
-
-        req = getRequest('GET', 'another')
-        result = r._get_callback(req)
-        self.assert_(result)
-        self.assertEqual(result[0], 3)
-
-        req = getRequest('PUT', 'another')
-        result = r._get_callback(req)
-        self.assertEqual(result, (None, None))
-
-    def test_callback(self):
-        marker = object()
-        def cb(request):
-            return marker
-        r = APIResource()
-        r.register('GET', 'regex', cb)
-        req = getRequest('GET', 'regex')
-        result = r.getChild('regex', req)
-        self.assertEqual(result.render(req), marker)
-
-    def test_longerpath(self):
-        marker = object()
-        r = APIResource()
-        def cb(request):
-            return marker
-        r.register('GET', '/regex/a/b/c', cb)
-        req = getRequest('GET', '/regex/a/b/c')
-        result = r.getChild('regex', req)
-        self.assertEqual(result.render(req), marker)
-
-    def test_args(self):
-        r = APIResource()
-        def cb(request, **kwargs):
-            return kwargs
-        r.register('GET', '/(?P<a>[^/]*)/a/(?P<b>[^/]*)/c', cb)
-        req = getRequest('GET', '/regex/a/b/c')
-        result = r.getChild('regex', req)
-        self.assertEqual(sorted(result.render(req).keys()), ['a', 'b'])
-
-    def test_order(self):
-        r = APIResource()
-        def cb1(request, **kwargs):
-            kwargs.update({'cb1':True})
-            return kwargs
-        def cb(request, **kwargs):
-            return kwargs
-        # Register two regexes that will match
-        r.register('GET', '/(?P<a>[^/]*)/a/(?P<b>[^/]*)/c', cb1)
-        r.register('GET', '/(?P<a>[^/]*)/a/(?P<b>[^/]*)', cb)
-        req = getRequest('GET', '/regex/a/b/c')
-        result = r.getChild('regex', req)
-        # Make sure the first one got it
-        self.assert_('cb1' in result.render(req))
-
-    def test_no_resource(self):
-        r = APIResource()
-        r.register('GET', '^/(?P<a>[^/]*)/a/(?P<b>[^/]*)$', None)
-        req = getRequest('GET', '/definitely/not/a/match')
-        result = r.getChild('regex', req)
-        self.assert_(isinstance(result, NoResource))
-
-    def test_all(self):
-        r = APIResource()
-        def get_cb(r): return 'GET'
-        def put_cb(r): return 'PUT'
-        def all_cb(r): return 'ALL'
-        r.register('GET', '^path', get_cb)
-        r.register('ALL', '^path', all_cb)
-        r.register('PUT', '^path', put_cb)
-        # Test that the ALL registration picks it up before the PUT one
-        for method in ('GET', 'PUT', 'ALL'):
-            req = getRequest(method, 'path')
-            result = r.getChild('path', req)
-            self.assertEqual(result.render(req), 'ALL' if method=='PUT' else method)
-
-
-class TestResource(Resource):
-    isLeaf = True
-    def render(self, request):
-        return 'aresource'
-
-
-class TestAPI(APIResource):
-
-    @GET('^/(?P<a>test[^/]*)/?')
-    def _on_test_get(self, request, a):
-        return 'GET %s' % a
-
-    @PUT('^/(?P<a>test[^/]*)/?')
-    def _on_test_put(self, request, a):
-        return 'PUT %s' % a
-
-    @GET('^/gettest')
-    def _on_gettest(self, request):
-        return TestResource()
-
-
-class DecoratorsTest(unittest.TestCase):
-    def _listen(self, site):
-        return reactor.listenTCP(0, site, interface="127.0.0.1")
-
-    def setUp(self):
-        r = TestAPI()
-        site = Site(r, timeout=None)
-        self.port = self._listen(site)
-        self.portno = self.port.getHost().port
-
-    def tearDown(self):
-        return self.port.stopListening()
-
-    def getURL(self, path):
-        return "http://127.0.0.1:%d/%s" % (self.portno, path)
-
-    @inlineCallbacks
-    def test_get(self):
-        url = self.getURL('test_thing/')
-        result = yield getPage(url, method='GET')
-        self.assertEqual(result, 'GET test_thing')
-
-    @inlineCallbacks
-    def test_put(self):
-        url = self.getURL('test_thing/')
-        result = yield getPage(url, method='PUT')
-        self.assertEqual(result, 'PUT test_thing')
-
-    @inlineCallbacks
-    def test_resource_wrapper(self):
-        url = self.getURL('gettest')
-        result = yield getPage(url, method='GET')
-        self.assertEqual(result, 'aresource')
-
-
-def test_suite():
-    import unittest as ut
-    suite = unittest.TestSuite()
-    suite.addTest(ut.makeSuite(DecoratorsTest))
-    suite.addTest(ut.makeSuite(APIResourceTest))
-    suite.addTest(unittest.doctest.DocFileSuite(os.path.join('..', 'README.rst')))
-    return suite
-

yunohost

@@ -1,134 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-import os
-import sys
-import argparse
-import gettext
-import getpass
-try:
-    import yaml
-except ImportError:
-    sys.stderr.write('Error: Yunohost CLI Require yaml lib\n')
-    sys.stderr.write('apt-get install python-yaml\n')
-    sys.exit(1)
-import json
-if not __debug__:
-    import traceback
-
-gettext.install('YunoHost')
-
-try:
-    from yunohost import YunoHostError, YunoHostLDAP, str_to_func, colorize, pretty_print_dict, display_error, validate, win, parse_dict
-except ImportError:
-    sys.stderr.write('Error: Yunohost CLI Require YunoHost lib\n')
-    sys.exit(1)
-
-
-def main():
-    """
-    Main instructions
-
-    Parse the action_dict and execute the action-specific function,
-    then print json or pretty result if executed in a tty :)
-
-    Returns:
-        int -- 0 or error code
-
-    """
-
-    if len(sys.argv) < 2:
-        sys.argv.append('-h')
-
-    with open('action_map.yml') as f:
-        action_map = yaml.load(f)
-
-    admin_password_provided = False
-    json_print = False
-    write_ldap = True
-    postinstall = False
-
-    for key, arg in enumerate(sys.argv):
-        if arg == '--admin-password':
-            admin_password_provided = True
-            admin_password = sys.argv[key+1]
-            sys.argv.pop(key)
-            sys.argv.pop(key)
-        if arg == '--no-ldap':
-            write_ldap = False
-            sys.argv.pop(key)
-        if arg == '--json':
-            json_print = True
-            sys.argv.pop(key)
-
-    try:
-        try:
-            with open('/etc/yunohost/installed') as f: pass
-        except IOError:
-            postinstall = True
-            if len(sys.argv) < 3 or sys.argv[1] != 'tools' or sys.argv[2] != 'postinstall':
-                raise YunoHostError(17, _("YunoHost is not correctly installed, please execute 'yunohost tools postinstall'"))
-
-        args = parse_dict(action_map)
-        args_dict = vars(args).copy()
-        for key in args_dict.keys():
-            sanitized_key = key.replace('-', '_')
-            if sanitized_key is not key:
-                args_dict[sanitized_key] = args_dict[key]
-                del args_dict[key]
-        del args_dict['func']
-        try:
-            with open('/etc/yunohost/passwd') as f:
-                admin_password = f.read()
-                admin_password_provided = True
-        except IOError: pass
-        if postinstall:
-            result = args.func(**args_dict)
-        elif admin_password_provided:
-            with YunoHostLDAP(password=admin_password):
-                result = args.func(**args_dict)
-        elif os.isatty(1) and write_ldap:
-            admin_password = getpass.getpass(colorize(_('Admin Password: '), 'yellow'))
-            with YunoHostLDAP(password=admin_password):
-                try:
-                    with open('/var/run/yunohost.pid', 'r'):
-                        raise YunoHostError(1, _("A YunoHost command is already running"))
-                except IOError:
-                    with open('/var/run/yunohost.pid', 'w') as f:
-                        f.write('ldap')
-                        os.system('chmod 400 /var/run/yunohost.pid')
-                    with open('/etc/yunohost/passwd', 'w') as f:
-                        f.write(admin_password)
-                        os.system('chmod 400 /etc/yunohost/passwd')
-                    try:
-                        result = args.func(**args_dict)
-                    except KeyboardInterrupt, EOFError:
-                        raise YunoHostError(125, _("Interrupted"))
-                    finally:
-                        os.remove('/etc/yunohost/passwd')
-                        os.remove('/var/run/yunohost.pid')
-        else:
-            with YunoHostLDAP(anonymous=True):
-                result = args.func(**args_dict)
-    #except TypeError, error:
-        #if not __debug__ :
-            #traceback.print_exc()
-        #print(_("Not (yet) implemented function"))
-        #return 1
-    except YunoHostError, error:
-        display_error(error, json_print)
-        return error.code
-    else:
-        if json_print or not os.isatty(1) and result is not None:
-            if len(win) > 0:
-                result['success'] = win
-            print(json.dumps(result))
-        elif result is not None:
-            pretty_print_dict(result)
-        else:
-            pass
-
-    return 0
-
-if __name__ == '__main__':
-    sys.exit(main())

yunohost.tac

@@ -1,273 +0,0 @@
-# -*- mode: python -*-
-import os
-import sys
-import gettext
-import ldap
-import yaml
-import json
-
-sys.path.append('/usr/share/pyshared')
-
-from twisted.python.log import ILogObserver, FileLogObserver, startLogging, msg
-from twisted.python.logfile import DailyLogFile
-from twisted.web.server import Site, http
-from twisted.internet import reactor
-from twisted.application import internet,service
-from txrestapi.resource import APIResource
-from yunohost import YunoHostError, YunoHostLDAP, str_to_func, colorize, pretty_print_dict, display_error, validate, win, parse_dict
-import yunohost
-
-if not __debug__:
-    import traceback
-
-gettext.install('YunoHost')
-
-dev = False
-installed = True
-action_dict = {}
-api = APIResource()
-
-def http_exec(request, **kwargs):
-    global installed
-
-    request.setHeader('Access-Control-Allow-Origin', '*') # Allow cross-domain requests
-    request.setHeader('Content-Type', 'application/json') # Return JSON anyway
-
-    # Return OK to 'OPTIONS' xhr requests
-    if request.method == 'OPTIONS':
-        request.setResponseCode(200, 'OK')
-        request.setHeader('Access-Control-Allow-Headers', 'Authorization, Content-Type')
-        request.setHeader('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS')
-        return ''
-
-    # Simple HTTP auth
-    elif installed:
-        authorized = False
-        pwd = request.getPassword()
-        if request.getUser() == 'admin' and pwd != '':
-            authorized = True
-        if dev and 'api_key' in request.args:
-            pwd = request.args['api_key'][0]
-            authorized = True
-        if authorized:
-            try: YunoHostLDAP(password=pwd)
-            except YunoHostError: authorized = False
-        if not authorized:
-            request.setResponseCode(401, 'Unauthorized')
-            request.setHeader('Access-Control-Allow-Origin', '*')
-            request.setHeader('www-authenticate', 'Basic realm="Restricted Area"')
-            return 'Unauthorized'
-
-    path = request.path
-    if request.method == 'PUT':
-        given_args = http.parse_qs(request.content.read(), 1)
-    else:
-        given_args = request.args
-    if kwargs:
-       for k, v in kwargs.iteritems():
-           dynamic_key = path.split('/')[-1]
-           path = path.replace(dynamic_key, '{'+ k +'}')
-           given_args[k] = [v]
-
-    #msg(given_args)
-    # Sanitize arguments
-    dict = action_dict[request.method +' '+ path]
-    if 'arguments' in dict: possible_args = dict['arguments']
-    else: possible_args = {}
-    for arg, params in possible_args.items():
-        sanitized_key = arg.replace('-', '_')
-        if sanitized_key is not arg:
-            possible_args[sanitized_key] = possible_args[arg]
-            del possible_args[arg]
-            arg = sanitized_key
-        if arg[0] == '_':
-            if 'nargs' not in params:
-                possible_args[arg]['nargs'] = '*'
-            if 'full' in params:
-                new_key = params['full'][2:]
-            else:
-                new_key = arg[2:]
-            new_key = new_key.replace('-', '_')
-	    possible_args[new_key] = possible_args[arg]
-            del possible_args[arg]
-
-    try:
-
-        # Validate arguments
-        validated_args = {}
-        for key, value in given_args.items():
-           if key in possible_args:
-               # Validate args
-               if 'pattern' in possible_args[key]:
-                   validate(possible_args[key]['pattern'], value)
-               if 'nargs' not in possible_args[key] or ('nargs' != '*' and 'nargs' != '+'):
-                   value = value[0]
-               if 'choices' in possible_args[key] and value not in possible_args[key]['choices']:
-                   raise YunoHostError(22, _('Invalid argument') + ' ' + value)
-               if 'action' in possible_args[key] and possible_args[key]['action'] == 'store_true':
-                   yes = ['true', 'True', 'yes', 'Yes']
-                   value = value in yes
-               validated_args[key] = value
-
-        func = str_to_func(dict['function'])
-        if func is None:
-            raise YunoHostError(168, _('Function not yet implemented : ') + dict['function'].split('.')[1])
-
-        # Execute requested function
-        try:
-            with open('/var/run/yunohost.pid', 'r'):
-                raise YunoHostError(1, _("A YunoHost command is already running"))
-        except IOError:
-            if dict['function'].split('.')[1] != 'tools_postinstall':
-                try:
-                    with open('/etc/yunohost/installed'): pass
-                except IOError:
-                    raise YunoHostError(1, _("You must run postinstall before any other actions"))
-                with open('/var/run/yunohost.pid', 'w') as f:
-                    f.write('ldap')
-                    os.system('chmod 400 /var/run/yunohost.pid')
-            with open('/etc/yunohost/passwd', 'w') as f:
-                f.write(request.getPassword())
-                os.system('chmod 400 /etc/yunohost/passwd')
-            try:
-                result = func(**validated_args)
-            except KeyboardInterrupt, EOFError:
-                raise YunoHostError(125, _("Interrupted"))
-            finally:
-                try:
-                    os.remove('/etc/yunohost/passwd')
-                    os.remove('/var/run/yunohost.pid')
-                except: pass
-        if result is None:
-            result = {}
-        if len(yunohost.win) > 0:
-            result['win'] = yunohost.win
-            yunohost.win = []
-
-        # Build response
-        if request.method == 'POST':
-            request.setResponseCode(201, 'Created')
-            if not installed:
-                installed = True
-        elif request.method == 'DELETE':
-            request.setResponseCode(204, 'No Content')
-        else:
-            request.setResponseCode(200, 'OK')
-
-    except YunoHostError, error:
-
-        # Set response code with function's raised code
-        server_errors = [1, 111, 168, 169]
-        client_errors = [13, 17, 22, 87, 122, 125, 167]
-        if error.code in client_errors:
-            request.setResponseCode(400, 'Bad Request')
-        else:
-            request.setResponseCode(500, 'Internal Server Error')
-
-        result = { 'error' : error.message }
-
-    return json.dumps(result)
-
-def api_doc(request):
-    request.setHeader('Access-Control-Allow-Origin', '*') # Allow cross-domain requests
-    request.setHeader('Content-Type', 'application/json') # Return JSON anyway
-
-    # Return OK to 'OPTIONS' xhr requests
-    if request.method == 'OPTIONS':
-        request.setResponseCode(200, 'OK')
-        request.setHeader('Access-Control-Allow-Headers', 'Authorization')
-        return ''
-
-    if request.path == '/api':
-        with open('doc/resources.json') as f:
-            return f.read()
-
-    category = request.path.split('/')[2]
-    try:
-        with open('doc/'+ category +'.json') as f:
-            return f.read()
-    except IOError:
-        return ''
-
-def favicon(request):
-    request.setHeader('Access-Control-Allow-Origin', '*') # Allow cross-domain requests
-    request.setResponseCode(404, 'Not Found')
-    return ''
-
-def is_installed(request):
-    global installed
-
-    try:
-        with open('/etc/yunohost/installed'):
-            installed = True
-    except IOError:
-        installed = False
-    request.setHeader('Access-Control-Allow-Origin', '*') # Allow cross-domain requests
-    request.setResponseCode(200, 'OK')
-    return json.dumps({ 'installed': installed })
-
-def main():
-    global action_dict
-    global api
-    global installed
-
-    # Generate API doc
-    os.system('python ./generate_api_doc.py')
-
-    # Register API doc service
-    api.register('ALL', '/api', api_doc)
-
-    # favicon.ico error
-    api.register('ALL', '/favicon.ico', favicon)
-
-    # Load & parse yaml file
-    with open('action_map.yml') as f:
-        action_map = yaml.load(f)
-
-    # Register only postinstall action if YunoHost isn't completely set up
-    try:
-        with open('/etc/yunohost/installed'):
-            installed = True
-    except IOError:
-        installed = False
-
-    del action_map['general_arguments']
-    for category, category_params in action_map.items():
-        api.register('ALL', '/api/'+ category, api_doc)
-        for action, action_params in category_params['actions'].items():
-            if 'action_help' not in action_params:
-                action_params['action_help'] = ''
-            if 'api' not in action_params:
-                action_params['api'] = 'GET /'+ category +'/'+ action
-            method, path = action_params['api'].split(' ')
-            # Register route
-            if '{' in path:
-                path = path.replace('{', '(?P<').replace('}', '>[^/]+)')
-            api.register(method, path, http_exec)
-            api.register('OPTIONS', path, http_exec)
-            action_dict[action_params['api']] = {
-                'function': 'yunohost_'+ category +'.'+ category +'_'+ action.replace('-', '_'),
-                'help'    : action_params['action_help']
-            }
-            if 'arguments' in action_params:
-                action_dict[action_params['api']]['arguments'] = action_params['arguments']
-
-    api.register('ALL', '/installed', is_installed)
-
-
-
-if __name__ == '__main__':
-    if '--dev' in sys.argv:
-        dev = True
-        startLogging(sys.stdout)
-    else:
-        startLogging(open('/var/log/yunohost.log', 'a+')) # Log actions to file
-    main()
-    reactor.listenTCP(6787, Site(api, timeout=None))
-    reactor.run()
-else:
-    application = service.Application("YunoHost API")
-    logfile = DailyLogFile("yunohost.log", "/var/log")
-    application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
-    main()
-    internet.TCPServer(6787, Site(api, timeout=None)).setServiceParent(application)