# Server Development

Getting started guide for the server development

# Preamble

This whole guide is based on Ubuntu 18.04 LTS with python 3.6. Ubuntu 12.04+ LTS and Debian based systems should be similar if not even identical. Our current supported python versions are python 3.6+.

We assume that you already have somewhere a Postgres database running. If not follow the guide to setup postgres.

# Installation

  1. Install some generic stuff

    sudo apt-get update
    sudo apt-get install -y \
            git \
            libyaml-dev \
            libpython3-dev \
            libpq-dev \
            libffi-dev \
            python3-dev \
            python3-pip \
            python3-psycopg2 \
            postgresql-client
    sudo pip3 install --upgrade pip
    
  2. Clone git repository

    git clone https://gitlab.com/psono/psono-server.git ~/psono-server
    
  3. Checkout new branch

    cd ~/psono-server
    git fetch
    git checkout develop
    git checkout -b [name_of_your_new_branch]
    
  4. Install python requirements

    sudo pip3 install -r requirements.txt
    sudo pip3 install -r requirements-dev.txt
    
  5. Create settings folder

    mkdir ~/.psono_server
    
  6. Create a settings.yaml in ~/.psono_server/ with the following content

    # generate the following six parameters with the following command
    # python3 ~/psono-server/psono/manage.py generateserverkeys
    SECRET_KEY: 'SOME SUPER SECRET KEY THAT SHOULD BE RANDOM AND 32 OR MORE DIGITS LONG'
    ACTIVATION_LINK_SECRET: 'SOME SUPER SECRET ACTIVATION LINK SECRET THAT SHOULD BE RANDOM AND 32 OR MORE DIGITS LONG'
    DB_SECRET: 'SOME SUPER SECRET DB SECRET THAT SHOULD BE RANDOM AND 32 OR MORE DIGITS LONG'
    EMAIL_SECRET_SALT: '$2b$12$XUG.sKxC2jmkUvWQjg53.e'
    PRIVATE_KEY: '302650c3c82f7111c2e8ceb660d32173cdc8c3d7717f1d4f982aad5234648fcb'
    PUBLIC_KEY: '02da2ad857321d701d754a7e60d0a147cdbc400ff4465e1f57bc2d9fbfeddf0b'
    
    # The URL of the web client (path to e.g activate.html without the trailing slash)
    # WEB_CLIENT_URL: 'https://www.psono.pw'
    
    # Switch DEBUG to false if you go into production
    DEBUG: True
    
    # Adjust this according to Django Documentation https://docs.djangoproject.com/en/2.2/ref/settings/
    ALLOWED_HOSTS: ['*']
    
    # Should be your domain without "www.". Will be the last part of the username
    ALLOWED_DOMAINS: ['psono.pw']
    
    # If you want to disable registration, you can comment in the following line
    # ALLOW_REGISTRATION: False
    
    # If you want to disable the lost password functionality, you can comment in the following line
    # ALLOW_LOST_PASSWORD: False
    
    # If you want to enforce that the email address and username needs to match upon registration
    # ENFORCE_MATCHING_USERNAME_AND_EMAIL: False
    
    # If you want to restrict registration to some email addresses you can specify here a list of domains to filter
    # REGISTRATION_EMAIL_FILTER: ['company1.com', 'company2.com']
    
    # Should be the URL of the host under which the host is reachable
    # If you open the url and append /info/ to it you should have a text similar to {"info":"{\"version\": \"....}
    HOST_URL: 'https://www.psono.pw/server'
    
    # The email used to send emails, e.g. for activation
    # Not necessary if you do not plan to develop around the user activation
    # ATTENTION: If executed in a docker container, then "localhost" will resolve to the docker container, so
    # "localhost" will not work as host. Use the public IP or DNS record of the server.
    EMAIL_FROM: '[email protected]'
    EMAIL_HOST: 'localhost'
    EMAIL_HOST_USER: ''
    EMAIL_HOST_PASSWORD : ''
    EMAIL_PORT: 25
    EMAIL_SUBJECT_PREFIX: ''
    EMAIL_USE_TLS: False
    EMAIL_USE_SSL: False
    EMAIL_SSL_CERTFILE:
    EMAIL_SSL_KEYFILE:
    EMAIL_TIMEOUT:
    
    # In case one wants to use mailgun, comment in below lines and provide the mailgun access key and server name
    # EMAIL_BACKEND: 'anymail.backends.mailgun.EmailBackend'
    # MAILGUN_ACCESS_KEY: ''
    # MAILGUN_SERVER_NAME: ''
    
    # In case you want to offer Yubikey support, create a pair of credentials here https://upgrade.yubico.com/getapikey/
    # and update the following two lines before commenting them in
    # YUBIKEY_CLIENT_ID: '123456'
    # YUBIKEY_SECRET_KEY: '8I65IA6ASDFIUHGIH5021FKJA='
    
    # If you have own Yubico servers, you can specify here the urls as a list
    # YUBICO_API_URLS: ['https://api.yubico.com/wsapi/2.0/verify']
    
    # Cache enabled without belows Redis may lead to unexpected behaviour
    
    # Cache with Redis
    # By default you should use something different than database 0 or 1, e.g. 13 (default max is 16, can be configured in
    # redis.conf) possible URLS are:
    #    redis://[:password]@localhost:6379/0
    #    rediss://[:password]@localhost:6379/0
    #    unix://[:password]@/path/to/socket.sock?db=0
    # CACHE_ENABLE: False
    # CACHE_REDIS: False
    # CACHE_REDIS_LOCATION: 'redis://127.0.0.1:6379/13'
    
    # Disables Throttling (necessary for unittests to pass) by overriding the cache with a dummy cache
    # https://docs.djangoproject.com/en/2.2/topics/cache/#dummy-caching-for-development
    # THROTTLING: False
    
    # Enables the management API, required for the psono-admin-client / admin portal
    # MANAGEMENT_ENABLED: False
    
    # Enables the fileserver API, required for the psono-fileserver
    # FILESERVER_HANDLER_ENABLED: False
    
    # Enables files for the client
    # FILES_ENABLED: False
    
    # Allows that users can search for partial usernames
    # ALLOW_USER_SEARCH_BY_USERNAME_PARTIAL: True
    
    # Allows that users can search for email addresses too
    # ALLOW_USER_SEARCH_BY_EMAIL: True
    
    # Disables central security reports
    # DISABLE_CENTRAL_SECURITY_REPORTS: True
    
    # Configures a system wide DUO connection for all clients
    # DUO_INTEGRATION_KEY: ''
    # DUO_SECRET_KEY: ''
    # DUO_API_HOSTNAME: ''
    
    # If you are using the DUO proxy, you can configure here the necessary HTTP proxy
    # DUO_PROXY_HOST: 'the-ip-or-dns-name-goes-here'
    # DUO_PROXY_PORT: 80
    # DUO_PROXY_TYPE: 'CONNECT'
    # If your proxy requires specific headers you can also configure these here
    # DUO_PROXY_HEADERS: ''
    
    # Normally only one of the configured second factors needs to be solved. Setting this to True forces the client to solve all
    # MULTIFACTOR_ENABLED: True
    
    # Allows admins to limit the offered second factors in the client
    # ALLOWED_SECOND_FACTORS: ['yubikey_otp', 'google_authenticator', 'duo']
    
    # Your Postgres Database credentials
    # ATTENTION: If executed in a docker container, then "localhost" will resolve to the docker container, so
    # "localhost" will not work as host. Use the public IP or DNS record of the server.
    DATABASES:
        default:
            'ENGINE': 'django.db.backends.postgresql_psycopg2'
            'NAME': 'psono'
            'USER': 'psono'
            'PASSWORD': 'password'
            'HOST': 'localhost'
            'PORT': '5432'
    # for master / slave replication setup comment in the following (all reads will be redirected to the slave
    #    slave:
    #        'ENGINE': 'django.db.backends.postgresql_psycopg2'
    #        'NAME': 'YourPostgresDatabase'
    #        'USER': 'YourPostgresUser'
    #        'PASSWORD': 'YourPostgresPassword'
    #        'HOST': 'YourPostgresHost'
    #        'PORT': 'YourPostgresPort'
    
    # Update the path to your templates folder
    TEMPLATES: [
        {
            'BACKEND': 'django.template.backends.django.DjangoTemplates',
            'DIRS': ['/home/psono/psono-server/psono/templates'],
            'APP_DIRS': True,
            'OPTIONS': {
                'context_processors': [
                    'django.template.context_processors.debug',
                    'django.template.context_processors.request',
                    'django.contrib.auth.context_processors.auth',
                    'django.contrib.messages.context_processors.messages',
                ],
            },
        },
    ]
    

    Update database credentials / secrets / paths like described in the comments

  7. Test E-Mail (optional)

    The most tedious step is usually for me to get e-mail working. To make this step easier, we offer a small test script which will send a test e-mail.

    To send a test e-mail to [email protected] execute:

    python3 ~/psono-server/psono/manage.py sendtestmail [email protected]
    

    If you receive this test e-mail, e-mail should be configured proper.

  8. Create our database

    python3  ~/psono-server/psono/manage.py migrate
    

# Run the dev server

From this point on forward, you can develop it like any other django application

To start the server you can do for example:

python3 ~/psono-server/psono/manage.py runserver 0.0.0.0:10100

This will start the Psono server on port 10100. If you open now http://your-ip:10100/info/ you should see something like this:

{"info":"{\"version\": \"....}

If not, please make sure you have no firewall on the server blocking you.

# Update database model

If you ever change parts of the model, then you can create the migration script with the following command

python3  ~/psono-server/psono/manage.py makemigrations restapi

and apply it then to your postgres installation with:

python3  ~/psono-server/psono/manage.py migrate

# Create user on the commandline:

If you are developing and don't want to setup email or go to the database to activate a user, you can just create a user on the command line with the following command:

python3 ./psono/manage.py createuser [email protected] myPassword [email protected]

Other useful commands can be found under Commands

# Run Unit Tests (with coverage)

To run unit tests, the database user needs CREATEDB rights.

coverage run --source='.' ./psono/manage.py test restapi.tests administration.tests fileserver.tests

To get a nice report one can do:

coverage report --omit=psono/restapi/migrations/*,psono/restapi/tests*,psono/administration/migrations/*,psono/administration/tests*,psono/fileserver/migrations/*,psono/fileserver/tests*

or:

coverage html --omit=psono/restapi/migrations/*,psono/restapi/tests*,psono/administration/migrations/*,psono/administration/tests*,psono/fileserver/migrations/*,psono/fileserver/tests*

The output of this command can be shown on https://your-ip/htmlcov/

# Run local dummy smtp server

If you want to debug e-mails, like those sent during the registration, one can start a local dummy smtp server with the following command

sudo python -m smtpd -n -c DebuggingServer localhost:25