Testing Laputa¶

Running tests¶

Default configuration parameters can be found in tests/common/parameters:

tests/common/parameters/
├── auth.json
├── config.yml
├── default_email_templates.yml
└── default_settings.yml

Note: by default, the database configuration (for both redis and mongo) is meaningless since they are set dynamically: a mongo and a redis containers are started automatically at the beginning of the test session and the configuration will be adjusted accordingly to match the random ports exposed by the containers.

Tests can be run with pytest directly:

pytest

If you want to use your own redis or mongo databases, or if you want to run your tests inside a container (as jenkins does for instance), you should use the --system-databases pytest option: in that case, pytest will use the database definition found in tests/common/parameters/config.yml.

Writing tests¶

Helpers¶

There are quite a few cases where you want to test an object structure but you can't test for strict equality because there are a few random or unpredictable values in it. This is exactly what the assertlib module in the tests/ directory is for. It provides "check-ables" that can be used to test structures.

A check-able is an instance of an assertlib.AbstractCheckable. It can be used to test a data structure using either its .match method, or just by testing equality:

from tests.assertlib import InstanceOf, OfLength

assert 12 == InstanceOf(int)
assert [1, 2] == OfLength(2)
assert [1, 2] != OfLength(3)

These check-ables can be logically combined using | and & operators:

from tests.assertlib import InstanceOf, OfLength, Almost

assert [1, 2] == InstanceOf(list) & OfLength(2)
assert 12.00000001 == Almost(21, 1) | Almost(12, 0.1)

The ListContaining and DictContaining check-ables are quite useful to test nested structures, when only some properties are of interest. They can be combined recursively if needed, and with other check-ables:

from tests.assertlib import DictContaining, ListContaining, InstanceOf, OfLength

assert DictContaining({
  'name': 'Delta',
  'order': InstanceOf(int),
}) == {
  'name': 'Delta',
  'order': 4,
  'uid': '123-456-789',
  'last_update': '2022-02-11'
}

assert ListContaining([
  DictContaining({ 'name': 'Delta' }),
  DictContaining({ 'name': 'Epsilon' }),
]) & OfLength(6) == [
  { 'name': 'Alpha', 'char': 'α'},
  { 'name': 'Beta', 'char': 'β'},
  { 'name': 'Gamma', 'char': 'γ'},
  { 'name': 'Delta', 'char': 'δ'},
  { 'name': 'Epsilon', 'char': 'ε'},
  { 'name': 'Zeta', 'char': 'ζ'},
]

The package provides two other helpers with a similar function:

is_seq_like(tested, model) to compare sequence structures,
is_dict_like(tested, model) to compare dict structures.

The model can use also any combination of check-ables:

from tests.assertlib import is_dict_like, InstanceOf, Optional

assert is_dict_like({'a': 12, 'b': b'foo', 'c': True}, {
    'a': 12,
    'b': InstanceOf(str) | InstanceOf(bytes),
    'c': True,
    'd': Optional(str)
})

The main difference between is_dict_like and DictContaining is that the first one will fail if there is any extra fields not defined in the model (hence the frequent use of Any or Optional). The same idea applies to is_seq_like and ListContaining: the first will expect the model and the tested structure to have the same length.

Using config fixtures¶

In tests, we use a custom TestConfig class that extends the standard laputa.common.config.Config one. This class is just a small extension to the base class that adds:

helper methods to add users or smallapps dynamically to the config,
a custom filesystem storage handler to make sure we never clutter the default template directory in tests/storage.

Using the "global" config fixture¶

config is a session-scoped fixture that creates a TestConfig instance with default parameters found in tests/common/parameters. If you want to perform API queries on it, you should use the testutil.client_session_from_config helper that accepts 2 parameters:

config: the TestConfig instance,
username: the username the session should use to make api calls.

WARNING Since it's a session-scoped configuration, it means that possible changes pushed on the underlying database will impact subsequent tests. You should avoid using it as much as possible unless performing read-only calls. Consider the local_config fixture instead.

Example¶

from ..testutil import client_session_from_config

def test_get_users_list_with_superadmin(config):
    """It should return the users list with superadmin if possible"""
    with client_session_from_config(config, username='appadmin') as api_client:
        res = api_client.get('/users?superadmin=true')
        assert res.status_code == 200
        assert 'appsuperuser' not in [user['username'] for user in res.json]

    with client_session_from_config(config, username='appsuperuser') as api_client:
        res = api_client.get('/users?superadmin=true')
        assert res.status_code == 200
        assert 'appsuperuser' in [user['username'] for user in res.json]

Using the "local_config" fixture¶

This is an alternative to the config fixture. It behaves exactly as config BUT runs on a local database created specifically for a single test. Therefore, changes made to the database are local to this test.

Example¶

from ..testutil import client_session_from_config

def test_get_users_list_with_superadmin(local_config):
    """It should return the users list with superadmin if possible"""
    with client_session_from_config(local_config, username='appadmin') as api_client:
        api_client.put('/laputa/url', json={'url': 'test_url'})

Switching between multiple users¶

Do not use client_session_from_config multiple times. Instead, create only one flask app from the config, and create different sessions:

from tests.testutil import bound_flaskapp, client_session

def test_get(local_config):
    with bound_flaskapp(local_config) as flaskapp:
        with client_session(flaskapp, username='rick') as api_client:
            api_client.get('/') # call using rick user

        with client_session(flaskapp, username='appuser') as api_client:
            api_client.get('/') # call using morty user

Using the "config_factory" fixture¶

When you don't want to load the full configuration defined in tests/common/parameters because you don't want to remember which users or smallapps preexist, you should use this fixture that will create an "empty" test configuration. "Empty" actually means without smallapp and only with a appsuperuser super user. All the other configuration parameters (e.g. email templates) are loaded. However, would you wish to override them, you could pass custom parameters to the factory.

As for the local_config fixture, configurations created with this factory will be bound to a database temporarily created for the test it is used on.

Example¶

from ..testutil import client_session_from_config

def test_get_privileges_list(config_factory):
    """It should return the privileges list"""
    config = config_factory().add_user('u1@userland.fr', role='USER')
    with client_session_from_config(config, username='u1@userland.fr') as api_client:
        res = api_client.get('/privileges')
        assert res.status_code == 200
        assert res.json == ['none', 'view', 'validate', 'contribute']

Caveats¶

If you are used to the old API and use data= parameters in your API calls expecting that the api_client would do the magic by itself to set the content-type to application/json, this is bad! If you do so with the new API, you'll likely get an error such as:

  TypeError: add_file() got an unexpected keyword argument 'username'

In that case, just use json= instead of data= and it should be good.

Example of bad API call:

    res = api_client.post('/users', data={'attributes': user})

You should instead do:

    res = api_client.post('/users', json={'attributes': user})