Skip to content

Datamodel

datamodel

XXX FILL ME

Instance level metadata

  • users
  • groups

Smallapp level metadata

name and id

Small app are identified by there name and an id. NB: By default, the id is the slugify version of the name

The id is used for: - the url of the small app is its id - collections related to the small app are prefix by the id For example, the collection for a domain is name: f"{small_app_id}".slide-data-{domain_name}

The id must not be longer than 30bytes - cf. restriction on domain name in Data section.

database collections

Each small app has its own db in mongo, named after the instance prefix and the small app id. It contains multiple collections, such as: - permissions - queries - etc. (to be completed)

This set of collection is duplicated for each stage: production and staging. For the production stage, collections have no suffix. For the staging stage, collections are suffixed by -staging.

When releasing from one stage to another, temporary collections are created for the copy (suffixed -temp). They are all renamed when the copy process and index creation (which can take some time) are finished.

These collections are handled either by the database_manager (legacy) or special MongoEngine classes.

As MongoEngine classes usually work with only one db in mongo, there is a small "trick" to construct a set of MongoEngine classes with a different database. All these classes are available in an object SmallAppDatabase, which can be constructed or retrieved by using laputa.models.mongo_connection.get_mall_app_db. An example of its usage can be found in tests/models/small_app_db/test_small_app_db.py:

from laputa.models.mongo_connection import get_small_app_db

# Access a production collection:
get_small_app_db(small_app_id).staging.queries

# Access its staging counterpart:
get_small_app_db(small_app_id).staging.queries

Another util, laputa.models.mongo_connection.get_db, can return either a SmallAppDatabase object (if a small app uid parameter is given) or a MainDatabase object (representing the default db). Example usage:

from laputa.models.mongo_connection import get_db

# Access instance-level email templates:
get_db().email_templates

# Access small app-level email_templates:
get_db(small_app_id).email_templates

Warning

There is no mechanism that ensures that all small apps have their MongoEngine classes instantiated at the same time. This means identity constraints configured in MongoEngine won't necessarily be applied from one small app database to another, or from the main database to a the small app one.

Example: if, in small app databases, a permission is linked to a UserGroup, defined in the main db. If the Permission collection has a cascasing delete rule on its UserGroup, deleting the UserGroup will not necessarily delete all the associated permissions across all small apps.

For now, just avoid using these kind of constraints for collections in different databases.

Small app data

  • slide data

domain name should not be longer than 70bytes.

From https://docs.mongodb.com/manual/reference/limits/#Restriction-on-Collection-Names mongo collection must be less than 120bytes. We name slide-data collection with the following convention: f"{small_app_id}".slide-data-{domain_name}-staging We chose the following repartition: 30 (small-app-id) + 1 (.) + 11 (slide_data-) + 70 (domain-name)+ 8 (-staging) = 120bytes

  • staging / public collections
  • configuration

Sensitive fields

To prevent leakage of secrets like passwords or API keys, it's best to encrypt them before storing then in MongoDB.

Symmetrical encryption

For fields of which the original value must be used, the EncryptedStringField can be used in MongoEngine models. It automatically encrypt values before passing them to the DB, and decrypt them on retrieval.

To use EncryptedStringField, one should provide a concrete Encryptor implementation. We have one in laputa.common.utils.encryptor, that uses AES-256 (LaputaEncryptor). The secret is read from the config.yml file, key db_encryption_secret.