Throttle-Rate¶

We implemented a security system to limit the number of requests for an ip address on a running Toucan Toco instance.

Previously we were using fail2ban on our nginx configuration, but we moved that logic in the application code, we needed: - flask-limiter which is a dependency inside this project. - A redis instance for saving requests's clients, timings and the temporary blacklist.

Of course we added a feature flag for this security feature that you can found on config.yml

app_params:
    ...
    ....
    throttle_rate:
        status: enable
        limit: 5/minute

is_behind_proxy: enable

throttle_rate > status : can be enable or disable
throttle_rate > limit : is the number of requests allowed in the allowed amount of time (here it's 5 requests per minute(can be second, hour, day or month))
is_behind_proxy : is a parameter that will tell to the limiter, if the instance is running behind a proxy or not so that it could try to get the ip address from the header of the request

How does it works¶

The flask limiter is implemented in laputa/api/throttle_rate.py and we create an instance of the limiter with all required params coming from the configuration file (config.yml) and the custom callback get_custom_remote_addr() function that will extract from a request's header the real ip address forwarded by the nginx, using the key X-Forwarded-For .

def get_custom_remote_addr():
    """
    We need to get the real ip address from the headers sent by nginx
    """
    if config.is_behind_proxy():
        if 'X-Forwarded-For' in request.headers:
            forward_ip = request.headers['X-Forwarded-For']
            return forward_ip.split(",")[0]
    return request.remote_addr or '127.0.0.1'

The limiter object will be used as a decorator for all resources where we want the throttle_rate to work.

limiter = Limiter(
    app,
    key_func=get_custom_remote_addr,
    ...
)

How to use it¶

According to the limiter's package documentation, when we want to add our limiter object on a single route, we just have to do :

@app.route("/slow")
@limiter.limit("3 per day")
def slow():
    return ":)"

But because on laputa repo, we're dealing with Resource from flask_restful, we implement that by surcharging decorators of the resource, let say for example we want to add the throttle_rate on a custom resources, we're going to do like this:

from laputa.api.throttle_rate import get_limit_config, limiter
from flask_restful import Resource


class CustomResource(Resource):
    decorators = [limiter.limit(**get_limit_config())]

    def get(self):
        ...

    def post(self):
        ...

The get_limit_config is the method that will just return all configurations fetched from the config.yml such as :

{
    'limit_value': config.get_throttle_rate_limit(),
    'exempt_when': lambda: not config.get_throttle_rate_status(),
    'error_message': 'Too many requests in a short time, please wait a bit and try again.',
}