In my previous post I showed a simple NetBox webhook listener using FastAPI. In this post I’ll show the configurations to run the API using systemd. Let’s get into it, on my Debian 10 system.
sudo apt install nginx python3-venv curl mkdir webhook cd webhook python3 -m venv venv . venv/bin/activate pip install -U pip wheel pip install fastapi gunicorn uvicorn uvloop httptools
Let’s save the webhook_listener.py from the previous post in the same folder (webhook).
Now we can try starting the webhook listener:
(venv) markku@btest:~/webhook$ gunicorn -w 2 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 webhook_listener:app [2020-11-22 19:15:18 +0200] [30175] [INFO] Starting gunicorn 20.0.4 [2020-11-22 19:15:18 +0200] [30175] [INFO] Listening at: http://0.0.0.0:8000 (30175) [2020-11-22 19:15:18 +0200] [30175] [INFO] Using worker: uvicorn.workers.UvicornWorker [2020-11-22 19:15:18 +0200] [30178] [INFO] Booting worker with pid: 30178 [2020-11-22 19:15:18 +0200] [30179] [INFO] Booting worker with pid: 30179 [2020-11-22 19:15:18 +0200] [30178] [INFO] Started server process [30178] [2020-11-22 19:15:18 +0200] [30178] [INFO] Waiting for application startup. [2020-11-22 19:15:18 +0200] [30178] [INFO] Application startup complete. [2020-11-22 19:15:18 +0200] [30179] [INFO] Started server process [30179] [2020-11-22 19:15:18 +0200] [30179] [INFO] Waiting for application startup. [2020-11-22 19:15:18 +0200] [30179] [INFO] Application startup complete.
It started two workers successfully. In my system TCP port 8000 was not yet listening for anything else so that worked fine.
Ctrl-C shuts down the app, and you can use deactivate to exit the venv.
Now let’s create the systemd service file, /etc/systemd/system/webhook_listener.service:
[Unit] Description=Webhook listener by Markku After=network.target [Service] User=markku Group=markku WorkingDirectory=/home/markku/webhook Environment="PATH=/home/markku/webhook/venv/bin" ExecStart=/home/markku/webhook/venv/bin/gunicorn --workers 3 --worker-class uvicorn.workers.UvicornWorker --bind 127.0.0.1:8000 webhook_listener:app [Install] WantedBy=multi-user.target
Then create the configuration file for NGINX, /etc/nginx/sites-available/webhook_listener:
server {
listen 80;
server_name btest.example.com;
location / {
include proxy_params;
proxy_pass http://127.0.0.1:8000;
}
}
Final setup for the files and settings:
sudo ln -s /etc/nginx/sites-available/webhook_listener /etc/nginx/sites-enabled/ sudo nginx -t sudo systemctl daemon-reload sudo systemctl start webhook_listener sudo systemctl enable webhook_listener sudo systemctl restart nginx
Let’s test our setup:
markku@btest:~$ curl http://btest.example.com/webhook/
{"detail":"Method Not Allowed"}markku@btest:~$
This is normal as curl issued a GET request and our API only implemented POST handler. So let’s try that:
markku@btest:~$ curl http://btest.example.com/webhook/ -X POST
{"detail":[{"loc":["header","content-length"],"msg":"field required","type":"value_error.missing"},{"loc":["body"],"msg":"field required","type":"value_error.missing"}]}markku@btest:~$
Let’s use jq to make it more readable:
markku@btest:~$ sudo apt install jq
...
markku@btest:~$ curl http://btest.example.com/webhook/ -X POST -s | jq .
{
"detail": [
{
"loc": [
"header",
"content-length"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body"
],
"msg": "field required",
"type": "value_error.missing"
}
]
}
markku@btest:~$
Basically it reported that we didn’t send anything. Let’s try with partially correct NetBox webhook request (long command split for clarity):
markku@btest:~$ curl http://btest.example.com/webhook/ -X POST -s
-d '{"username": "markku", "event": "created", "model": "device"}' | jq .
{
"detail": [
{
"loc": [
"body",
"data"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body",
"timestamp"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body",
"request_id"
],
"msg": "field required",
"type": "value_error.missing"
}
]
}
markku@btest:~$
Note that the reported missing values correspond to our WebhookData class: we sent username, event and model fields, but data, timestamp and request_id were still missing. Let’s add them:
markku@btest:~$ curl http://btest.example.com/webhook/ -X POST -s
-d '{"username": "markku", "event": "created", "model": "device",
"timestamp": "2020-01-01 00:00:00",
"request_id": "12345678-1234-1234-1234-123456789012",
"data": {"url": "/api/xxx"}}' | jq .
{
"result": "ok"
}
markku@btest:~$
It worked! And the results can be shown in webhook-handler.log:
2020-11-22 20:06:13,707 webhook-listener INFO: No message signature to check
2020-11-22 20:06:13,707 webhook-listener INFO: WebhookData received:
2020-11-22 20:06:13,707 webhook-listener INFO: Raw data: username='markku' data={'url': '/api/xxx'} event='created' timestamp=datetime.datetime(2020, 1, 1, 0, 0) model='device' request_id=UUID('12345678-1234-1234-1234-123456789012')
2020-11-22 20:06:13,707 webhook-listener INFO: Request ID: 12345678-1234-1234-1234-123456789012
2020-11-22 20:06:13,707 webhook-listener INFO: Username: markku
2020-11-22 20:06:13,708 webhook-listener INFO: Event: created
2020-11-22 20:06:13,708 webhook-listener INFO: Timestamp: 2020-01-01 00:00:00
2020-11-22 20:06:13,708 webhook-listener INFO: Model: device
2020-11-22 20:06:13,708 webhook-listener INFO: Data: {'url': '/api/xxx'}
2020-11-22 20:06:13,708 webhook-listener INFO: URL in data: /api/xxx
In case of errors, sudo journalctl -e is the command to use first.
For the record, these were the package versions installed by the commands in this demo:
(venv) markku@btest:~/webhook$ pip list Package Version ----------------- ------- click 7.1.2 fastapi 0.61.2 gunicorn 20.0.4 h11 0.11.0 httptools 0.1.1 pip 20.2.4 pkg-resources 0.0.0 pydantic 1.7.2 setuptools 40.8.0 starlette 0.13.6 typing-extensions 3.7.4.3 uvicorn 0.12.3 uvloop 0.14.0 wheel 0.35.1