запуск python скрипта через systemd
ilyasst / python_systemd.md
Run a python script forever
In this section, we are going to show how to run a python script as a systemd service, allowing you to boot it at the start of a Linux machine and to maintain it alive.
Test method: Telegram Bot
We are going to use a very basic Telegram bot in order to test that our script will:
/Temp, you should already have venv we installed it at the beginning.
Let’s also update pip, and install a pip we need.
Let’s now make this script into a systemd service.
Important
All the paths in your scripts have to be absolute paths, there can be no relative path in your scripts. If there are relative paths that you must keep, you will have to change your current working directory by retrieving
Note that ExecStart is the path to the Python file directly if you made it an executable using the right virtual environment. If you did not, then you have to specify a python binary to execute it.
We have to add an ExecStartPre delay otherwise the service keeps tries to start before internet is even available and we get this error:
Use Ctrl X + Y to save and exit when you finished editing.
If you try ths script (in your Virtual Environment not as a service), you will see that the script will return the /help command, but it will simply crash if you try to run the /kill command which tries to print a variable a that was never defined. Because python sees each telegram bot function as a separate function, it does not check that all variables exist before, as a variable can be defined with an incoming Telegram message.
In order to delete the service:
sudo systemctl disable dummy.service
Freqtrade on Dietpi
UPDATE (March 2020)
In addition, the DefaultStrategy was removed from freqtrade. You will thus get an error if you try to run it, use BBRSI instead.
Use Ctrl+X then Y followed by Enter to save the new serviced.
Enable the service sudo systemctl enable freqtrade.service
Reboot the Raspberry pi: sudo reboot now
UPDATE (March 2020)
In addition, the DefaultStrategy was removed from freqtrade. You will thus get an error if you try to run it, use BBRSI instead.
Use Ctrl+X then Y followed by Enter to save the new serviced.
Setup a python script as a service through systemctl/systemd
I’ll be setting this up on an Ubuntu 18.10 machine.
Almost all versions of Linux come with systemd out of the box, but if your’s didn’t come with it then you can simply run the following command:
To check which version of systemd you have simply run the command:
Create a python file whatever you like. I’m going to call mine test.py.
The above script will write the current timestamp in the file after every 10 seconds. Let’s write the service now.
sudo nano /etc/systemd/system/test.service (name of the service which is test in this case)
Insert the username in your OS where is written. The ExecStart flag takes in the command that you want to run. So basically the first argument is the python path (in my case it’s python3) and the second argument is the path to the script that needs to be executed. Restart flag is set to always because I want to restart my service if the server gets restarted. For more information on this, you can go to this link. Now we need to reload the daemon.
Let’s enable our service so that it doesn’t get disabled if the server restarts.
And now let’ start our service.
Now our service is up and running.
Note: The file will be written in the root directory (/) because the program will write in the path from the perspective of systemd. To change that simply edit out the file path. For example:
There are several commands you can do to start, stop, restart, and check status.
To stop the service.
This was a very basic introduction to systemd aimed at beginners who want to get started with writing their own systemd services for python. If you want a deep dive into systemd and systemctl, here is a detailed guide by the digital ocean.
NOTE: This doesn’t only apply to python scripts. You can basically run any program with this regardless of the programming language your program is written in.
Systemd за пять минут
Наша компания занимается администрированием веб-серверов на базе CentOS. Довольно часто наши клиенты используют веб-приложения на базе python, ruby или java. Для автозапуска подобных приложений есть готовые шаблоны для написания стартап-скриптов. Но прогресс не стоит на месте, вышел уже второй релиз CentOS 7 и, следуя старой традиции «не ставить dot-zero релизы на продакшен», мы начинаем предлагать клиентам сервера на базе CentOS 7.1 (1503).
В CentOS7, так же как и в его родителе RHEL7, используется systemd — менеджер системы и служб для Linux, совместимый со скриптами инициализации SysV и LSB. systemd обеспечивает возможности агрессивной параллелизации и много всего прочего.
Огромный монстр с множеством возможностей, гибкими настройками и мегабайтами документации…
Но что делать, если стоит задача быстро-быстро, вот прямо вчера, сделать автозапуск некоего сервиса?
Давайте выжмем из документации минимально необходимый набор информации для создания простых старт-стоп скриптов.
Systemd запускает сервисы описанные в его конфигурации.
Конфигурация состоит из множества файлов, которые по-модному называют юнитами.
Все эти юниты разложены в трех каталогах:
/usr/lib/systemd/system/ – юниты из установленных пакетов RPM — всякие nginx, apache, mysql и прочее
/run/systemd/system/ — юниты, созданные в рантайме — тоже, наверное, нужная штука
/etc/systemd/system/ — юниты, созданные системным администратором — а вот сюда мы и положим свой юнит.
[Название секции в квадратных скобках]
имя_переменной = значение
Для создания простейшего юнита надо описать три секции: [Unit], [Service], [Install]
В секции Unit описываем, что это за юнит:
Названия переменных достаточно говорящие:
Далее следует блок переменных, которые влияют на порядок загрузки сервисов:
Запускать юнит после какого-либо сервиса или группы сервисов (например network.target):
After=syslog.target
After=network.target
After=nginx.service
After=mysql.service
В итоге переменная Wants получается чисто описательной.
Если сервис есть в Requires, но нет в After, то наш сервис будет запущен параллельно с требуемым сервисом, а не после успешной загрузки требуемого сервиса
В секции Service указываем какими командами и под каким пользователем надо запускать сервис:
(по умолчанию): systemd предполагает, что служба будет запущена незамедлительно. Процесс при этом не должен разветвляться. Не используйте этот тип, если другие службы зависят от очередности при запуске данной службы.
systemd предполагает, что служба запускается однократно и процесс разветвляется с завершением родительского процесса. Данный тип используется для запуска классических демонов.
Также следует определить PIDFile=, чтобы systemd могла отслеживать основной процесс:
Команды на старт/стоп и релоад сервиса
Тут есть тонкость — systemd настаивает, чтобы команда указывала на конкретный исполняемый файл. Надо указывать полный путь.
Таймаут в секундах, сколько ждать system отработки старт/стоп команд.
Попросим systemd автоматически рестартовать наш сервис, если он вдруг перестанет работать.
Контроль ведется по наличию процесса из PID файла
В секции [Install] опишем, в каком уровне запуска должен стартовать сервис
multi-user.target или runlevel3.target соответствует нашему привычному runlevel=3 «Многопользовательский режим без графики. Пользователи, как правило, входят в систему при помощи множества консолей или через сеть»
Вот и готов простейший стартап скрипт, он же unit для systemd:
myunit.service
Кладем этот файл в каталог /etc/systemd/system/
Смотрим его статус systemctl status myunit
Если нет никаких ошибок в юните — то вывод будет вот такой:
Запуск python скрипта через systemd
Writing a systemd Service in Python
Many Linux distributions use systemd to manage the system’s services (or daemons), for example to automatically start certain services in the correct order when the system boots.
Writing a systemd service in Python turns out to be easy, but the complexity of systemd can be daunting at first. This tutorial is intended to get you started.
When you feel lost or need the gritty details, head over to the systemd documentation, which is pretty extensive. However, the docs are distributed over several pages, and finding what you’re looking for isn’t always easy. A good place to look up a particular systemd detail is systemd.directives, which lists all the configuration options, command line parameters, etc., and links to their documentation.
Aside from this README.md file, this repository contains a basic implementation of a Python service consisting of a Python script ( python_demo_service.py ) and a systemd unit file ( python_demo_service.service ).
System and User Services
systemd supports both system and user services. System services run in the system’s own systemd instance and provide functionalities for the whole system and all users. User services, on the other hand, run in a separate systemd instance tied to a specific user.
Even if your goal is to develop a system service it is a good idea to start with a user service, because it allows you to focus on getting the service up and running before dealing with the complexities of setting up a system service. Most of this tutorial targets user services, but there’s a section at the end on how to go from a user service to a system service once you’re ready.
Creating a User Service
To create a systemd service you need to create a corresponding unit file, which is a plain-text, ini-style configuration file. For this tutorial we will use a simple self-contained unit file, see systemd.unit for advanced approaches.
Unit files for user services can be put in several places. Some of these require root access, but there are multiple possible places in your home directory. As far as I can tell, there is no established default choice for these, so for this tutorial we are going to use
Therefore, store the following unit description as
Once you have done this, systemd will find our service:
The unit options for systemd services are documented in systemd.service.
Connecting the Service to a Python Script
We can now start to write the actual Python code for the service. Let’s start small with a script that simply prints a message every 5 seconds. Store the following script as python_demo_service.py in a directory of your choice:
To link our service to our script, extend the unit file as follows:
Manually Starting and Stopping the Service
Now our service can be started:
Depending on your systemd version, you may need to reload the user daemon so that our service can be found and started.
Note that this command returns immediately. This is because systemd has created a separate process that runs our script. This means that we don’t have to care about the nasty details of correctly forking into a daemon process ourselves, since systemd does all the work for us. Yay!
We can check that our service is running:
In the first line of the output we can see the Description from our unit file. The output also tells us the state of our service and the PID it is running as.
Obviously our service can also be stopped:
You might have noticed that the output of our script’s print calls did not show up on your terminal. This is because systemd detached the service process from that terminal and also redirected the process’s STDOUT and STDERR streams.
One thing to remember is that in Python, STDOUT and STDERR are buffered. When running in a terminal, this means that output will only show up after a newline ( \n ) has been written. However, our service’s STDOUT and STDERR are pipes, and in this case the buffer is only flushed once it is full. Hence the script’s messages only turn up in systemd’s logs after it has produced even more output.
To avoid this effect we need to disable the buffering of STDOUT and STDERR, and one possibility to do so is to set the PYTHONUNBUFFERED environment variable. This can be done directly in our unit file by adding the following line to the [Service] section:
As always when you change your unit file you need to tell systemd to reload its configuration, and (if your service is currently running), restart the service:
The output from our script should now show up in systemd’s logs, which by default are redirected to syslog:
Another way to display your service’s output is via
There are many more possible configurations for logging. For example, you can redirect STDOUT and STDERR to files instead. See systemd.exec for details.
Automatically Starting the Service during Boot
Many services are intended to be started automatically when the system boots. This is easy to achieve using systemd. First we need to attach our service to a suitable target: targets are special systemd units that are used for grouping other units and for synchronization during startup. See systemd.target for details about targets in general and systemd.special for a list of built-in targets.
For user services, the default.target is usually a good choice. Add the following to your unit file:
Our service is now ready to be started automatically, but for that to actually happen we have to enable the service first:
If you restart your system now then our service will be started automatically once you log in. After your last session is closed, your user’s systemd instance (and with it, our service) will shutdown. You can make your user’s systemd instance independent from your user’s sessions (so that our service starts at boot time even if you don’t log in and also keeps running until a shutdown/reboot) via
To disable autostart, simply disable your service:
Note that simply enabling a service does not start it, but only activates autostart during boot-up. Similarly, disabling a service doesn’t stop it, but only deactivates autostart during boot-up. If you want to start/stop the service immediately then you still need to do that manually as described above in addition to enabling/disabling the service.
To check whether your service is enabled, use
Automatically Restarting the Service after Failure
As with any other software, your service might crash. In that case, systemd can automatically try to restart it. By default, systemd will not do that, so you have to enable this functionality in your unit file.
systemd has several options to precisely configure under which circumstances your service should be restarted. A good starting point is to set Restart=on-failure in the [Service] section of your unit file:
We can simulate a crash by killing our service using the SIGKILL signal:
Afterwards, the logs will show that systemd restarted our service:
Notifying systemd when the Service is Ready
Often, a service needs to perform some initializiation before it is ready to perform its actual work. Your service can notify systemd once it has completed its initialization. This is particularly useful when other services depend on your service, since it allows systemd to delay starting these until your service is really ready.
The notification is done using the sd_notify system call. We’ll use the python-systemd package to execute it, so make sure it is installed. Then add the following lines to our script:
You can then see the notification in action by (re-)starting the service: systemctl will wait for the service’s notification before returning.
You can do a lot more via sd_notify, see its documentation for details.
Creating a System Service
Once you have a working user service you can turn it into a system service. Remember, however, that system services run in the system’s central systemd instance and have a greater potential for disturbing your system’s stability or security when not implemented correctly. In many cases, this step isn’t really necessary and a user service will do just fine.
Stopping and Disabling the User Service
Before turning our service into a system service let’s make sure that its stopped and disabled. Otherwise we might end up with both a user service and a system service.
Moving the Unit File
Previously, we stored our unit file in a directory appropriate for user services (
/.config/systemd/user/ ). As with user unit files, systemd looks into more than one directory for system unit files. We’ll be using /etc/systemd/system/ ‘, so move your unit file there and make sure that it has the right permissions
Moving the Python Script
Until now you have probably stored the service’s Python script somewhere in your home directory. That was fine for a user service, but isn’t optimal for a system service. A separate subdirectory in /usr/local/lib is a better choice:
Obviously we also need to change the script’s location in our unit file: update the ExecStart=. line to
Using a Dedicated Service User
A good choice for the name of the service user is the name of the service. To create the user we will use the useradd command:
Once you have created the user, add the following line to the [Service] section of your unit file:
After reloading the systemd configuration restarting our service, we can check that it runs as the correct user:
Where to go from here
We now have a basic implementation of a system systemd service in Python. Depending on your goal, there are many ways to go forward. Here are some ideas:
And of course, if you find an error in this tutorial or have an addition, feel free to create an issue or a pull request.