Thursday, September 1, 2016

How to Set Up an Ultimate Backup System, Using Duplicity, Systemd and Thunar on Arch Linux Part Two

 

How to Automate Script Calls Using Systemd:

How to Write Systemd Service Units:

Now that we have a working script for performing a system backup, we want to automate it. To get familiarized with systemd I would recommend to read through the wiki pages of arch Linux they are pretty great [1], [2], [3]. The back_sys_up.sh is responsible for backing up the system, independent of any users. That is why we want it to run whenever the machine is powered, and not depend on whether the user is logged in or not. To do so we need to create our service for calling the backup script in  /etc/systemd/system folder:
cd  /etc/systemd/systemsudo 
sudo vim duplicity-system-remote-backup.service
This file must contain the following content:

It is also recommended to find out which services are responsible for the network connection on your machine and add the requirements in this file at the bottom of the [Unit] section:
Requires=your_network_connection.service 
After=yout_network_connection.service
This will ensure that your machine is connected to the network before running the backup service. The [Unit] section contains all of the parameters responsible for the systemd unit, and it is universal for all the systemd units. In our case we just have a short description of what the unit does and its requirements. The [Service] section is unique for systemd service units. We have only two parameters here. Type specifies the type of the service in our case oneshot, which means that the service will stop upon the completion of the script. Since we are going to be using a timer to activate the script every time this is a desired behavior.  The second parameter is the ExecStart, which specifies the script to be executed.

How to Write Systemd Timer Units:

Now lets configure the timer for our service. To do so execute the following command:
sudo vim duplicity-system-remote-backup.timer
It is important that the timer and the service files have the same name. The content of the timer should be as follows:

Here there are two important parameters. OnBootSec specifies the time that should pass after boot in order for the service to start, in this case 5 minutes. OnUnitActiveSec specifies the time that should pass from the last time the unit was active in order to start the service, in this case 1 hour. Now it is time to enable the timer and do some testing. Run the following command to enable the timer:
sudo systemctl enable duplicity-system-remote-backup.timer
In order to test you need to restart. After reboot open two terminal windows. In one of them type:
 journalctl -f
Leave this terminal visible, here you can monitor all the systemd activities. In the second terminal you can type:
systemctl list-timers 
 To see which timers are currently present, how much time is left until they execute, when was the list time they have been active, etc.

Housekeeping:

Since full backups can take a long time on slow networks, and since there is no guaranty that your machine will stay up for the duration of this time, duplicity allows the restarting of interrupted backup processes from the point of interruption. To do so duplicity divides the bulk of your files into blocks and uploads them block by block. Every time a block is being uploaded a lock file is being created by duplicity. This can cause issues when duplicity is unexpectedly interrupted, i.e. the machine is turned off while duplicity is performing a backup, or something along this lines. The problem is that after reboot the lock file is still in place, and duplicity will refuse to perform backups if there is a lock file from another instance. To overcome this issue a small script must be written that will be activated every time after boot, and that will look for the lock files and delete them before duplicity is activated. To do so create the following file:
sudo vim /root/scripts/system_backup/remove_locks.sh
and fill it in as follows:

In order to save time I took a shortcut and also added the remove functionality for the my_user in to this script as well. By default the lock files are stored in ~/.cache/duplicity folder. If you did not explicitly specify a different folder this script should do the trick. To automate this process we need to create the systemd service and timer units for this script. To do so create duplicity-cleanup-onboot.service  duplicity-cleanup-onboot.timer  files in /etc/systemd/system/ folder, with the following content:

Now you should have s fully functioning system backup using duplicity. In the next part we will describe the creation of the back-end as well as the front-end for the backup system of the user.

No comments:

Post a Comment