Example of configuring scheduled backup

Purpose of customization⁠

Create a script that will regularly run the console client, archive and move important data to object storage.

What you need to customize⁠

console client (in the example S3cmd with crontab automation tool);
cloud or dedicated server with Ubuntu version 18.04 or higher installed;
user with access to object storage.

Result of customization⁠

The script will create a backup of the file or directory using tar and upload the backup to the object store using s3cmd.

Steps to customize⁠

Create script.
Transfer files to object storage.
Configure flow-control.
Check script.
Optional: automate backups via crontab or Cyberduck.

1. Create a script⁠

Open the home directory on your server:
```
cd ~
```
Use the nano editor to create an empty file (for example, named bkupscript):
```
nano bkupscript.sh
```
Start writing a backup script in a text editor with shebang:
```
#!/bin/bash
```
Shebang is an interpreter directive that allows scripts or data files to be run as commands and looks like a sequence of two characters: # and !. Thanks to the shebang at the beginning of the script, the shell runs the file's commands in bash.
Under shebang at the top of the text file, add variables to the script:
```
#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3
```
Here:
- DATETIME — the timestamp to attach to the received file name. Each file backed up in the space will have a unique name. This timestamp is created by calling the date command and formatting the output to display the last two digits of the year (% y), the last two digits of the month (% m), the last two digits of the day (% d), the hour (% H), the minute (% M), and the second (% S);
- SRC — the source path for the file or folder to which the backup is being made. $1 indicates that the value is taken from the first parameter passed to the script;
- DST is the destination of the file. In the example, this is the name of the space to which the backup is loaded. This name will be obtained from the second parameter passed to the script, as indicated by $2;
- GIVENNAME — user-selected name for the destination file. The resulting file name will start with GIVENNAME, and DATETIME will be appended to it. This name comes from the third parameter passed to the $3 script.

Add a showhelp function to the backup script to display messages if the script fails:

#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
}

Gather the files you need and combine them into a single package that can be downloaded:
```
#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
){

}

tarandzip(){
    echo "\n##### Gathering files #####\n"
    tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC
}
```
When the if instruction is called, the script executes the tar instruction and waits for the result. If the command is successful, the lines after the then statement will be executed:
- output a message that the tar process has successfully terminated;
- return error code 0 so that the part of the code calling this function knows that everything is working fine.
The else part of the script will only be executed if the tar command detects an execution error:
- output a message that the tar command was not executed;
- returning error code 1, indicating that something has gone wrong.

End the if/then/else script with the fi phrase. View of the completed tarandzip function:

tarandzip(){
    echo "\n##### Gathering files #####\n"
    if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
        echo "\n##### Done gathering files #####\n"
        return 0
    else
        echo "\n##### Failed to gather files #####\n"
        return 1
    fi
}

2. Transfer files to object storage⁠

Add a function to the backup script to transfer the movetoSpace file to the selected space. The previously declared variables are used to create a command that will place the backup files in the selected space:
```
movetoSpace(){
    /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
}
```
Here:
- /bin/s3cmd — calls s3cmd, a command line tool used to manage object store segments;
- put — used by s3cmd to load data into the container;
- $ GIVENNAME- $ DATETIME.tar.gz is the name of the backup that will be loaded into the space. It consists of the fourth and first declared variables followed by .tar.gz, and is created by the tarandzip function;
- s3: // $ DST ; — the location where the file will be uploaded;
- s3: // is a schema of type URI used to describe where objects are stored on the network, and $DST; is the third variable declared earlier.

Add a notification that the file transfer process has started:

movetoSpace(){
    echo "\n##### MOVING TO SPACE #####\n"
    /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
}

Customize the output of the message about the result of command execution:

if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
    echo "\n##### Done moving files to s3://"$DST" #####\n"
    return 0
else
    echo "\n##### Failed to move files to the space #####\n"
    return 1
fi

3. Customize flow control⁠

If the script is configured correctly, when it starts, it should read the input command, assign values from it to each variable, execute the tarandzip function, and then execute movetoSpace.

If the script fails at any step, it should print the output of the showhelp function to help troubleshoot the problem.

At the end of the file, add a conditional instruction if / then / else:

if [ ! ! -z "$GIVENNAME" ]; then
    if tarandzip; then
        movetoSpace
    else
        showhelp
    fi
else
    showhelp
fi

4. Check the script⁠

Check the script:

#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(){
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
}

tarandzip(){
    echo "\n##### Gathering files #####\n"
    if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
        echo "\n##### Done gathering files #####\n"
        return 0
    else
        echo "\n##### Failed to gather files #####\n"
        return 1
    fi
}
movetoSpace(){
    echo "\n##### MOVING TO SPACE #####\n"
    if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
        echo "\n##### Done moving files to s3://"$DST" #####\n"
        return 0
    else
        echo "\n##### Failed to move files to the Spac #####\n"
        return 1
    fi
}

if [ ! -z "$GIVENNAME" ]; then
    if tarandzip; then
        movetoSpace
    else
        showhelp
    fi
else
    showhelp
fi

Before exiting nano, close the file (Ctrl+X) and save your changes (Y+Enter).

5. Optional: automate backups⁠

Crontab
Cyberduck

Set up a cron job that will use a script to make regular backups to the selected space. For the purposes of this example, backups will be performed every minute.
Make the script executable:
```
chmod +x bkupscript.sh
```
Edit the crontab file so that the script runs every minute:
```
crontab -e
```

The first time you run the crontab -e command, you will be prompted to select an editor from the list:

no crontab for root - using an empty one
Select an editor.  To change later, run "select-editor".
1. /bin/ed
2. /bin/nano <---- easiest
3. /usr/bin/vim.basic
4. /usr/bin/vim.tiny
Choose 1-4 [2]:

Choose the default nano or another text editor.

In crontab, add a line at the end of the script:

* * * * * * * ~/bkupscript.sh ~/backupthis nameofyourspace cronupload

Close the file (Ctrl+X) and save your changes (Y+Enter).
If you leave the cron job running unchanged, a new file will be copied to the selected space every minute. If cron is running successfully, reconfigure crontab to back up files at the desired interval.

On the local machine, there is a directory whose contents need to be periodically copied to object storage. This can be done with a script and a Cron job that sends a backup to the repository every day at a specified time.

Example job (additionally you can use other Cyberduck options):

#!/bin/bash
SELECTEL_ID=<selectel_account>
USERNAME=<username>
PASSWORD=<password>
BACKUP_PATH=<container_name>/<prefix>
LOCAL_PATH=<path>

duck --upload selectel:/<container_name>/<prefix> <path> --existing rename --username <selectel_account>:<username> --password <password> -q -y

Specify:

<selectel_account> — Selectel Control Panel account number;
<username> — service user name;
<password> — service user password. If you forget your password, create a new one;
<container_name> is the name of the container;
<prefix> — object prefix;
<path> — the path to the local directory.

Here:

The --existing key of the duck command specifies what to do with files already in the repository;
The rename option renames an existing backup, adding the time and date to the name.

Cyberduck Options⁠

When automate backups via Cyberduck, you can use additional options:

compare — perform differential backups:
```
duck --upload selectel:/<container_name>/<prefix> <path> --existing compare --username <selectel_account>:<username> --password <password>
```
The uploaded backup will be compared to the existing backup by size, modification date, and checksum. If the parameters are different, the old version will be deleted and the new version will be uploaded to the repository;
skip — only new files (those that appeared in the folder on the local machine after the previous upload) will be uploaded to the storage. Already existing files will not be downloaded, even if they have been modified on the local machine;
overwrite — deletes an existing backup from the storage and loads a new one.

Example of configuring scheduled backup

Purpose of customization⁠​

What you need to customize⁠​

Result of customization⁠​

Steps to customize⁠​

1. Create a script⁠​

2. Transfer files to object storage⁠​

3. Customize flow control⁠​

4. Check the script⁠​

5. Optional: automate backups⁠​

Cyberduck Options⁠​