Backup To AWS S3 Deep Archive

Backup to AWS S3 with storage class Deep Archive is a cheap solution for a disaster backup. The following explanations and configuration is based Ubuntu 20.04.6 LTS and duplicity 1.2.2. Duplicity is installed via snap.

The script is split in two parts. The secret values like access keys are stored in a .env file. With this, no credentials and keys needs to be stored in the script.

# .env file
AWS_ACCESS_KEY_ID="ENTER-YOUR-AWS_ACCESS_KEY_ID"
AWS_SECRET_ACCESS_KEY="ENTER-YOUR-AWS_SECRET_ACCESS_KEY"
GPG_KEYID="GPG-KEY-ID"

BUCKET_NAME="AWS-BUCKET-NAME"

The variable AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY defines the access values for AWS S3, GPG_KEYID is used to encrypt the backup with a GPG Key. Everything is stored in the bucket, configured in BUCKET_NAME.

 1| #!/bin/bash
 2| 
 3| if [ $(whoami) != "root" ]; then
 4|    echo "please execute as root user"
 5|    exit 5
 6| fi
 7| 
 8| AWS_ACCESS_KEY_ID="<set me in .env file>"
 9| AWS_SECRET_ACCESS_KEY="<set me in .env file>"
10| GPG_KEYID="<set me in .env file>"
11| BUCKET_NAME="<set me in .env file>"
12| export $(cat /absolute/path/to/.env | xargs)
13| 
14| LOGFOLDER="/logfolder/dupli_logs"q
15| 
16| DATESTR=`date +%Y-%m-%d_%H%M`
17| 
18| backup_to_duplicity() {
19| 
20|    SOURCE_FOLDER=${1}
21|    DEST_FOLDER=${2}
22| 
23|    LOG_ERR="${LOGFOLDER}/dupli.${DEST_FOLDER}.${DATESTR}.err.log"
24|    LOG_STD="${LOGFOLDER}/dupli.${DEST_FOLDER}.${DATESTR}.std.log"
25| 
26| 
27|    echo "BACKUP ${SOURCE_FOLDER}" | tee -a ${LOG_ERR} ${LOG_STD}
28|    date | tee -a ${LOG_ERR} ${LOG_STD}
29| 
30| 
31|    dupl_cmd="/snap/bin/duplicity --full-if-older-than 180D \
32|       --encrypt-key=${GPG_KEYID} \
33|       --progress \
34|       --s3-use-deep-archive \
35|       --s3-use-multiprocessing \
36|       --volsize 700 \
37|       ${SOURCE_FOLDER} \
38|       boto+s3://${BUCKET_NAME}/${DEST_FOLDER}/ \
39|       2>> ${LOG_ERR} 1>> ${LOG_STD}"
40| 
41|    echo "${dupl_cmd}"
42|    bash -c "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} ${dupl_cmd}"
43| 
44| }
45| 
46| 
47| 
48| backup_to_duplicity "/media/my/mounted/folder" "FolderInAWS"
49| 
50| 
51| unset AWS_ACCESS_KEY_ID
52| unset AWS_SECRET_ACCESS_KEY