Tuesday, August 04, 2020

Enable automated backups for Nextcloud - 4 backups

Once I had the entire nextcloud setup with mobile client sync, It was important to have the backup process setup before I add more users/files. This makes the installation more reliable as a personal data storage. This allows a point in time recovery for all the data in your nextcloud.

The target is to setup 3-2-1 backup rule for my nextcloud instance at a minimum. if you are not familiar with this general backup guideline I suggest you read more on it before you proceed.
  • 3 copies of production data
  • 2 media types
    • Elastic File System (non-versioned)
    • Object Storage (versioned)
  • 1 offline copy (versioned) - setting up the offline copy isnt covered in this page

There are following options to choose from when deciding how to use the object storage as the versioned backup solution:
  1. Setup object storage as filesystem (s3fs)
    1. (pro) get a point in time copy of data that isn't cloud provider dependent
    • (pro) allows the backup shell script to take care of object rotation. 
    • (con) unnecessarily expose all backup data in the filesystem.
    • (con) makes it harder to use the object storage from the context of another client application
  2. Use s3cmd for uploading the backup
    1. (pro) can use the S3cmd sync for storage/filesystem backup
    2. (con) need to manage lifecycle/versions of data outside the backup script
  3. Use cloud provider native snapshots
    1. (pro) use provider specific APIs to take snapshots of compute and block storage
    2. (pro) easiest/fastest recovery option
    3. (con) snapshots arent useful outside the cloud provider
I prefer option2 as of now, and given theres flexibility to change the backup design as the service scales I don't see the need to over-engineer it as of now. In near future I will have both option2 and option3 running in parallel. 

Heres the steps I followed:
  1. Setup the DigitalOcean Space and get a managed key
  2. Install and Configure S3cmd
    1. https://www.digitalocean.com/docs/spaces/resources/s3cmd/
    2. sudo apt-get update
    3. sudo apt-cache madison s3cmd (make sure the version is above 2)
    4. sudo apt-get install s3cmd -y
    5. sudo s3cmd --configure (go through the interactive process)
  3. Create Backup folder
    1. sudo mkdir -p /media/nextcloud_data/backups
    2. sudo mkdir -p /media/nextcloud_data/backups/logs
    3. sudo mkdir -p /media/nextcloud_data/backups/tmp
  4. Create backup script
    1. sudo vim /usr/sbin/nextcloudbackup.sh
    2. Todo: get script from github
    3. sudo chmod +x /usr/sbin/nextcloudbackup.sh
    4. sudo /usr/sbin/nextcloudbackup.sh
  5. Cron setup
    1. sudo visudo
    2. opuser ALL=(ALL) NOPASSWD: /usr/sbin/nextcloudbackup.sh
    3. sudo crontab -u <username> -e
    4. 0 2 * * * sudo /usr/sbin/nextcloudbackup.sh
  6. Backup pod and volume

Reading material:

No comments: