We can all agree that taking backups is important. We also agree that backups on the same medium as the backed up content are as good as no backup at all. This has led a lot of us to store our backups to the cloud, predominantly on cheap Amazon S3 storage. But how can we make sure that should the content server be compromised our backups will not be abused or deleted?
The plausible threat
I will approach this from the perspective of web site backups but this applies to any network accessible server that stores backups to Amazon S3. Should the server be compromised by a hacker they will be able to recover your Amazon S3 access and secret keys. Armed with these they can download your backups, replace them with compromised copies or delete them. If you have made the mistake of using the default, full access credentials they will be able to create very expensive GPU instances on Amazon EC2, charging you thousands of dollars until you notice something is up.
The easiest way to avoid this nightmare scenarios is using write-only Amazon S3 credentials. This is a feature offered by Amazon through their IAM policies tool. Using the tool we can create credentials which only work with Amazon S3 and only allow you to understand load objects, therefore eliminating most of the aforementioned issues. End though it sounds scary it’s relatively easy to implement if someone shows you how.
Create policies
The first step is to create IAM policies. These tell Amazon which permissions to give. We will need two policies, one to list the names of your buckets and one to upload stuff to a specific bucket (or a directory thereof).
Base policy (list S3 buckets)
Log in to your AWS Console and click on IAM, Create Policy.
Click on Select, located next to Create Your Own Policy.
Policy Name: AllowBucketList
Description: Allows the user to list all of my S3 buckets
Policy document:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowGroupToSeeBucketListInTheConsole", "Action": [ "s3:ListAllMyBuckets" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::*" ] } ] }
Backup Operator policy to bucket EXAMPLEBUCKET
The next policy allows us to write to files on a specific bucket or even a specific folder thereof. Please remember to change EXAMPLEBUCKET in the code and names below to the actual name of your bucket!
Go to your AWS Console, click on IAM, Create Policy.
Click on Select, located next to Create Your Own Policy.
Policy Name: EXAMPLEBUCKETWriteAccess
Description: Allows the user to write to EXAMPLEBUCKET, but not read from it or delete any of its contents
Policy document:
You have two options. The first is the strictest one where the user can only list the contents of the bucket and write to the bucket, but not delete from it. This means that deleting older files (e.g. using Akeeba Backup’s remote quota settings) will fail. You will have to do that manually. As always, security is at odds with convenience.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1416670692010", "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets", "s3:ListBucket", "s3:PutObject", "s3:PutObjectAcl", "s3:ListBucketMultipartUploads", "s3:ListMultipartUploadParts", "s3:AbortMultipartUpload" ], "Resource": [ "arn:aws:s3:::EXAMPLEBUCKET/*" ] } ] }
The second option also allows deleting from the bucket. This allows, for example, Akeeba Backup / Akeeba Solo to apply remote quotas and delete remote files. However, if your site gets hacked then it's very likely that the attacker will be able to delete your backups to prevent you from easily recovering your site. I only recommend this option if you are already copying the backups to a local NAS every day, therefore having a backup of your backups. Speaking of which, it’s a good idea to do it nevertheless. You can never have too many backups.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1416670692010", "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets", "s3:ListBucket", "s3:PutObject", "s3:PutObjectAcl", "s3:ListBucketMultipartUploads", "s3:ListMultipartUploadParts", "s3:AbortMultipartUpload", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::EXAMPLEBUCKET/*" ] } ] }
If you want to restrict access to a specific folder of a bucket, instead of the entire bucket, change the EXAMPLEBUCKET/*
part of the code with EXAMPLEBUCKET/folderName/*
where folderName is the path to your folder. Attentive readers may have figured out by now that you can also limit by file name parts as well. I’ll leave that as an exercise to the reader.
Create a group
Technically, creating a group is not necessary. You can always attach policies directly to a user. I have, however, found it much easier to have groups for different access levels. This allows better organization when you start dealing with multiple servers and types of content per server being backed up.
Log in to your AWS Console and go to IAM, Groups
Click on the Create New Group button.
Group Name: EXAMPLEBUCKETBackupOperatorGroup
Attach Policy: attach the AllowBucketList and EXAMPLEBUCKETWriteAccess policies you created above.
Create the user
Now we are getting there! We will create a user, assign it to the group we have created and get the special credentials for our write only user.
Go to your AWS Console and click IAM, Users, Add User.
User name: EXAMPLEBUCKETBackupOperator
Access Type: Programmatic access
Now click on Next: Permissions.
Select the group EXAMPLEBUCKETBackupOperatorGroup and click on Next: Review. Finally click on Create User.
Get the access credentials (keys).
You can only have two sets of access keys per user. Therefore I recommend using a different user (and, ideally, a different bucket or at least a different folder) for each server / content type you are backing up. If the site is compromised or you are no longer in control of it you can remove its user from the AWS control panel.
Click on the user. Click on Security Credentials. Under Access Keys click on Create Access Key. Write down the keys and download the CSV. Please remember that the secret key will NOT be shown to you again!
You can now use this set of access and secret keys in your backup application, e.g. Akeeba Backup, to ensure maximum security of your backups.
Increasing security
The instructions above offer only a modicum of protection. At worst, you have created separation, i.e. made sure that compromising a single server's Amazon S3 credentials will only let the attacker delete and overwrite backups for that server only. At best, you have restricted the problem to overwriting backups instead of deleting them outright. Still, your backups are not entirely safe from successful attacks.
The best defense against this eventuality and a sane practice is to download all your backups locally, shortly after they are created. A very cheap Raspberry Pi single board computer with a cheap USB hard drive is more than enough for this. You can set up a CRON job to have s3cmd download the backup archives for you.
However, since you are using Amazon S3, you can use the platform's tools to your advantage. You can set up your Amazon S3 bucket so that the archives are transferred to Amazon Glacier after a predetermined amount of time (or even immediately) using lifecycle policies. What's even better is that Amazon Glacier allows you to implement Vault Locking. This means that you only allow the files to be written to once. Even if the attacker gets your Amazon S3 credentials and modify or delete the files on S3 the backup archive files on Glacier will NOT be overwritten / deleted. The only downside of Glacier is that it takes anywhere between 3 to 5 hours to retrieve files from it. Considering that retrieving a backup from there would be your last resort it's not that bad.