Hackviking He killed Chuck Norris, he ruled dancing so he took up a new hobby…

15Feb/160

AWS EC2 Linux: Simple backup script

I have a small EC2 box running a couple of WordPress sites. To back them up I wrote a quick bash script that dumps out the databases and also zips up all the files. This script is running daily, the amount of disc space doesn't really matter since the sites are really small. If you have larger sites you might want to split the script into one for weekly backup for files and a daily one for databases, the principal are the same.

Prerequisites

  • Credentials for MySQL with access to the database in question. In this example I run with root.
  • A folder where you want to store your backups. You should a separate folder since we clean out old backups in the script.
  • Path to the www root folder for the sites.

Script

First we create a script and open it for editing in nano:

nano backup.sh

First line in any bash script is:

#! /bin/bash

Then we create a variable to keep the current date:

_now=$(date +"%m_%d_%Y")

Then we dump out the databases to .sql files:

mysqldump --user root --password=lamedemopassword wp_site_1 > /var/www/html/backup/wp_site_1_$_now.sql
mysqldump --user root --password=lamedemopassword wp_site_2 > /var/www/html/backup/wp_site_2_$_now.sql

Here we use the $_now variable we declared in the beginning of the script. So we can easily find the correct backup if we need to do a restore. Next step is to zip up the www contents of each site:

tar -zcvf /var/www/html/backup/www_backup_site1_$_now.tar.gz /var/www/html/wp_site_1
tar -zcvf /var/www/html/backup/www_backup_site2_$_now.tar.gz /var/www/html/wp_site_2

Once again we use the $_now variable to mark the file name with the current date for the backup.

Then we want to clean out backups older then x days. In this example I remove all backup files older then 7 days.

find /var/www/html/backup/* -mtime +7 -exec rm {} \;

The first part find /var/www/html/backup/* -mtime +7  then we use the -exec to pipe the result into a command, in this case rm. The {} inserts the files found and the \ escapes the command to prevent it to expand the shell. Then we finalize the row with a ;. So this means that for each file found it will execute the rm command and remove that file.

Save the backup.sh file and exit nano. Now we need to make the script executable:

chmod 755 backup.sh

Then we can do a test run of the script:

sudo ./backup.sh

Now check the folder that the files was created and contain the actual data. If your satisfied with the result you can move the script into the cron.daily folder.

sudo mv /home/ec2-user/backup.sh /etc/cron.daily/

Now the daily cronjob will create a backup of the WordPress sites. Both files and databases.

11Feb/160

AWS EC2 Linux: Enable SSH password logon

Amazon AWS EC2 instances are by default secured with ssh certificates. This is great for security until you need to provide a UAT duplicate for an external user or developer. You don't want to share your certificate with them and setting up a new one is more work than this quick fix. The security isn't as important on a UAT or test system as it is on a production system so for this use case we can go for lower security.

To enable users to access we first need to set a password on the ec2-user account. It's highly recommended that you select a strong password!

sudo passwd ec2-user

Then we need to allow password only connections. We edit the ssh settings, find the line PasswordAuthentication no and change it to PasswordAuthentication yes.

sudo nano /etc/ssh/sshd_config

Then we need to restart the ssh service.

sudo service sshd restart

Now you can login to you Amazon AWS EC2 instance with only a password. To secure the server again just change the PasswordAuthentication line back to no.

4Feb/150

Amazon EC2 Linux – Add additional volumes

EBS Mappings

Adding additional storage to your Amazon EC2 instance have several advantages. You can select the right storage type for the use. Why use a fast SSD backed volume for storing nightly backups instead of magnetic storage, that ar slower but come at a much lower price.

First you need to provision storage and assign it to your instance. Amazon provides a good guide on how to add additional volumes to your instances. There are several advantages to using several different volumes. As I wrote in my guide to move mysql storage you will not risk running the boot disk full witch will make the system halt. Other advantages include the selection of storage fit for your purpose and price range, as mentioned above. External volumes can also easily be migrated between instances if and when you get a need for that. It is also easier when you need to extend your storage space. Instead of making a snapshot of the entire instance and then launching a new one with a bigger drive you can attach new storage and migrate the data. This approach will make the downtime much shorter.

When selecting the correct storage for you solution there are a few things to keep in mind. When it comes to EBS it comes in three basic flavors. All with there benefits and disadvantages, it is there for important to make an educated decision.
Continue reading...

4Feb/150

Move MySQL database storage location

It's always a good idea to keep storage away from the boot device. If you run out of space on the boot device the system will halt. If you make a new install it's easy enough to move your storage and you can do it from a cloud-init script like this:

- mkdir /var/db
- chown -R mysql:mysql /var/db
- sed -i 's:datadir=/var/lib/mysql:datadir=/var/db:g' /etc/my.cnf
- service mysqld start

If the installation is all ready up and running you have to add steps for stopping the MySQL server and copy the database files:

mkdir /var/www/db
service mysqld stop
mv /var/lib/mysql/* /var/db
chown -R mysql:mysql /var/db
sed -i 's:datadir=/var/lib/mysql:datadir=/var/db:g' /etc/my.cnf
service mysqld start

In these examples I have user /var/db where I mounted the second storage device. You can however use any location you see fit. Points of interest in the command sequence.

chown -R mysql:mysql /var/db

Make sure that the mysql deamon have access to the storage location.

sed -i 's:datadir=/var/lib/mysql:datadir=/var/db:g' /etc/my.cnf

sed is a simple tool for search and replace inside text/config files directly from the command line. Here it searches for the line specifying the MySQL datadir location and replaces it with the new value.

3Feb/150

Unattended use of mysql_secure_installation

After installing MySQL on any Linux distribution you run the mysql_secure_installation script, or at least you should! It will prompt you to set a new root password, remove anon access and a few other things. But if you want this configuration to be done in a deployment or cloud-init script? The mysql_secure_installation command/script doesn't accept any parameters, so it can't be used for unattended install. How ever you can execute the same commands via the mysql command line tool as long as the service is started.

mysql -e "UPDATE mysql.user SET Password=PASSWORD('{input_password_here}') WHERE User='root';"
mysql -e "DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');"
mysql -e "DELETE FROM mysql.user WHERE User='';"
mysql -e "DROP DATABASE test;"
mysql -e "FLUSH PRIVILEGES;"

I use this to provision new MySQL servers in the Amazon EC2 environment and it works like a charm. If this is used in a cloud-init script make sure to execute the sudo service mysqld start first!