Drupal 7 and Aegir server setup

Setting up a high performance Drupal server can be a complex task, which consist of configuring different caching technologies (APC, MemCache and Varnish) and the LAMP stack. This guide explains how I set up me Debian servers to run Drupal sites with the Aegir hosting system. If you are not familiar with the Aegir hosting system, take a look at their community site at http://community.aegirproject.org/. I use the Aegir hosting system to better utilize the caching and make it easier for administrators to perform regular tasks such as creation, migration, update and backup of Drupal sites.

The server that I use in this guide is a linode (http://linode.com) virtual private server with 1 GB of memory running Debian 7.

This guide is a work in progress and will continue to be so, because server management will always be a on-going process.


Part I: Basic server configuration

First part in configuring a new Debian server consist of setting up the basic system to your preferred setting (time-zone, language etc.).


If you default nationality is out-side the US (default for linode servers), you may want to update your servers locales to reflect your nationality.

~# dpkg-reconfigure locales

Software repositories

The default software repositories in Debian, do not always have the newest version of the LAMP stack components. So we are going to add the Dotdeb repository. Edit the file in /etc/apt/sources.list and insert the lines below and run the commands to fetch the GnuPG key.

deb http://packages.dotdeb.org wheezy all
deb-src http://packages.dotdeb.org wheezy all
~$ wget http://www.dotdeb.org/dotdeb.gpg
~$ cat dotdeb.gpg | apt-key add -
~$ apt-get update

Basic programs

Not all of these program are required, but they will be nice to have later on for debugging the setup and in compiling software if needed.

~$ apt-get install nmap mc lynx rcconf zip build-essential automake autoconf uptimed bsd-mailx rsync unzip git-core openssl-blacklist sudo

Setting up mail

Aegir needs to be able to send e-mails during installation and your Drupal sites needs this to.

~$ apt-get install exim4-daemon-light mailutils
~$ dpkg-reconfigure exim4-config


Select the following settings in the on screen selections

  1. internet site; mail is sent and received directly using SMTP
  2. system mail name:
  4. localhost; localhost.localdomain;
  5. leave empty
  6. leave empty
  7. No
  8. Maildir format in home directory
  9. No
  10. root


You might want to update our timezone, so that cron job etc. is executed as expected. Run the command below and select you timezone in the interface.

~$ sudo dpkg-reconfigure tzdata

Part II: Apache, MySQL and PHP

Before Aegir can be installed, we need to have the LAMP stack installed and configured on the server. The first step is to install the software and some extension that will be used to configure the LAMP stack. Run the command below to preform the installation.

~$ apt-get install apache2 php5 php5-cli php5-gd php5-mysql php5-sqlite mysql-server php-pear php5-dev percona-toolkit

Apache configuration

To run Aegir and Drupal with clean URLs Apache rewrite is needed, if you do not want to have SSL support you can skip that below.

~$ a2enmod rewrite
~$ a2enmod expires
~$ a2enmod ssl
~$ a2ensite default-ssl
~$ /etc/init.d/apache2 restart

If you want to have SSL support for your virtual host (sites) you will needed to set up Apache to listen port 443 for every virtual host. You should notice that you can only have one SSL certificate for each IP, which is a limitation of the way the SSL protocol works. One way to solve this issue is to use the secure pages module and redirect all SSL traffic through one domain name, but this is outside the scoop of this guide.

Edit /etc/apache2/ports.conf and change it to reflect the lines below.

NameVirtualHost *:80
Listen 80

<IfModule mod_ssl.c>
  NameVirtualHost *:443
  Listen 443

Restart the Apache server to reload the configuration.


The server has an limit amount of memory and Apache uses 60-120 Mb of memory for each thread (the size is very different base on the Drupal modules enabled etc.), so we need to set an appropriate number of threads. To to set the number of threads (clients) edit /etc/apache2/apache2.conf. You should look for "MaxClients" its listed more than once. One for each type of worker mode. Change it to about 10 (this should be adjusted to the actual memory usage). Reload Apache to apply the configuration

~$ /etc/init.d/apache2 reload

PHP configuration

Some of the Drupal installation profiles and modules requires a higher PHP memory limit, then the default one that PHP comes with. The maximum file upload size limit is also set low for most sites. So edit /etc/php5/apache2/php.ini and changes the lines to reflect the ones below and restart Apache.

memory_limit = 196M
upload_max_filesize = 16M
post_max_size = 20M

PHP have some problems with time zones, so you should set your time zone in the ini file as well (see http://php.net/manual/en/timezones.php for a list of time zones).

PHP extensions

Install the upload progress extension to show the upload progress for users, when they upload images.

~$ pecl install uploadprogress
~$ nano -w /etc/php5/conf.d/uploadprogress.ini

Add the line "extension=uploadprogress.so" to the ini file and restart Apache.

MySQL configuration

The basic MySQL configuration can be used without preforming any changes, but you can get better performance by adding the file "/etc/mysql/conf.d/innodb.cnf" and adjusting the values below. These settings I normally only use on larger servers, with 4+ Gb of memory and setting the pool size to 512 Mb.


Part III: Installing Aegir

This part of the guide is based on the manual install guide for Aegir, which can be found at http://community.aegirproject.org/. When this guide was written Aegir was version 1.4.

NOTE: That Aegir 2.x can run with the new drush 6.x.

Create Aegir user

Aegir provision framework needs to have an user, which Aegir can use to create sites on the server and set the correct permissions for files. The user needs to belong to the web server group (www-data), but at the same time not have access to the whole system.

~$ adduser --system --group --home /home/www/aegir aegir
~$ usermod -a -G www-data aegir

Sudo configuration

The Aegir user needs to have access to restart/reload the Apache process, which can be archived by using sudo. Edit the file "/etc/sudoers" and add the line below.

aegir ALL=NOPASSWD: /usr/sbin/apache2ctl


We need to install Drush (Drupal shell) as it is used by Aegir.

~$ cd /opt
~$ wget install http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz
~$ tar -zxvf drush-7.x-4.5.tar.gz
~$ rm drush-7.x-4.5.tar.gz
~$ ln -s /opt/drush/drush /usr/local/bin/drush
~$ pear install console_table

Change user

Before you continue the installation you should become the aegir user.

~$ su -s /bin/bash - aegir

Install provision

Install the last version of the provision drush extension by running the command below.

~$ drush dl --destination=/home/www/aegir/.drush provision-6.x

Install hostmaster

~$ drush hostmaster-install


  1. Aegir frontend URL:
  2. MySQL privileged user ("root") password: ************
  3. Admin user e-mail [xxxx@xxxx.xx]:

Reviewing your settings and accept the install process. When the process has completed please follow the link give in the last message in the console, but before this you need to run the commands below as root.

~$ ln -s /home/www/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf
~$ /etc/init.d/apache2 restart

PHP timezone

If you aegir print the message "warning: strtotime(): It is not safe to rely on" when you log in you have to go into "/etc/php5/apache2/php.ini and set your timezone. An list over PHP timezones can be found here http://dk.php.net/manual/en/timezones.php.

Part IV: Caching (APC and MemCache)

The translation off PHP into machine code is a relative "slow" process, so a large speed up can be gained by using an up-code cache to store the compiled PHP code. This guides explains how to install Alternative PHP Cache (APC).


If you use the http://dotdeb.org repository as explained in the first part of this article, you can simply used the package manager to install APC. It's also possible to user pecl to install the newest version of APC from http://pecl.php.net/package/APC.

~$ apt-get install php5-apc
~$ nano -w /etc/php5/conf.d/apc.ini

I use the following configuration for my installation of APC for more information about configure APC see http://php.net/manual/en/book.apc.php.

apc.shm_size=30M ;Change the amont of memory, based on the memory usage.


Another typical bottleneck is the amount of data that Drupal loads form the database, in this guide currently MySQL. This data retrieve from the database can be optimized by inserting a cache between Drupal and the database, namely MemCache. MemCache is extreme fast memory based cache and can be used to many other things then MySQL caching. You should be aware that MemCache is available via the servers network and the data is not protected, so you should make sure to close the network port in the firewall (you can find the port number(s) in the configuration file). It do however, as default, only list to local host.

~$ apt-get install memcached php5-memcached


The configuration file is located at "/etc/memcached.conf", normally I raises the memory limit to match the resources available and needed. Be aware that MemCache always uses all the memory that you can give it :-)

To use MemCache together with Drupal you need to use the MemCache module, which can be found here http://drupal.org/project/memcache

Part V: Security

I want to be able to use drush to synchronize files from the server to my local development environment, which requires that I’m able to log onto the server as the aegir user. This is due to the fact that all files in an aegir site is owned by aegir and is not readable by other the aegir user and www-data (Apache). So I will configure the server to allow SSH for the aegir user, but I will not permit root to login over SSH and the aegir user is not allow to change user. The means that I also will add a new day-to-day user that can be used to administrate the server outside aegir.

Furthermore I'll limit SSH to use public/private keys to enhance security and protected the server as much as possible. So add the new user and insert the users public key. Note that commands should be executed as the root user.

~$ adduser <username>
~$ su - <username>
~$ mkdir .ssh
~$ nano -w .ssh/authorized_keys

You should place your public key in the authorized_keys file and test that you are able to login before changing the SSH configuration in the SSH section. If you use http://linode.org as your server provider you can use their virtual console to log into the system, if you lock your self out of the system. I want the to use this user to administrate the server, so I'll add the user to the sudo group.

~$ usermod -a -G sudo <username>


When using sudo with your administrator user created above, you will be asked for your password every time. You can change this behaviour by adding the line below to "/etc/sudoers".



We can enhance SSH by limiting which methods may be used to log into the system and not allow root login. Edit the configuration file "/etc/ssh/sshd_config" and change the lines below from yes to no.

PermitRootLogin no
PasswordAuthentication no

Aegir user

The "default" aegir user, we created earlier in this article, do not have permission to log onto the system do to the fact that the user does not have a shell. You can edit "/etc/passwd" and change the word false to bash in the end of the aegir users. Another method (and more safe) is to login as root and change user to aegir and execute the commands below. The two last lines below is used to add an public key to the aegir user, so you can log into the system as this user over SSH.

~$ passwd
~$ chsh  -s /bin/bash
~$ su -s /bin/bash - aegir
~$ mkdir .ssh
~$ nano -w .ssh/authorized_keys


I uses the Linux IP tables to build a firewall that allows incoming connections on port 22, 80 and 443, it's base on the guide found here http://www.faqs.org/docs/iptables/index.html. If you are interested in knowing how IP tables works, I recommend that you read the documentation. To get a firewall up and running now... you can download the file below an run the "rc.firewall.sh" script.

Download scripts here: firewall.tar.gz

Restore iptables after reboot

In Debian 6 iptables are not restored automatically on reboot, which means that your firewall may not be present after reboot (or boot). I normally adds a init script that restores iptables. First start by running the firewall script and save the configuration.

~$ iptables-save > /etc/iptables.conf
~$ touch /etc/init.d/firewall
~$ chmod 755 /etc/init.d/firewall
~$ nano -w /etc/init.d/firewall

Add the following code to the file.

# Provides:          firewall
# Required-Start:    $local_fs $remote_fs $network $syslog $named
# Required-Stop:     $local_fs $remote_fs $network $syslog $named
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# X-Interactive:     true
# Short-Description: Start/stop apache sole search framework


case $1 in
        echo "Starting firewall"
        $IPT_RESTORE < $CONF
        echo "Removeing firewall"
        $IPT -F
        $IPT -X
        echo "Saving current firewall rules"
        $IPT_SAVE > $CONF
        $0 stop
        sleep 1
        $0 start
        echo "Usage: $0 {start|stop|restart|save}" >&2
        exit 1

Next make sure that the script is executed at boot time adding it to the run levels using the command below.

~$ rcconf

Improve protection of SSH

To further enhance the security I install fail2ban (http://www.fail2ban.org), which is used to protected against brute-force password attacks. If you want to tweak fail2ban you can find the configuration file at "/etc/fail2ban/jail.conf".

~$ apt-get install fail2ban


I uses git, which creates the .git folder, so to prevent our software repositories in begin download through Apache. Edit "/etc/apache2/apache2.conf" and add the lines below to the file.

# The following lines prevent .git and .git* files from being viewed by Web clients.
<Files ~ "^.git">
  Order allow,deny
  Deny from all

<DirectoryMatch "/.git/">
  Order allow,deny
  Deny from all
RedirectMatch 404 /.git(/|$)

To further enhance security of Apache I changes the server tokens to production and sets the signature to off by editing "/etc/apache2/conf.d/security" and changing the values below.

ServerTokens Prod
ServerSignature Off

Further enhancements

For more information about securing Debian see http://www.debian.org/doc/manuals/securing-debian-howto/ch4.en.html

Part VI: Backup

This part of the article is about backing up the server either locally or to a remote server. If you don't have another server you can backup to, you can at least backup the databases locally. As I use another server to backup the web-server(s) I have to create a users that can login without requiring a password, namely a public key with a password less private key on the backup server. So this user have to be somewhat limited in its access.

I call this user rdiff, named after the backup software later in this article, so create that user with the commands below.

~$ adduser rdiff
~$ usermod -a -G www-data rdiff
~$ usermod -a -G aegir rdiff

Mysql (Zmanda)

I uses the Zmanda community edition MySQL database backup program, which can be download from http://www.zmanda.com/download-zrm.php. Zmanda can be configured to in a variety of ways, personally I use a 7 day cycle and backs up all databases into the rdiff users home folder.

~$ apt-get install libxml-parser-perl libdbi-perl

Download Zmanda ZRM Debian package and install it using the package manager.

~$ wget http://www.zmanda.com/downloads/community/ZRM-MySQL/3.0/Debian/mysql-zrm_3.0.0_all.deb
~$ dpkg -i mysql-zrm_3.0.0_all.deb

Zmanda can be configured to run as many backup sets that you want, I prefer to only have one backup set.

~$ mkdir -p /home/rdiff/mysql/backupset
~$ chown -R rdiff:rdiff /home/rdiff/mysql
~$ chmod g+rx /home/rdiff/mysql
~$ mkdir /etc/mysql-zrm/backupset/
~$ cp /etc/mysql-zrm/mysql-zrm.conf /etc/mysql-zrm/backupset/
~$ nano -w /etc/mysql-zrm/backupset/mysql-zrm.conf

The last line above opens the configuration file for the backup set, I have change/uncomment the following configuration lines in my backup sets configuration. The e-mail address entered in the "mailto" line will receive an e-mail every time the backup set is executed.


I'm going to use the post-backup-plugin to change the group and permissions for the backed up databases, because the backup process is being run by root and is not readable by the backup user. So edit the "post-backup.pl" perl file defined above and add the lines below.

my $path = $ARGV[2];
system('chown -R root:rdiff '.$path);
system('find '.$path.' -type d -exec chmod 750 {} \;');
system('find '.$path.' -type f -exec chmod 640 {} \;');


Create the first backup (to test that every ting works) and add a scheduler to run the backup set every night.

~$ mysql-zrm-scheduler --backup-set backupset --backup-level 0 --now
~$ mysql-zrm-scheduler --add --backup-set backupset --backup-level 0 --interval daily --start-time 01:00


It's all very well that we can backup the databases, but what about restoring the individual databases. It's easy just follow the steps below, which assumes that you are the root users when executing the first command.

~$ sudo -i
~$ su - rdiff
~$ cd /home/rdiff/mysql
~$ mysql-zrm-reporter --show restore-info --where backup-set=backupset

You now have a list over available backups, so now we want to recover a database from one of the backups.

~$ mysql-zrm --action restore --backup-set backupset --source-directory /home/rdiff/mysql/backupset/<date> --databases "<database name>"

Remote diff backup (rdiff)

In order to backup this server to a remote machine I use rdiff backup (http://www.nongnu.org/rdiff-backup/), which allows you to take backup over SSH. If you are going to user rdiff backup you will need to install it one both machines. The backup program is based on python, so we need to install some python libraries, before installing rdiff backup.

~$ apt-get install python-dev librsync-dev python-pylibacl python-pyxattr python-support

Download rdiff backup from http://savannah.nongnu.org/download/rdiff-backup and use the newest version, which of this writing was 1.3.3.

~$ wget http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.3.3.tar.gz
~$ tar -zxvf rdiff-backup-1.3.3.tar.gz
~$ cd rdiff-backup-1.3.3
~$ python setup.py install --prefix=/opt/rdiff-backup-1.3.3
~$ ln -s /opt/rdiff-backup-1.3.3/bin/* /usr/local/bin/
~$ ln -s /opt/rdiff-backup-1.3.3/lib/python2.7/site-packages/* /usr/local/lib/python2.7/dist-packages/


To backup the new server onto the backup server into "/mnt/backup//(aegir/mysql)/" I use the commands below. The backup is executed through SSH for maximum security and because we have add the backup users public key to the rdiff user on the server the process can be executed as cron jobs.

rdiff-backup rdiff@<servername>::/home/www/aegir/platforms /home/rdiff/backup/<servername>/aegir
rdiff-backup rdiff@<servername>::/home/rdiff/mysql /home/rdiff/backup/<servername>/mysql

The backup is an incremental backup, so only the changes since last run is transferred over the network and information is stored with time information. As I'm only interested in the last 7 days I use the commands below to consolidate the backup and delete every thing older then 7 days.

rdiff-backup --force --remove-older-than 7D /home/rdiff/backup/<servername>/aegir
rdiff-backup --force --remove-older-than 7D /home/rdiff/backup/<servername>/mysql

I also backup the MySQL backup files, better safe then sorry, to the backup server as you can see above. So I manually can copy them back if the server have been compromised and use Zmanda to restore the databases.


The data can be restored in a range of different ways, will start by looking at restoring all files for the whole aegir folder including sites.

Restore all data from the current state, namely "now".

rdiff-backup --restore-as-of now /home/rdiff/backup/<servername>/aegir/  rdiff@<serveranme>::/home/www/aegir

Restore all data from yesterday, where the xD is how many days back the backup should be restored.

rdiff-backup --restore-as-of 1D /home/rdiff/backup/<servername>/aegir/  rdiff@<serveranme>::/home/www/aegir

If you want to restore a single file or folder in the backup just specify the folder/file in the backup set.

rdiff-backup --restore-as-of 1D /home/rdiff/backup/<servername>/aegir/<(file|folder)>  rdiff@<serveranme>::/home/www/aegir/<(file|folder)>


Rdiff backup has alot more possibilities then the once I have mentioned here, such as search the backup for files etc. So I recommend that you go to the rdiff homepage and read the examples listed there. There is also a web-interface which could be installed on the backup server (see http://www.rdiffweb.org/),

Remote backup server

This pages explains how to configure a backup server on a remote location to backup your Aegir server(s). The backup server is based on Debian 6 running on a small linode (http://linode.com) virtual private server with 512 Mb of memory.

Installation Debian

This step is the same as for the Aegir server (Part I: Basic server configuration) without the dotdeb repositories. You can also drop installing the git-core package, but it will not hurt to install it.

Install rdiff backup

We can use another part of the guide Part VI: Backup under the section about installing rdiff backup. The process is the same for all machines that have to be backed up.

SSH connection

I don't have the backup server on the same network as the aegir server, so if that network get compromised or are hit by accidents my backup is secured at another location. So to preform the backup in a secure manager it will run over SSH. I want the backup to be performed automatically every night, so I use a password less private SSH key to connect to the aegir server from the backup server.

So add a rdiff user and generate the password less SSH key.

~$ adduser rdiff
~$ su - rdiff
~$ ssh-keygen
~$ cat .ssh/id_rsa.pub

SSH into the Aegir server(s), as a user that can sudo into root, when logged in as root add the public SSH key outputted above to the rdiff user. After adding the key you should be able to ssh from the backup server user rdiff into the other server(s) as rdiff, without being asked to enter a password.

~$ sudo -i
~$ su - rdiff
~$ mkdir .ssh
~$ nano -w .ssh/authorized_keys


We should now be able to SSH into the aegir server as rdiff, so lets create the local file structure on the backup server to make it easier to find the files when we need to do a restore.

~$ mkdir -p /home/rdiff/backup/<servername>/aegir
~$ mkdir -p /home/rdiff/backup/<servername>/mysql

Try to run a manual backup a test that every thing works as you expect. The first two commands synchronize the files and the last two check for out-date backups, which should return that non was found.

~$ rdiff-backup rdiff@<servername>::/home/www/aegir/platforms /home/rdiff/backup/<servername>/aegir
~$ rdiff-backup rdiff@<servername>::/home/rdiff/mysql /home/rdiff/backup/<servername>/mysql
~$ rdiff-backup --force --remove-older-than 7D /home/rdiff/backup/<servername>/aegir
~$ rdiff-backup --force --remove-older-than 7D /home/rdiff/backup/<servername>/mysql

Automate the backup

As mentioned earlier I configure the backup to run automatically a least every night. To do this I simply uses cron jobs to run a script that executes the rdiff-backup commands shown above. First step is, as root, to allow the rdiff user to run cron jobs by adding the username (rdiff) to the file cron.allow.

~$ nano -w /etc/cron.allow
~$ su - rdiff
~$ nano -w /home/rdiff/backup.php

I have add the lines below to the shell script, where it's important to notice that it have to be full paths to the programs (rdiff-backup) that you want to execute.


// Check if its CLI else exit fast.
define('ISCLI', PHP_SAPI === 'cli');
if (!ISCLI) {
  echo '<b>This script is only for command line use.<b>';

* Configuration parameters.
$rdiff = '/usr/local/bin/rdiff-backup';
$rdiff_options = '--print-statistics';
$rdiff_target_root_path = '/home/rdiff/backup/';

// Servers that should be backed-up.
$servers = array(
  '<FQDN>' => array(
    'user' => 'rdiff',
    'name' => 'Server 1',
    'paths' => array(
      '/home/www/aegir/platforms' => $rdiff_target_root_path . 'server_1/aegir',
      '/home/rdiff/mysql' => $rdiff_target_root_path . 'server_1/mysql',
    'time' => '7D',

// A status mail will be sent to this address for each server.
$mail_receive = 'root@fqdn.com';

// Loop over the servers and there paths running backup on each path.
foreach ($servers as $servername => $server) {
  $message = '';
  foreach ($server['paths'] as $src => $target) {
    // Start backup message.
    $message .= '******** BACKING UP: ' . $src . "\n\n";

    // Run backup.
    $output = array();
    $cmd = $rdiff . ' ' . $rdiff_options . ' ' . $server['user'] . '@' . $servername . '::' . $src . ' ' . $target . ' 2>&1';
    exec($cmd, $output);
    $message .= implode("\n", $output);

    // Run clean-up.
    $output = array();
    $cmd = $rdiff . ' --force --remove-older-than ' . $server['time'] . ' ' . $target . ' 2>&1';
    exec($cmd, $output);
    $message .= "\n\n******** REMOVE OLD COPIES ********\n\n" . implode("\n", $output) . "\n\n";

  // Send mail.
  mail($mail_receive, 'Backup executed (' . $server['name'] . ')', $message);


* Send mail with free space information.
$output = array();

// Get information about disk usage.
$cmd = '/usr/bin/du -h --max-depth=1 ' . $rdiff_target_root_path . ' 2>&1';
exec($cmd, $output);
$message = implode("\n", $output);

// Get information about free disk.
$output = array();
$cmd = '/bin/df -h /dev/xvda'; // Change to match you disk.
exec($cmd, $output);
$message .= "\n\n******** DISK FREE ********\n\n". implode("\n", $output) . "\n\n";;

// Send the mail.
mail($mail_receive, 'Backup disk usages', $message);

Edit the rdiff user crontab and add a line to execute the backup every night at 1 am.

~$ crontab -u rdiff -e
0 1 * * * /home/rdiff/backup.php


Again this is basically the same as for the Aegir server (Part V: Security) with the exception of the Aegir user and Apache parts. You can download a modified version of the firewall script here: firewall_backup.tar.gz. It only allow connection through port 22 (SSH).