Infrastructure

Introduction

Ever since I learned how to properly host a website myself (i.e. rent my own VM, setup a web server and firewall, point DNS to server, serve web pages), I wanted to also host my own email. When I looked at this 10-15 years ago for menucosm, all the advice I found online could be summed up as: don’t bother, it’s too much hassle and not worth the €60/year it cost at the time to pay Google to host it, so that’s what I did.

Last year, I started working on a side-hustle, selling custom Lego sets I designed myself. Since this was a side-hustle and I was already paying for my own web server, I decided to try and keep things as lean as possible, which meant hosting the e-commerce site myself. No issues here: I found a PHP e-commerce package, installed it and wrote my own custom theme. Obviousy I would need email for this, for payments, orders, contacts etc.

I’d dockerized my setup on my web server over the years, so I now have a number of sites, all dockerized, sitting behind a reverse proxy webserver, powered by the excellent nginx-proxy. One of the great things about nginx-proxy is that it can handle SSL certificates for you automatically via Let’s Encrypt, for all your sites behind the proxy server, very convenient.

The great thing about docker and virtualization is that plenty of people have built complicated but reusable images. Since it was 2025, I knew there must be dockerized email setups and sure enough there are. The one I went for in the end was Docker Mailserver. I also needed a UI for webmail and configuration, so after I read about Rainloop, I found a docker image for that also. Now I just needed to set it all up.

For the sake of this blog post, let’s say my website is example.com and my mail server will be mail.example.com.

I’ve been using this setup for about 9 months so far, and as far as I can see, all has been running smoothly. From the very beginning, Google was accepting my test emails (i.e. from my new webmail to my existing gmail account), not marking anything as spam. I receive emails directly and through the website and these seem to be delivered successfully also. All good as far as I know.

One important thing before starting

Many hosting providers, including my own, have port 25 blocked for all outgoing connections. This is to prevent a server maliciously getting taken over and used for spam. I had to contact the support team, tell them what I was up to, promise I’d be dilligent and then ask them to open the port, which they did. I wasted about 1 – 1.5 weeks trying to unsuccessfully work with port 25 closed, which I was poorly advised to try by their support team, before finally realising it MUST be open and they were subsequently OK about opening it!

Docker Mailserver

The documentation for this package is fantastic. It’s ready to use out of the box with configuration all going into the provided mailserver.env file. We’ll be working with the env file and it’s compose.yaml. So first off, get those 2 files from the repo and recreate them in a local mailserver folder. Also create a docker-data folder in there, which the mail server will use for persistence.

SSL Certs

As mentioned above, the nginx-proxy package handles all my SSL via Let’s Encrypt. To get this to work, you need to add some environment variables to your compose.yaml, see the new environment block below. Docker Mailserver needs to be able to read the various SSL files that nginx-proxy creates, so we also need to map a new volume for the folder that nginx-proxy writes to, that Docker Mailserver will know to read; see the final line under the volume block of docker-compose below.

compose.yaml

As mentioned above, this is largely ready to go out of the box. Here is my slightly edited one (see comments for this blost starting with #DC), with the environment variables for nginx-proxy & Let’s Encrypt, and the volume where SSL certs are stored:

mailserver.env

There are a huge number of options here which are well explained in the file and in the documentation. I’ll detail the ones I thought were important:

Bring up the server

At this point we’re ready to start the mailserver. This all just runs on the CLI, there’s no GUI until we install Rainloop. So, from within our mailserver folder, we simply do:

Creating first accounts

Before setting up all the DNS records, the documentation recommends to create an email account, so let’s do that. We’ll also create a ‘postmaster’ alias for this; it’s standard practice to have a ‘postmaster@’ account, it’s up to you whether or not to have it separate or alias it. We’ll be using the postmaster account in the next section.

DNS Records

In order to prove you’re sending valid email, i.e. you are who you say you are and you’re not spoofing a service, certain DNS records need to be setup. For this, you’ll need to access your control panel in your DNS provider (usually whoever you registered your domain with).

A Records

First up, let’s setup up some A records and the MX record. Using your server’s IP address, I would suggest setting up A records for:

• mail.example.com
• webmail.example.com
• mta-sts.example.com

For the last of these domains, you also need to configure your primary webserver to also host it (i.e. to respond to requests for mta-sts.example.com), along with your www site. The site doesn’t need to look perfect at this domain, just respond to the initial request.

Next we want to create an MX record. You can leave the Host name blank, use the default TTL (mine’s 1 day), priority 10 and set Result to mail.example.com.

DKIM, DMARC and SPF

These next parts are key for sending email successfully. I’ll roughly outline what to do, but more info can be found at the Docker Mailserver guide for this.

DKIM

For DKIM, as per the guide, you can set this up via

Assuming you’ve kept the same volume structure from compose.yaml above, this will output files into ./docker-data/dms/config/rspamd/dkim. Have a look at the contents of rsa-2048-mail-example.com.public.dns.txt and copy them to the clipboard. Go back to your Control Panel and create a new TXT record, using mail._domainkey as the Host name value, paste in your copied data to the Result field, use the same TTL as before and save.

DMARC

For DMARC, you just need to create another TXT record. Use _dmarc for Host name. For Result, the guide gives you some defaults, as well as linking to a generator where you can come up with your own. You’ll want to use the postmaster@example.com alias for this: this is how other mailservers will send you reports on issues they’ve encountered with your service. For reference, I went with the following: v=DMARC1;p=quarantine;pct=100;rua=mailto:postmaster@example.com;ruf=mailto:postmaster@example.com;adkim=s;aspf=r

SPF

This is one is simple enough. Simply create another TXT record with an empty Host name and the following for Result: v=spf1 ip4:your.ip.address -all

Other Records

Above, I said to create an A record for mta-sts.example.com. In addition to this, we want to create a TXT record with Host name set to _mta-sts and Result set to v=STSv1; id=yyyymmddxx, where yyyymmddxx is the value of the Serial Number field in your DNS control panel. Once this is done, you need to create a corresponding .well-known/mta-sts.txt file in your primary website’s root folder, i.e. so it’s accessible at https://www.example.com/.well-known/mta-sts.txt and https://mta-sts.example.com/.well-known/mta-sts.txt. Set the contents of this file to:

Finally, I also have the following TXT record, so I suggest you set it up also: Host name of _smtp._tls and Result of v=TLSRPTv1; rua=mailto:postmaster@example.com.

Reverse IP Address

Once we’re done with all the DNS records, we need to setup a reverse IP lookup. We do this in the admin/control panel of your hosting provider. Each hosting provider is different, but with mine it was pretty straight forward. I found the page I wanted under Network > Public Networks. In here, I added a new entry to the existing table, setting IP Address as my server’s IP and Reverse DNS Name as mail.example.com.

At this point, restart your mailserver docker machine (just to make sure all changes are in effect) and at that point, if all goes well, you’re actually hosting a proper mail server! Now we just want a way to interact with it…

Aside: Connect to nginx-proxy

My nginx-proxy server connects to an internal docker network, to which all sites behind the proxy must connect. Let’s say mine is called DCNET, so once you’ve restarted the image, connect to the network via

Rainloop

Again we’re going to be using a docker image and a github repo here, wernerfred/docker-rainloop. Above we created a mailserver folder; for Rainloop, I created a rainloop sub-folder in here, but you can create it at the same level as mailserver if you prefer. We’re going to create a second compose.yaml file; you could ultimately merge this with your compose.yaml from above, but that’s beyond the scope of this article. My rainloop instance will run on port 9009.

Here is what I have for my rainloop/compose.yaml:

If you look under volumes, you’ll see 3 local files mapped in to the image. These configure Rainloop’s internal server (running Apache) to work off the correct port, set log formats and more. For simplicity, I’ll just paste the contents of each:

000-default.conf

ports.conf

apache2.conf

This file is a bit long, so what I suggest is to simply copy the contents of /etc/apache2/sites-available/000-default.conf from inside the rainloop image, save it as 000-default.conf and append the following line:

Bring up the server and enjoy!

OK, hopefully everything will be configured correctly, so now all you need to do is bring up the server. So, within the rainloop directory you can do

then head on over to webmail.example.com and you should see your nice webmail interface! You’ll want to configure it straight away by going to webmail.example.com/?admin, log in using admin / 12345 as the username and password, change your admin password and then set any other configuration you like.

Finally, going back to the root webmail login, you should be able to log in with the account you created above. At this point, you should be up and running, able to send and receive emails. Now, think of what you’re going to spend the €60/year you’re saving on!

Read more...

TL;DR The MultiViews option in Apache automatically will map e.g. /xfz/ to /xyz.php

I was recently creating a new section of the website I work for and decided to opt for tidy URLs, for SEO purposes, instead of our standard.long?url=format URLs that we have elsewhere. Let’s say the new section I was creating was called David’s Boxes, so I wanted to have relative URLs like /davids-boxes/big/blue map to davids-boxes.php?size=big&colour=blue. Purely co-incidentally, there happened to be a defunct davids-boxes folder in our www directory, which contained an old WordPress install, which I prompty deleted (more on this later). Then, I set up rewrite rules in our www/.htacess to do the example mapping above.

Everything was working fine locally: /davids-boxes/ matched to /davids-boxes.php and /davids-boxes/big/blue mapped to /davids-boxes.php?size=big&bolour=blue, all as expected. However, when I put the .htaccess file onto our test server, I couldn’t get the rules to match properly: everything mapped to the basic /davids-boxes.php, i.e. with no extra GET parameters. I tried different order of rules, moving the rules to the top of the .htaccess etc., but nothing worked. Then I simply deleted the rules from the .htaccess, expecting /davids-boxes/ not to map to anything, but it still strangely mapped to /davids.boxes.php as before. This led me to believe there was another rewrite rule somewhere else (a fact that was also helped by the previous WordPress install). Searching the entire codebase, which includes all ‘sub-‘.htaccess files, yielded no results, so then I began thinking it might be the server…

I had a look in our sites-available Apache configs, expecting there may be some sort of obvious generic rewrite to map any e.g. /xyz/ to xyz.php; no such luck. Going through each line in the config, I noticed we had the FollowSymLinks and MultiViews options enabled in the <Directory> tag. I was familiar with the former, but not the latter. Investigating into MultiViews, it turns out this was the thing doing the automatic mapping I was experiencing! The documentation states “if /some/dir has MultiViews enabled, and /some/dir/foo does not exist, then the server reads the directory looking for files named foo.*, and effectively fakes up a type map which names all those files”. Such relief to figure it out. I checked with our CTO, he didn’t know how it got there, so after removing it on testing and doing a quick test, we got rid of it everywhere and my problems were solved.

Read more...

At work we use memcache as our local variable cache and the excellent memcache.php from Harun Yayli to give us a simple way of viewing what’s in the cache.

One use case we came up with that was missing from the original memcached.php script was a way to group similar variables and see how much of the cache they’re taking up. For example, for searches done on the site, we generate a key by concatenating search- to an md5 of the SQL, then store the result of that query in the cache with that key. Another example might be to cache an ad, so the key could be ad-1234, for the ad with ID 1234. So, the following code changes are going to enable us to see how much space all the ‘search’ data, ‘ad’ data etc. takes up in comparison to each other.

It works by starting off with a list of known key prefixes (i.e. search- and ad- in the examples above), then uses existing memcache commands to get a list of slabs, then queries each slab for each item it contains. From this list of items, it looks for our known keys, calculates the size of the item and adds it to a running total. Once it has all the totals, it generates a nice pie chart with a legend, using Google’s Chart API.

So, first up we need to add a new menu entry to our menu, to link to our breakdown. This is simply done by editing the getMenu function in src/display.functions.php and adding a new menu entry to it, as follows:

Next up, we need to add the big block of code that’s going to generate our pie chart. You’ll see in memcache.php a switch block around $_GET['op']. This is where we want to add our block for our new operation 16, as follows:

So, now you should see a new menu option and clicking on it, should hopefully bring up a nice pie chart, something like the screenshot below (I’ve had to blur out our cache variable names).

Memcache Breakdown

Read more...

On the website I work for, when a user uploads an image for an ad, we generally keep 3 versions of that image, each a different size, simply referred to as ‘small’, ‘main’ or ‘large’. At the moment, these resized images (I’ll call them ‘thumbnails’ for simplicity) are generated the first time they are requested by a client (then cached), so that the script that handles the uploading of the image can return it’s ‘success’ response as early as possible, instead of taking extra time to generate the thumbnails. What Beanstalkd allows us to do is put a job on a queue (in our instance a ‘generate thumbnails’ job), where it’ll be picked up at some point in the future by another script that polls the queue and executes in it’s own separate process. So, my uploading script is only delayed by say the 0.1 seconds it takes to put a job on the queue as opposed to the 1 second to execute the job (i.e. generate the thumbnails). This blog post is how I got the whole thing to work on a Ubuntu 12.04 server, using PHP.

This post was largely inspired by an article on the blog Context With Style, which was written for a Mac. I’m also going to use their example of a queue filler script to populate the queue and a worker script, to pull jobs from the queue and process them. I recommend you read that post for a better idea.

One other thing, most of these UNIX commands need to be run as root, so I’ll assume you’re in super-user mode.

Beanstalkd

Installing Beanstalkd is pretty straightforward:

We don’t need to start it just yet, but for reference, to run it you can do

Pheanstalk

Pheanstalk is a PHP package to interface with a Beanstalk daemon. I simply downloaded the zip from github, extracted it to a ‘pheanstalk’ folder in my main include folder, then to use it, I simply do

Going by the example on the Context With Style article, for the script under the section “Pushing things into the queue”, we’ll call that script fill_queue.php. We’ll call the script in “Picking up things from the queue” worker.php. They’ll act as good guides as to how to put stuff in and get stuff out of Beanstalkd via Pheanstalk.

So, the idea is we’ll have our worker.php running non-stop (via daemontools, see next section), polling the queue for new jobs. Once we know our worker.php is ready, we can manually run fill_queue.php from the command line to populate the queue. The worker should then go through the queue, writing the data it reads to a log file in ./log/worker.txt. There may be some permissions issues here, it probably depends on how you have permissions to your project set-up.

Daemontools

First up we need to install daemontools, which is

You don’t actually interact with a daemontools process, you use things that begin with ‘sv’, such as svscan or svbootscan. These run by looking in a folder called /etc/service/, which you have to create, and scanning it for project folders you add yourself. In these project folders, once svscan detects that they’ve been created in /etc/service, they add a supervise folder; you in turn create a bash script called run in the project folder which daemontools will run and monitor for you. Don’t worry, all these steps are outlined below!

Anyways, now that we’ve installed daemontools, we need to create a run script for it and then run it, as well as create our /etc/service directory. Some of these tips are thanks to this post.

Hopefully, now if you do a ps aux | grep sv, you’ll see at least svscan running.

Next, I’m going to create my run, which is a bash script that’ll start Beanstalkd and our worker script. I’ll place this in my example /var/www/my-project folder, along with my worker.php, fill_queue.php and log/worker.txt files. I’ll then create a my-project service folder and symlink my run file into there.

Hopefully when you do the tail, you’ll see data that corresponds with the output from fill_queue.php. This will indicate that your worker is running, polling the queue for new jobs. If you re-run fill_queue.php, your log file should expand accordingly.

Read more...

So, I recently started a new job as Lead Developer on carsireland.ie and one of the first things I was tasked with was moving the codebase from a simple PC running Linux to the cloud, so that it could be accessed remotely, outside the office. Now, while I do prefer Git, SVN is still reasonably popular, especially with websites older than a few years, hence the CTO wanted to stick with it, for the time being at least! Needless to say, most of the following is best done as root, or at least with sudo privileges. Also, this is done on Ubuntu, hence the use of apt-get.

1. Setting up Apache for HTTPS

Apache was already running on the server, but it had to be enabled for HTTPS. Firstly You need to generate self-signed SSL certificates. You’ll be asked for a passphrase; enter one and note it down:

Move the certificates to somewhere that Apache expects to find it:

Enable SSL for Apache

2. SVN

Install SVN and it’s Apache module

Create a new folder for the code (we’ll call the folder ‘svn’):

Create the repository:

Tell Apache about the repository:

This opens up the pretty simple nano editor. At the bottom of the file, before the final <VirtualHost>, add:

You may need to enable your SSL site, so if the files /etc/apache2/sites-enabled/000-default-ssl or /etc/apache2/sites-enabled/default-ssl don’t exist, do:

For Apache to be able to read/write to the repository, we need to change it’s owner to www-data:

Next, we need to add some login details for users, i.e. developers (you’ll be asked to enter a password):

For subsequent users, drop the -c flag above.

Restart Apache (however you want to do it). Following from above:

You should now be able to view the initial empty repository at http://server.locaton/svn where ‘server.location’ is either an IP address or a domain, depending on how you’ve set-up the server.

If you have an SVN dump of your repository and you want to load it into the new one, you can simply do:

At this point, your SVN server should be up and running and ready to take commits. You may need to play around with the permissions of your /home/svn directories, making certain ones executable/writeable to Apache. If I’ve left anything else out, please let me know in the comments.

Read more...