posted 2018-04-08 by Thomas Kooi
I have various Terraform modules for many purposes and often they end up using CentOS with a Docker install script. While demoing something to a co-worker, we had to wait ~7 minutes for a Terraform apply it to finish with an install script. This should be a lot faster, so I sat down this sunday and looked into Packer.
Since I am already making heavy use of Terraform, taking a peak at Packer is a small next step. Turns out, it’s nice and easy, though initially the documentation of Packer seemed a bit confusing (I expected it to be similair to Terraform).
So my main use case is installing some specific versions of Docker CE and Kubeadm. Pretty much all of my Terraform projects involve a Docker installation. To run the full installation script takes ~5 minutes, excluding any additional steps for configuration or host specific installs. Now I know I could simply use the provided Docker image on DigitalOcean, but where’s the fun in that (besides, I wanted specific versions of Docker CE).
Most of the time I first provision a bunch of master machines (Docker Swarm mode masters, etcd, or kubernetes api servers), followed by a couple of worker nodes. This means I often have to wait ~10 minutes for the entire thing to finish, on top of all the other things that need installing and configuring…
I figured I’d start out simple and take the DigitalOcean builder example from Packer.
{
"type": "digitalocean",
"api_token": "YOUR API KEY",
"image": "ubuntu-14-04-x64",
"region": "nyc3",
"size": "512mb",
"ssh_username": "root"
}
And turn it into a packer config file:
{
"builders": [
{
"type": "digitalocean",
"api_token": "MY_TOKEN",
"image": "centos-7-x64",
"region": "ams3",
"size": "s-1vcpu-1gb",
"ssh_username": "root"
}
],
"provisioners": [
{
"type": "shell",
"script": "digitalocean/scripts/install-kubeadm.sh"
}
]
}
And the contents of digitalocean/scripts/install-kubeadm.sh:
#!/bin/bash
# install docker
yum update -y
yum install -y docker
systemctl enable docker && systemctl start docker
# get kubernetes repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
# install kubernetes
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
Building the image turns out to be as simple as:
$ packer build docker-kubeadm.json
And off we go;
digitalocean output will be in this color.
==> digitalocean: Creating temporary ssh key for droplet...
==> digitalocean: Creating droplet...
==> digitalocean: Waiting for droplet to become active...
==> digitalocean: Waiting for SSH to become available...
==> digitalocean: Connected to SSH!
==> digitalocean: Provisioning with shell script: digitalocean/scripts/install-kubeadm.sh
digitalocean: Loaded plugins: fastestmirror
digitalocean: Determining fastest mirrors
digitalocean: * base: mirror.denit.net
digitalocean: * extras: mirror.nforce.com
digitalocean: * updates: centos.mirror.triple-it.nl
...
==> digitalocean: Gracefully shutting down droplet...
==> digitalocean: Creating snapshot: packer-1523190279
==> digitalocean: Waiting for snapshot to complete...
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
Build 'digitalocean' finished.
==> Builds finished. The artifacts of successful builds are:
--> digitalocean: A snapshot was created: 'packer-1523190279' (ID: 33289213) in regions ''
Now when I look at the available images using doctl:
$ doctl compute image ls
ID Name Type Distribution Slug Public Min Disk
33289213 packer-1523190279 snapshot CentOS false 25
I have a couple of flavours that I usually use; Just plain Docker CE, one of the latest stable builds, and Docker with kubeadm.
A simple change to the standard Docker CE installation script gave me a shared script to install Docker CE:
#!/bin/bash
# Installing Docker CE On CentOS distributions
# https://docs.docker.com/install/linux/docker-ce/centos/#install-docker-ce
export VERSION=${VERSION:-18.03.0.ce-1.el7.centos}
yum update -y
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce-$VERSION
systemctl start docker
Now I just needed to add some configuration for creating an image for multiple docker versions. I started with Docker v17.12.0 and Docker v18.03.0.
{
"builders": [
{
"type": "digitalocean",
"api_token": "MY_TOKEN",
"image": "centos-7-x64",
"region": "ams3",
"size": "s-1vcpu-1gb",
"ssh_username": "root",
"snapshot_name": "docker-v17.12.0-ce",
"snapshot_regions": ["ams3"]
}
],
"provisioners": [
{
"type": "shell",
"environment_vars": ["VERSION=17.12.0.ce-1.el7.centos"],
"script": "digitalocean/scripts/install-docker-ce.sh"
}
]
}
When building this with Packer for a couple different Docker versions, I ended up with the following images:
$ doctl compute image ls
ID Name Type Distribution Slug Public Min Disk
33289213 packer-1523190279 snapshot CentOS false 25
33289232 docker-v17.12.0-ce snapshot CentOS false 25
33289248 docker-v18.03.0-ce snapshot CentOS false 25
In Provisioning a Swarm mode cluster I talked about the Terraform module I use for creating a quick Docker swarm mode lab. I always either used the Docker image provided by DigitalOcean or ran an install script using user_data.
Instead I am now able to point towards the image I just created using Packer:
data "digitalocean_image" "docker_image" {
name = "docker-v17.12.0-ce"
}
module "swarm-cluster" {
source = "thojkooi/docker-swarm-mode/digitalocean"
version = "0.1.0"
total_managers = 1
total_workers = 1
domain = "do.example.com"
do_token = "${var.do_token}"
manager_ssh_keys = "${var.ssh_keys}"
worker_ssh_keys = "${var.ssh_keys}"
manager_os = "${data.digitalocean_image.docker_image.image}"
worker_os = "${data.digitalocean_image.docker_image.image}"
provision_user = "root"
manager_tags = ["${digitalocean_tag.cluster.id}", "${digitalocean_tag.manager.id}"]
worker_tags = ["${digitalocean_tag.cluster.id}", "${digitalocean_tag.worker.id}"]
}
So Packer is a nice and simple tool to use. I will definely be using this more. Storing a snapshot on DigitalOcean is only around $0.05 at time of writing.
Looking at the size of the images I just created, this is about $0.30 a month, and it saves me around 5 minutes when provisioning a new droplet.
thojkooi ~ $ doctl compute snapshot ls
ID Name Created at Regions Resource ID Resource Type Min Disk Size Size
33289213 packer-1523190279 2018-04-08T12:29:23Z [ams3] 88805641 droplet 25 1.57 GiB
33289232 docker-v17.12.0-ce 2018-04-08T12:36:51Z [ams3] 88806114 droplet 25 1.34 GiB
33289248 docker-v18.03.0-ce 2018-04-08T12:45:46Z [ams3] 88806835 droplet 25 1.37 GiB