Why? Why would you run Docker on ZFS?
I discovered ZFS when I was playing with LXD, because it’s the recommended storage driver for it.
While ZFS has a lot of great features, the ones I like the most are the RAM cache, compression, and snapshots.
After moving from LXD to Docker (so leaving ZFS), I felt the difference in speed the cache gave me, and I saw some files like databases being 3 times bigger.
Well, running Docker on ZFS shouldn’t be difficult, right? No, it really isn’t!
In fact, there is even a page dedicated in the docs.
Install Docker
First, we need Docker, obviously.
According the the docs:
curl -fsSL get.docker.com -o get-docker.sh
sh get-docker.sh
Stop it while we mess with ZFS:
service docker stop
Remove the docker data:
rm -rf /var/lib/docker
Create a zpool
You can help yourself with this article of mine to setup a ZFS pool.
You have two choices. If you intend to only store Docker data on your pool, mount it on /var/lib/docker.
If you intent to store other stuff, just mount it anywhere you want, we’ll user an option later to specify the Docker data directory.
As for me, my zpool is structured like this:
/zpool
├── docker
└── srv
The equivalent of /var/lib/docker is in /zpool/docker, and my data is in /zpool/srv (bind mounts etc).
Use ZFS as Docker’s storage driver
Let’s check what storage driver Docker uses by default:
root@hinata ~ # docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.03.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
To use ZFS as the storage driver, we need to specify it to the deamon.
Create /etc/docker/daemon.json and add:
{
"storage-driver": "zfs"
}
If your zpool is not mounted on /var/lib/docker your need to use the “graph” option like this in my case:
{
"storage-driver": "zfs",
"graph": "/zpool/docker"
}
You can now start docker. Docker will copy basic data into the data folder:
root@hinata ~ # service docker start
root@hinata ~ # ls /zpool/docker/
builder containerd containers image network plugins runtimes swarm tmp trust volumes zfs
Make sure Docker is using your zpool:
root@hinata ~ # docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.03.0-ce
Storage Driver: zfs
Zpool: zpool-docker
Zpool Health: ONLINE
Parent Dataset: zpool-docker
Space Used By Parent: 81920
Space Available: 143545778176
Parent Quota: no
Compression: ons
When we launch a new container, we can see it’s mounted into the zpool:
root@hinata ~# zfs list -r zpool-docker
NAME USED AVAIL REFER MOUNTPOINT
zpool-docker 10.7G 123G 10.7G /zpool
zpool-docker/41fb183f4bdc6c4a965261a8700e0c9f9faf6777a35f8ed5fc5040ebef735518 20.5K 123G 20.5K legacy
zpool-docker/accec25e0803671ce053645c93efcab5afcd6f290115d3583a54adfb3c99acee 10K 123G 22.5K legacy
zpool-docker/accec25e0803671ce053645c93efcab5afcd6f290115d3583a54adfb3c99acee-init 19K 123G 22.5K legacy
That’s it, you can now enjoy Docker on ZFS!
As I said, I have all my data and Docker data in my zpool. I saw a performance improvement by moving back my containers to ZFS, especially when they’re IO intensive (obviously), like databases.
The documentation says you should use Ubuntu for ZFS, but I knew it worked fine on Debian because I tested with LXD, and I confirm it works fine with Docker too.
Again, you should really read the documentation because it’s more detailed and goes more in depth on how ZFS works.