Getting Docker On ZFS

So I've been working with docker more and more at work, and I decided to give the docker integration on my antsle a go.

So, it works pretty well in that docker is installed and usable.  However, 1) don't expect the docker containers to appear in AntMan.  (I didn't but I figure this should get mentioned) and 2) the default installation puts your images and containers in /var/lib/docker which is on the 16GB root partition.

It took me a bit to find all the pieces and parts to get this in order.  It's all out there if you want to wade through the tutorials (which you should, you should wade through the tutorials) but here is how I solved the problem... by moving my docker directory to a zfs mount in /antlets.  Probably not the best place for it, but it was quick and easy.

Quick guide on moving this to into ZFS:

1. Create a zfs volume

root@myantsle:~ # zfs create antlets/docker

2. Back up your current docker directory.

root@myantsle:~ # cp -r /var/lib/docker /antlets/docker/olddockerdir

3. Now you'll need to let docker know to use this new mountpoint.  Create and edit the file /etc/docker/daemon.json and add these lines. (The default seems to be to not include this file.  So I'm going to assume that if you have an /etc/docker/daemon.json file, you'll know how to add what's needed:

{
"storage-driver":"zfs",
"graph":"/antlets/docker"
}

4.  Now you'll just need to restart docker:

root@myantsle:~ # rc-service docker restart

Note that if you want to run a local registry that supports http then you'll want your /etc/docker/daemon.json file to look like this:

{
"storage-driver":"zfs",
"graph":"/antlets/docker",
"insecure-registries":["myantsle:5050"]
}

Just replace the 5050 with the port you've mapped your registry to.  IIRC, the default on the registry image from docker is 5000 but I already had something running on that port.

Also note: I didn't have many running containers, so I didn't bother figuring out how to move my existing containers to the new directory.  I tried the obvious (copying /var/lib/docker/* to /antlets/docker) but that didn't work and I didn't want to spend a lot of time figuring out the process.  If anyone already has it, please let me know.

D'oh, I forgot to mention:

It appears that antman uses a docker container for the in-browser ssh session.  Using the process above, the in-browser ssh session won't start.

For me that is ok, but if you use that then you'll need to figure how to transfer that container.

Hi @cawsllyffant,

This was most likely a mount issue from factory. Docker should have been using zfs by default. This was a known issue in older antsles (~1 year back), maybe that was your case?

Anyway, we do something similar. We create a docker file system mouned at /var/lib/docker by running:

zfs create antlets/_docker -o mountpoint=/var/lib/docker

With the underscore ( _ ) as prefix. That's our standard for non antlet filesystems.

Then /etc/docker/daemon.json, we set:

{
"storage-driver": "zfs"
}

My system is from 2017, so certainly falls into <1 year old category.

Still going strong too, despite falling on my big toe from ~3 feet high.  Thing is built (and weighs) like a tank. :)