Cleaning Up Space after Docker

Photo by Wil Stewart on Unsplash

You know how I love Docker and try to move everything inside containers. But a couple days ago I got a system warning that I have about 1Gb left on the root partition. That was a surprise. At the time I was in a Zoom call. And the first thing that came to my mind was “eh, must be Zoom eating up space”. But in the span of 30 minutes 1Gb turned to 100Mb. So I panicked a little, turned everything off, rebooted and got angry that it was still less than 100Mb on /.

Imagine my amazement when I discovered that /var/lib/docker/ was eating around 20Gb! What could it be? Turns out it was.. logs! Tons of logs from all the containers I have. And I only have around 10 running daily. Plus 5 more that I spin up when I need them. Such as Elastic, Kafka and Logstash.

Thankfully, the solution was quite easy:

sudo sh -c "truncate -s 0 /var/lib/docker/containers/*/*-json.log"

This will remove all the logs. That alone freed me 16Gb! My-my…

But I don’t want to run this command with Crontab. Or, even worse, run it manually from time to time. There must be a way to set Docker logging policy, I thought. And there sure is. Official documentation says how to configure logging and set the max number of files with their size. So, now my /etc/docker/daemon.json looks like this:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "1",
    "env": "os,customer"
  }
}

Meaning I’ll have only one file under 100Mb. Which I think should be suffice for the logs.

But, this will not affect already created containers! You’ll need to recreate every single one in order for them to respect the new logging config.

Hope this will save you some time and nerves if/when you’ll encounter a similar issue. So you can just focus on writing quality code 😊

Originally published at https://c-v-ya.github.io.