Linux Incident Investigation
Contents
This is a living blog post where I collect shell commands that have proven useful when trying to fix a faulty Linux server.
Full Disk
When you run out of disk space this is a good way to find directory using up the most space:
du -cha --max-depth=1 / | grep -E "M|G"If you're using Docker a good hunch is to also check for too many stored images with docker image ls -a or for large volumes with docker system df -v.
If the images/volumes are unused, runnint docker prune can be enough to clean them up. Otherwise you have to manually wade through the command output, see which service is using too much storage and take appropriate steps.