Sometimes you might find yourself in the odd situation where Linux is reporting that you’re out of disk space. But when you examine the output of the du -h command it turns out that you have lots of space.
So what gives?
The answer is probably inodes.
inodes are internal file system data structures that describe files and directories.
The number of possible inodes is limited and set during partition creation. That means you can run out of them and be unable to create any new files, even if there is a lot of available space on the device.
Normally this isn’t an issue. BUT, if a process ends up creating a lot of tiny files without deleting them then you’re likely to run out of inodes.
To find out if you’re out of inodes, run:
sudo df -i
The output will look something like this:
The column IUse will contain the percent of inodes that have been used. Chances are that, if you’re reading this article, one of those entries will contain ‘100%’.
So how do you find out where these millions of tiny files are located?
In the case of WordPress and the world of webservers, the issue is likely to involve the OPENLITESPEED webserver and PHP.
In particular, the failure of LITESPEED/PHP to delete PHP session files.
Session files are located in the /var/lib/lsphp/session/lsphp?? folders. eg: for php 8.1 they are located in the /var/lib/lsphp/session/lsphp81 folder.
To find out if there are a lot of files in these folders, navigate to /var/lib/lsphp/session/lsphp81:
Then run the following command:
sudo find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n
The output will include the number of files below each php folder.
In the example above, you can see that the lsphp81 folder has over 15 million files!
If you were to navigate into it and list out the files you will see that the files all start with ‘sess_’.
You might think that the obvious solution would be to try a command such as rm sess_*. Except that will not work – you’ll end up with an error along the lines of
bash: /usr/bin/rm: Argument list too long
This is because Linux/bash is expanding the asterisk into individual file names before processing the command. (It’s just how the bash interpreter is designed to work).
Instead, you have to navigate into the problematic folder and then use the following command:
sudo find . -name "sess_*" -delete
With millions of files, it will likely take a long time to run. And you’ll probably need to repeat it for all the other PHP version folders.
Once you’ve finished cleaning up, running df -i again should show a low percent usage of inodes.