Resolving 'Root Partition Is Filling Up' Issue on Sophos UTM Firewall

This is a quick guide on how to resolve an occasional problem that could happen to neglected Sophos UTM firewall with default settings, where updates are automatically downloaded to device, filling up storage after some time.

An email alert similar to the below is sent to administrator when it happens:

From: Firewall Notification System [do-not-reply@fw-notify.net]
Sent: 18 June 2016 11:58 PM
To: Administrator [admin@wandersick.ws]
Subject: [SOPHOS][INFO-150] Root partition is filling up - please check

Root partition is filling up - please check. Current usage: 86%
--
System Uptime      : 362 days 1 hour 2 minutes
System Load        : 0.05
System Version     : Sophos UTM 9.210-20

Please refer to the manual for detailed instructions.

To solve it, one way is to
  1. SSH into Sophos UTM firewall using root login
  2. Remove content under Sophos update directory (Main focus of this blog post)
  3. Configure firmware download interval for System Up2Date packages from Automatic to Manual

1. SSH into Sophos UTM firewall using root with public key authentication

In order for root login to be made possible, it is required to first generate a private key and set up public key authentication. You may refer to any guide found on the Internet such as this one.
After enabling root login, we can then authenticate using the private key file using SCP/SSH clients such as WinSCP, PuTTy by specifying private key file as shown below.

a. Specifying private key file in WinSCP SCP client:

image

b. Specifying private key file in PuTTy SSH client:

image

2. Remove content under Sophos update directory

Commands for determining storage utilization on directory /var

  • cd /var
  • du -shx  *

After root SSH login, executing the above directory size checking commands revealed the /var/up2date folder (containing downloaded firmware updates) took 2GB of space on the device.

image

Mitigate the situation by removing the update files under /var/up2date/sys. Below screenshot shows a way to do it using WinSCP. (Backing up the files is recommended prior to deletion.)

image

Command for listing space utilization statistics on each file system mounted on device

  • df -h

Before removing the files, the disk utilization was 86%:

image

After removing the files, the disk utilization lowered to 46%:

image

3. Configure download interval for System Up2Date packages as Manual

To prevent this problem from recurring, firmware download interval can be set to Manual on the firewall. After configuring the setting, firewall updates will not be automatically downloaded, hence device storage won’t be filled up as easily.

Tip: Be sure to do this quickly after moving/deleting the files. Otherwise, the device storage will be filled up again by the update download next time

clip_image002

Hope this post provides a quick way to resolve this problem for those who encounter it.

Update in April 2017: Fix it as early as we can! It was found that if we leave the problem unresolved until the firewall disk partition becomes 100% full, web administration console (httpd) will fail with the error below, and we must recover the firewall via console logon (or using Windows PE/Linux boot disk).

Internal Server Error

The server encounted an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at admin to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

Reference

let's find where the problem is, from the command line as root:

cd /
du -shx *

And then go into the largest in the list and do du -shx * again.

It looks like you're now on 9.318, so I would set the Download interval to "Manual" and delete everything in /var/up2date/sys

Comments

  1. I was faced the same problem, and has been rectified.

    thanks its working.

    ReplyDelete
  2. Great ... nailed it
    thanks

    ReplyDelete
  3. Thanks very much for taking the time to put this post together, it was very helpful for me to fix this "root partition filling up" issue on a Sophos UTM device with a poor history of applying updates.

    ReplyDelete

Post a Comment