Security Onion: Delete or Reset Snort and Trisul Data Directories

We are building a portable IDS that we take from location to location to assess different legs of the network.  The concept was to build the box, test it out in the office, configure and apply upgrades, take an image of it in case we needed to restore it, and then send it out into the wild.  The problem we were having is that we couldn’t image it easily because the amount of data we accrued during test in the office environment.  So we needed a way to reset the box to remove all sensor data.  We were primarily using snort and trisul network analytics on security onion.

Trisul

For trisul this is very simple.  Check the size of your trisul data directory first:

du -hs /nsm/trisul_data

Then you can reset it with:

cd /usr/local/share/trisul
./cleanenv -f -saveinit

You may need to supply additional arguments, particularly if you are working with contexts.  the “-saveinit” argument is important if you have defined any changes, but includes your interfaces and home networks, so whether or not you include this is up to you, particularly on a portable box.  Get more info on trisul cleanenv script.

Snort

As for snort, security onion makes sure disk use is below 90% with an hourly cron job, but if you need to delete all data, right now, so you can change networks or image a disk you are out of luck with that cron.  I’ve run the contents manually using:

/usr/local/sbin/nsm_sensor_clean --force-yes

That would recover some disk space, but not for our purposes.

According to the Security Onion FAQ, pcaps are stored in /nsm/sensor_data/NAME_OF_SENSOR/dailylogs/ and you can verify their disk usage with du -hs.  Ours was 293G.  You can delete these files by replacing NAME_OF_SENSOR with your sensor name and issuing the following command as root:

rm -rf /nsm/sensor_data/NAME_OF_SENSOR/dailylogs/*

MySQL

Something a little more tricky.  For whatever reason, security onion or snort stores data for each day and interface in their own set of tables.  This is a pain to clean by hand so don’t try.  First find out if this is an issue by logging in to mysl from shell using:

mysql -uroot

There is no root password.  You can find out the disk size of each of your MySQL databases using this:

SELECT
   table_schema, count(*) TABLES,
   concat(round(sum(table_rows)/1000000,2),'M')
   rows,concat(round(sum(data_length)/(1024*1024*1024),2),'G')
   DATA,concat(round(sum(index_length)/(1024*1024*1024),2),'G')
   idx,concat(round(sum(data_length+index_length)/(1024*1024*1024),2),'G')
   total_size,round(sum(index_length)/sum(data_length),2) idxfrac
FROM
   information_schema.TABLES group by table_schema;

I can see that my big databases are snorby and securityonion_db. You can find out which are the big tables if you like using this:

SELECT table_name,
           round(((data_length + index_length) / (1024*1024)),2) as "MBytes"
FROM information_schema.tables
WHERE table_schema = "securityonion_db";

That will give you some file sizes so we know the effects of our commands. Also, you can check the entire directory like so.

du -hs /var/lib/mysql

Mine was 1.4G – not acceptable for taking an image.  Now you can purge the sguil data from security onion using a provided script at /usr/local/bin/sguil-db-purge.  My suggestion to you is to copy this file into your home directory and call it just “db-purge” so you don’t get confused.

In order for this to work, you must find the lines:

DAYSTOKEEP=365
source /etc/nsm/securityonion.conf

….and change them to:

source /etc/nsm/securityonion.conf
DAYSTOKEEP=0

First, if you don’t change the order, you’ll cause the securityonion.conf script to override the variable.  Changing it to 0 will delete all the sguil archives. Now run that script:

/usr/local/bin/sguil-db-purge

That’s good.  It got my /var/lib/mysql down to 678M, but still have more work to do. There’s still a matter of snorby.  However for me this made up only about 250MB and I wasn’t able to get it to work so I left it.  But the suggestion on the list was to run the following command, but it just caused an error for me.

bundle exec rake snorby:hard_reset

That’s good enough for me.  You could keep on going if you want by moving into the /nsm directory and deleting logs for things like httpry. I’m prepared to move along at this point with a mostly fresh install, already configured, and ready for imaging.