Event retention helps QRadar administrators keep up and organize the data collected by their SIEM system.
Retention window.
Click the Admin tab Retention window to configure the buckets applicable to your deployment. By default, the Event Retention functionality provides a default retention bucket and ten not configured retention buckets.
System stores all events in the default retention bucket if you don’t configure any event retention bucket, You can check files in /store/ariel/events/payloads/year/month/day/hour, where each minute file has added postfix with number 0 to its name. Each file keeps one-minute events stored in the specific bucket as seen on the picture. Once, you configure buckets to store your data, then you will be able to find files with other numbers, indicating bucket number, where the specific data kept by QRadar.
After that, each configured bucket has own retention policy for events and flows that match custom filter requirements. When QRadar receives events or flows, each piece of data system assigns to configured retention bucket matching policy or stored in default container.
Managing event retention bucket sequence.
You can change the order of the retention buckets to make sure that data matches against the retention buckets in the order of your requirements. Priority of retention buckets is from the top to the bottom. Bucket keeps data until reached policy of time period. There are two options for setting data cut policy in the bucket. The Bucket can keep data as long as there is free space available or cut data just after configured storage time elapse.
There are ten default free buckets, which let you configure multiple retention policies. Incoming events check configured policy of retention buckets from the top to the bottom. The first bucket matching filters for incoming record keeps this record despite that the next one could also have similar positive filter criteria. Default retention bucket stores the record which does not match any of the configured buckets policy. Default bucket always lies on the bottom of the list available buckets. There are multiple properties, for which you can configure policy, including Custom Event Properties (properties extracted from payload)
Configuring event retention buckets
From the top menu, you can edit a retention bucket. When you disable or enable a bucket, any new data that matches the requirements for the disabled bucket is stored in the next bucket that matches the properties or eventually in the default bucket. The data is not removed from the system, after drops of the configured bucket, but still containing data. There are only filters definitions cut and default storage keeps the data.
Compression of data (only for QRadar below version 7.2.7)
Prior QRadar version 7.2.7, if you wanted to enable data compression, you could click the Allow data in this bucket to be a compressed drop down menu and then select the applicable time frame from the list. After the configured time, system compressing all events found in the retention bucket. When the used disk space usage reached 83% for payloads and 85% for records (see /opt/qradar/conf/arielConfig.xml) then compression could occur. By default, the Compression setting was one week. The minimum setting was Never and the longest setting was two weeks. Each new versions of QRadar have fully compressed each piece of data and this option is no more available.