Manage Versa Analytics Log Archives
For supported software information, click here.
The Analytics log collector nodes in an Analytics cluster receive logs in IPFIX format and store them in files, called log files, in a local log storage directory, which, by default, is /var/tmp/log. By default, the Analytics log collector node runs a cron job once per hour that archives the log files stored in subdirectories under /var/tmp/log into subdirectories in the /var/tmp/archive directory. If you configure a log collector node to store log files in a different directory, one other than /var/tmp/log, you must create an additional cron job to archive the log files. Each Analytics log collector node has its own log storage directories, log archive directories, and cron jobs. You perform log archive tasks separately on each node.
To manage logs and log files, you can perform the following tasks:
- Create and edit archive cron jobs
- Display archive file details for a tenant
- Restore logs from the archive
- Transfer archived logs to another system
- Delete archived logs
This article describes how logs are archived and how to display, restore, transfer, and delete archived logs. This article also describes how to manage archive cron jobs.
Log Processing and Archiving
To understand archive and restoration commands, it is helpful to know how logs are processed in an Analytics log collector node.
An Analytics log collector node processes incoming logs in the following sequence:
- The local collector on the Analytics log collector node receives logs sent from Versa Operating SystemTM (VOSTM) devices.
- The local collector stores the logs in clear text files in its log storage directory, with one subdirectory for each organization. Each organization subdirectory contains a further subdirectory named for the routing instance that forwarded the log, and the incoming logs are collected into log files in these subdirectories.
- Any log files created in the routing instance subdirectories under /var/tmp/log are automatically processed into the cluster datastores by the Versa Analytics driver.
- After processing the log files, the Versa Analytics driver moves the log file into a backup directory under the /var/tmp/log directory.
- A cron job stored in /etc/cron.d/log-archive periodically archives all log files stored in the backup directories under /var/tmp/log. Also, any additional log archive cron jobs that you have configured archive logs stored in non-default log storage directories. The log archive cron jobs convert the clear text files in the log storage directories to compressed gzip format and move them to a log archive directory.
For information about Analytics configuration terms, such as log collector and Versa Analytics driver, see Versa Analytics Configuration Concepts.
Manage Log Archive Cron Jobs
The default archive cron job archives logs from the /var/tmp/log directory to the /var/tmp/archive directory. /var/tmp/log is the default log storage directory, and /var/tmp/archive is the default log archive directory. You can create additional cron jobs to archive logs from a non-default log storage directory to a non-default log archive directory.
Manage the Default Log Archive Cron Job
You can modify the default log archive cron job to change the log directory locations and the frequency of archiving.
You can change the /etc/cron.d/log-archive cron entry using scripts on the Analytics log collector node. To change the archive time interval, and to change the directories where the cron job places the log files to be archived and stores the archived log files, issue the following command:
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-start source-directory destination-directory (hourly | daily | weekly)
Enter the following information in this command:
| Field | Description |
|---|---|
| source-directory | Directory where the log files to be archived are stored. The default is /var/tmp/log. |
| destination-directory | Directory where log files are archived. The default is /var/tmp/archive. |
| (hourly | daily | weekly) | Enter the archiving time interval. |
To disable the archiving of log files, issue the following command:
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-stop
Manage a Non-Default Log Archive Cron Job
You can create or modify an additional log archive cron job. You create additional cron job to archive log files from a non-default log storage directory to a non-default log archive directory. If you later need to process logs from the non-default log archive directory into the Analytics datastores, you can restore the logs to /var/tmp/log, where the Versa Analytics driver automatically processes them. To restore logs from a non-default log archive directory, use the request system storage CLI command. For information about the request system storage command, see Restore Analytics Logs Using the CLI.
To archive logs from a non-default directory, create a cron job from a shell on the Analytics node:
- Create a directory in which to store the archive logs.
admin@Analytics$ sudo mkdir /var/tmp/flow-archive
- Create a cron archive job name and start it. (If you type the command on a single line, omit the \.) job-name is the name of the cron job. If you do not specify a job name, the cron job overwrites the default log archive job that archives logs in the /var/tmp/log directory.
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-start source-directory destination-directory \ (hourly|daily|weekly) [active | backup] [job-name] [checksum]
For example:
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-start /var/tmp/flow-log /var/tmp/flow-archive \ hourly active flow-log-archive
To check that the archive cron job has been created, display the files in the cron.d directory:
admin@Analytics$ ls -tlr /etc/cron.d/
total 20
-rw-r--r-- 1 root root 488 Mar 3 2017 munin-node
-rwxr-xr-x 1 root root 172 Aug 26 12:24 log-archive
-rwxr-xr-x 1 root root 103 Aug 26 12:24 vandb-repair
-rwxr-xr-x 1 root root 135 Sep 2 10:55 vandb-auto-delete
-rwxr-xr-x 1 root root 210 Sep 2 11:01 flow-log-archive
To analyze the archived flow logs, issue the request system storage archive restore command to restore the flow logs. For example:
admin@Analytics> request system storage archive restore start src-path /var/tmp/flow-archive dest-path /home/versa/logs start-date 2020/07/20 end-date 2020/07/20 no-confirm status success result log-restore job has been started; please check /var/log/versa/versa-log-restore.log
Display Archive File Information for a Tenant
For Releases 21.2.1 and later.
To display information about archived log files for a tenant from the Director GUI:
- In Director view, select the Analytics tab in the top menu bar.
- Select Administration > Maintenance > Log Archives in the left menu bar. The following screen displays.
- In the Driver Hosts field, select the Analytics log collector node containing the archive log files.
- Click View Archive Logs Details. The following screen displays.
- In the Select a Tenant field, select an organization, and then select a VOS device in the Select an Appliance field. The number of archive files matching the organization and appliance displays, along with the filenames of the oldest and newest archive files. The archive file name reflects the date the archive file was created.
- Click Close.
Restore Analytics Logs
Restoring Analytics logs restores archived logs to a destination directory. If the destination directory is /var/tmp/log, logs are automatically processed into the Analytics cluster datastores by the Versa Analytics driver. The restore operation restores archived logs on a single Analytics log collector node to a directory on the same node.
You can restore logs for a specific time period, organization, and VOS device.
Restore Analytics Logs Using the Director GUI
For Releases 21.2.1 and later.
To restore archived logs from the Director GUI:
- In Director view, select the Analytics tab in the top menu bar.
- Select Administration > Maintenance > Log Archives in the left menu bar. The following screen displays.
- In the Driver Hosts field, select the Analytics log collector node containing the logs to be restored. The archived logs must be in the /var/tmp/archive directory on the Analytics log collector node. You can restore them to any directory on the Analytics log collector node.
- Click Restore. The Restore Archive Logs screen displays. Enter information for the following fields.
Field Description Select Duration Select the starting and ending dates of the logs to restore. Click the field to display a calendar. Click a start date, then click an end date, and then click Apply to copy the dates to the field. Select a Tenant Select the organization whose archived logs to restore. If omitted, logs from all organizations are restored. Select an Appliance Select the VOS device whose archived logs to restore. This is a subcategory of the organization selected in the Select a tenant field. If omitted, logs from all appliances for the selected organization are restored. Destination Path (required) Enter the name of the directory in which to place the restored logs. - Click Confirm. The status of the restoration process displays on the screen.
Restore Analytics Logs Using the CLI
For Releases 21.1.1 and later.
When restoring Analytics logs from the CLI, you can start the restore operation, stop the operation, and view the status of the operation. The restoration process takes time, and you can stop the operation before it has completed.
To start an Analytics log operation, issue the following command from the CLI on the Analytics log collector node containing the logs:
admin@Analytics> request system storage archive restore start Possible completions: appliance - Appliance name of which logs need to be restored dest-path - Destination Path to which logs need to be restored end-date - End date till when logs need to be restored <yyyy/mm/dd> no-confirm - Do not prompt for confirmation src-path - Source path from where logs need to be restored start-date - Start date from when logs need to be restored <yyyy/mm/dd> tenant - Tenant name of which logs need to be restored
For example:
admin@Analytics> request system storage archive restore start src-path /var/tmp/archive dest-path /home/versa/logs start-date 2020/07/20 end-date 2020/07/20 no-confirm status success result log-restore job has been started; please check /var/log/versa/versa-log-restore.log
Enter the following information in this command:
| Field | Description |
|---|---|
| appliance appliance-name | Name of the VOS device that generated the logs. |
| dst-path destination-directory | Directory into which to place the restored logs. The default is /var/tmp/log. |
| end-date date | End date of the logs to be restored, in the format yyyy/mm/dd. |
| no-confirm | Restore the logs without prompting for confirmation. |
| src-path source-directory |
Location of the directory where the archives to be restored are located. |
| start-date date | Start date of the logs to be restored, in the format yyyy/mm/dd. |
| tenant tenant-name | Name of the tenant or customer. |
The restored Analytics logs are placed in the source directory that you specify in the src-path option. The source directory contains one directory for each tenant, and each tenant directory contains a subdirectory for each VOS device.
To view the status of an Analytics log restore operation, issue the following command:
admin@Analytics> request system storage archive restore status
For example:
admin@Analytics> request system storage archive restore status status success result log-restore job is still running PID: 5289
To stop an Analytics log restore operation that is in progress, issue the following command:
admin@Analytics> request system storage archive restore stop
For example:
admin@Analytics> request system storage archive restore stop Will stop all log-restore jobs Are you sure? [no,yes] yes status success result Stopped log-restore job
When you stop an Analytics log restore operation that is in progress, the restore operation is stopped midstream. The logs that were already restored before you issued the stop command remain restored, but no additional logs are restored. This means that some archives are restored, while others are not.
Restore Analytics Logs Using Scripts
To restore the Analytics logs, issue the following command from a shell on the Analytics log collector node:
/opt/versa/scripts/van-scripts/log-restore.py --src source-directory --dst destination-directory --tenant tenant-name --appliance appliance-name --start-date date --end-date date
Enter the following information in this command:
| Field | Description |
|---|---|
| src source-directory | Directory that contains the archived logs files. The default is /var/tmp/archive. |
| dst destination-directory | Directory into which to place the restored files. The default is /var/tmp/log. |
| tenant tenant-name | Name of the tenant or customer. |
| appliance appliance-name | Name of the appliance that generated the logs. |
| start-date date | Start date of the logs to be restored, in the format yyyy/mm/dd. |
| end-date date | End date of the logs to be restored, in the format yyyy/mm/dd. |
The following example shows how to issue the command:
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-restore.py --src /var/tmp/archive/ --dst /var/tmp/log --tenant tenant-Customer1 --appliance VSN0-Branch1 --start-date 2017/06/12 --end-date 2017/06/13
Transfer Archived Logs
To transfer the Analytics archived log data to an external storage device, issue the following command:
/opt/versa/scripts/van-scripts/log-archive-transfer.py --src source-directory --user username --dst-host destination-host --dst destination-directory [--ssh-key ssh-key-filename]
Enter the following information in this command:
| Field | Description |
|---|---|
|
src source-directory |
Local directory that contains the archived logs files. The default is /var/tmp/archive. |
| user username |
Username to use to log in to the remote storage device. |
| dst-host destination-host | IP address or hostname of the remote storage device. |
|
dst destination-directory |
Remote directory into which to place the restored files. |
|
sh-key ssh-key-filename |
(Optional) Directory path and filename that contains the SSH key to use to access the remote storage device. The SSH key allows you to log in to the remote device without a password. If this option is not included, the ~/.ssh/id_rsa.pub file is used. |
Transferring the archived logs copies them to the external storage device. It does not remove the files from the Analytics node.
You can call the log-archive-transfer.py script from within a cron job. The script output is stored in the /var/log/versa/versa-log-archive-transfer.log file.
The following example shows how to issue the command:
admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-transfer.py --src /var/tmp/archive --dst /var/tmp --dst-host 10.40.24.3 --user versa
Delete Archived Logs
Delete Archived Logs from the Director GUI
For Releases 21.2.1 and later.
To delete archived Analytics logs using the Director GUI
- In Director view, select the Analytics tab in the top menu bar.
- Select Administration
> Maintenance > Log Archives in the left menu bar. The following screen displays.
- In the Driver Hosts field, select the Analytics log collector node containing the logs to be deleted. The logs must be in the /var/tmp/archive directory on the Analytics log collector node.
- Click Delete. The Delete Archive Logs screen displays. Enter information for the following fields.
Field Description Select Duration Select the starting and ending dates of the logs to be restored. Click the field to display a calendar. Click a start date, then click an end date, then click Apply to copy the dates to the field. Select a Tenant Select the organization whose archived logs to delete. If omitted, logs from all organizations are deleted. Select an Appliance Select the VOS device whose archived logs to delete. This is a subcategory of the organization selected in the Select a tenant field. If omitted, logs from all appliances for the selected organization are deleted. - Click Confirm. The status of the deletion process displays on the screen.
Delete Archived Logs Using the CLI
For Releases 21.1.1 and later.
To delete archived Analytics logs, issue the following command from the CLI:
admin@Analytics> request system storage archive delete start Possible completions: directory - Directory from which to delete archive logs end-date - End date till when archive logs need to be deleted <yyyy/mm/dd> no-confirm - Do not prompt for confirmation start-date - Start date from when archive logs need to be deleted <yyyy/mm/dd>
For example:
admin@Analytics> request system storage archive delete start start-date 2020/07/01 end-date 2020/07/01 directory /var/tmp/archive/tenant-Corp-Inline-Customer-1/VSN0-Bangalore-ECT-DC-Active Will clear all log files from '2020/07/01' to '2020/07/01' in '/var/tmp/archive/tenant-Corp-Inline-Customer-1/VSN0-Bangalore-ECT-DC-Active' Are you sure? [no,yes]yes status success result Clearing archive logs, please check /var/log/versa/log-archive-cleanup.log for details
Enter the following information in this command:
| Field | Description |
|---|---|
| directory directory-name | Directory from which to delete the archived logs. |
| end-date date | End date of the logs to be deleted, in the format yyyy/mm/dd. |
| no-confirm | Delete the logs without prompting for confirmation. |
| start-date date | Start date of the logs to be deleted, in the format yyyy/mm/dd. |
To view the status of an Analytics log deletion operation, issue the following command:
admin@Analytics> request system storage archive delete status
For example:
admin@Analytics> request system storage archive delete status last-run-time : 18-08-2020 10:00:01 AM archive-disk-utilization : /var/tmp/archive - 4.0K archive-delete-job : Archive delete job is not running
Supported Software Information
Releases 20.2 and later support all content described in this article, except:
- For Releases 21.1.1 and later, you can restore Analytics logs from the CLI. You can delete archived logs.
- For Releases 21.2.1 and later, you can restore and delete archived log files from the Director GUI. You can display details of archived log files based on their organization and VOS device.