Configure Log Collectors and Log Exporter Rules

Configure a Remote Collector

To configure a remote collector, you define the destination IP address of the remote server and information about the connection to the remote server. Analytics log collector nodes stream logs to remote destinations using TCP and UDP as the underlying connection mechanisms. For Releases 21.1 and later, you can stream logs securely over a Transport Layer Security (TLS) connection. For TLS connections, the certificates must be in Privacy Enhanced Mail (PEM) format, and you cannot use certificate chains for certificate verification.

For remote collectors that use a Kafka cluster as their destination, you must configure them from the CLI. See Configure a Kafka Remote Log Collector, below.

To allow the Analytics log collector node to stream logs and alarms to a Director node, you configure a remote collector with the destination port TCP 20514 and the destination address of the Director node. For more information, see Export Alarm Logs to Director Nodes.

To configure a remote collector:

1. In Director view, select the Analytics tab in the top menu bar.
2. Select Administration > Configurations > Log Collector Exporter in the left menu bar.
3. Click the Remote Collector tab. The following screen displays.
   
   
    
4. In the Driver Hosts field, select the Analytics log collector node .
5. Click the Add icon. The Remote Collector popup window displays. Enter information for the following fields.
   
   

   Field	Description
   Name (Required)	Enter a name for the remote collector.
   Description	Enter a text description for the remote collector.
   Destination Address	Enter the destination IP address of the remote system.
   Destination Port	Enter the port number to use to connect to the remote system. To stream logs and alarms to a Director node, configure TCP port 20514.
   Destination FQDN	(For Releases 21.1.1 and later.) Enter the fully qualified domain name (FQDN) of the destination system. To use FQDN, the DNS server listed in the /etc/resolv.conf file must be reachable from the log collector to perform the name resolution. Alternately, add the hostname and IP address to the /etc/hosts file.
   Pending Queue Limit	Enter the maximum number of log messages that can be queued for sending to the remote collector. Default: 2048
   Source Address	Enter the IP address of the Analytics log collector node. This is the address of the southbound interface on the Analytics node.
   Transmit Rate	Enter the maximum rate at which to transmit log messages to the remote collector. Default: 1000 per second
   Type	Select the transport type to use on the connection between the Analytics log collector node and the remote system: TCP TLS (for Releases 21.1.1 and later)—Stream the logs securely. UDP If you select TLS for secure transport, TLS Attributes fields display. Enter information for the following fields, all of which are required.   CA Certificate Path—Enter the path to the CA certificate. Private Key Path—Enter the path to the private key. Client Certificate Path—Enter the path to the client certificate.
   Template	Select the remote collector template to apply. For more information, see Configure a Remote Template, above.

6. Click Save.

To configure a remote collector from the CLI, issue the following commands:

set log-collector-exporter remote collectors director-collector destination-address ip-address
set log-collector-exporter remote collectors director-collector destination-port port-number
set log-collector-exporter remote collectors director-collector ip-address
set log-collector-exporter remote collectors director-collector transport transport
set log-collector-exporter remote collectors director-collector template director-template

You can also configure a remote collector to be the primary collector, which is useful when you also configure a remote collector group to provide HA. In a remote collector group, the primary collector receives the log stream. (For Releases 21.1.1 and later, you can select a primary collector from Director when you create a log collector group.) You can configure the primary log collector from the CLI, by issuing the following command:

set log-collector-exporter remote collectors primary-collector collector-name

To verify the remote collector configuration, issue the following command:

admin@Analytics% show configuration log-collector-exporter remote collectors | find director-collector
director-collector {
   destination-address  192.168.101.3:
   destination-port     20514;
   source-address       192.168.101.4;
   transport            tcp;
   template             director-template;
}

Configure a Remote Collector Group

To provide HA for remote collectors, you can configure multiple remote collectors, as described in the previous section, and then place them into a remote collector group. You can configure one of the remote collectors to be the primary collector. The logs are streamed to the primary collector when it becomes active. When the connection to the active collector is unavailable, logs are streamed to next collector in the group. Logs continue to stream to this collector even if the initial active collector again becomes available.

To configure a remote collector group:

1. In Director view, select the Analytics tab in the top menu bar.
2. Select an Analytics cluster node:
   • For Releases 22.1.1 and later, hover over the Analytics tab and then select an Analytics node.
   • For Releases 21.2 and earlier, select an Analytics node in the horizontal menu bar.
3. Select Administration > Configurations > Log Collector Exporter in the left menu bar.
4. Select the Remote Collector Group tab. The following screen displays.
   
   
    
5. In the Driver Hosts field, select the Analytics log collector node.
6. Click the Add icon. The Remote Collector Group popup window displays. Enter information for the following fields.
   
   

   Field	Description
   Name	Enter a name for the remote collector group.
   Description	Enter a text description for the remote collector group.
   Collectors	Highlight the remote collectors to include in the group by clicking on the remote collector name. Control-click to select or deselect individual remote collectors.
   Primary Collector	Select a primary remote collector.

7. Click OK.

Configure a Remote Collector Group List

To provide redundancy, you use a remote collector group list to export logs to multiple remote systems. To do this, you place two or more remote collector groups into a list, and then logs are streamed to the active collector of each remote collector group. You define the list on the same screen on which you define a remote profile. See Configure a Remote Profile, below.

Configure a Remote Profile

You configure a remote profile to assign a name to an individual remote collector, a remote collector group, or a remote collector group list. You then reference the remote profile when you configure a log exporter policy.

To configure a remote profile:

1. In Director view, select the Analytics tab in the top menu bar.
2. Select an Analytics cluster node:
   • For Releases 22.1.1 and later, hover over the Analytics tab and then select an Analytics node.
   • For Releases 21.2 and earlier, select an Analytics node in the horizontal menu bar.
3. Select Administration > Configurations > Log Collector Exporter in the left menu bar.
4. Select the Remote Profile tab. The following screen displays.
   
   
    
5. In the Driver Hosts field, select the Analytics log collector node.
6. Click the Add icon. The Remote Profile popup window displays. Enter information for the following fields.
   
   
    

   Field	Description
   Name (Required)	Enter a name for the remote profile.
   Description	Enter a text description for the remote profile.
   Collector Fields (Required)	Choose one of the collector fields.
   Collector	Click, and then select the name of a remote collector.
   Collector Group	Click, and then select the name of a remote collector group.
   Collector Group List	Click to associate the remote profile with a remote collector group list. Then, control-click individual remote collector groups to add them to the list.

7. Click Save Changes.

Configure Log Exporter Rules

To define the specific type of logs to stream from a local collector to a remote collector, you configure log exporter rules that match log types and tenants and export them to the remote collector defined in a remote profile. When you define the rules, ensure that the match criteria are unique in the local collector. If you configure overlapping match criteria, the rule that matches first is used first and the other rules are ignored.

To configure log exporter rules:

1. In Director view, select the Analytics tab in the top menu bar.
2. Select an Analytics cluster node:
   • For Releases 22.1.1 and later, hover over the Analytics tab and then select an Analytics node.
   • For Releases 21.2 and earlier, select an Analytics node in the horizontal menu bar.
3. Select Administration > Configurations > Log Collector Exporter in the left menu bar.
4. Select the Exporter Rules tab. The following screen displays.
   
   
    
5. In the Driver Hosts field, select the Analytics log collector node.
6. Click the Add icon. The Exporter Rules popup window displays. Enter information for the following fields.
   
   
    

   Field	Description
   Name (Required)	Enter a name for the log exporter rule.
   Local Collector	Select the local collector from which the log messages are exported to the remote collector.
   Log Types	Select one or more log types to export. If you do not select any log types, all the listed log types are exported. For a description of the log types, see Analytics Log Collector Log Types Overview.
   Tenants	Enter the names of one or more tenants to export logs only for those tenants. Separate the tenant names with commas. If you do not enter any tenant names, logs for all tenants are exported.
   Remote Collector Profile	Select the name of the remote profile you configured, as described in Configure a Remote Profile, above. The remote profile specifies the remote collector, remote collector group, or remote collector group list that receives the exported logs.

7. Click Save Changes.

For Releases 21.1 and later, you can match log subtypes for some of the log types, and you can configure subtype match parameters only from the CLI, in the alarm-types option. The following example shows how to configure a log exporter policy that includes subtypes from the Analytics CLI:

admin@Analytics% show log-collector-exporter
exporter {
    rules {
        r1 {
            match {
                # Specify the local collector from which logs are received
                local-collector lc1;
 
                # Match on the specific log type.
                # If omitted, all logs are exported.  
               log-types [ alarm-log ]
 
                # Match a specific log field value.
                # If omitted, all configured log types or all logs are exported.
                # Example here matches specific alarm type in an alarm log.
                # alarm-type option available in Releases 21.1.1 and later.
                features {
                    alarm-log {
                        alarm-type [ interface-down ];
                    }
                }
 
                # Optional match list of tenants.
                # If omitted, logs of all tenants are exported. 
                tenants [ t1 t2 ]
            }
            set {
                remote-collector-profile rp1;
            }
        }
    }
}

For Releases 22.1.1 and later, you can configure export rules to match alarm logs based on the alarm text. You can match up to four alarm text strings in match criteria per exporter rule. Issue the following CLI command on the Analytics node running the log collector exporter:

admin@Analytics% set log-collector-exporter exporter rules alarm-rule match features alarm-log alarm-text list

Example: Configure a Primary Log Collector in a Log Collector Group

This section provides an example of how to configure from the CLI a log collector group that has a primary log collector. For Releases 21.2.1 and later, you can configure a primary log collector from the Director GUI when you configure a remote collector group. (See Configure a Remote Collector Group, above.) Here, the log collector group is called cg1, and it has two remote log collectors, rc1 and rc2, with rc2 being the primary log collector:

admin@Analytics% show log-collector-exporter remote collector-groups
cg1 {
    collectors        [ rc1 rc2 ];
    primary-collector rc2;
}

When the Analytics node initiates connections to the remote log collectors, rc2 is chosen as the primary collector, and the Analytics node streams logs to rc2, which also becomes the active collector. To check the status of the two remote log collectors, issue the following command:

admin@Analytics% show log-collector-exporter remote collector-groups cg1 status
status
primary-collector rc2
active-collector  rc2

If the primary collector, rc2, goes down, the other collector in the group, rc1, becomes the active collector, and the Analytics node streams logs rc1:

admin@Analytics% show log-collector-exporter remote collector-groups cg1 status
status
primary-collector rc2
active-collector  rc1

If the primary collector, rc2, comes up back up again, the logs are not again streamed to rc2 even though it is primary collector. Instead, the Analytics node continues to stream logs to rc1 because that connection is still up. This is illustrated by the following commands. First, you can see that the connections to both remote log collectors are up (their Connection State is Established):

admin@Analytics% show log-collector-exporter remote collectors connection
REMOTE
COLLECTOR  CLIENT  SERVER        SERVER  CONNECTION
NAME       SOCKET  ADDRESS       PORT    STATE
------------------------------------------------------
rc1        15      10.40.93.3    516     established
rc2        23      10.40.199.55  515     established

Then, you can see that rc1 is still the active log collector even through rc2 is the primary collector and is available:

admin@Analytics% show log-collector-exporter remote collector-groups cg1 status
status
primary-collector rc2
active-collector  rc1

Configure a Kafka Remote Log Collector

You can configure a remote collector to connect to a Kafka cluster defined by list of brokers. You can use Kafka clusters to manage the persistence and replication of message data. You can define Kafka topics, which represent the named stream of records. Messages are mapped to these Kafka topics based on topic map configuration. If there is no such mapping, the default topic is used for all logs.

For Releases 22.1.1 and later, when you configure export rules for the log collector exporter, you can enable streaming of logs to a Kafka cluster with different Kafka topics per exporter rule.

To enable streaming to a non-default Kafka topic, issue the following commands on the Analytics node running the log collector exporter. Here, "threat-rule" is the unique exporter rule name, the the topic map number is an integer between 1 and 7 that corresponds to a topic map number assigned in the remote collector Kafka configuration.

admin@Analytics$ cli
admin@Analytics> configure 
admin@Analytics% set log-collector-exporter exporter rules threat-rule set parameters topic topic-map-number

The following example associates topic map number 1 to topic name HI-PRI for a remote collector named kafka1:

admin@Analytics% set log-collector-exporter remote collectors kafka1 kafka topic-map 1 topic HI-PRI

To display the Kafka remote collector configuration, issue the show log-collector-exporter remote collectors CLI command. For example:

admin@Analytics% show log-collector-exporter remote collectors kafka-west

transmit-rate 10000;         # Number of messages that would be transmitted per second
pending-queue-limit 100000;  # After the limit is hit, messages are dropped
template syslog-template;    # Template used for the message
kafka {
    brokers kafka-server:9092,kafka-server1:9092; # Broker list
    default-topic LO-PRIO;                        # Default topic, if no mapping found
    message-max-retries 3;   # Number of message retries by Kafka layer if ack not received
    health-monitoring {      # Health monitoring parameters
        error-threshold 20;                      # If number of errors exceeds this threshold
                                                 # in the last evaluation interval, health is set as
                                                 # “conn-err-exceeds-threshold“ and triggers switchover
        eval-interval 120;                       # Evaluation interval after which errors are reset
        clear-interval 300;                      # Interval after which connection health state is cleared 
    }
topic-map 0 {                # Topic map configuration for logs received with flag 0
    topic HI-PRIO;           # Topic name mapped to logs received with flag 0
                             # of acks the leader broker must receive from ISR brokers
                             # before responding to the request:
    required-acks -1;        # 0 = No ack to client, 1 = Only the leader broker acks,
                             # -1 or all = broker blocks until message is committed by all in-sync
                             # replicas (ISRs) or broker's in.sync.replicas setting before sending response
    request-timeout-ms 5000; # How long broker wait to try to meet the required acks before
                             # sending an error to the client
    }
topic-map 1 {                # Topic map configuration for logs received with flag 1
    topic LO-PRIO;
    required-acks -1;
    request-timeout-ms 5000;
    }
}

To display the Kafka remote collector group configuration, issue the following CLI command:

admin@Analytics% show log-collector-exporter remote collector-groups

kafka-cg {
    collectors        [ kafka-east kafka-west ]; # List of clusters
    primary-collector kafka-west;                # Optional primary collector provides preference
                                                 # to the set collector
}

admin@Analytics% set log-collector-exporter remote collectors kafka1 kafka topic-map 1 topic HI-PRI

Configure Kafka SSL and SASL Authentication

You can stream logs to a Kafka cluster securely over a TLS connection. SSL authentication uses public keys such as client, server, and certificate authority for authentication, and Simple Authentication and Security Layer (SASL) authentication uses the username and password for authentication.

To configure Kafka SSL and SASL authentication:

1. Enable TLS transport for Kafka SSL and SASL authentication:

admin@Analytics% set log-collector-exporter remote collectors kafka-east transport tls

2. Verify that the transport is set to TLS:

admin@Analytics% show log-collector-exporter remote collectors kafka-east transport
transport           tls;

3. Configure TLS attributes for SSL authentication. All TLS attributes are mandatory.

[edit log-collector-exporter remote collectors kafka-east tls-attributes]
admin@Analytics% show
ca-cert-path           /home/versa/certs/rootCA.crt;         # CA certificate in PEM format 
priv-key-path          /home/versa/certs/kafka-prod.key;     # Private key
client-cert-path       /home/versa/certs/kafka-prod.crt;     # Client certificate (PEM)
auth-security-protocol ssl;                                  # TLS/SSL or SASL

4. Configure TLS attributes for SASL authentication. All TLS attributes are mandatory.

[edit log-collector-exporter remote collectors kafka-east tls-attributes]
admin@Analytics% show
sasl-username          versa          # Username for SASL
sasl-password          password      # Password for SASL
sasl-mechanism         scram-sha-256  # SASL mechanism to be used
auth-security-protocol sasl-ssl       # SASL or TLS/SSL

View Kafka Remote Log Collector Status Statistics

By default, Kafka stores alert logs in the /var/log/syslog directory.

To check the Kafka log collector statistics:

1. Verify the connection status:

admin@Analytics> show log-collector-exporter remote collectors kafka-connection 
REMOTE                                                                                    
COLLECTOR                                                                   BROKER        
NAME        BROKER LIST                                   BROKER STATE      HEALTH        
------------------------------------------------------------------------------------------
kafka-east  kafka-server-sec:9092,kafka-server1-sec:9092  all-brokers-down  conn-healthy  
kafka-west  kafka-server:9092,kafka-server1:9092          established       conn-healthy

2. Verify the group status:

admin@Analytics> show log-collector-exporter remote collector-groups kafka-cg kafka-status
kafka-status 
 primary-collector kafka-west
 active-collector  kafka-west

3. Verify the group switchover status:

admin@Analytics> request log-collector-exporter remote collector-groups kafka-cg switchover to-collector kafka-east
status Switchover from kafka-west to kafka-east

4. Verify the Kafka statistics:

admin@Analytics> show log-collector-exporter remote collectors kafka-west kafka-statistics
kafka-statistics 
 message-sent               32794 =>  # of messages posted to kafka layer
 message-send-failures      0     =>  # of messages that could not be posted to kafka layer
 logs-in-queue              0     =>  # of logs pending to be processed in collector queue
 message-delivery-pass      32794 =>  # of messages that have got acked by the kafka broker
 message-delivery-fail      0     =>  # of messages that did not get ack from kafka broker
 message-diverted           0     =>  # of messages that are diverted to another broker as it could not be sent 
 message-size-exceeds-limit 0     =>  # of messages exceeding transmit buffer limit. Not applicable for kafka
 message-exceeds-qlimit     0     =>  # of messages exceeding queue limit as it could not be processed fast enough
 message-params-invalid     0     =>  # of messages with invalid parameters 
 transport-errors           0     =>  # of transport errors seen by this broker
 all-brokers-down           0     =>  # of times all brokers down message seen
 authentication-errors      0     =>  # of times authentication errors seen
 exceeds-threshold-errors   0     =>  # of times connection errors exceeds threshold (can trigger switchover)
                  MESSAGE   
TOPIC    MESSAGE  SEND      
NAME     SENT     FAILURES  
----------------------------
HI-PRIO  32794    0               => # of messages sent and message sent failures per topic
LO-PRIO  0        0  

5. Clear the Kafka statistics:

admin@Analytics> request log-collector-exporter remote collectors kafka-west clear statistics
status Ok 

Debug the Kafka Remote Log Collector Connection

To debug Kafka connections, issue the following CLI commands:

admin@Analytics> show log-collector-exporter remote collectors kafka-connection
admin@Analytics> show log-collector-exporter remote collectors kafka-connection broker-health
admin@Analytics> show log-collector-exporter remote collectors kafka-connection broker-state
admin@Analytics> show log-collector-exporter remote collectors kafka-statistics

Send Email Alerts and Notifications about Logs

For critical security events, you can configure an Analytics log collector node to send email alerts and notifications to users.

To configure an Analytics log collector node to stream email notifications, you do the following:

• Configure a remote collector to send logs to the system containing the notification agent.
• Install the notification agent.
• Configure the notification agent.

admin@Analytics% show log-collector-exporter remote
templates {
    syslog {
        type   syslog;
        format kvp;
    }
}
collectors {
    notif-collector {
        destination-address 192.168.1.21;   <-- Local IP address
        destination-port    514;
        transport           tcp;
        template            syslog;
    }
}
profiles {
    notif-profile {
        collector notif-collector;
    }
}

admin@Analytics% show log-collector-exporter exporter
rules {
    notif-rule {
        match {
            local-collector local-coll;
            log-types       [idp-log urlf-log ];
        }
        set {
            remote-collector-profile notif-profile;
        }
    }
}

admin@Analytics$ cat /opt/versa_van/apps/van-notif-agent/etc/van-notif-agent.conf
[COLLECTOR]
# Directory where the notification events are temporarily stored
directory: /var/tmp/notifier
# Listen IP address and port to bind (typically local IP) where the
# log collector/exporter will export the logs.
# If 0.0.0.0 is specified, it will not bind to any specific IP.
ip: <ip>
port: <port>
# Transport on which logs are received from log collector/exporter
# Options: tcp/udp
protocol: tcp

# SMTP configuration
[SMTP]
# SMTP server hostname or IP address
host: hostname
# SMTP server port
port: port-number
# Email address of sender
sender: sender@versa-networks.com
# Subject of email
subject: Appliance Events
# Password of sender used to login to SMTP server
password: xxxxxxx

# Global Configuration
[GLOBAL]
# Comma-separated list of log types to process
threat_types: idpLog, ipfLog, urlfLog, dosThreatLog
# Comma-separated list of log subtypes to process
threat_subtypes: suspicious, high_risk, attempted-user
# Comma-separated list of log subtypes to skip
subtype_skip_list: allow, trustworthy
# Interval, in minutes, at which to send email notification
email_interval: 15
# If true, logs are printed in email
print_logs: false
# Comma-separated list of notification recipients
recipients: abc@versa-networks.com
# Number of events per email (recommended value = 25)
batch_size: 25
# Global timezone
timezone: UTC

# Per-tenant configuration
[tenant_tenant-name]
# Comma-separated list of log types to process
threat_types: idpLog, ipfLog, urlfLog, dosThreatLog
# Comma-separated list of log subtypes to process
threat_subtypes: suspicious, high_risk, attempted-user
# Comma-separated list of log subtypes to skip
subtype_skip_list: allow, trustworthy
# Interval, in minutes, at which to send email notification
email_interval: 15
# If true, logs are printed in email
print_logs: false
# Comma-separated list of notification recipients
recipients: abc@versa-networks.com
# Number of events per email (recommended value = 25)
batch_size: 25
# Tenant timezone
timezone: US/Pacific

# [tenant_tenant-name]
# threat_types: idpLog, ipfLog, urlfLog, dosThreatLog
# threat_subtypes: suspicious, high_risk, attempted-user
# Comma-separated list of log subtypes to skip
# subtype_skip_list: allow, trustworthy
# email_interval: 1
# print_logs: false
# recipients: email
# batch_size: 25
# timezone: Japan

admin@Analytics$ sudo service van-notif-agent restart

admin@Analytics> request system notification-agent restart
status success
result Restart initiated, check if process is restarted by status option

admin@Analytics$ sudo tail -f /var/log/versa/van-notif-agent.log
2020-07-17 14:30:30,098 - ParserService - INFO - Parser-stats  [Tenant-Common] [current=0] [errors=0] [total=0]
2020-07-17 14:30:30,099 - ParserService - INFO - Parser-stats  [Versa] [current=0] [errors=0] [total=0]
2020-07-17 14:30:30,099 - ParserService - INFO - Parser-stats  [Corp-Inline-Customer-1] [current=0] [errors=0] [total=200]
2020-07-17 14:30:34,915 - CollectorService - DEBUG - Event     [2020-07-17T21:30:34+0000] [urlfLog] [Corp-Inline-Customer-1]
2020-07-17 14:30:34,916 - CollectorService - INFO - Event-stats[Corp-Inline-Customer-1] [current=32] [total=232]
2020-07-17 14:30:35,106 - ParserService - INFO - Parser-stats  [Corp-Inline-Provider] [current=0] [errors=0] [total=0]
2020-07-17 14:30:35,107 - ParserService - INFO - Parser-stats  [Tenant-Common] [current=0] [errors=0] [total=0]
2020-07-17 14:30:35,107 - ParserService - INFO - Parser-stats  [Versa] 

admin@Analytics$ /opt/versa_van/apps/van-notif-agent/src/helper.py timezones

Modify or Add a Local Collector

You can modify or add local collectors in the Versa Director GUI. Local collectors can identify flow logs and store them in a separate directory from other log types. This allows you to store the flow logs but not automatically process them into the Analytics datastore nodes. If required, you can manually process the flow logs into the Analytics datastore nodes by restoring them from the archive.

To modify or add a local log collector:

1. In Director view, select the Analytics tab in the top menu bar.
2. Select an Analytics cluster node:
   • For Releases 22.1.1 and later, hover over the Analytics tab and then select an Analytics node.
   • For Releases 21.2 and earlier, select an Analytics node in the horizontal menu bar.
3. Select Administration > Configurations > Log Collector Exporter in the left menu bar.
4. In the Driver Hosts field, select the Analytics log collector node.
5. Click the Local Collector tab. The following screen displays.
   
   
    
6. Click the name of the log collector to modify, or click the Add icon to add a new log collector. To display and Add in the popup window, enter information for the following fields.
   
   
    

   Field	Description
   Collector Name (Required)	Enter a name for the local log collector.
   Address	Enter the IP address of the local log collector.
   Clients	Enter the IP prefix of the clients of the local log collector. Separate multiple clients with a comma.
   Port	Enter the TCP port number of the local log collector.
   Maximum Connections	Enter the maximum number of connections to the local log collector. When this limit is reached, the log collector stops accepting new connections. If you increase the maximum number, the load on the log collector also increases, which may result in a backlog in processing the logs. To determine whether the log collector can handle the higher log rate, monitor for backlogs using the headend monitoring tools. For more information, see Configure Versa Analytics Platform Monitoring. Default: 512
   Protocol	(Releases 22.1.3 and later.) Enter the protocol number.
   Transport	(Releases 22.1.3 and later.) Select the transport type.
   Storage (Group of Fields)	Configure the storage information for the local log collector.
   Directory	Enter the directory in which to store log files. The default is /var/tmp/log. If you are using a different directory, see Set Up an Additional Log Storage Directory, below. Default: /var/tmp/log
   Format	Select the format in which to store log files: Syslog
   File Generation Interval	Enter how often to create a new log file.
   Maximum Logs per File	Enter the maximum number of log messages for each log file.
   Enable Category	Click to configure flow logs.
   Category Name	Select the category of logs to store in the directory listed in the next field. Flow
   Directory	Enter the name of the directory in which to store flow logs. If you are storing the flow logs in a directory other than /var/tmp/log, see Set Up an Additional Log Storage Directory, below.
   Throttle	Enter how many logs across all the connections to allow per minute. Any flows exceeding this value are discarded until the beginning of the next minute. This settings applies only to logs being written to disk on the local system. Logs streamed to remote servers are not affected.
   Disable	Click to not store any flow logs, either in the default or the configured directory.

7. Click Save.

admin@Analytics$ sudo mkdir /var/tmp/flow-log

admin@Analytics$ sudo mkdir /var/tmp/flow-archive

sudo /opt/versa/scripts/van-scripts/log-archive-start source-directory destination-directory (hourly|daily|weekly) 
[active|backup] [job-name] [checksum]

admin@Analytics$ sudo /opt/versa/scripts/van-scripts/log-archive-start /var/tmp/flow-log
/var/tmp/flow-archive hourly active flow-log-archive

admin@Analytics$ ls -tlr /etc/cron.d
total 20
-rw-r--r-- 1 root root 488 Mar  3  2017 munin-node
-rwxr-xr-x 1 root root 172 Aug 26 12:24 log-archive
-rwxr-xr-x 1 root root 103 Aug 26 12:24 vandb-repair
-rwxr-xr-x 1 root root 135 Sep  2 10:55 vandb-auto-delete
-rwxr-xr-x 1 root root 210 Sep  2 11:01 flow-log-archive

Supported Software Information

Releases 20.2 and later support all content described in this article, except:

• For Release 21.1.1, you can match subtypes in a log collector exporter rule; you can configure TLS on connections to remote log collectors; you can configure email streaming and streaming to Kafka clusters; you can restore Analytics logs from the CLI; you can delete archived logs; you can specify a destination FQDN for a remote collector instead of a destination IP address; you can identify a primary remote collector in a remote collector group from the CLI; you can configure remote collector alarms.
• For Release 21.2.1, you can identify a primary collector when configuring a remote collector group from the Director GUI; local collectors can accept logs from VOS devices running newer versions of the IPFIX template.
• For Release 21.2.3 , you can configure a local collector to receive syslog format logs over TLS transport; you can configure local collectors to accept logs only from specified tenants; you can configure syslog priority values in a remote template.
• For Releases 22.1.1, you can filter alarm logs by alarm text in a log exporter rule; syslog identifiers flowMonDNSLog and saseWebLog are available; you can configure an exporter rule option to stream logs to a specific Kafka topic.
• Release 22.1.3 supports SASE web and DNS-monitoring logs.

Additional Information

Analytics Log Collector Log Types Overview
Configure Log Export Functionality
Configure Versa Analytics Platform Monitoring
Configure VOS Device Alarms
Manage Versa Analytics Log Archives
Perform Initial Software Configuration
Versa Analytics Configuration Concepts
Versa Analytics Log Collector Log Types
Versa Analytics Scaling Recommendations