Format Recognizer Specification

In many cases, events associated with a device or application will contain a unique pattern. You can enter a regular expression in the Format Recognizer section of the parser XML file to search for this pattern, which, if found, will then parse the events according to the parser instructions. After the first match, the event source IP to parser file map is cached, and only that parser file is used for all events from that source IP. A notable exception is when events from disparate sources are received via a syslog server, but that case is handled differently.

While not a required part of the parser specification, a format recognizer can speed up event parsing, especially when one parsing pattern file among many pattern files must be chosen. Only one pattern check can determine whether the parsing file must be used or not. The other less efficient option would be to examine patterns in every file. At the same time, the format recognizer must be carefully chosen so that it is not so broad to misclassify events into wrong files, and at the same time, not so narrow that it fails at classifying the right file. 

Order in Which Parsers are Used

 FortiSIEM parser processes the files in the specific order listed in the file parserOrder.csv.

Format Recognizer Syntax

The specification for the format recognizer section is:

<eventFormatRecognizer><![CDATA[regexpattern]]></eventFormatRecognizer>

In the regexpattern block, a pattern can be directly specified using regex or a previously defined pattern (in the pattern definition section in this file or in the GeneralPatternDefinitions.xml file) can be referenced.

Example Format Recognizers

Cisco IOS

All Cisco IOS events have a %module name pattern.

<patternDefinitions>    <pattern name="patCiscoIOSMod" list="begin"><![CDATA[FW|SEC|SEC_LOGIN|SYS|SNMP|]]></pattern>    <pattern name="patCiscoIOSMod" list="continue"><![CDATA[LINK|SPANTREE|LINEPROTO|DTP|PARSER|]]></pattern>    <pattern name="patCiscoIOSMod" list="end"><![CDATA[CDP|DHCPD|CONTROLLER|PORT_SECURITY-SP]]></pattern></patternDefinitions><eventFormatRecognizer><![CDATA[: %<:patCiscoIOSMod>-<:gPatInt>-<:patStrEndColon>:]]></eventFormatRecognizer>

Cisco ASA

All Cisco ASA events have the pattern ASA-severity-id pattern, for example ASA-5-12345.

<eventFormatRecognizer><![CDATA[ASA-\d-\d+]]></eventFormatRecognizer>

Palo Alto Networks Log Parser

In this case, there is no unique keyword, so the entire message structure from the beginning to a specific point in the log must be considered.

Event
<14>May 6 15:51:04 1,2010/05/06 15:51:04,0006C101167,TRAFFIC,start,1,2010/05/06 15:50:58,192.168.28.21,172.16.255.78,::172.16.255.78,172.16.255.78,rule3,,,icmp,vsys1,untrust,untrust,ethernet1/1,ethernet1/1,syslog-172.16.20.152,2010/05/06 15:51:04,600,2,0,0,0,0,0x40,icmp,allow,196,196,196,2,2010/05/06 15:50:58,0,any,0

<eventFormatRecognizer><![CDATA[<:gPatTime>,\w+,(?:TRAFFIC|THREAT|CONFIG|SYSTEM)]]></eventFormatRecognizer>