Parser Examples

The followng example is based on Cisco IOS Syslog Parser. The objective is to parse this syslog message:

<190>91809: Jan  9 02:38:47.872: %SEC-6-IPACCESSLOGP: list testlog permitted tcp 192.168.20.33(3438) -> 69.147.86.184(80), 1 packet

Complete these steps to create an appropriate parser.

Add Device Type

Create a CiscoIOSParser.xml file with this content:

<eventParser name="CiscoIOSParser">
   <deviceType>
      <Vendor>Cisco</Vendor>
      <Model>IOS</Model>
      <Version>ANY</Version>
   </deviceType>
</eventParser>

Create the Parser Specification and Add Local Patterns

Create the parser XML file with this content, and add the pattern definition patCiscoIOSMod for detecting IOS modules such as SEC.

<eventParser name="CiscoIOSParser">
   <deviceType>
       <Vendor>Cisco</Vendor>
      <Model>IOS</Model>
      <Version>ANY</Version>
   </deviceType>
   <patternDefinitions>
      <pattern name="patCiscoIOSMod" list="begin">  <![CDATA[FW|SEC|SEC_LOGIN|SYS|SNMP|]]></pattern>
     <pattern name="patCiscoIOSMod" list="continue"><![CDATA[LINK|SPANTREE|LINEPROTO|DTP|PARSER|]]></pattern>
     <pattern name="patCiscoIOSMod" list="end"><![CDATA[CDP|DHCPD|CONTROLLER|PORT_SECURITY-SP]]></pattern>
     <pattern name="patStrEndColon"><![CDATA[[^:]*]]></pattern>
     <pattern name="patComm"><![CDATA[[^,]+]]></pattern>
  </patternDefinitions>
</eventParser>

Define the Format Recognizer

Add this format recognizer for detecting %SEC-6-IPACCESSLOGP, which is a signature of Cisco IOS syslog messages. 

<eventParser name="CiscoIOSParser">
   <deviceType>
      <Vendor>Cisco</Vendor>
      <Model>IOS</Model>
      <Version>ANY</Version>
   </deviceType>
   <patternDefinitions>
     <pattern name="patCiscoIOSMod" list="begin">  <![CDATA[FW|SEC|SEC_LOGIN|SYS|SNMP|]]></pattern>
     <pattern name="patCiscoIOSMod" list="continue"><![CDATA[LINK|SPANTREE|LINEPROTO|DTP|PARSER|]]></pattern>
     <pattern name="patCiscoIOSMod" list="end"><![CDATA[CDP|DHCPD|CONTROLLER|PORT_SECURITY-SP]]></pattern>
     <pattern name="patStrEndColon"><![CDATA[[^:]*]]></pattern>
     <pattern name="patComm"><![CDATA[[^,]+]]></pattern>
   </patternDefinitions>
   <eventFormatRecognizer>
     <![CDATA[: %<:patCiscoIOSMod>-<:gPatInt>-<:patStrEndColon>:]]>
   </eventFormatRecognizer>
</eventParser>

Parse the Syslog Header

A syslog message consists of a syslog header and a body. For better organization, first parse the syslog header and event type. Subsequent code will include event type specific parsing, which is why event type is extracted in this step. In this example, the header is in boldface.

<190>91809: Jan 9 02:38:47.872: %SEC-6-IPACCESSLOGP: list testlog permitted tcp 192.168.20.33(3438) -> 69.147.86.184(80), 1 packet

The XML code for parsing the header does the following:

  1. Matches the pattern <190>91809: Jan 9 02:38:47.872: %SEC-6-IPACCESSLOGP:
  2. Sets the eventType attribute to IOS-SEC- IPACCESSLOGP.
  3. Sets deviceTime.
  4. Sets event severity (1-7 scale in Cisco IOS, 1=> most severe, to normalized 1-10 scale in FortiSIEM where 10=>most severe)
  5. Saves the event list testlog permitted tcp 192.168.20.33(3438) -> 69.147.86.184(80), 1 packet in a temporary variable _body.

Note that the patterns gPatSyslogPRI, gPatMon, gPatDay, gPatTime, gPatInt, and gPatmesgBody are global patterns that are defined in the GeneralPatternDefinitions.xml file:

<generalPatternDefinitions>
 <pattern name="gPatSyslogPRI"><![CDATA[<\d+>]]></pattern>
 <pattern name="gPatMesgBody"><![CDATA[.*]]></pattern>
 <pattern name="gPatMon"> <![CDATA[Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec|\d{1,2}]]></pattern>
 <pattern name="gPatDay"><![CDATA[\d{1,2}]]></pattern>
 <pattern name="gPatTime"><![CDATA[\d{1,2}:\d{1,2}:\d{1,2}]]></pattern>
 <pattern name="gPatInt"><![CDATA[\d+]]></pattern>
</generalPatternDefinitions>

This parser file XML fragment for parsing the example syslog message looks like this:

<parsingInstructions>
    <collectFieldsByRegex src="$_rawmsg"><regex><![CDATA[<:gPatSyslogPRI>?<:gPatMon>\s+<:gPatDay>\s+<:gPatTime> %<evIdPrefix:patCiscoIOSMod>-<_severity:gPatInt>-<_evIdSuffix:patStrEnd
Colon>: <_body:gPatMesgBody>]]></regex>
    </collectFieldsByRegex>
    <setEventAttribute attr="eventType">combineMsgId("IOS-",$_evIdPrefix, "-", $_evIdSuffix)</setEventAttribute>
    <choose>
        <when test='$_severity IN "6, 7"'>
            <setEventAttribute attr="eventSeverity">1</setEventAttribute>
        </when>
        <when test='$_severity = "1"'>
           <setEventAttribute attr="eventSeverity">10</setEventAttribute>
        </when>
        <when test='$_severity = "2"'>
           <setEventAttribute attr="eventSeverity">8</setEventAttribute>
        </when>
        <when test='$_severity IN "3, 4"'>
          <setEventAttribute attr="eventSeverity">5</setEventAttribute>
      </when>
      <when test='$_severity = "5"'>
          <setEventAttribute attr="eventSeverity">2</setEventAttribute>
      </when>
    </choose>
<parsingInstructions>

Parse the Syslog Body

The parsing is done on an eventType by eventType basis, because the formats are eventType-specific. Parsing the syslog body involves three steps:

  1. Parsing the action string. Based on the action staring value (permit or denied), modify the eventType by appending the action string value at the end, and also modify the eventSeverity values.
  2. Parsing the protocol, source, and destination IP, port, and totalPackets.
  3. Converting the protocol string to a protocol integer.
<choose>
   <when test='$eventType IN "IOS-SEC-IPACCESSLOGP,IOS-SEC-IPACCESSLOGDP, IOS-SEC-IPACCESSLOGRP"'>
       <collectAndSetAttrByRegex src="$_body">
       <regex><![CDATA[list <_aclName:gPatStr>\s+<_action:gPatWord>\s+<_proto:gPatWord>\s+<srcIpAddr
:gPatIpV4Dot>\(<srcIpPort:gPatInt>\)<:gPatMesgBody>->\s+<destIpAddr:gPat
IpV4Dot>\(<destIpPort:gPatInt>\),\s+<totPkts:gPatInt> <:gPatMesgBody>]]>
                  </regex>
       </collectAndSetAttrByRegex>
       <choose>
          <when test='$_action = "permitted"'>
              <setEventAttribute attr="eventType">combineMsgId("IOS-",$_evIdPrefix, "-", $_evIdSuffix, "-PERMITTED")</setEventAttribute>
        <setEventAttribute attr="eventSeverity">1</setEventAttribute>
          </when>
          <when test='$_action = "denied"'>
               <setEventAttribute attr="eventType">combineMsgId("IOS-",$_evIdPrefix, "-", $_evIdSuffix, "-DENIED")</setEventAttribute>
               <setEventAttribute attr="eventSeverity">3</setEventAttribute>
          </when>
       </choose>
       <setEventAttribute attr="ipProto">convertStrToIntIpProto($_proto)</setEventAttribute>
   </when>
</choose>

Final Parser

<eventParser name="CiscoIOSParser">
    <deviceType>
      <Vendor>Cisco</Vendor>
      <Model>IOS</Model>
      <Version>ANY</Version>
    </deviceType>
    <patternDefinitions>
        <pattern name="patCiscoIOSMod" list="begin"> <![CDATA[FW|SEC|SEC_LOGIN|SYS|SNMP|]]></pattern>
        <pattern name="patCiscoIOSMod" list="continue"> <![CDATA[LINK|SPANTREE|LINEPROTO|DTP|PARSER|]]></pattern>
        <pattern name="patCiscoIOSMod" list="end"><![CDATA[CDP|DHCPD|CONTROLLER|PORT_SECURITY-SP]]></pattern>
        <pattern name="patStrEndColon"><![CDATA[[^:]*]]></pattern>
        <pattern name="patComm"><![CDATA[[^,]+]]></pattern>    
</patternDefinitions>
    <parsingInstructions>
    <!—parse header -->
    <collectFieldsByRegex src="$_rawmsg"><regex><![CDATA[<:gPatSyslogPRI>?<:gPatMon>\s+<:gPatDay>\s+<:gPatTime>
%<_evIdPrefix:patCiscoIOSMod>-<_severity:gPatInt>-<_evIdSuffix:patStrEnd
Colon>: <_body:gPatMesgBody>]]></regex>
    </collectFieldsByRegex>
    <setEventAttribute attr="eventType">combineMsgId("IOS-",$_evIdPrefix, "-", $_evIdSuffix)</setEventAttribute>
    <choose>
        <when test='$_severity IN "6, 7"'>
            <setEventAttribute attr="eventSeverity">1</setEventAttribute>
        </when>
        <when test='$_severity = "1"'>
           <setEventAttribute attr="eventSeverity">10</setEventAttribute>
        </when>
        <when test='$_severity = "2"'>
           <setEventAttribute attr="eventSeverity">8</setEventAttribute>
        </when>
        <when test='$_severity IN "3, 4"'>
          <setEventAttribute attr="eventSeverity">5</setEventAttribute>
      </when>
      <when test='$_severity = "5"'>
          <setEventAttribute attr="eventSeverity">2</setEventAttribute>
      </when>
    </choose>
    <!—parse body -->
    <choose>
      <when test='$eventType IN "IOS-SEC-IPACCESSLOGP,IOS-SEC-IPACCESSLOGDP, IOS-SEC-IPACCESSLOGRP"'>
          <collectAndSetAttrByRegex src="$_body">
       <regex><![CDATA[list
<_aclName:gPatStr>\s+<_action:gPatWord>\s+<_proto:gPatWord>\s+<srcIpAddr
:gPatIpV4Dot>\(<srcIpPort:gPatInt>\)<:gPatMesgBody>->\s+<destIpAddr:gPat
IpV4Dot>\(<destIpPort:gPatInt>\),\s+<totPkts:gPatInt> <:gPatMesgBody>]]>
                  </regex>
          </collectAndSetAttrByRegex>
          <choose>
              <when test='$_action = "permitted"'>
                 <setEventAttribute attr="eventType">combineMsgId("IOS-", $_evIdPrefix, "-", $_evIdSuffix,
"-PERMITTED")</setEventAttribute>
          <setEventAttribute attr="eventSeverity">1</setEventAttribute>
              </when>
              <when test='$_action = "denied"'>
                 <setEventAttributeattr="eventType">combineMsgId("IOS-", $_evIdPrefix, "-", $_evIdSuffix,
"-DENIED")</setEventAttribute>
                 <setEventAttribute attr="eventSeverity">3</setEventAttribute>
              </when>
          </choose>
          <setEventAttribute attr="ipProto">convertStrToIntIpProto($_proto)</setEventAttribute>    
</when>
    </choose>
<parsingInstructions>

Parsed Output

Input syslog:

<190>91809: Jan 9 02:38:47.872: %SEC-6-IPACCESSLOGP: list testlog permitted tcp 192.168.20.33(3438) -> 69.147.86.184(80), 1 packet

Parsed fields:

  1. phRecvTime: the time at which the event was received by FortiSIEM
  2. phDeviceTime: Jan 9 02:38:47 2010
  3. eventType: SEC-IPACCESSLOGP-PERMITTED
  4. eventSeverity: 3
  5. eventSeverityCategory: LOW
  6. aclName: testlog
  7. ipProto: 6
  8. srcIpAddr: 192.168.20.33
  9. destIpAddr: 69.147.86.184
  10. srcIpPort: 3438
  11. destIpPort: 80
  12. totPkts: 1