Collect and Set Functions
These functions can be used to extract fields from log fragments.
The following are deprecated
collectFieldsByRegex
Description: This function extracts event fields from a raw message using regular expression.
Syntax:
<collectFieldsByRegex src="$Arg1">
<regex>
<Arg2>
</regex>
</collectFieldsByRegex>
Arg1: Input string from which fields are to be extracted.
Arg2: Regular expression for extracting the fields from input string.
Example:
Suppose the following log fragment is saved in _body variable.
Configured from console by vty1 (192.168.29.8)
The following example uses collectFieldsByRegex to extract user and srcIpAddr variables from _body.
<collectAndSetAttrByRegex src="$_body"> <regex><![CDATA[Configured from console by <user:gPatWord> \(<srcIpAddr:gPatIpAddr>\)]]> </regex> </collectAndSetAttrByRegex>
After execution:
-
useris set tovty1. -
srcIpAddris set to192.168.29.8.
collectFieldsByKeyValuePair
Description: This function extracts Key values when logs are structured as
Key1 = value1 <separator> Key2 = value2 <separator> ....
You can specify the separator between key and value (e.g. '=') and also the separator between Key-value construct (e.g. <separator> above).
Note:
Regarding collectFieldsByKeyValuePair vs. collectAndSetAttrByKeyValuePair per the lines below:
-
<collectFieldsByKeyValuePair sep="$Arg3" kvsep="$Arg2" src="$Arg1">
-
<collectAndSetAttrByKeyValuePair sep="$Arg2" src="$Arg1">
In general, 2(collectAndSetAttrByKeyValuePair) can be added before 1(collectFieldsByKeyValuePair) as 2 is simpler, working for most cases, whereas 1 is more efficient, but also more strict.
Syntax:
<collectFieldsByKeyValuePair sep="Arg3" kvsep="Arg2" src="$Arg1">
<attrKeyMap attr="Arg5" key="Arg4"/>
<attrKeyMap attr="Arg7" key="Arg6"/>
</collectFieldsByKeyValuePair>
Arg1: Input string from which fields are to be extracted.
Arg2: Character for separating Key and Value.
Arg3: Character for separating two key value pairs.
Arg4: Key in the event.
Arg5: FortiSIEM event attribute to which the value should be set.
Arg6: Key in the event.
Arg7: FortiSIEM event attribute to which the value should be set.
Example:
Suppose the following log fragment is stored in local _body variable.
time=21:02:30 filename=readme.txt mname="MSIL/Small.AXD!tr.dldr"
The collectFieldsByKeyValuePair function extracts readme.txt and MSIL/Small.AXD!tr.dldr from the _body attribute. The separator is space character.
<collectFieldsByKeyValuePair sep=" " kvsep="=" src="$_body">
<attrKeyMap attr="fileName" key="filename"/>
<attrKeyMap attr="virusName" key="mname"/>
</collectFieldsByKeyValuePair>
After execution:
-
filenameis set toreadme.txt. -
virusNameis set toMSIL/Small.AXD!tr.dldr.
collectAndSetAttrByKeyValuePair
Description: This function extracts Key values when logs are structured as
Key1 = value1 <separator> Key2 = value2 <separator> ....
Note: Regarding collectFieldsByKeyValuePair vs. collectAndSetAttrByKeyValuePair per the lines below:
-
<collectFieldsByKeyValuePair sep="$Arg3" kvsep="$Arg2" src="$Arg1">
-
<collectAndSetAttrByKeyValuePair sep="$Arg2" src="$Arg1">
In general, 2(collectAndSetAttrByKeyValuePair) can be added before 1(collectFieldsByKeyValuePair) as 2 is simpler, working for most cases, whereas 1 is more efficient, but also more strict.
Syntax:
<collectAndSetAttrByKeyValuePair sep="$Arg2" src="$Arg1"> <attrKeyMap attr="Arg4" key="Arg3"/> <attrKeyMap attr="Arg6" key="Arg5"/> </collectFieldsByKeyValuePair>
Note:
Arg1: Input string from which fields are to be extracted.
Arg2: Character for separating two key value pairs.
Arg3: Key in the event.
Arg4: FortiSIEM event attribute to which the value should be set.
Arg5: Key in the event.
Arg6: FortiSIEM event attribute to which the value should be set.
Example:
Suppose the following log fragment is stored in local _body variable.
[time]=21:02:30 [filename]=readme.txt [mname]="MSIL/Small.AXD!tr.dldr"
The collectFieldsByKeyValuePair function extracts readme.txt and MSIL/Small.AXD!tr.dldr from the _body attribute. The separator is " [".
<collectAndSetAttrByKeyValuePair sep=" [" src="$_body"> <attrKeyMap attr="fileName" key="[filename]="/> <attrKeyMap attr="virusName" key="[mname]="/> </collectAndSetAttrByKeyValuePair>
After execution:
-
filename is set to readme.txt.
-
virusName is set to MSIL/Small.AXD!tr.dldr.
collectAndSetAttrByKeyValuePairMultiValue
Description: This function extends collectAndSetAttrByKeyValuePair, by allowing multiple values for the same key. If the log contains multiple values for the same key, the function will store all the values for the same key in a comma separated string format and assign it to the specified variable.
Syntax:
<collectAndSetAttrByKeyValuePairMultiValue src="$Arg1" sep="Arg2">
<attrKeyMap attr="Arg4" key="Arg3"/>
<attrKeyMap attr="Arg6" key="Arg5"/>
</collectAndSetAttrByKeyValuePairMultiValue>
Arg1: Input string from which fields are to be extracted.
Arg2: Character separator for multiple values.
Arg3: Key in the event.
Arg4: FortiSIEM event attribute to which the value of Arg3 should be set.
Arg5: Key in the event.
Arg6: FortiSIEM event attribute to which the value of Arg5 should be set.
Example:
Suppose the following log fragment is set in _body variable. Note there are 2 values for NetworkDeviceGroups key.
... , NetworkDeviceGroups=Routers; NetworkDeviceGroups=Firewalls, ...
The following example extracts the values of all NetworkDeviceGroups= from _body
<collectAndSetAttrByKeyValuePairMultiValue src="$_body" sep=";">
<attrKeyMap attr="deviceType" key="NetworkDeviceGroups="/>
</collectAndSetAttrByKeyValuePairMultiValue>
After execution:
deviceType: Routers; Firewalls
collectAndSetAttrByPos
Description: Certain application logs, such as those from Microsoft IIS, are structured as a list of values with a separator. There are no corresponding Keys in the log. These logs can be parsed using the collectAndSetAttrByPos function. It splits the string by the separator, and extracts values by position.
Syntax:
<collectAndSetAttrByPos sep="Arg2" src="$Arg1">
<attrPosMap attr="Arg4" pos="Arg3"/>
<attrPosMap attr="Arg6" pos="Arg5"/>
</collectAndSetAttrByPos>
Arg1: Input string from which fields are to be extracted.
Arg2: Character separator for multiple values.
Arg3: Position in the event.
Arg4: FortiSIEM event attribute to which the value in position Arg3 must be set.
Arg5: Key in the event.
Arg6: FortiSIEM event attribute to which the value in position Arg5 must be set.
Example:
Suppose the following log fragment is set in _detail variable:
OB JOY *FILESRV CRTSTRMFIL QPWFSERVSO LNS0811 000112 00023
The following example extracts the 2 attributes from _detail variable, based on position and using space as a separator.
<collectAndSetAttrByPos sep=" " src="$_detail"> <attrPosMap attr="opName" pos="4"/> <attrPosMap attr="seqNum" pos="8"/> </collectAndSetAttrByPos>
After execution:
opName: CRTSTRMFIL
seqNum: 00023
collectAndSetAttrByPosWithQuotes
Description: This function extends the collectAndSetAttrByPos function with the added functionality that it can extract quoted string values.
Syntax:
<collectAndSetAttrByPosWithQuotes src="$Arg1" sep="Arg2" quo="Arg3">
<attrPosMap attr="Arg5" pos="Arg4"/>
<attrPosMap attr="Arg7" pos="Arg6"/>
</collectAndSetAttrByPosWithQuotes>
Arg1: Input string from which fields are to be extracted.
Arg2: Character separator for multiple values.
Arg3: Quote Character.
Arg4: Position in the event.
Arg5: FortiSIEM event attribute to which the value in position Arg4 must be set.
Arg6: Key in the event.
Arg7: FortiSIEM event attribute to which the value in position Arg6 must be set.
Example:
Suppose the following log fragment is set in _detail variable:
0,,"Routing and Remote Access Service Policy",
The following example extracts the 2 attributes from _msg string, based on position and using comma as separator. The double quote here escaped to " as for XML standard.
<collectAndSetAttrByPosWithQuotes src="$_detail" sep="," quo=""">
<attrPosMap attr="nasId" pos="1"/>
<attrPosMap attr="policyName" pos="3"/>
</collectAndSetAttrByPosWithQuotes>
After execution:
nasId: 0
policyName: Routing and Remote Access Service Policy
collectAndSetAttrByPosWithNestedSep
Description: This function extends collectAndSetAttrByPosWithQuotes function by allowing the values to be inside any character pair, e.g. [..] or ".." or {..}.
Syntax:
<collectAndSetAttrByPosWithNestedSep src="$Arg1" L1Sep="Arg2" L2Sep="Arg3">
<attrPosMap attr="Arg5" pos="Arg4"/>
<attrPosMap attr="Arg7" pos="Arg6"/>
…..
</collectAndSetAttrByPosWithNestedSep>
Arg1: Input string from which fields are to be extracted.
Arg2: Character separator for multiple values.
Arg3: Start and End character pairs separated by comma – values within (start,end) is set to one event attribute, e.g. in the example below: "" and [] separated by comma.
Arg4: Position in the event.
Arg5: FortiSIEM event attribute to which the value in position Arg3 must be set.
Arg6: Key in the event.
Arg7: FortiSIEM event attribute to which the value in position Arg5 must be set.
Example:
Suppose the following log fragment is set in _body variable:
[25/Sep/2015:17:39:43 +0100] GET "http://example.net/ping?" HTTP/1.1 200 356 921 "http://example.com/news/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
Note that
-
time is within []:
[25/Sep/2015:17:39:43 +0100] -
URL is within "":
http://example.net/ping?
To parse this function, you need to specify
-
L1Sep: Level 1 separator (in this case space character)
-
L2Sep: Level 2 separator (in this case "" or [])
<collectAndSetAttrByPosWithNestedSep src="$_body" L1Sep=" " L2Sep=""",[]">
<attrPosMap attr="_devTime" pos="1"/>
<attrPosMap attr="httpMethod" pos="2"/>
<attrPosMap attr="uriStem" pos="3"/>
<attrPosMap attr="httpVersion" pos="4"/>
<attrPosMap attr="httpStatusCode" pos="5"/>
<attrPosMap attr="recvBytes64" pos="6"/>
<attrPosMap attr="sentBytes64" pos="7"/>
<attrPosMap attr="httpReferrer" pos="8"/>
<attrPosMap attr="httpUserAgent" pos="9"/>
</collectAndSetAttrByPosWithNestedSep>
After execution:
_devTime: 25/Sep/2015:17:39:43 +0100
httpMethod: GET
uriStem: http://example.net/ping?
httpVersion: HTTP/1.1
httpStatusCode: 200
recvBytes64: 356
sentBytes64: 921
httpReferrer: http://example.com/news/
httpUserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36
collectFieldsByCsvFile
Description: This function replaces the value of one or more event attributes by looking up a CSV file. A typical use case is to replace error codes in parsed events to descriptions.
Syntax:
<collectFieldsByCsvFile file="Arg1" key="$Arg2" reloadInterval="Arg3">
<attrKeyMap attr="Arg5" column="Arg4"/>
<attrKeyMap attr="Arg7" column="Arg6"/>
…
</collectFieldsByCsvFile>
Arg1: CSV File specified with absolute path containing value mappings.
Arg2: FortiSIEM event attribute whose value must be looked up.
Arg3: Time in seconds specifying how often the file should be re-read to capture new values.
Arg4: Column in CSV file.
Arg5: FortiSIEM event attribute to which the value in position Arg4 must be set.
Arg6: Column in CSV file.
Arg7: FortiSIEM event attribute to which the value in position Arg6 must be set.
Note: Column number starts from 0.
Example:
Suppose the csv file: /tmp/winLogonFailCode2.csv has 3 columns: Failure Code, Description, Severity:
| Failure Code | Description | Severity |
|---|---|---|
| 0XC000005E | Login failed – No logon servers | 3 |
|
0XC0000064 |
Login failed - Misspelled or bad user account |
10 |
Suppose the subStatus field in a log is parsed to a value that matches the first column "Failure Code". The following function will look up the corresponding value in the second (respectively third) column and set to errReason (respectively eventSeverity) field in the log.
Note: The column number starts from 0. So, to get the first column's value, column="0", second column: column="1".
<collectFieldsByCsvFile file="/tmp/winLogonFailCode2.csv" key="$subStatus" reloadInterval="3600">
<attrKeyMap attr="errReason" column="1"/>
<attrKeyMap attr="eventSeverity" column="2"/>
</collectFieldsByCsvFile>
If subStatus is parsed as 0XC000005E, then
-
errReasonwill be set to"Login failed – No logon servers." -
eventSeveritywill be set to3.
collectFieldsByCsvValue
Description: This function is similar to collectFieldsByCsvFile function, but allows you to look up multiple log fields. This is typically done to replace codes to descriptions in parsed events.
Syntax:
<collectFieldsByCsvValue file="Arg1" pattern="Arg2" delimiter="Arg3" reloadInterval="Arg4">
<attrKeyMap attr="Arg5"/>
<attrKeyMap attr="Arg6"/>
….
</collectFieldsByCsvValue>
Arg1: CSV file specified with absolute path.
Arg2: Required pattern for FortiSIEM event attributes.
Arg3: Delimiter for updated value, when there are multiple matches.
Arg4: Time in seconds specifying how often the file should be re-read to capture new values.
Arg5: FortiSIEM event attribute whose value should be matched and updated.
Arg6: FortiSIEM event attribute whose value should be matched and updated.
Example:
Suppose the csv file: /opt/phoenix/data-definition/WindowsServerMessageTable.csv has 2 columns: Code, Description.
|
Code |
Description |
|---|---|
| %%1538 | READ_CONTROL |
| %%1539 | WRITE_DAC |
| %%1540 | WRITE_OWNER |
<collectFieldsByCsvValue file="/opt/phoenix/data-definition/WindowsServerMessageTable.csv" pattern="^%%\d+$" delimiter="," reloadInterval="3600">
<attrKeyMap attr="userPerm"/>
<attrKeyMap attr="groupPerm"/>
<attrKeyMap attr="otherPerm"/>
</collectFieldsByCsvValue>
If userPerm is parsed as %%1538, then this function will update userPerm to "READ_CONTROL".
If userPerm is parsed as %%1538,%%1539, then this function will update userPerm to "READ_CONTROL,WRITE_DAC".
If groupPerm is parsed as %%1540, then this function will update groupPerm to "WRITE_OWNER".
If groupPerm is parsed as 1540, then this function will not update the value since the pattern (pattern="^%%\d+$" ) does not match.
collectFieldsBySNMPTrap
Description: This function collects fields from SNMP Traps. This is a version of collectFieldsByKeyValuePair specialized for SNMP Trap.
Syntax:
<collectFieldsBySNMPTrap src="$Arg1">
<attrKeyMap attr="Arg3" key="Arg2"/>
<attrKeyMap attr="Arg5" key="Arg4"/>
….
</collectFieldsBySNMPTrap>
Arg1: Input string from which fields are to be extracted.
Arg2: SNMP OID.
Arg3: FortiSIEM event attribute to which the value corresponding to Arg2 must be set to.
Arg4: SNMP OID.
Arg5: FortiSIEM event attribute to which the value corresponding to Arg4 must be set to.
Example:
Suppose the following log fragment is set in _body variable:
... SNMPv2-SMI::enterprises.25597.1.1.2.1.9.417081 = STRING: "01:02:03:04:05:06" SNMPv2-SMI::enterprises.25597.1.1.2.1.36.417081 = STRING: "1.2.3.4" ...
The function searches for the Keys in the logs and extracts trailing values, e.g. ignores 417081 in SNMPv2-SMI::enterprises.25597.1.1.2.1.9.417081.
<collectFieldsBySNMPTrap src="$_body">
<attrKeyMap attr="srcMACAddr" key="SNMPv2-SMI::enterprises.25597.1.1.2.1.9"/>
<attrKeyMap attr="srcIpAddr" key="SNMPv2-SMI::enterprises.25597.1.1.2.1.36"/>
</collectFieldsBySNMPTrap>
After execution:
srcMACAddr: 01:02:03:04:05:06
srcIpAddr: 1.2.3.4
collectAndSetAttrBySymbol
Description: This function collects fields from logs generated by FortiSIEM itself. This is a version of collectFieldsByKeyValuePair specialized for FortiSIEM Logs. These logs are in key, value format and the keys are exact event attributes within [], e.g. [srcIpAddr]=192.168.20.45.
Syntax:
<collectAndSetAttrBySymbol src="$Arg1" sep="Arg2" symStart="Arg3" symEnd="Arg4">
<excludeAttr>Arg5</excludeAttr>
</collectAndSetAttrBySymbol>
Arg1: Input string from which fields are to be extracted.
Arg2: Separator between Attribute, Value pairs.
Arg3: Start of FortiSIEM Event Attribute.
Arg4: End of FortiSIEM Event Attribute.
Arg5: Comma separated list of FortiSIEM Event Attributes that should not be extracted.
Example:
Suppose the following log fragment is set in _body variable:
[phEventCategory]=2,[srcIpAddr]=192.168.20.45,[phCustId]=1,[sessionId]=0ab028e5b665b36e5944ae48b2f5, [phLogDetail]= this is a standard log
The function looks for FortiSIEM Event attributes between "[" and "]=" characters: e.g. [phEventCategory]=2 search results in phEventCategory.
<collectAndSetAttrBySymbol src="$_body" sep=",[" symStart="[" symEnd="]=">
<excludeAttr>phLogDetail</excludeAttr>
</collectAndSetAttrBySymbol>
After execution:
phEventCategory: 2
srcIpAddr: 192.168.20.45
phCustId: 1
Note: phLogDetail is not parsed as it is in <excludeAttr>.
collectAndSetAttrByXPath
Description: This function collects fields from XML formatted logs generated by FortiSIEM. This is similar to collectAndSetAttrBySymbol, but for FortiSIEM generated XML formatted logs, such as FortiSIEM Incidents. The keys are exact event attributes, e.g. <deviceTime>1409271060</deviceTime>.
Syntax:
<collectAndSetAttrByXPath src="$Arg1" xpath="Arg2"/>
Arg1: Input string from which fields are to be extracted
Arg2: The children of this path should be mapped
Example:
Suppose the following log fragment is set in _body variable:
<event name="phRuleIncident">
<deviceTime>1409271060</deviceTime>
<firstSeenTime>1409271060</firstSeenTime>
<count>1</count>
</event>
The following example searches for XPath /event/* and iterates all the returned XML nodes. If the tag name is a FortiSIEM Event Attribute, then it is set to the tag value.
<collectAndSetAttrByXPath src="$_body" xpath="/event/*"/>
After execution:
deviceTime: 1409271060
firstSeenTime: 1409271060
count: 1
collectFieldsByXPath
Description: This function collects fields from general XML formatted logs. This is a general version of collectAndSetAttrByXPath.
Note: It is not a full-featured XPath function. It is mainly for WinOSXmlParser, and does not support non-leaf node with attribute value, such as
/root/node[@Name='A']/leaf
This is due to performance concerns.
Syntax:
<collectFieldsByXPath src="$Arg1">
<attrKeyMap attr="Arg3" key="Arg2"/>
<attrKeyMap attr="Arg5" key="Arg4"/>
…
</collectFieldsByXPath>
Arg1: Input string from which fields are to be extracted.
Arg2: an XPath in the event.
Arg3: FortiSIEM Event attribute to which the value of Arg2 is set.
Arg4: an XPath in the event.
Arg5: FortiSIEM Event attribute to which the value of Arg4 is set.
Example:
Suppose the following log fragment is set in _body variable:
<Event> <System> <EventID>5156</EventID> <Computer>WIN2019</Computer> </System> <EventData> <Data Name='Application'>System</Data> </EventData> <RenderingInfo Culture='en-US'> <Keywords> <Keyword>Audit Success</Keyword> </Keywords> </RenderingInfo> </Event>
The following example evaluates each key as XPath, and sets the returned values to specified attributes.
<collectFieldsByXPath src="$_xml">
<attrKeyMap attr="type" key="/Event/RenderingInfo/Keywords/Keyword"/>
<attrKeyMap attr="appName" key="/Event/EventData/Data[@Name='Application']"/>
<attrKeyMap attr="hostName" key="/Event/System/Computer"/>
</collectFieldsByXPath>
After execution:
type: Audit Success
appName: System
hostName: WIN2019
collectAndSetAttrFromAnotherEvent
Description: This function enables you to correlate two events by using a common attribute (typically some sort of ID) to set attributes in a latter event from an earlier event. A common case is where the earlier event has user or source IP, but the latter event does not and there is a common logonID to correlate the two events. By populating key information in the latter event, analysts can focus on the second event without having to join the two events during analysis.
Syntax:
<collectAndSetAttrFromAnotherEvent AnotherEventType="Arg1">
<when test="Arg3 = $AnotherEvent.Arg2">
<setEventAttribute attr="Arg5">$AnotherEvent.Arg4</setEventAttribute>
<setEventAttribute attr="Arg7">$AnotherEvent.Arg6</setEventAttribute>
</when>
</collectAndSetAttrFromAnotherEvent>
Arg1: Other Event Type.
Arg2: Key Attribute of Other Event.
Arg3: Attribute of current event that must match Arg2.
Arg4: Attribute of Other Event.
Arg5: Attribute of Current Event that must be set to the value in Arg4.
Arg6: Attribute of Other Event.
Arg7: Attribute of Current Event that must be set to the value in Arg6.
Example:
ASA-302013 (inbound TCP connection) – has ipConnId, and srcIpAddr.
<134>Nov 28 2007 17:20:48: %ASA-6-302013: Built outbound TCP connection 76118 for outside:207.68.178.45/80 (207.68.178.45/80) to inside:192.168.20.31/3530 (99.129.50.157/5967)
ASA-302014 (teardown TCP connection) - has ipConnId only (the directionality of the network traffic is not clear in the log and has to be derived from the ASA-302013 log).
<134>Nov 28 2007 17:22:08: %ASA-6-302014: Teardown TCP connection 76118 for outside:77.67.126.83/80 to inside:192.168.20.31/3531 duration 0:01:05 bytes 6410 TCP Reset-I
After receiving ASA-302014, the following function looks for ASA-302013 with same ipConnId (76118 in the example above), and sets ASA-302013's srcIpAddr to ASA-302014.
<when test="$eventType = 'ASA-302014'">
<collectAndSetAttrFromAnotherEvent AnotherEventType="ASA-302013">
<when test="$ipConnId = $AnotherEvent.ipConnId">
<setEventAttribute attr="srcIpAddr">$AnotherEvent.srcIpAddr</setEventAttribute>
</when>
</collectAndSetAttrFromAnotherEvent>
</when>
After execution, the ASA-302014 event will have ASA-302013's srcIpAddr value.
collectAndSetAttrByJSON
Description: This function collects fields from a JSON formatted log. This is like collectFieldsByXPath, but for JSON.
Syntax:
<collectAndSetAttrByJSON src="$Arg1">
<attrKeyMap attr="Arg3" key="Arg2"/>
<attrKeyMap attr="Arg5" key="Arg4"/>
…..
</collectAndSetAttrByJSON>
Arg1: Variable storing the Input string from which fields are to be extracted.
Arg2: JSON Object in the event.
Arg3: FortiSIEM Event attribute to which the value corresponding the path Arg2 must be mapped to.
Arg4: JSON Object in the event.
Arg5: Variable to which the value corresponding the path Arg4 must be mapped to.
Note: Variable can be a local variable or a FortiSIEM event attribute.
Example:
Suppose the following log fragment is set in _body variable:
{
"geo": {
"city": "ASHBURN",
"country": "US"
},
"httpMessage": {
"bytes": "266",
"host": "www.example.com",
"method": "GET",
"path": "/",
"port": "80"
}
}
The following function parses host and country:
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="hostName" key="httpMessage.host"/>
<attrKeyMap attr="srcGeoCountry" key="geo.country"/>
</collectAndSetAttrByJSON>
After execution:
hostName: www.example.com
srcGeoCountry: US
collectAndSetAttrByJsonArray
Description: This function collects multiples values for a key in a JSON array and sets the concatenated values to an event attribute.
Syntax:
<collectAndSetAttrByJsonArray src="$Arg1" sep="Arg2">
<attrKeyMap attr="Arg4" key="Arg3"/>
<attrKeyMap attr="Arg6" key="Arg5"/>
…
</collectAndSetAttrByJsonArray>
Arg1: Variable storing the Input string from which fields are to be extracted.
Arg2: Separator for concatenated matched value.
Arg3: JSON Object in the event.
Arg4: Variable to which the value corresponding the path Arg3 must be mapped to.
Arg5: JSON Object in the event.
Arg6: Variable to which the value corresponding the path Arg5 must be mapped to.
Note: Variable can be a local variable or a FortiSIEM event attribute.
Example:
Suppose the following log fragment is set in _body variable:
{
"categories": [
{
"id": 23,
"type": "content",
"label": "Search Engines",
"integration": false,
"deprecated": true
},
{
"id": 25,
"type": "content",
"label": "Software/Technology",
"integration": false,
"deprecated": true
}
]
}
The following example extracts all matched values and joins by comma:
<collectAndSetAttrByJsonArray src="$_body" sep=",">
<attrKeyMap attr="webCategories" key="label"/>
</collectAndSetAttrByJsonArray>
After execution:
webCategories: Search Engines,Software/Technology
collectAndSetAttrByJsonSymbol
Description: This is specialized function for collecting fields for FortiSIEM generated JSON log. This function is similar to collectAndSetAttrBySymbol, but for JSON events. This function can be also considered a special version of collectAndSetAttrByJSON where you do not have to specify all the parsed fields, if the key is an FortiSIEM event attribute, then the field is automatically parsed.
Syntax:
<collectAndSetAttrByJsonSymbol src="$Arg1">
<excludeAttr>Arg2</excludeAttr>
</collectAndSetAttrByJsonSymbol>
Arg1: Variable storing Input string from which fields are to be extracted.
Arg2: Comma separated list of values that must be excluded from mapping.
Note: Variable can be a local variable or a FortiSIEM event attribute.
Example:
Suppose the following log fragment is set in _body variable:
{
"collectorId": 10007,
"computer": "WIN-ANQ35T4NKLL",
"user": "Administrator",
"phLogDetail":"Standard log"
}
The following function parses all the fields except phLogDetail. You do not need to specify the parsed fields one by one.
<collectAndSetAttrByJsonSymbol src="$_json">
<excludeAttr>phLogDetail</excludeAttr>
</collectAndSetAttrByJsonSymbol>
After execution:
collectorId: 10007
computer: WIN-ANQ35T4NKLL
user: Administrator
collectAndSetAttrByRegex (Deprecated)
This is the same as collectFieldsByRegex and is provided for backward compatibility. We recommend the use of collectFieldsByRegex function instead of collectAndSetAttrByRegex function.
collectAndSetAttrByKeyValPair (Deprecated)
This is the same as collectAndSetAttrByKeyValuePair and is provided for backward compatibility. We recommend the use of collectAndSetAttrByKeyValuePair function instead.