Extraction Function

Details of the following extraction function is available.

EXTRACT Function

EXTRACT(<field>,<pattern>) extracts a value from <field> after applying regular expression <pattern>, and then returns the extracted value. The returned type is determined from the regular expression. The EXTRACT function helps to parse new fields from the historical raw log which the parser may have missed.

Syntax

EXTRACT(<field>,<pattern>)

  • <field> is a string valued eventAttribute. <pattern> is a regular expression pattern for value extraction

EXTRACT(FUNCTION(<field>),<pattern>)

  • FUNCTION returns a string value. <pattern> is a regular expression pattern for value extraction.

Elasticsearch does not support nested functions.

In ClickHouse, these nested function categories are allowed:

  • Conversion Functions, Evaluate-and-Set Function, Extraction Functions, String Manipulation Functions

Scope

Available for ClickHouse and Elasticsearch queries from 7.0.0 onwards. Can extract only one value. Multiple extractions not supported

Example

rawEventMsg

EXTRACT(rawEventMsg,".*\[recvBytes64\]=(\d+).*" )

<134>Sep 19 17:55:01 172.30.52.10 java: [PH_DEV_MON_VM_NET_INTF_UTIL]: [eventSeverity]=PHL_INFO,[vmName]=CentOS7-7.2.1, [recvPkts64]=0, [sentPkts64]=0,[recvBytes64]=23000, [sentBytes64]=54999

23000

<134>Sep 19 17:55:01 172.30.52.10 java: [PH_DEV_MON_VM_NET_INTF_UTIL]: [eventSeverity]=PHL_INFO,[vmName]=CentOS7-7.2.1, [recvPkts64]=0, [sentPkts64]=0,[recvBytes64]= 368640, [sentBytes64]=54999

368640