一、The Content
- Describe the Splunk CIM
- List the knowledge objects included with the Splunk CIM Add-On
- Use the CIM Add-On to normalize data
二、Start
2.1.Workshop objective
- The content of this workshop primarily serves the purpose of enablement
- It should NOT be understood as Best Practice on CIM, data normalization
- Focuses
1、CIM
2、Normalization
3、Props.conf transform, IFX, regular expression, field aliases, and tags
4、Pivot usage
2.2.CIM
- CIM defines relationship in the underlying data using tags and fields mapping over the fields generated by the feature ‘schema on-the-fly’
- Maintain consistency on fields name and value across different data sources
2.3.Benefit of on-the-fly Normalization
- Normalization is not necessary during indexing, easy to maintain and avoid costly ETL
- Retain the machine data integrity
- Reduce the workload on building correlation between difference data sources, vendor independence
- Improve consistency on data presentation
2.4.Common Information Model
2.5.Field Normalization
2.6.Value Normalization
Ensure the same fields contains consistent value across the CIM model
2.7.The result
2.8.Resolving Field Conflict
- The field ‘action’ in ‘web’ of CIM refers to the action taken by the server or
proxy and carries possible values of ‘allowed’, ‘blocked’ and ‘unknown’ - The field ‘action’ is access_combined_cookie is a KV pair inside the URL
- Let’s try to resolved the issue using props.conf, before importing the data to the
system
2.9.Props.conf Configuration
- Go to $SPLUNK_HOME/etc/apps/search/local
- Create or edit the props.conf as below
[access_combined_wcookie]
SEDCMD-cartAction = s/action/cartAc/g
- Restart the Splunk
- Type $SPLUNK_HOME/bin/ splunk show config props to verify it’s included under
[access_combined_wcookie]
3.0.Map the Fields
3.1.Create the Aliases
3.2.Verify the Field Aliases
3.3.Using Tag to Link the ‘web’ Data Model
3.4.Create the Tag
3.5.Verify the Tag
3.6.Verify using Pivot
3.7.Customer Requirement
- IT department
1、Web request baseline and trend
2、Top 5 users having access issue - Marketing department
Are we referred by the search engine effectively
3.8.Understanding the ‘authentication’ Data Model
3.9.Create the Missing Fields
4.0.The src
4.1.The auth_status
4.2.The username
Adjust the Regex
We want to avoid incorrect extractions such as “invalid user appserver”,
“session opened for user root”
We can enhance the generated Regex to
4.3.Field/Value Normalization
Normalize auth_status’ to the Field ‘action’
