Splunk

Splunk Enterprise Fundamental Part2 Module4

一、The Content

  • Describe the Splunk CIM
  • List the knowledge objects included with the Splunk CIM Add-On
  • Use the CIM Add-On to normalize data

二、Start

2.1.Workshop objective

  • The content of this workshop primarily serves the purpose of enablement
  • It should NOT be understood as Best Practice on CIM, data normalization
  • Focuses
    1、CIM
    2、Normalization
    3、Props.conf transform, IFX, regular expression, field aliases, and tags
    4、Pivot usage

2.2.CIM

  • CIM defines relationship in the underlying data using tags and fields mapping over the fields generated by the feature ‘schema on-the-fly’
  • Maintain consistency on fields name and value across different data sources

2.3.Benefit of on-the-fly Normalization

  • Normalization is not necessary during indexing, easy to maintain and avoid costly ETL
  • Retain the machine data integrity
  • Reduce the workload on building correlation between difference data sources, vendor independence
  • Improve consistency on data presentation

2.4.Common Information Model

07cim

2.5.Field Normalization

07vim

2.6.Value Normalization

07gim
Ensure the same fields contains consistent value across the CIM model

2.7.The result

07rim

2.8.Resolving Field Conflict

  • The field ‘action’ in ‘web’ of CIM refers to the action taken by the server or
    proxy and carries possible values of ‘allowed’, ‘blocked’ and ‘unknown’
  • The field ‘action’ is access_combined_cookie is a KV pair inside the URL
  • Let’s try to resolved the issue using props.conf, before importing the data to the
    system

2.9.Props.conf Configuration

  • Go to $SPLUNK_HOME/etc/apps/search/local
  • Create or edit the props.conf as below
[access_combined_wcookie]
SEDCMD-cartAction = s/action/cartAc/g
  • Restart the Splunk
  • Type $SPLUNK_HOME/bin/ splunk show config props to verify it’s included under
    [access_combined_wcookie]

3.0.Map the Fields

07nbn

3.1.Create the Aliases

07ggh

3.2.Verify the Field Aliases

07yyh

3.3.Using Tag to Link the ‘web’ Data Model

07ffg

3.4.Create the Tag

07web

3.5.Verify the Tag

07wer

3.6.Verify using Pivot

07piot

3.7.Customer Requirement

  • IT department
    1、Web request baseline and trend
    2、Top 5 users having access issue
  • Marketing department
    Are we referred by the search engine effectively

3.8.Understanding the ‘authentication’ Data Model

3.9.Create the Missing Fields

4.0.The src

07reg
07xxx
07edt
07frt

4.1.The auth_status

07abc
07def
07far
07fff

4.2.The username

07vvc
07eve
Adjust the Regex
We want to avoid incorrect extractions such as “invalid user appserver”,
“session opened for user root”
We can enhance the generated Regex to
07brt
07gff
07ssv

4.3.Field/Value Normalization

Normalize auth_status’ to the Field ‘action’
07wew
07sav
07src
07auth
07tag
07rre

支付宝扫码打赏 微信打赏

若你觉得我的文章对你有帮助,欢迎点击上方按钮对我打赏

扫描二维码,分享此文章

linuxwt's Picture
linuxwt

我叫王腾,来自武汉,2016年毕业后在上海做了一年helpdesk,自学了linux后回武汉从事系统运维的工作,从2017年开始写博客记录自己的学习工作,现在正在进行数据迁移到此博客,目前就职于北京神州新桥科技有限公司,个人的座右铭是:逃脱舒适区才能在闲暇的时候惬意的玩耍。

武汉光谷 https://linuxwt.com

Subscribe to 今晚打老虎

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!

Comments