Splunk

Splunk Enterprise Fundamental Part2 Module2

一、Course content

0799

Creating Data Models

  • Describe the relationship between data models and pivot
  • Identify data model attributes
  • Create a data model
  • Use a data model in pivot

二、Start

2.1.What is a Data Model ?

  • Describes how underlyingmachine data is represented and accessed
  • Defines meaningful relationships in the data
  • Enables single authoritative view of underlying raw data
  • Allow business user/analyst to create report, dashboard
  • Enable Column based acceleration

2.2.About Data Model Objects

  • Data models are composed of one or more objects
  • Each object is a dataset that corresponds in some manner to a set of data in your index
  • Objects in data models can be arranged in parent/child relationships. Each top-level or root object can have child objects which inherit the constraints and attributes of the parent and have additional constraints and attributes of their own
  • Objects break down into four types
    1、Events objects (can be accelerated persistently)
    2、Search objects (ad-hoc acceleration only)
    3、Transaction objects (ad-hoc acceleration only)

2.3.Create a DM using Instant Pivot

071

2.4.The Root

  • Root define the events for the model based on condition (constraint)
  • It’s already created based on the search from the instant pivot.

2.5.About Attributes

Define what are the fields available in the the Root according to the user
category of the data model

  • business users are only interested on product and marketing related info but not network status
  • By default, only _time, host, source and sourcetype is available

2.6.Add the Relevant Fields

072

073

2.7.Data Enrichment

  • Status type details is not necessary for business users. They only concern if the transaction is successful or failed
  • Use Evaluated expression to generate a business users friendly field on-the-fly

2.8.Transaction Status

07e34
07e5

2.9.Data Enrichment - Lookup

  • Create the Lookup Definition using prices.csv
    1、Remove the Automatic Lookup if there is
    2、Set the Lookup permission to 'App'
  • Edit the Data Model and add the Lookup field

3.0.Adding DM Lookup

07fr
07ytyt

07gt
07vt

3.1.Hierarchical Constraints

Child object access the events defined by the parent, root in this
example. It can also inherit the parent’s attributes
07bb
07n

3.2.Add 2 nd Level Child Objects

07ggb

3.3.Completed Data Model

07mmmm

3.4.Enable Acceleration

  • Edit the Permission setting
    1、Set the "Display For" to the App
    2、Everyone: Read-Only; admin: Read-Write
  • Edit the acceleration setting
    Enable acceleration and set the summary range reasonably

3.5.Using SPL to access Data Model

07ooo

3.6.Let's make a report

07llll

3.7.tstats: using SPL to access accelerated DM

Comparing to Transparent/Report Acceleration, Creating new report
using different fields inside the accelerated DM does not require to
disable/enable the acceleration
07ppo

3.8.Accelerated Search

07mnb
Fields and Values generated on the fly using Eval Expression/Lookup can
be used as well
07ijn

3.9.Accelerated data: summariesonly=true

By default, tstats retrieves events from accelerated index (tsidx) and normal
index, i.e. summariesonly=false
In Search App, type "|tstats summariesonly=true count from
datamodel=DM_Cartstatus"
07hhj

4.0.Accelerated Only

  • Set the permission of lookup definition to App
  • Rebuild the DM and Wait for 100% complete
  • In Search App, type "|tstats summariesonly=true count from
    datamodel=DM_Cartstatus"

07ui

4.1.A few more on DM Acceleration

  • Summary search happens every 5 minutes
  • If acceleration cannot be completed properly, system will skip and defer it to the next iteration
    If the next search can be completed and clean the backlog, it’s fine. Otherwise, the non-accelerated portion grows,There are different reasons, e.g. maxsearches limit reached, maxAutosummary limit reached,Out of search disk space
  • Parallel summarization is enabled by default since 6.3
    acceleration.max_concurrent = 2

4.2.Performance on Summary search

In ES, the audit tab provides useful information, such as Data Model
Audit, Search Audit

支付宝扫码打赏 微信打赏

若你觉得我的文章对你有帮助,欢迎点击上方按钮对我打赏

扫描二维码,分享此文章

linuxwt's Picture
linuxwt

我叫王腾,来自武汉,2016年毕业后在上海做了一年helpdesk,自学了linux后回武汉从事系统运维的工作,从2017年开始写博客记录自己的学习工作,现在正在进行数据迁移到此博客,目前就职于北京神州新桥科技有限公司,个人的座右铭是:逃脱舒适区才能在闲暇的时候惬意的玩耍。

武汉光谷 https://linuxwt.com

Subscribe to 今晚打老虎

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!

Comments