3.2.manage and explore datasets
Dataset types and usage
- Data model datasets
- Table datasets
search app -> datasets -> specify dataset
- In the Search & Reporting app, click Datasets to open the Datasets listing page.
- Find a dataset that you want to review
- Click the > symbol in the first column to expand the row of the dataset details
Visualize a dataset with Pivot
- In the Search & Reporting app, click Datasets
- Find a dataset that you want to work with in Pivot
- Select Explore > Visualize with Pivot
3.3.creating data models
About data models
In building a typical data model, knowledge managers use knowledge object types such as lookups, transactions, search-time field extractions, and calculated fields
what is data model?
To create an effective data model, you must understand your data sources and your data semantics
Data model datasets can get their fields from custom field extractions that you have defined. Data model datasets can get additional fields at search time through regular-expression-based field extractions, lookups, and eval expression
relationship between data models, data model datasets, and searches in the following subsections
Data models generate searches:
- Dataset constraints determine the first part of the search through
1、Simple search filters (Root event datasets and all child datasets).
2、Complex search strings (Root search datasets)
3、transaction definitions (Root transaction datasets).
The fields you select are added to the search that the dataset generates
The fields can include calculated fields, user-defined field extractions, and fields added to your data by lookups
Root datasets can be defined by a search constraint, a search, or a transaction
- Root event datasets
the most commonly-used type of root data model dataset
- Root search datasets
define a base dataset that includes one or more fields that aggregate over the entire dataset
- Root transaction datasets
Before you create a transaction dataset you must already have some event or search dataset trees in your model.
data model acceleration
To accelerate a data model, it must contain at least one root event dataset, or one root search dataset that only uses streaming commands. Acceleration only affects these dataset types and datasets that are children of those root datasets. You cannot accelerate root search datasets that use nonstreaming commands (including transforming commands), root transaction datasets, and children of those datasets. Data models can contain a mixture of accelerated and unaccelerated datasets.
- For a root event dataset or a child dataset of any type, the constraint looks like a simple search, without additional pipes and search commands
- For a root search dataset, the constraint is the dataset search string
- For a root transaction dataset, the constraint is the transaction definition
Dataset field types
You can only add auto-extracted fields to root datasets,Child datasets can inherit them, but they cannot add new auto-extracted fields of their own. Auto-extracted fields divide into three groups
- Eval Expression
- Regular Expression
- Geo IP
The Data Model Editor groups data model dataset fields into three categories
All data model datasets have inherited fields
Root event, search, and transaction datasets also have inherited fields. These inherited fields are default fields that are extracted from every event, such as _time, host, source, and sourcetype.