Splunk手册学习之Search Tutorial

一、The Content

1. Getting started
2. Uploading the tutorial data
3. Using the Splunk Search app
4. Searching the tutorial data
5. Enriching events with lookups
6. Creating reports and charts
7. Creating dashboards

二、Start

2.1.Getting started

下载Splunk Enterprise，免费使用60天，每天有500MB的流量

2.2.Uploading the tutorial data

• The kind of data
  The datas which is indexed are like the content of next graphic
  
• the place of datas which are stored
  When you add data to splunk deployment,the data is processed and transform into a series of individual events that you can view,search,analyze.
  The process of transforming the data is called indexing.During indexing,the incoming data is processed to enable fast searching and analysis.The processed results are stored in the index as events.
  Events are stored in the index as a group of files that fall into two categories:

1. Raw data ,which is the data that you add to the splunk deployment.The raw data is stored in a compresseed format
2. index files,which include some metadata files that point to the raw data

• what is the tutorial data

1. access.log

175.44.24.82 - - [22/Feb/2019:18:44:40] "POST /product.screen?productId=WC-SH-A01&JSESSIONID=SD7SL9FF5ADFF5066 HTTP 1.1" 200 3067 "http://www.buttercupgames.com/product.screen?productId=WC-SH-A01" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; BOIE9;ENUS)" 307
142.233.200.21 - - [22/Feb/2019:19:20:13] "GET show.do?productId=SF-BVS-01&JSESSIONID=SD6SL8FF4ADFF5218 HTTP 1.1" 404 1329 "http://www.buttercupgames.com/cart.do?action=purchase&itemId=EST-13" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 674

2. secure.log

Thu Mar 22 2019 00:15:06 mailsv1 sshd[60445]: pam_unix(sshd:session): session opened for user mdubios by (uid=0)
Thu Mar 22 2019 00:15:06 mailsv1 sshd[3759]: Failed password for djohnson from 194.8.74.23 port 3769 ssh2
Thu Mar 22 2019 00:15:08 mailsv1 sshd[5276]: Failed password for invalid user appserver from 194.8.74.23 port 3351 

3. vendor_sales.log

[13/Apr/2019:18:23:07] VendorID=5037 Code=C AcctID=5317605039838520
[13/Apr/2019:18:23:22] VendorID=9108 Code=A AcctID=2194850084423218
[13/Apr/2019:18:23:49] VendorID=1285 Code=F AcctID=8560077531775179
[13/Apr/2019:18:23:59] VendorID=1153 Code=D AcctID=4433276107716482

• Use the Add Data wizard
  
  
  
  
  
  
  

2.3.using the splunk search app

simple instruction:
1、app menu
2、splunk bar
3、app bar
4、search bar
5、time range picker
6、how to search
7、what to search
8、history search

Host，Source,Sourcetype describle the original data

tutorialdata contains next three sourcetypes:

• access_combined_wcookie
• secure
• vendor_sales

we run a search and display next graphic:
1、app bar
2、search bar
3、time range picker
4、search action buttons which you can printing、sharing and exporting by
5、search results tab
6、search mode menu
7、timeline
8、fields sidebar
9、events viewer
10、save as menu

2.4.specifying time ranges

explore data from Buttercup Games online store using different time ranges
The tutorial data for the Buttercup Games store contains events for a seven day period. The dates of the events are based on the date that you downloaded the tutorial data file
Presets time

Because the data for the Buttercup Games online store is a snapshot of historical data, you will not use the '''Real-time''' preset time ranges in this tutorial

Custom time ranges

if select option "latest" is beginning of today,display 12:00:00AM

2.5.search the tutorialdata

The search assistant
The search assistant will display the terms which contain you content you type in the search bar
retrieve data from index

buttercuogames ( error OR fail* OR severe)

Understanding the results
the result bar contains four kinds of result tab
events patterns statistics visualization
The Events tab displays the Timeline of events, the Display options, the Fields sidebar, and the Events viewer

change display of the event view
List is default
you can change it to table or raw
field sidebar
it contains three part
selected fields
interested fields
all fields
Timeline
it can make you zomm the data

Other resukt tabs
patterns(样式) ：by the search results,summary the common patterns
statistics: get thesearch result by searching with transforming commands
visualization: Searches with transforming commands also populate the Visualization tab. The results area of the Visualizations tab includes a chart and the statistics table that is used to generate the chart

2.6.using the fields to search

a field can be multivalued in single event
extracted fields
index time
Default fields and timestamps are extracted, and transforms are applied.
search time
search time field extraction, field aliasing, source type renaming, event type matching, and so on.
search with fields
the filed should apply next three criteria:
filed name is sensitive,but field name is not
filed value can contain wildzard
quotation marks should be needed if filed value contain space

When you first run a search the Selected Fields list contains the default fields host, source, and sourcetype

Run targeted searches

sourcetype=access_combined* status=200 action=purchase

sourcetype=access_* status!=200 action=purchase

(error OR fail* OR severe) OR (status=404 OR status=500 OR status=503)

sourcetype=access_* status=200 action=purchase categoryId=simulation

2.7.using the search languages

view results in statistics tab

sourcetype=access_* status=200 action=purchase | top categoryId

format results on visualization tab

click part "STRATEGY" of above graphic

2.8.using a subsearch

requirements: find the frequent shopper who access the Buttercup Game store
search without a subsearch

sourcetype=access_* status=200 action=purchase | top limit=1 clientip

sourcetype=access_* status=200 action=purchase  clientip="87.194.216.51" | stats count,dc(productId),values(productId) by clientip

search with a subsearch

sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip

Because you specified only the clientip field with the table command, that is the only field returned. The count and percent fields that the top command generated are discarded from the output

sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count, distinct_count(productId), values(productId) by clientip

By default, subsearches return a maximum of 10,000 results and have a maximum runtime of 60 seconds
Make the search syntax easier to read
ctrl + \
Make the search results easier to understand

sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count AS "Total Purchased", distinct_count(productId) AS "Total Products", values(productId) AS "Product IDs" by clientip | rename clientip AS "VIP Customer"

2.9.Enabling field lookups

There are five key steps to enabling field lookups:
1、Upload the lookup file
2、Share the uploaded file with the applications
3、Create a lookup definition
4、Share the lookup definition with the applications
5、Optional. Make the lookup definition automatic
Download and uncompress the tutorial lookup file
Prices.csv

Find lookups manager

upload lookup table file

share the lookup table file

add the field lookup definition

sharing the lookup definition with all apps

make the lookup automatic

3.0.search with lookup

• Display the product names and prices

sourcetype=access_* | stats values(price) AS Price By productName | rename productName AS "Product Name"

• Display the VIP client purchases

sourcetype=access_* status=200 action=purchase 
    [ search sourcetype=access_* status=200 action=purchase 
    | top limit=1 clientip 
    | table clientip] | stats count AS "Total Purchased",dc(productId) AS "Total Products",values(productId) AS "Product IDs" By clientip | rename clientip AS "VIP Customer"

3.1.create reports and charts

Save a search as a report
run a search

sourcetype=access_* status=200 action=purchase 
    [ search sourcetype=access_* status=200 action=purchase 
    | top limit=1 clientip 
    | table clientip] | stats count AS "Total Purchased",dc(productId) AS "Total Products",values(productName) AS "Product IDs" By clientip | rename clientip AS "VIP Customer"

edit report


find and share report

Sharing is set to Private. Only you can view and edit the report. You can allow other apps to view, edit, or both view and edit the report by changing the report permission

3.2.create a basic chart

customer requirements:

• The number of times each product is viewed
• The number of times each product is added to the cart
• The number of times each product is purchased

sourcetype=access_* status=200 | chart count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productName | rename productName AS "Product Name", views AS "Views", addtocart AS "Adds to Cart", purchases AS "Purchases"

3.3.Create an overlay chart and explore visualization options

use the stats command to count the user actions. The eval command is used to calculate the conversion rates for those actions. For example, how often someone who viewed a product also added the product to their cart

sourcetype=access_* status=200 | stats count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productName | eval viewsToPurchases=(purchases/views)*100 | eval cartToPurchases=(purchases/addtocart)*100 | table productName views addtocart purchases viewsToPurchases cartToPurchases | rename productName AS "Product Name", views AS "Views", addtocart as "Adds To Cart", purchases AS "Purchases"

Format the X-Axis and Y-Axis labels

Format the second Y-Axis
conversion rates such as viewsToPurchases are shown separately

save the revised chart as report
The save operation is the same as above operation of saving the result as a report

3.4.Create a report from a custom chart

sourcetype=access_* | timechart count(eval(action="purchase")) by productName usenull=f useother=f

uses the usenull and useother arguments to ensure that the timechart command counts events that have a value for productName Events that have null values for productName are not included

3.5.Create a report from a sparkline chart

For searches that use the stats and chart commands, you can add sparkline charts to the results table

sourcetype=access_* status=200 action=purchase| chart sparkline(count) AS "Purchases Trend" count AS Total BY categoryId | rename categoryId AS Category

3.6.Create dashboards and panels

create dashboard

sourcetype=access_* status=200 action=purchase | top categoryId

edit dashboard
there two ways to edit dashboard

• UI
  
  you can use add Panel option to create new panel
  you can use add Input option to choose from a list of controls to add to the dashboard, including text, a checkbox, and a time range picker
  you can use the Dark Theme option to change the background appearance of the dashboard
• Source
  you can edit the XML source for the panel directly

Add controls to a dashboard

3.7.add more panels to dashboards

you add those saved reports to an existing dashboard
Add saved reports to a dashboard



Adding a search to an existing dashboard

If the panel is based on an ad hoc search, you can connect the panel to the shared Time Range Picker. If the panel is a report, you cannot connect it to the shared Time Range Picker. Reports can be scheduled to run at a set time interval

三、Source Download

tutorial data download
prices.csv download

赏 支付宝扫码打赏 微信打赏

若你觉得我的文章对你有帮助,欢迎点击上方按钮对我打赏

扫描二维码,分享此文章