一、The Content
- Getting started
- Uploading the tutorial data
- Using the Splunk Search app
- Searching the tutorial data
- Enriching events with lookups
- Creating reports and charts
- Creating dashboards
二、Start
2.1.Getting started
下载Splunk Enterprise，免费使用60天，每天有500MB的流量
2.2.Uploading the tutorial data
- The kind of data
The datas which is indexed are like the content of next graphic
- the place of datas which are stored
When you add data to splunk deployment,the data is processed and transform into a series of individual events that you can view,search,analyze.
The process of transforming the data is called indexing.During indexing,the incoming data is processed to enable fast searching and analysis.The processed results are stored in the index as events.
Events are stored in the index as a group of files that fall into two categories:
- Raw data ,which is the data that you add to the splunk deployment.The raw data is stored in a compresseed format
- index files,which include some metadata files that point to the raw data
- what is the tutorial data
- access.log
175.44.24.82 - - [22/Feb/2019:18:44:40] "POST /product.screen?productId=WC-SH-A01&JSESSIONID=SD7SL9FF5ADFF5066 HTTP 1.1" 200 3067 "http://www.buttercupgames.com/product.screen?productId=WC-SH-A01" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; BOIE9;ENUS)" 307
142.233.200.21 - - [22/Feb/2019:19:20:13] "GET show.do?productId=SF-BVS-01&JSESSIONID=SD6SL8FF4ADFF5218 HTTP 1.1" 404 1329 "http://www.buttercupgames.com/cart.do?action=purchase&itemId=EST-13" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 674
- secure.log
Thu Mar 22 2019 00:15:06 mailsv1 sshd[60445]: pam_unix(sshd:session): session opened for user mdubios by (uid=0)
Thu Mar 22 2019 00:15:06 mailsv1 sshd[3759]: Failed password for djohnson from 194.8.74.23 port 3769 ssh2
Thu Mar 22 2019 00:15:08 mailsv1 sshd[5276]: Failed password for invalid user appserver from 194.8.74.23 port 3351
- vendor_sales.log
[13/Apr/2019:18:23:07] VendorID=5037 Code=C AcctID=5317605039838520
[13/Apr/2019:18:23:22] VendorID=9108 Code=A AcctID=2194850084423218
[13/Apr/2019:18:23:49] VendorID=1285 Code=F AcctID=8560077531775179
[13/Apr/2019:18:23:59] VendorID=1153 Code=D AcctID=4433276107716482
- Use the Add Data wizard
2.3.using the splunk search app
simple instruction:
1、app menu
2、splunk bar
3、app bar
4、search bar
5、time range picker
6、how to search
7、what to search
8、history search
Host，Source,Sourcetype describle the original data
tutorialdata contains next three sourcetypes:
- access_combined_wcookie
- secure
- vendor_sales
we run a search and display next graphic:
1、app bar
2、search bar
3、time range picker
4、search action buttons which you can printing、sharing and exporting by
5、search results tab
6、search mode menu
7、timeline
8、fields sidebar
9、events viewer
10、save as menu
2.4.specifying time ranges
explore data from Buttercup Games online store using different time ranges
The tutorial data for the Buttercup Games store contains events for a seven day period. The dates of the events are based on the date that you downloaded the tutorial data file
Presets time
Because the data for the Buttercup Games online store is a snapshot of historical data, you will not use the '''Real-time''' preset time ranges in this tutorial
Custom time ranges
if select option "latest" is beginning of today,display 12:00:00AM
2.5.search the tutorialdata
The search assistant
The search assistant will display the terms which contain you content you type in the search bar
retrieve data from index
buttercuogames ( error OR fail* OR severe)
Understanding the results
the result bar contains four kinds of result tab
events patterns statistics visualization
The Events tab displays the Timeline of events, the Display options, the Fields sidebar, and the Events viewer
change display of the event view
List is default
you can change it to table or raw
field sidebar
it contains three part
selected fields
interested fields
all fields
Timeline
it can make you zomm the data
Other resukt tabs
patterns(样式) ：by the search results,summary the common patterns
statistics: get thesearch result by searching with transforming commands
visualization: Searches with transforming commands also populate the Visualization tab. The results area of the Visualizations tab includes a chart and the statistics table that is used to generate the chart
2.6.using the fields to search
a field can be multivalued in single event
extracted fields
index time
Default fields and timestamps are extracted, and transforms are applied.
search time
search time field extraction, field aliasing, source type renaming, event type matching, and so on.
search with fields
the filed should apply next three criteria:
filed name is sensitive,but field name is not
filed value can contain wildzard
quotation marks should be needed if filed value contain space
When you first run a search the Selected Fields list contains the default fields host, source, and sourcetype
Run targeted searches
sourcetype=access_combined* status=200 action=purchase
sourcetype=access_* status!=200 action=purchase
(error OR fail* OR severe) OR (status=404 OR status=500 OR status=503)
sourcetype=access_* status=200 action=purchase categoryId=simulation
2.7.using the search languages
view results in statistics tab
sourcetype=access_* status=200 action=purchase | top categoryId
format results on visualization tab
click part "STRATEGY" of above graphic
2.8.using a subsearch
requirements: find the frequent shopper who access the Buttercup Game store
search without a subsearch
sourcetype=access_* status=200 action=purchase | top limit=1 clientip
sourcetype=access_* status=200 action=purchase clientip="87.194.216.51" | stats count,dc(productId),values(productId) by clientip
search with a subsearch
sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip
Because you specified only the clientip field with the table command, that is the only field returned. The count and percent fields that the top command generated are discarded from the output
sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count, distinct_count(productId), values(productId) by clientip
By default, subsearches return a maximum of 10,000 results and have a maximum runtime of 60 seconds
Make the search syntax easier to read
ctrl + \
Make the search results easier to understand
sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count AS "Total Purchased", distinct_count(productId) AS "Total Products", values(productId) AS "Product IDs" by clientip | rename clientip AS "VIP Customer"
2.9.Enabling field lookups
There are five key steps to enabling field lookups:
1、Upload the lookup file
2、Share the uploaded file with the applications
3、Create a lookup definition
4、Share the lookup definition with the applications
5、Optional. Make the lookup definition automatic
Download and uncompress the tutorial lookup file
Prices.csv
Find lookups manager
upload lookup table file
share the lookup table file
add the field lookup definition
sharing the lookup definition with all apps
make the lookup automatic
3.0.search with lookup
- Display the product names and prices
sourcetype=access_* | stats values(price) AS Price By productName | rename productName AS "Product Name"
- Display the VIP client purchases
sourcetype=access_* status=200 action=purchase
[ search sourcetype=access_* status=200 action=purchase
| top limit=1 clientip
| table clientip] | stats count AS "Total Purchased",dc(productId) AS "Total Products",values(productId) AS "Product IDs" By clientip | rename clientip AS "VIP Customer"
3.1.create reports and charts
Save a search as a report
run a search
sourcetype=access_* status=200 action=purchase
[ search sourcetype=access_* status=200 action=purchase
| top limit=1 clientip
| table clientip] | stats count AS "Total Purchased",dc(productId) AS "Total Products",values(productName) AS "Product IDs" By clientip | rename clientip AS "VIP Customer"
edit report
find and share report
Sharing is set to Private. Only you can view and edit the report. You can allow other apps to view, edit, or both view and edit the report by changing the report permission
3.2.create a basic chart
customer requirements:
- The number of times each product is viewed
- The number of times each product is added to the cart
- The number of times each product is purchased
sourcetype=access_* status=200 | chart count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productName | rename productName AS "Product Name", views AS "Views", addtocart AS "Adds to Cart", purchases AS "Purchases"
3.3.Create an overlay chart and explore visualization options
use the stats command to count the user actions. The eval command is used to calculate the conversion rates for those actions. For example, how often someone who viewed a product also added the product to their cart
sourcetype=access_* status=200 | stats count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productName | eval viewsToPurchases=(purchases/views)*100 | eval cartToPurchases=(purchases/addtocart)*100 | table productName views addtocart purchases viewsToPurchases cartToPurchases | rename productName AS "Product Name", views AS "Views", addtocart as "Adds To Cart", purchases AS "Purchases"
Format the X-Axis and Y-Axis labels
Format the second Y-Axis
conversion rates such as viewsToPurchases are shown separately
save the revised chart as report
The save operation is the same as above operation of saving the result as a report
3.4.Create a report from a custom chart
sourcetype=access_* | timechart count(eval(action="purchase")) by productName usenull=f useother=f
uses the usenull and useother arguments to ensure that the timechart command counts events that have a value for productName Events that have null values for productName are not included
3.5.Create a report from a sparkline chart
For searches that use the stats and chart commands, you can add sparkline charts to the results table
sourcetype=access_* status=200 action=purchase| chart sparkline(count) AS "Purchases Trend" count AS Total BY categoryId | rename categoryId AS Category
3.6.Create dashboards and panels
create dashboard
sourcetype=access_* status=200 action=purchase | top categoryId
edit dashboard
there two ways to edit dashboard
- UI
you can use add Panel option to create new panel
you can use add Input option to choose from a list of controls to add to the dashboard, including text, a checkbox, and a time range picker
you can use the Dark Theme option to change the background appearance of the dashboard - Source
you can edit the XML source for the panel directly
Add controls to a dashboard
3.7.add more panels to dashboards
you add those saved reports to an existing dashboard
Add saved reports to a dashboard
Adding a search to an existing dashboard
If the panel is based on an ad hoc search, you can connect the panel to the shared Time Range Picker. If the panel is a report, you cannot connect it to the shared Time Range Picker. Reports can be scheduled to run at a set time interval
