In Elastic Stack, we use Kibana to discover data, manage indices in Elasticsearch, use developer tools to develop queries, and a few other features. Kibana lets users visualize data with charts and graphs in Elasticsearch. In this article, we will walk through the steps of creating our own graphs. We will start by creating a pie chart.
This article is an excerpt from the book Mastering Python Networking, Third Edition by Eric Chou – The one-stop solution to using Python for network automation, programmability, and DevOps.
A pie chart is great at visualizing a portion of the component in relation to the whole. Let’s create a pie chart based on the Filebeat index that graphs the top 10 source IP addresses based on the number of record counts. We will select Visualization -> New Visualization -> Pie:
Figure 1: Kibana pie chart
Then we will type netflow in the search bar to pick our [Filebeat NetFlow] xindices:
Figure 2: Kibana pie chart source
By default, we are given the total count of all the records in the default time range. The time range can be dynamically changed:
Figure 3: Kibana time range
We can assign a custom label for the graph:
Figure 4: Kibana chart label
Let’s click on the Add option to add more buckets. We will choose to split the slices, pick the terms for aggregation, and select the source.ip field from the drop-down menu. We will leave the option for descending but increase the size to 10.
The change will only be applied when you click on the apply button at the top. It is a common mistake to expect the change to happen in real time when using a modern website and not by clicking on the apply button:
Figure 5: Kibana play button
We can click on the Options link at the top to turn off Donut and turn on Show labels:
Figure 6: Kibana chart options
The final graph is a nice pie chart showing the top IP sources based on the number of document counts:
Figure 7: Kibana pie chart
As with Elasticsearch, the Kibana graph is also an iteration process that typically takes a few tries to get right. What if we split the result into different charts instead of slices on the same chart? Yeah, that is not very visually appealing:
Figure 8: Kibana split chart
Let’s stick to splitting things into slices on the same pie chart and change the time range to the Last 1 hour, then save the chart so that we can come back to it later. Note that we can also share the graph either in an embedded URL (if Kibana is accessible from the shared location) or a snapshot:
Figure 9: Kibana save chart
We can also do more with the metrics operations. For example, we can pick the data table chart type and repeat our previous bucket breakdown with the source IP. But we can also add a second metric by adding up the total number of network bytes per bucket:
Figure 10: Kibana metrics
The result is a table showing both the number of document counts as well as the sum of the network bytes. This can be downloaded in CSV format for local storage:
Figure 11: Kibana tables
Kibana is a very powerful visualization tool in the Elastic Stack. We are just scratching the surface of its visualization capabilities. Besides many other graph options to better tell the story of your data, we can also group multiple visualizations onto a dashboard to be displayed. We can also use Timelion (https://www.elastic.co/guide/en/kibana/7.4/timelion.html) to group independent data sources for a single visualization, or use Canvas (https://www.elastic.co/guide/en/kibana/current/canvas.html) as a presentation tool based on data in Elasticsearch.
Kibana is typically used at the end of the workflow to present our data in a meaningful way. In this article, we have covered the visualization part of the Elastic Stack: Kibana.
Additional Resources:
Mastering Python Networking, Third Edition
Automation & Programmability Training
