Integrating Bro IDS with the ELK Stack – Part 2

存储架构 2018-03-13

In part 1 of this series , we described how to set up the integration between Bro and the ELK Stack. In this part, we will look into some examples of how to make use of Kibana’s analysis and visualization capabilities to gain insight from the log data Bro makes available.

Let’s start with the end-result. Using Kibana, you can set up a comprehensive dashboard that gives you a nice overview of your network traffic. The dashboard below was created for the conn.log we shipped in the previous article, reporting on a network’s TCP/UDP/ICMP connections.

But let’s begin with the basics.

Querying Bro logs

You’ve got your Bro logs flowing into Elasticsearch and displayed in Kibana. What now?

First, it’s important you understand what you’re looking at and what the different fields parsed by Logstash (or signify.

Bro’s documentation on the subject can help with that, so first be sure to do some research.

Next, start reviewing the data in the Discover page in Kibana. You can open one message as an example and take a look at the fields and their values.

We can see the geographical fields that were added to our messages based on the enrichment we performed in our pipeline configurations.

Here is a list of some of the useful fields available in these logs:

  • conn_state – a Bro-assigned state of the connection.
  • service – the application protocol being sent over the connection (e.g. DNS).
  • duration – how long the connection lasted.
  • protocol – transport layer protocol of the connection (e.g. TCP/UDP).
  • orig_bytes – number of bytes the originator sent.
  • resp_bytes – number of bytes the responder sent.
  • orig_pkts – number of packets the originator sent.
  • resp_pkts – number of packets the responder sent.
  • Ip_orig_h – address of the originator (the field we geo-enriched).
  • History – the state history of connections (displayed as a string of letters).

Select some fields from the list of available fields — this will give you some visibility into the messages in the main display area. For example, select the protocol , conn_state and uid fields (we can remove the message field from the list).

We can enter some queries to search for specific events. For example, say you want to look for TCP connections only:


Or, perhaps rejected TCP connections:

protocol:tcp AND conn_state:REJ

Or, rejected TCP connections originating from China:

Visualizing Bro logs

Once you’re more acquainted with your data, you will find it easier to visualize it. Kibana is renowned for its visualization capabilities, allowing you to slice and dice your data in any way you want.

Here are some examples.

Map of connection originators

Using our geographical fields, we can create a Coordinate Map visualization that gives us a depiction of where in the world connections to our network our originating from.

Top IPs per bytes sent

We can create a simple Pie Chart visualization that gives us a breakdown of the IPs sending the most bytes. This is useful for identifying potential malicious behavior. To do this, we will use a Sum aggregation of the orig_bytes field, and a Terms aggregation of the id_orig_h field.

Bytes/Packets Transmitted

Using Metric visualizations, we can sum up key indicators of traffic volume. Here is an example of a metric visualization summing up resp_bytes . We could do the same for the other bytes/packets metrics.

Connection duration

A Line Chart visualization can be created to depict the average duration of connections over time. Again, this could be useful for identifying abnormal network activity. We will use an Average of the duration field, and a Date Histogram for the X Axis.

Connection State History

The history field records the state history of connections as a string of letters, each signifying a different event type. For example, a connection with a history value of Cd means that there was a bad checksum from the originator and a payload attached.

You can create a Line Chart visualization that gives you a breakdown of the different connections states over time.

Again, these are just examples. Kibana has plenty more tools to use for visualizing the data and exploring these options is part of the fun. Once you have your separate visualizations lined up, add them all into one dashboard.

To help you get started, this dashboard is available in ELK Apps —’s library of pre-made visualizations and dashboards. Simply open the ELK Apps page, search for “bro-conn” and install the dashboard with one click.


Bro is an extremely powerful NIDS tool, and logging is one of its major components. In this series we analyzed only one single log type — Bro connection logs, but as mentioned, the tool offers many more log types that can be analyzed and monitored. For example, Bro logs HTTP traffic from your network to an http.log file which can also be used for analysis and auditing purposes.

In any case, hooking up Bro with a centralized logging platform such as the ELK Stack is necessary for easier analysis and visualization, and to make the most out of this data.

To complement the “traditional” analysis steps described in this series with proactive methodologies, you will want to implement an alerting mechanism that will notify you in real-time whenever there is suspicious or abnormal network activity. Bro’s notification framework can be used for this purpose, but requires some configuration and is somewhat limited in scope.

Alerting does not exist in the open source ELK Stack, unless you decide to use the X-Pack or configure your own workaround. provides a built-in alerting engine that allows you to configure and create query-based alerts that will notify you either email or any other messaging application you might be using. Future posts will delve into this aspect further.

Want to do more to protect your data and prevent DDoS attacks? can help!

Find out More

Daniel Berman

Daniel Berman is Product Evangelist at He is passionate about log analytics, big data, cloud, and family and loves running, Liverpool FC, and writing about disruptive tech stuff. Follow him @proudboffin .


AWS Cost and Usage Report Analysis with an... This post is the second part of a series about AWS Cost and Usage report analysis with and the ELK Stack. Part 1 explored how to ship...
Inserting Document In Bulk Into Elastic Search Usi... What is Logstash? Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously t...
基于Docker的日志分析平台(四)平台整合... 在上一篇中我们基本上完成了 ELK 和 Kafka 环境的安装,并且也通过几个简单的例子入门。现在我们就把搭建好的架构中加入 Kakfa 作为缓冲区。再来说一下,首先 Logstash 从日志源读取日志并且存储到 Kafka,然后 Logstash 再从 Kafka 中读取日志存储到 Elastic...
ELK 实时日志分析平台环境搭建 ELK(ElasticSearch, Logstash, Kibana),三者组合在一起搭建实时的日志分析平台,目前好多公司都是这套! Elasticsearch 是个开源分布式搜索引擎,它的特点有:分布式,零配置,自动发现,索引自动分片,索引副本机制,restful 风格接...
利用 ELK 搭建 Docker 容器化应用日志中心... 封面图片 概述 应用一旦容器化以后,需要考虑的就是如何采集位于Docker容器中的应用程序的打印日志供运维分析。典型的比如SpringBoot应用的日志 收集。本文即将阐述如何利用ELK日志中心来收集容器化应用程序所产生的日志,并且可以用可视化的方式对日志进行查询与分...