Kelson Martins Blog

Recently, a post was published on how to setup Elasticsearch to allow federated searches across different Elasticsearch clusters. On this post, we will be approaching Kibana capability of displaying data from different Elasticsearch clusters, which is a feature that is built upon Elasticsearch Cross Site Search.
Elasticsearch Cross Site Search was introduced in version 5.3, while Kibana Cross Site Search was introduced in version 5.4, so keep this in mind while going through this article as your Stack needs to be at least in the mentioned versions.

Elasticsearch Setup

Before diving into Kibana setup, let’s recapitulate the configuration required for Cross Cluster Search to function.
The first thing is to ensure that you have an Elasticsearch cluster that is configured to support Cross Cluster Search and have a remote cluster registered. For a refresher, see this post.
In our scenario, I will be utilizing 2 nodes, one in a cluster entitled “America” and one in a cluster entitled “Europe”.
My primary host which will be serving these requests is the “America” cluster, which is responding requests on port 9201. This can be confirmed by requesting its remote cluster information.
curl -XGET localhost:9201/_remote/info?pretty
{
  "America" : {
    "seeds" : [
      "172.18.0.2:9300"
    ],
    "http_addresses" : [
      "172.18.0.2:9200"
    ],
    "connected" : true,
    "num_nodes_connected" : 1,
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s"
  },
  "Europe" : {
    "seeds" : [
      "172.18.0.5:9300"
    ],
    "http_addresses" : [
      "172.18.0.5:9200"
    ],
    "connected" : true,
    "num_nodes_connected" : 1,
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s"
  }
}
Please note that we have 2 nodes from different clusters registered and they are both connected and properly registered on Elasticsearch Cross Cluster Search.
Performing a regular query against, I can see some data from both cluster being aggregated.
curl -XGET -H 'Content-Type: application/json' localhost:9201/America-Europe:fluentd-*/_search?pretty -d '
{
  "query": {
    "match_all": {}
  }
}
'

{
  "took": 54,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 111,
    "max_score": 1.0,
    "hits": [
      {
        "_index": "America:fluentd-containers-20171207",
        "_type": "fluentd",
        "_id": "rh1MMWABXGLJCNLx5ngI",
        "_score": 1.0,
        "_source": {
          "container_id": "26e21001eceedef506bf7f7e5e4bff915ccd5f6875945cceec617fddd216d98e",
          "container_name": "/elksinglecompose_kibana_1",
          "source": "stdout",
          "hostname": "localhost.localdomain",
          "@timestamp": "2017-12-07T14:06:49.000000000+00:00"
        }
      },
      {
        "_index": "Europe:fluentd-containers-20171213",
        "_type": "access_log",
        "_id": "aeykUGABdenm2FpkY8eG",
        "_score": 1.0,
        "_source": {
          "log": "[2017-12-13T16:10:37,774][WARN ][o.e.d.i.m.UidFieldMapper ] Fielddata access on the _uid field is deprecated, use _id instead",
          "container_id": "b92a0ea267b9587547b45bb41f7c8c6d17171d91106ee310c806c986bdb39ec8",
          "container_name": "/compose03_elasticsearch03_1",
          "source": "stdout",
          "hostname": "localhost.localdomain",
          "@timestamp": "2017-12-13T16:10:37.000000000+00:00"
        }
      }
    ]
  }
}
Note that 2 results were retrieved, one from the “America:fluentd-containers-20171207” index and one from the “Europe:fluentd-containers-20171213” index.
Great, this confirms our Elasticsearch setup.

Kibana, Show me the data

Now, wouldn’t be great if I could see data from these 2 clusters on Kibana? Well, since Kibana 5.4 you can, and the setup is amazingly simple (Thank you Elastic).
As we have our Elasticsearch properly setup, all we need to do is to create specific patterns in Kibana to search across the clusters or our choosing. In our scenario, this translates to “America” and “Europe”.
Using similar syntax that the one used on the curl queries against Elasticsearch, we can use the following pattern in Kibana index management:
America,Europe:fluentd-*
Alternatively,  you can use the the wildcards in the cluster name so a possible option for our scenario would be the use use of:
*:fluentd-*

The above configuration can be seen in the following screenshots under Kibana 6.0.0 Management > Index Management > Create New Index:

Once you create the index, you may navigate to the “Discover” option and once there, you will access your newly created Cross Cluster Index.
Note that under the “_index” field, we can identify that our data is being retrieved by 2 indexes from different hosts.

Conclusion

If you work with Elasticsearch and have used its Cross Cluster Search feature, you have a lot to gain by adding Cross Cluster Search index into Kibana. Saying that this post presented such Kibana feature which allows you to easily browse through sets of data from remote clusters while being also able to use its dashboard & analytics capabilities on top of the retrieved data.
For more details on Kibana Cross Cluster Search, visit the official Elastic documentation for the feature here.
Stay tuned.

Software engineer, geek, traveler, wannabe athlete and a lifelong learner. Works at @IBM

Next Post