Image from: https://www.elastic.co/blog/azure-cloud-monitoring-with-the-elastic-stack
I have installed and did some basic setup of ELK in the last post. In this post, I will talk about how to integrate/send data from Azure services to ELK. Elatic.co did a good explanation about how it works. You can refer here for conceptual reading.
As we can see from the image above, you can retrieve data directly from Azure Monitor or from Azure Event Hub then send it to ELK. In this post, I will use Azure Application Gateway as example, getting data from Azure monitor using Elastic Azure module and send it to ELK. Let’s get started!
Steps
- Get Azure service principal.
- Install and configure Metricbeat.
- Verify the result.
From Elastic.co article, we understand that Metricbeats pulls data from Azure monitor, therefore we need to assign Metribeat permission to access and collect data from Azure monitor. Once permission is setup, we need to configure Metricbeat so that it knows what services and metrics it needs to pull back.
Configure Metricbeat
Below is the snippet of Metricbeat Azure module configuration file. Modify the file at /etc/metricbeat/modules.d/azure.yml.disabled
Line 9 ~ 12 is the information obtained from Azure service principal
Line 16 ~ 19 is the place where you tell Metricbeat what data to pull from Azure monitor
Save configuration, enable the module and restart Metricbeat.
metricbeat modules enable azure
Verify result
Login to your kibana at HTTP://YOUR_KIBANA_IP:5601. If everything is setup correctly, you will see data pumping into ELK!
If you started your kibana fresh new, there are couple steps to do before you can see the data. I will show it in next post as well as setting up a custom dashboard showing data retrieved from Azure Application Gateway looks below.
