Azure Data Explorer -Overview
What is Azure Data Explorer?
Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry. It is ideal for analyzing huge volumes of data from any source, such as Web application and IOT devices. This huge data can be used for diagnostics, monitoring and analytics.
Why Azure Data Explorer?
Azure Data Explorer can handle large amounts of structured, semi-structured (JSON-like nested types) and unstructured (text) data equally well. It allows you to search for specific text terms, locate particular events and perform metric-style calculations on data. It can answers analytic ad-hoc queries on it with seconds of latency.
How to query Azure Data Explorer?
Azure Data Explorer supports native SQL queries, but still for better performance Azure uses KQL (Kusto Query Language) to query the data.
Kusto is a service for storing and running interactive analytics over Big Data. It is based on relational database model supporting entities such as databases, tables and columns. You can also perform searching & filtering of rows and even group by-aggregates & joins.
Let’s explore some real time data
To ingest data and perform operation, we need a Azure Data Explorer cluster. You can create one by visiting https://portal.azure.com/#home
Search for Azure Data Explorer, and select Azure Data Explorer Cluster. To create a cluster & database follow the azure document https://docs.microsoft.com/en-us/azure/data-explorer/create-cluster-database-portal
You can also use Azure’s sample database, where you can perform queries on real-time data.
https://dataexplorer.azure.com/clusters/help
So here, you can see I have executed the KQL count query to fetch the number of records in the table Covid19.
Query to get all Covid19 data from country India & display columns as State, Confirmed & Active
Covid19
| sort by Timestamp
| where Country == “India”
| project State, Confirmed, Active;
Now we can try to represent data in graphical manner
Covid19
| where Country == “India”
| project Timestamp, Active
| render timechart ;
As you can see Azure Data Explorer allows you to represent the queried data in form of charts such as time-chart, bar-chart, area-chart and so on. The above screen-shot gives you the data for rise in Covid19 in India by date.
All this computation took only 1.404 seconds.
You can find more examples at — https://docs.microsoft.com/en-us/azure/data-explorer/write-queries
Conclusion
To sum up in one sentence, Azure Data Explorer is a big data analytics cloud platform optimized for interactive, ad-hoc queries on top of fast flowing data. To get an idea of the cost, check out the cost estimator and note that you only pay for what you use, you wont be charged when your cluster is stopped.
