Real-Time Intelligence in Microsoft Fabric: Streaming Analytics Without the Complexity

A manufacturing client operating plants in Ontario and Germany needed to monitor equipment sensor data — vibration, temperature, pressure — in near real time to detect early signs of mechanical failure before production lines went down. The existing architecture: IoT sensors sending data to Azure Event Hubs, a Stream Analytics job applying windowed aggregations, results landing in an Azure SQL Database, and a Power BI report refreshing every 15 minutes. The 15-minute refresh window meant that by the time an anomaly appeared in a dashboard, maintenance had already missed the optimal intervention window.

This kind of architecture — assembled from five or six loosely coupled Azure services — represents the prior generation of Azure real-time analytics. Microsoft Fabric’s Real-Time Intelligence workload consolidates Eventstream, KQL Database, Real-Time Dashboards, and Data Activator into a single platform, with a meaningful reduction in operational complexity.

What Real-Time Intelligence Replaces

Before Fabric, building a real-time analytics pipeline on Azure required assembling:

• Azure Event Hubs or Azure IoT Hub for event ingestion
• Azure Stream Analytics for windowed aggregations and real-time transformations
• Azure Data Explorer (ADX) for time-series and log data storage and KQL querying
• Azure Monitor Workbooks or Power BI for visualisation
• Azure Logic Apps or Azure Functions for alerting and downstream actions

Each of these services required separate provisioning, separate access control, separate monitoring, and explicit pipeline connections between them. A data engineer maintaining this stack spent significant time on infrastructure management rather than analytics logic.

Fabric’s Real-Time Intelligence workload replaces this assembly with:

• Eventstream for ingestion (replacing Event Hubs + Stream Analytics for many use cases)
• KQL Database for time-series storage and querying (ADX under the hood)
• Real-Time Dashboards for second-level-refresh visualisation
• Data Activator for trigger-based alerting and actions

Eventstream: Code-Free Event Ingestion

Eventstream is a no-code event ingestion and routing service. You configure sources — Azure Event Hubs, Azure IoT Hub, Azure Service Bus, Apache Kafka endpoints, a custom REST API source, or Fabric’s own Real-Time Hub — and destinations — KQL Databases, Lakehouses (writing events to OneLake Files or Tables), or external storage.

Between source and destination, Eventstream provides a visual stream processing editor: filter events by property values, project specific fields, aggregate over tumbling or hopping windows, join multiple event streams. For the manufacturing sensor use case, this means: receive sensor events from IoT Hub, filter to events with anomalous vibration readings, enrich with a device metadata lookup, and route to a KQL Database — all without writing Spark Structured Streaming or Flink code.

For more complex transformations that exceed Eventstream’s visual editor capabilities, a Spark Structured Streaming notebook in Fabric can consume from the same Event Hubs source and write results to a KQL Database or OneLake. The two approaches are complementary, not mutually exclusive.

KQL Database: The Right Engine for Time-Series Data

The KQL Database is the storage and query engine for Real-Time Intelligence. Architecturally, it is Azure Data Explorer (ADX) running as a Fabric-managed service — the same columnar, time-series-optimised database that Microsoft uses internally for Azure Monitor, Application Insights, and Microsoft 365 telemetry.

KQL (Kusto Query Language) is a read-oriented query language optimised for log and time-series analytics. It is not SQL — it has a pipe-based syntax that chains operators left to right. For data engineers coming from SQL, KQL requires an adjustment period, but its expressiveness for time-series analysis exceeds SQL for most sensor, log, and event analytics use cases.

A brief KQL primer for common data engineering patterns:

Basic time-series aggregation (average sensor reading over 5-minute windows):

SensorReadings
| where ingestion_time() > ago(1h)
| summarize avg(temperature) by bin(timestamp, 5m), device_id
| render timechart

Anomaly detection using the built-in series_decompose_anomalies() function:

SensorReadings
| make-series avg_vibration = avg(vibration) on timestamp from ago(24h) to now() step 5m by device_id
| extend anomalies = series_decompose_anomalies(avg_vibration, 1.5)
| mv-expand timestamp, avg_vibration, anomalies
| where anomalies != 0

Session analysis (grouping related events by session):

UserEvents
| where timestamp > ago(7d)
| summarize session_duration = max(timestamp) - min(timestamp), event_count = count() by user_id, session_id

For SQL practitioners, the adjustment is in thinking in pipelines of transformations rather than joins and WHERE clauses. The ADX query engine is extraordinarily fast on these patterns at scale — millions of events per second ingestion with sub-second query latency on billions of rows is the design target.

Real-Time Dashboards: Visuals That Actually Update in Real Time

Power BI reports in Fabric refresh on a minimum cycle that depends on capacity SKU — even with DirectLake, the visual refresh cycle has a minimum interval. For manufacturing floor displays, trading desks, or logistics operations centres where second-by-second data matters, this is not sufficient.

Real-Time Dashboards in Fabric are purpose-built for continuous refresh. Tiles on a Real-Time Dashboard query a KQL Database on a configurable schedule: as frequently as every 30 seconds. The dashboard is not a Power BI report — it is a separate Fabric item with its own editor, KQL-based tile configuration, and display refresh logic.

Use cases where Real-Time Dashboards replace custom monitoring tools:

• Manufacturing equipment monitoring (OEE, sensor anomalies, line status)
• Retail inventory and transaction monitoring (sales velocity, stock alerts)
• Logistics (fleet position, delivery status, temperature monitoring for cold chain)
• IT operations (server metrics, error rates, latency percentiles)

For use cases where 15-minute Power BI refreshes are adequate, use DirectLake-connected Power BI reports on your Gold Lakehouse. For use cases requiring second-level freshness, Real-Time Dashboards on KQL Databases are the right tool.

Data Activator: Trigger-Based Actions Without Custom Lambda

Data Activator is a trigger-and-action service that monitors data in Power BI reports or KQL Databases and executes actions when defined conditions are met.

The core concept: you define a reflex — a condition applied to a data stream or visual — and an action to execute when that condition triggers. Actions include: send an email, send a Teams notification, trigger a Fabric pipeline, call a Power Automate flow.

For the manufacturing use case: a Data Activator reflex monitors the anomaly detection KQL query output, and when a device shows anomalous vibration readings for more than three consecutive 5-minute windows, it triggers a Teams message to the maintenance lead and a Fabric pipeline that creates a maintenance work order in the ERP system via an HTTP connector.

This replaces the custom Azure Functions + Logic Apps combination that would previously have been required. Data Activator is not a general-purpose event processing engine — it is designed for threshold-based monitoring and notification triggers on streaming data. For complex event processing logic, a Spark Structured Streaming notebook or Azure Functions remains more appropriate.

Fabric Real-Time Intelligence vs AWS Kinesis + OpenSearch + EventBridge

For AWS practitioners evaluating both stacks:

Capability	Fabric Real-Time Intelligence	AWS Kinesis + OpenSearch + EventBridge
Event ingestion	Eventstream (no-code routing)	Kinesis Data Streams + Kinesis Data Firehose
Stream processing	Eventstream visual editor + Spark Streaming	Kinesis Data Analytics (Flink)
Time-series storage	KQL Database (ADX)	OpenSearch Service + OpenSearch Dashboards
Query language	KQL (optimised for time-series)	OpenSearch DSL + SQL
Visualisation	Real-Time Dashboards (30s refresh)	OpenSearch Dashboards (near-real-time)
Alerting	Data Activator	Amazon EventBridge + SNS + Lambda
Integration with batch layer	OneLake (shared with Spark/Power BI)	S3 (requires Glue/Athena for batch)

AWS Kinesis + OpenSearch is a mature, production-proven combination with deep AWS ecosystem integration. Its disadvantage is complexity: ingestion, processing, storage, and visualisation are separate services with distinct SDKs, IAM policies, and scaling configurations.

Fabric Real-Time Intelligence’s advantage is integration cohesion: event data in a KQL Database is immediately accessible alongside Delta table data in the same Fabric workspace, with unified governance through Microsoft Purview. For organisations where the batch and streaming layers need to work closely together — combining event data with Lakehouse data for enriched analytics — Fabric’s unified platform reduces the integration work significantly.

See our guide on Microsoft Fabric data governance and Purview for how governance applies across both the streaming and batch layers in Fabric.

Conclusion

Microsoft Fabric’s Real-Time Intelligence workload represents a meaningful step forward for organisations that previously had to assemble a real-time analytics stack from five or more loosely coupled Azure services. Eventstream, KQL Database, Real-Time Dashboards, and Data Activator work together in a coherent platform — reducing the infrastructure management overhead that typically consumes a disproportionate share of data engineering time in streaming projects.

Evaluating Microsoft Fabric for your organisation? Infra IT Consulting helps Canadian and international businesses assess, architect, and implement modern data platforms. Book a discovery call →