Anomaly Detection

Abstract Conventional monitoring and alerting frameworks in distributed systems often rely on threshold-based metrics, which can produce excessive false positives and fail to capture complex dependencies between signals. This paper proposes an applied framework for using information-theoretic measures—specifically mutual information and the broader notion of entropy covariance—to detect anomalies and support governance in event-driven infrastructure. The approach shifts focus from isolated metrics to the relationships between uncertainties: when two signals that normally exhibit dependency diverge, the system can treat this as an indicator of instability. Leveraging existing mathematical foundations from information theory, this work evaluates the operational value of dependency-aware monitoring on real event streams (e.g., Kafka topics, MongoDB change data capture). Rolling measures of mutual information and covariance are integrated into monitoring pipelines, and their effectiveness is compared against conventional thresholds. ...

The concept of covariance of entropies can be understood as a way of quantifying how uncertainties in different signals vary together. Rather than monitoring each metric independently, the focus shifts to the relationships between sources of uncertainty. When signals that typically exhibit aligned behavior diverge, this can provide an early indicator of system anomalies. While this terminology is uncommon, the underlying idea overlaps strongly with established constructs in information theory, particularly mutual information. Mutual information measures the reduction in uncertainty about one random variable given knowledge of another, and has been widely applied to anomaly detection and monitoring tasks. ...