Extend the MongoDB → Kafka → ClickHouse pipeline with ServiceNow ticket data to provide early-warning signals for incidents, helping on-call engineers see problems before tickets are created.
ClickHouse schema definitions and example queries for the TicketSoon pilot, integrating MongoDB CDC, system events, and ServiceNow tickets into a unified event store.
This note outlines a potential PhD research direction focused on enabling large-scale event-driven systems to self-discover their operational structure, assess risk, and take safe, explainable actions. The work combines temporal modeling, machine learning, and governance principles, with applications in data infrastructure and AI safety. Problem Statement Modern data infrastructures (pipelines, schedulers, CDC systems) produce massive streams of events. Operators (SREs, data engineers) currently monitor, correlate, and intervene manually to handle failures or delays. The goal is to formalize this process: can a system learn from its own history to automatically surface what should happen, when, and what to do when things go wrong—without hand-maintained DAGs or crontabs? ...
A mathematical sketch of governance invariants built on dimensionless normalization and fractal (renormalization group) stability. Outlines entropy-free invariants, control laws, and universal scaling patterns for safe automation.
An entropy-first formulation of dimensionless, fractal governance. Uses normalized entropy, transfer entropy, multiscale entropy, and entropy production as invariants to detect cascades and shape safe, self-similar automation.