Inside the AI model: the data science and modeling approach
How engineered features, two learning layers, composite risk scoring and recall-first tuning turned high-frequency pump telemetry into ranked, explained alerts, for data scientists and technical evaluators.
On this page
Summary at a Glance
An unsupervised learned baseline drives detection; supervised failure-modality matching adds context
Domain-informed features derived across sensors, time and operating conditions
A composite risk score ranks each anomaly as Watch, Warning or Critical
Recall first; thresholds set by the cost of a missed failure against the cost of a false alarm
Chronological backtest in the proof of concept: 84% recall, 78% precision, typically 2–4 days of warning (how it was tested)
Python, deployed on the client’s Google Cloud
From a Learned Baseline to a Ranked, Explained Alert
The predictive-maintenance case study showed the key modeling decision: learn each pump’s normal operating behavior and flag meaningful departures from it. This page covers what sits underneath that decision, layer by layer.
Features come first, built from how the equipment actually runs. An unsupervised layer decides whether behavior is abnormal; a supervised layer suggests which known failure it resembles. A composite risk score ranks every anomaly, and tuning sets the thresholds by what each kind of error costs.
Connected to the case study. The learned baseline and the alert dashboard run interactively in the case study, on synthetic data.
Domain-Informed Feature Engineering
Raw sensor channels, modeled one at a time, miss most of what matters. We engineered derived features that capture relationships and changes tied to equipment behavior and potential mechanical degradation: signals no single sensor reports on its own.
They were built around how the equipment actually operates, in normal running and under stress. Meaningful signals came from relationships between measurements, changes over time, and the interaction of multiple operating variables.
We drew on the domain knowledge of the client’s field engineers and maintenance teams throughout, paired with our own analytical and engineering expertise. Neither would have been enough alone.
Supervised + Unsupervised: Complementary Layers
Unsupervised learned baseline
Asks: is this behavior abnormal? Flags meaningful departures, including failure patterns never seen before. Every alert starts here.
Supervised modality analysis
Asks: which known failure does this resemble? Compares flagged behavior with known, labeled failure-modality signatures to suggest a possible failure mode and what to check first.
Unsupervised multivariate anomaly detection
Catastrophic failures are rare, and some novel failure modes have no historical precedent. A conventional supervised classifier can only learn failure patterns already in its training data. Normal operating data was far richer than labeled failure data, so we chose unsupervised anomaly detection, which can flag a meaningful departure whether or not that failure signature had occurred before. The analytical framework combined several complementary techniques, including:
The aim was a framework suited to the equipment and the available data, with no single algorithm doing all the work.
Supervised learning for known failure modalities
To recognize what kind of failure a departure resembled, we trained supervised models on the failure events labeled by modality during the ground-truth reconstruction. They investigated the patterns, or signatures, associated with each known modality: which engineered features moved, in what combination, and how early.
When a flagged departure resembled a known signature, the alert said so; when it resembled none, it was still raised and ranked on its own evidence. With few labeled examples per modality, this layer never triggered an alert on its own: it sharpened the explanation, and detection stayed with the learned baseline.
Composite Risk Scoring Layer: Severity-Based Alerts
An accurate anomaly detector does not by itself make a useful maintenance system: a stream of undifferentiated flags leads to alert fatigue. We built a composite risk-scoring layer that combines the factors below into one score and ranks each alert as Watch, Warning or Critical, so maintenance teams work from a prioritized queue.
The first three measure how strong and how lasting a departure is; the last two describe its kind: which sensors moved, and whether their combined pattern resembled a known failure modality.
Telling operators whether an anomaly actually warrants a response was central to the design from the start. Each alert’s rank and the evidence behind it feed the alert dashboard and, in production, the alert-delivery rules, so planners act directly on a ranked queue, with no raw model output to interpret first.
Tuning: Balancing Detection With Operational Usability
Model performance has to be weighed against the operational consequences of false negatives and false positives. Missing a developing catastrophic failure can be very expensive, while excessive alerts create fatigue and erode trust.
Throughout tuning, of thresholds, hyperparameters and severity calibration, we weighed the cost of a missed failure against the burden of excessive alerts, both costed in the case study’s ROI calculator, and never optimized accuracy in isolation. Alert volume stayed at a level maintenance teams could realistically act on, and warning arrived early enough to schedule an intervention and avoid an emergency repair.
What was tuned.
- Detection thresholds. The anomaly-score cutoffs that decide when a departure from the learned baseline counts, kept sensitive enough to protect recall.
- Model hyperparameters. The settings of the unsupervised methods used here (isolation forest, density estimation, hierarchical clustering), chosen for how well the scores separated pre-failure behavior from normal operation.
- Severity calibration. Where the Watch, Warning and Critical boundaries sit on the composite risk score (persistence, recurrence, magnitude and sensor pattern), which sets how many alerts reach each tier.
The objective. Maximize recall (the share of failures flagged ahead of time) at an alert volume teams could act on, with the cost of a missed failure and of a false alarm weighed in dollars, and with enough lead time to schedule an intervention. Accuracy alone was never the target. The operating point is therefore a recall-first choice, not a balanced-accuracy one.
No tuning on test data. Tuning used the training sets only. The later held-out periods and the units left out of training (validation design) were scored afterwards, so the reported recall and precision are not fitted to them.
High recall, controlled precision and severity-based prioritization together made the model practically useful. The recall-first strategy behind these choices is in the case study.
Technology and Methodology
Data processing
Analytical methods
Model interpretation
Model optimization
Deployment technology
How the model was put into production, retrained and monitored is on the production and data engineering page.
Have Sensor Data, but No Clean Failure History?
The model is only as good as the ground truth beneath it. We can look at your telemetry and failure records with you, define the prediction problem, and tell you whether a defensible model can be built on them.