← Back to the problem

Romania ONIA 2026 · Correction by IOAI Training

Fire Distinguisher

This page gives the full approach. Read it once you have searched.

Metric
exact stats / tolerance / weighted F1
Evaluated on
held-out validation (official test labels are private)
Our score
weighted F1 0.967 (5-fold CV)
Expected level
weighted F1 >= 0.90 for full 50 points

Expected level reached

We solved the problem ourselves: downloaded the data, ran the experiments and measured the score below. The statement and data belong to their organisers.

The problem in brief

Test vehicles carry gas, temperature and flame sensors that report once per second. Each reading of the training set (3,232 rows) is labelled NORMAL, GAS_LEAK, OVERHEAT or FIRE; the test set has 800 unlabelled readings. Besides the useful sensors (raw gas, temperature, flame, their moving averages, a drifting gas sensor, the ambient temperature) the table contains distractors: sensor noise, timestamp, battery level, speed, weather and sensor ID. About 28 % of every numeric value is missing. The task has four parts: describe the classes, compute a Mahalanobis anomaly score for each test reading, turn it into LOW/MEDIUM/HIGH risk levels, and build a 4-class classifier.

How you are scored

subtaskwhatpoints
112 descriptive statistics, all must be right20
2Mahalanobis distance per test reading, proportional to the share within ±0.05\pm 0.0515
3risk level per test reading, proportional to the share correct15
4predicted hazard, weighted F150

Weighted F1 averages the per-class F1 scores with weights equal to the class frequencies nc/nn_c / n:

F1w=∑cncn⋅2 PcRcPc+Rc.\text{F1}_w = \sum_{c} \frac{n_c}{n}\cdot\frac{2\,P_c R_c}{P_c + R_c}.

Subtask 4 gives 50 points for F1w≥0.90\text{F1}_w \ge 0.90, 0 below 0.60 and a linear interpolation in between (statement, "Evaluation"). Subtasks 1 to 3 are deterministic computations: what matters is following the recipe exactly. Test labels are private, so subtask 4 is evaluated by 5-fold cross-validation on the training set.

First look at the data

The four classes are perfectly balanced (about 800 readings each) and the physics is simple:

Left: temperature against gas concentration, coloured by label: NORMAL is cold with little gas, GAS_LEAK cold with much gas, OVERHEAT hot with little gas, FIRE hot with much gas, plus a sprinkling of corrupted readings far from every cluster. Right: histogram of the Mahalanobis distance to the NORMAL profile on a log scale: NORMAL, GAS_LEAK and OVERHEAT overlap between 1 and 8, while all FIRE readings with flame = 1 sit at about 2310.

Two binary questions, "is it hot?" and "is there gas?", carve four quadrants, and the flame sensor fires for 807 of the 815 FIRE readings and never for the other classes. A few observations matter for what follows:

  • Redundant sensors. gas_sensor_drift is almost a copy of gas_raw (correlation 0.97) and ambient_temp of temp_c (0.99). The moving averages gas_avg and temp_avg are much noisier versions (correlation 0.4 with the raw values).
  • Corrupted readings. About 4 % of readings have absurd values (gas near 2000, temperature above 180 °C) whatever their label.
  • Distractor columns. Weather and sensor ID are spread evenly across classes; timestamp, noise, battery, speed and the rates dGas, dTemp have Spearman correlations below 0.025 with the label.

Subtask 1: hazard profile

Straight pandas, with the default behaviour of skipping missing values in the means (temp_c and gas_raw are the "temperature" and "gas concentration" of the statement):

NORMALGAS_LEAKOVERHEATFIRE
share of readings (%)25.1224.9424.7225.22
mean temp_c (°C)28.546.993.0112.0
mean gas_raw321.2867.4379.9825.1

The table confirms the quadrant picture: OVERHEAT and FIRE are hot, GAS_LEAK and FIRE are gassy. (The NORMAL mean temperature, 28.5 °C, is above its median, 19.7 °C, because of the corrupted hot readings: the mean is sensitive to outliers.)

Subtask 2: the Mahalanobis anomaly score

Feature selection. Encode the label as NORMAL =0= 0, GAS_LEAK =1= 1, OVERHEAT =2= 2, FIRE =3= 3 and rank the 13 numeric columns (all except id) by the absolute Spearman correlation with this code, computed on the available pairs:

featuretemp_avgflametemp_cambient_tempgas_avggas_sensor_driftgas_rawtimestamp_ms
∣ρ∣\lvert\rho\rvert0.7780.7720.7590.7490.3780.3610.3590.023

The top 6 are temp_avg, flame, temp_c, ambient_temp, gas_avg, gas_sensor_drift. Reassuringly, computing Spearman after median imputation reorders them (flame moves first) but selects the same six, and there is no tie. Temperature features rank higher than gas features because the code is ordinal: temperature increases along 0, 1, 2, 3 except for one step, while gas goes low, high, low, high.

Preparation. Fill missing values with the train median of each column, then standardise with the train mean and standard deviation (StandardScaler convention, population standard deviation).

Distance. With μ\mu and Σ\Sigma the mean and covariance (np.cov, denominator n−1n-1) of the standardised NORMAL training rows,

d(x)=(x−μ)⊤(Σ+10−6I)−1(x−μ).d(x) = \sqrt{(x-\mu)^\top (\Sigma + 10^{-6} I)^{-1} (x-\mu)}.

Unlike the Euclidean distance, Mahalanobis measures each direction in units of the NORMAL spread along it, and it discounts correlated features: temp_c and ambient_temp are nearly the same measurement and should not count twice. Geometrically, it is the Euclidean distance after "whitening" the NORMAL cloud into a round ball.

The surprise: why every fire scores 2310. No NORMAL reading has flame = 1, so the flame column is constant inside NORMAL: its variance there is exactly 0 and Σ\Sigma is singular. The 10−610^{-6} regulariser makes the matrix invertible, but along the flame direction the "allowed spread" is now 10−6=10−3\sqrt{10^{-6}} = 10^{-3}. A reading with flame =1= 1 is (1−0)/0.433=2.31(1 - 0)/0.433 = 2.31 standard deviations away from NORMAL on that axis (0.433 is the standard deviation of flame over the whole train), so its distance is about 2.31/10−3≈23102.31 / 10^{-3} \approx 2310, whatever the other sensors say. That is visible as the lone red bar in the figure.

This has a practical consequence for the ±0.05\pm 0.05 tolerance: for those readings, a tiny convention change (standard deviation with n−1n-1 instead of nn, for instance) changes the third significant digit, about 0.360.36 in absolute value, and fails the tolerance on about a quarter of the test set. We used the standard library conventions (StandardScaler then np.cov), and checked our formula against scipy.spatial.distance.mahalanobis.

The other surprise is in the opposite direction: GAS_LEAK and OVERHEAT readings get distances of only about 2.4, barely more than NORMAL (1.8). Two reasons. The NORMAL class contains the corrupted readings, which inflate its covariance: the standard deviation of gas_sensor_drift within NORMAL is 373, while its robust spread (interquartile range / 1.35) is only 122. And the median imputation moves 28 % of the values to the global median (560 for gas), halfway between "no gas" and "leak". A classical Mahalanobis distance is not robust: a robust covariance estimator (for example scikit-learn's MinCovDet) would separate the classes far better, but the statement fixes the recipe.

Subtask 3: risk levels

Apply exactly the same pipeline to all 3,232 training readings and take the 33rd and 66th percentiles of their distances (NumPy's default linear interpolation): p33=1.750p_{33} = 1.750, p66=3.862p_{66} = 3.862. A test reading is LOW if d<p33d < p_{33}, MEDIUM if p33≤d<p66p_{33} \le d < p_{66}, HIGH otherwise. Our test predictions: 223 LOW, 342 MEDIUM, 235 HIGH. Since the 807 fires alone fill a quarter of the train distribution, HIGH contains every flame reading plus the most unusual non-fire readings.

Subtask 4: the hazard classifier

A first baseline

A logistic regression on the 13 numeric columns (median imputation, standardisation) reaches weighted F1 0.864: that is 44 of 50 points already. It is limited because the quadrant structure is not linear in the raw features ("FIRE if hot and gassy"), and because median imputation hides missingness.

The key idea

The classes are axis-aligned boxes in (temperature, gas, flame) space, which is exactly what decision trees cut well, and missing values should be handled by the model, not papered over. HistGradientBoostingClassifier learns at every split which side missing values should go to, using the training loss. When temp_c is missing, the tree can fall back to ambient_temp or temp_avg in the next split, so the redundant sensors are used automatically as backups. Median imputation, in contrast, replaces a missing temp_c by the global median, 73.7 °C, a value halfway between the cold classes (about 20 to 40 °C) and the hot ones (about 90 to 115 °C), which is wrong for every class.

The full solution

Gradient-boosted trees (HistGradientBoostingClassifier, default parameters) on the 13 raw numeric columns, with native NaN handling. No feature engineering is needed; the distractor columns are simply never chosen for splits.

clf = HistGradientBoostingClassifier(random_state=0)
clf.fit(tr[NUMERIC], y)              # NaN allowed: the model learns where missing values go

Results

5-fold cross-validated weighted F1 on the training set (official test labels are private):

approachweighted F1points / 50
logistic regression, median imputation0.86444.0
fused features only (row-wise median of each sensor family + flame), gradient boosting0.93550
random forest, median imputation0.96150
gradient boosting, raw columns + averaged "best copies" of temperature and gas0.96550
gradient boosting, raw columns with native NaN (final)0.96750

Hand-made fusion of the sensor copies did not help: the trees already exploit the redundancy, and the row-wise median mixes the good copies with the noisy moving averages. The remaining errors (mostly OVERHEAT and GAS_LEAK readings predicted NORMAL) are concentrated on readings where a whole sensor family is missing: when temp_c, temp_avg and ambient_temp are all absent (or all three gas columns), the error rate is 12.4 % on these 476 readings, against 1.7 % for the others. No model can see a temperature that was not recorded.

With subtasks 1 to 3 computed exactly as specified, the expected total is 100 points.

What to remember

  • Follow recipe tasks to the letter, and know your library conventions. Population versus sample standard deviation, percentile interpolation and the order "impute, then scale" all change the numbers; check one result with an independent implementation (here scipy).
  • A constant feature makes a covariance matrix singular. A tiny ridge such as 10−6I10^{-6} I makes it invertible but turns any deviation along that feature into an enormous distance: understand what the regulariser does before trusting the scores.
  • Mahalanobis is only as good as its covariance estimate. Outliers inside the reference class inflate Σ\Sigma and hide true anomalies; robust estimators exist for this reason.
  • Let tree ensembles handle missing values natively when values are missing at random across redundant sensors: they learn backup routes that imputation destroys.
  • Look at the physics first. Two thresholds (hot? gassy?) plus the flame bit already explain the classes; the model only has to learn them robustly.
Full code (solution.py)
"""Fire Distinguisher (ONIA 2026, county round, grades 9-10).

Subtask 1: class shares, mean temperature and mean gas per hazard type.
Subtask 2: Mahalanobis distance of each test reading to the NORMAL profile (6 Spearman-selected features).
Subtask 3: LOW / MEDIUM / HIGH risk from the 33rd and 66th percentiles of the train distances.
Subtask 4: 4-class hazard classifier (weighted F1).

Run from the folder holding train.csv and test.csv; writes submission.csv (CPU, a few seconds).
"""
import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import f1_score
from sklearn.model_selection import StratifiedKFold, cross_val_predict

tr = pd.read_csv("train.csv"); te = pd.read_csv("test.csv")
LABELS = ["NORMAL", "GAS_LEAK", "OVERHEAT", "FIRE"]
y = np.asarray(tr["label"], dtype=object)
rows = []

# ---------------- Subtask 1: hazard profile ----------------
share = (tr["label"].value_counts(normalize=True) * 100).reindex(LABELS).round(2)
temp = tr.groupby("label")["temp_c"].mean().reindex(LABELS).round(1)    # NaN readings are skipped
gas = tr.groupby("label")["gas_raw"].mean().reindex(LABELS).round(1)
for k, v in enumerate(list(share) + list(temp) + list(gas), start=1):
    rows.append((1, k, v))
print("Subtask 1:", list(share), list(temp), list(gas))

# ---------------- Subtask 2: Mahalanobis anomaly score ----------------
NUMERIC = [c for c in tr.select_dtypes("number").columns if c != "id"]
code = tr["label"].map({l: i for i, l in enumerate(LABELS)})
rho = tr[NUMERIC].apply(lambda c: c.corr(code, method="spearman"))      # pairwise-complete observations
ranking = sorted(NUMERIC, key=lambda c: (-abs(rho[c]), c))               # strongest |rho|, ties alphabetical
FEATS = ranking[:6]
print("Top-6 Spearman features:", {c: round(rho[c], 4) for c in FEATS})

median = tr[FEATS].median()                                              # train statistics only
Z_tr = tr[FEATS].fillna(median)
mean, std = Z_tr.mean(), Z_tr.std(ddof=0)                                # StandardScaler convention
scale = lambda d: ((d[FEATS].fillna(median) - mean) / std).values
S_tr, S_te = scale(tr), scale(te)

normal = S_tr[y == "NORMAL"]
mu = normal.mean(axis=0)
sigma = np.cov(normal, rowvar=False) + 1e-6 * np.eye(len(FEATS))
inv = np.linalg.inv(sigma)


def mahalanobis(S):
    diff = S - mu
    return np.sqrt(np.einsum("ij,jk,ik->i", diff, inv, diff))


d_tr, d_te = mahalanobis(S_tr), mahalanobis(S_te)
rows += [(2, i, round(float(v), 2)) for i, v in zip(te["id"], d_te)]

# ---------------- Subtask 3: risk level ----------------
p33, p66 = np.percentile(d_tr, 33), np.percentile(d_tr, 66)
risk = np.where(d_te < p33, "LOW", np.where(d_te < p66, "MEDIUM", "HIGH"))
rows += [(3, i, r) for i, r in zip(te["id"], risk)]
print(f"Subtask 3 thresholds: p33={p33:.3f}  p66={p66:.3f}")
print("mean train distance by class:", pd.Series(d_tr).groupby(y).mean().reindex(LABELS).round(2).to_dict())

# ---------------- Subtask 4: hazard classifier ----------------
# Gradient-boosted trees handle the ~28 % missing values natively: at each split, NaN rows are sent
# to whichever side reduces the loss most, so "this sensor is missing" never gets confused with
# "this sensor reads the median".
clf = HistGradientBoostingClassifier(random_state=0)
cv = StratifiedKFold(5, shuffle=True, random_state=0)
pred_cv = cross_val_predict(clf, tr[NUMERIC], y, cv=cv)
print(f"Subtask 4 CV weighted F1: {f1_score(y, pred_cv, average='weighted'):.4f}")
clf.fit(tr[NUMERIC], y)
rows += [(4, i, p) for i, p in zip(te["id"], clf.predict(te[NUMERIC]))]

pd.DataFrame(rows, columns=["subtaskID", "datapointID", "answer"]).to_csv("submission.csv", index=False)
print("wrote submission.csv,", len(rows), "rows")