IOAI Training: Learn AI by solving problems

The problem in brief

You get hourly weather for Quezon City from 2020 to 2022 (26 304 hours, about 40 columns: temperature, humidity, pressure, wind, cloud cover, soil temperature and moisture, radiation, and a weather_code). The task is to predict the rainfall in millimetres for each of the 8 760 hours of 2023, whose other columns are given. The organisers warn that the sensors are "dirty" (999 % humidity readings, gaps, drift) and forbid external 2023 weather data. It was a 48-hour Kaggle sprint whose top 20 went to the national finals.

The 2023 rainfall is not published, so our score is on 2022, held out from training: we train on 2020 and 2021 and predict 2022, which is the same "next year" situation as the real test. We tried to rebuild the 2023 truth from the public Open-Meteo archive, which is clearly the source of the data, for scoring only. It is not identical: pressure matches, but temperature differs by about 0.2 °C and only 60 % of the weather codes agree, because the archive has been reprocessed since. We therefore did not use it.

How you are scored

The metric is the mean absolute error over the 8 760 test hours:

\text{MAE} = \frac{1}{n}\sum_{i=1}^{n} \lvert y_i-\hat y_i\rvert .

The private leaderboard (69.3 % of the hours) decided the ranking, and its scores are not public. MAE has one consequence worth knowing before you start: the best constant under absolute error is the median, not the mean. It rains in only about a third of the hours, so the median hour is dry, and "always 0" is the natural baseline.

First look at the data

The dirt. The training file has 263 humidity values equal to 999, 511 wind speeds equal to $-1$ (a sentinel), and 1 282 missing temperatures. The test file is clean. We set the sentinels to missing and rebuild the missing temperatures exactly from dew point and humidity, by inverting the Magnus formula $T_d = \frac{b\,\gamma}{a-\gamma}$ with $\gamma=\ln(RH/100)+\frac{aT}{b+T}$ . Tree models would cope with missing values anyway, but cleaning is cheap and avoids surprises.

The decisive column: weather_code. This is the WMO code that the weather service derives from its own precipitation field. Grouping the training hours by code tells the whole story:

code	meaning	hours	rain (mm/h)
0, 1, 2, 3	clear to overcast	17 208	exactly 0, every time
51 / 53 / 55	light / moderate / dense drizzle	7 630	0.1-0.4 / 0.5-0.9 / 1.0-1.2
61 / 63	slight / moderate rain	1 368	1.3-2.4 / 2.5-7.5
65	heavy rain	98	7.6-45.3

The code is a binned version of the target, with disjoint bins. All that remains is to place each hour inside its bin, and only codes 63 and 65 have wide bins.

Each code is a rain bin, and soil moisture places the hour inside it

A first baseline

approach (train 2020-2021, test 2022)	MAE
always 0 (the median)	0.2945
organisers' starter notebook	does not run: it uses columns `temp`, `humidity`, `pressure`, `wind_speed` that the files do not have
its idea, rebuilt (LightGBM, squared error, temperature/humidity/pressure/wind + hour, month)	0.3559, worse than always 0
LightGBM, squared error, every column except the codes	0.2319
LightGBM, absolute error, every column except the codes	0.2061
median rain of each weather code (a lookup table)	0.0727

Two lessons already. Squared-error models chase the mean and spread small positive predictions over dry hours, which MAE punishes. Switching the objective to L1 alone gains 10 %. And a lookup table of 10 medians beats every "real" model without the codes by a factor of three.

The key idea

Treat the code as a coarse measurement of the target, then refine inside each bin with physics. Two refinements carry almost all of the remaining gain.

Neighbouring hours. The model gets the codes of the 3 previous and 3 following hours. Inside a rain event, an hour of code 63 surrounded by 65s is likely near the top of its bin. Using future hours is legitimate here because the 2023 file gives every column for the whole year. It would not be possible in real forecasting.
The ground is a rain gauge. Rain wets the top 7 cm of soil, so the hour-to-hour change in soil_moisture_0_to_7cm measures how much water arrived. In the right panel above, the hours of code 63 show a clean rising branch, where moisture rises in proportion to rain. They also show a cloud near zero change: hours when the soil was already saturated and could absorb nothing more. A tree model separates the two cases easily, because it also sees the moisture level. This feature ends up as the most useful one after the code itself.

Then hard rules where the data is certain: codes 0-3 get exactly 0, every prediction is clipped to the range its code covered in training, and predictions are rounded to the 0.1 mm resolution of the data.

The full solution

Cleaning: the sentinels (999, $-1$ ) become missing values, and missing temperatures are rebuilt from dew point and humidity.
Features: all columns, hour, month, day of the year, codes at $t\pm1,2,3$ , the pressure change, and changes from the previous hour and to the next one for top and deeper soil moisture, cloud cover, humidity, temperature and solar radiation.
LightGBM with objective='l1', 1 500 trees, learning rate 0.02, 31 leaves, row and column subsampling 0.8, 2 threads.
Post-processing: codes 0-3 get 0, clip to the code's training range, round to 0.1 mm.
For the submission, refit on 2020-2022 and predict 2023 (8 760 rows with the integer id).

p = np.maximum(0, model.predict(test[F]))
p = np.clip(p, test.weather_code.map(lo), test.weather_code.map(hi))   # stay inside the code's bin
p[test.weather_code < 50] = 0                                         # dry codes: never any rain
p = np.round(p, 1)

Results

All scores: train on 2020-2021, MAE on the held-out year 2022 (8 760 hours).

approach	MAE 2022
always 0	0.2945
notebook-style LightGBM, squared error, 6 features	0.3559
LightGBM, squared error, all columns but codes	0.2319
LightGBM, absolute error, all columns but codes	0.2061
median per weather code	0.0727
LightGBM squared error, all columns + codes	0.0823
LightGBM absolute error, all columns + neighbouring codes	0.0704
+ clipping to the code's range	0.0690
+ soil-moisture, cloud and humidity changes, longer training	0.0515
+ rounding to 0.1 mm (final)	0.0511

The final model is 6 times better than "always 0" and 30 % better than the lookup table. Almost all of the remaining error sits in the two widest bins: codes 63 and 65 account for about 255 of the roughly 450 mm of total absolute error, in only 283 of the 8 760 hours.

What to remember

Read every column before modelling. A derived column like weather_code can contain the target almost directly. Here it was worth more than every sensor feature combined. (Asking whether such a column will exist at prediction time is the other half of the habit. It does here.)
Match the loss to the metric. For MAE, train with an absolute-error objective and remember that the best constant is the median. Under squared error, the model sprinkles small rain amounts over dry hours.
Use physics for features. Soil moisture rising from one hour to the next is a rain gauge, and its relation to rain depends on whether the soil is already saturated, which a tree model captures.
Encode what is certain as rules, not as something the model must learn: dry codes give 0, predictions stay inside the code's bin, and they are rounded to the data's resolution.
Validate as you will be tested. Hold out a whole future year, not random hours, so that seasonal patterns and neighbouring hours cannot leak between training and validation.

"""Quezon City Rainfall Sprint (IOAI Philippines 2026 semi-finals): hourly rainfall, MAE.

1. clean the sensor sentinels (humidity 999, wind -1, missing temperature rebuilt from dew point);
2. the WMO weather_code column already bins the rain (0-3 = dry, 51/53/55 drizzle, 61/63/65 rain):
   dry codes get exactly 0, the others are refined inside their bin;
3. LightGBM with the L1 (absolute error) objective on all columns + neighbouring hours' codes +
   hour-to-hour changes, especially of the top-soil moisture (rain wets the soil);
4. predictions are clipped to the range seen for that code in training.
Validation: train 2020-2021, test on 2022 (the 2023 truth is private). Then refit on 2020-2022.
Data: github.com/ioaiph/2026 Semi-Finals/Practical/Data -> work/
"""
import numpy as np, pandas as pd, lightgbm as lgb

D = 'work/'
DROP = ['city_name', 'datetime', 'precipitation', 'rain', 'snowfall', 'snow_depth', 'id']

def prepare(d):
    d = d.copy()
    d['datetime'] = pd.to_datetime(d['datetime'], format='mixed')
    d.loc[d.relative_humidity_2m > 100, 'relative_humidity_2m'] = np.nan      # the 999 % sensor
    d.loc[d.wind_speed_10m < 0, 'wind_speed_10m'] = np.nan                    # the -1 sentinel
    a, b = 17.625, 243.04                                                      # Magnus formula
    g = a * d.dew_point_2m / (b + d.dew_point_2m) - np.log(d.relative_humidity_2m / 100)
    d['temperature_2m'] = d.temperature_2m.fillna((b * g / (a - g)).round(1))
    d['hour'], d['month'], d['doy'] = d.datetime.dt.hour, d.datetime.dt.month, d.datetime.dt.dayofyear
    for k in (1, 2, 3):                                                        # neighbouring hours' codes
        d[f'wc_lag{k}'] = d.weather_code.shift(k)
        d[f'wc_lead{k}'] = d.weather_code.shift(-k)
    d['dp_change'] = d.pressure_msl.diff()
    for c in ['soil_moisture_0_to_7cm', 'soil_moisture_7_to_28cm', 'cloud_cover',
              'relative_humidity_2m', 'temperature_2m', 'shortwave_radiation']:
        d[c + '_d1'] = d[c].diff()                     # change since the previous hour
        d[c + '_dm1'] = d[c].shift(-1) - d[c]          # change to the next hour
    d['sm_d2'] = d.soil_moisture_0_to_7cm.shift(-1) - d.soil_moisture_0_to_7cm.shift(1)
    return d

def fit_predict(train, test):
    F = [c for c in train.columns if c not in DROP]
    m = lgb.LGBMRegressor(objective='l1', n_estimators=1500, learning_rate=0.02, num_leaves=31,
                          min_child_samples=20, subsample=0.8, subsample_freq=1, colsample_bytree=0.8,
                          n_jobs=2, random_state=0, verbose=-1)
    m.fit(train[F], train.precipitation)
    p = np.maximum(0, m.predict(test[F]))
    lo = train.groupby('weather_code').precipitation.min()
    hi = train.groupby('weather_code').precipitation.max()
    p = np.clip(p, test.weather_code.map(lo).fillna(0).values, test.weather_code.map(hi).fillna(50).values)
    p[test.weather_code.values < 50] = 0               # codes 0-3: never any rain in training
    return np.round(p, 1)                               # rain is recorded in 0.1 mm steps

if __name__ == '__main__':
    tr = prepare(pd.read_csv(D + 'ioai_qc_train.csv'))
    fit, val = tr[tr.datetime.dt.year < 2022], tr[tr.datetime.dt.year == 2022]
    mae = lambda p: np.abs(p - val.precipitation.values).mean()
    print('2022 hold-out MAE, predict 0           :', round(mae(np.zeros(len(val))), 4))
    med = fit.groupby('weather_code').precipitation.median()
    print('2022 hold-out MAE, median per code     :', round(mae(val.weather_code.map(med).values), 4))
    print('2022 hold-out MAE, LightGBM L1 + rules :', round(mae(fit_predict(fit, val)), 4))
    te = prepare(pd.read_csv(D + 'TEST_FINAL_INT.csv'))
    pd.DataFrame({'id': te['id'].astype(int), 'precipitation': fit_predict(tr, te)}).to_csv('submission.csv', index=False)
    print('submission.csv written:', len(te), 'rows')