IOAI Training: Learn AI by solving problems

The problem in brief

You get colour images of galaxies from a sky survey and must sort each one into one of ten shapes: disturbed, merging, round smooth, in-between round smooth, cigar-shaped smooth, barred spiral, unbarred tight spiral, unbarred loose spiral, edge-on without bulge, edge-on with bulge. The labels come from Galaxy Zoo volunteers. The contest (Cyprus AI Camp 2026, a 3-hour Kaggle session) forbids pretrained models and extra data, so the network has to be trained from scratch, and the score is plain accuracy on a hidden test set.

Rebuilt data. The Kaggle files need a login, so we rebuilt the task from the public dataset it comes from, Galaxy10 DECaLS (Hugging Face matthieulel/galaxy10_decals: 15 962 training and 1 774 test images of 256 x 256 pixels, labels included). Our split is therefore not the organisers' split. We kept 10 % of the training images as a validation set for every decision and scored the 1 774 test images once per model at the end.

How you are scored

Accuracy, the share of test galaxies whose predicted class is the true one:

\text{accuracy} = \frac{1}{N}\sum_{i=1}^{N} \mathbb{1}[\hat y_i = y_i].

There is no points formula on the competition page. As a reference, the public Kaggle leaderboard of the camp (read with the Kaggle CLI) has a best score of 0.822, followed by several contestants around 0.80. Keep in mind that it is measured on the organisers' test images, not on ours, so the comparison is only indicative.

First look at the data

Three training galaxies per class

The class is in the centre. The galaxy sits in the middle of the cutout, and the outer quarter of the image is mostly empty sky or neighbouring objects. We crop the central 75 % and downsample to 96 pixels, which keeps the shape and makes training 7 times cheaper than at 256 pixels.
Orientation means nothing. The sky has no "up": a spiral rotated by 90 degrees or seen in a mirror is still the same kind of spiral. This symmetry is the key to the problem.
Moderate imbalance. Cigar-shaped galaxies are rare (306 training images) while round smooth ones number 2 395.
Some classes are fuzzy. "Disturbed" galaxies, tight against loose spirals, or round smooth against in-between round smooth are judgment calls even for humans, so 100 % is out of reach.

A first baseline

A plain CNN trained from scratch for 15 epochs without augmentation (four blocks of two 3 x 3 convolutions, 32 to 256 channels, global average pooling) reaches 0.819 test accuracy. Its training loss keeps falling fast while validation accuracy grows more slowly: with 14 000 images and a network that sees each one in the same orientation every epoch, it starts memorising.

Even this model gains a lot from one cheap trick at prediction time: averaging its predictions over the 8 rotated and mirrored versions of each test image raises it to 0.846. That is a strong hint that the network has learned orientation-specific details that do not generalise.

The key idea

Build the symmetry of the sky into the training: use the 8 symmetries of the square as data augmentation, and average over them at test time.

The rotations by 0, 90, 180 and 270 degrees, each with or without a mirror flip, form the dihedral group $D_4$ . They map the square pixel grid onto itself exactly (no interpolation, no blur), and none of them can change the class of a galaxy. So for every training image we get 8 equally valid images for free, and we tell the network, through the data, that its answer must not depend on orientation:

f(g \cdot x) \approx f(x) \quad \text{for every } g \in D_4 .

At test time we compute the class probabilities for all 8 transformed copies of each galaxy and average them. Each copy gives a slightly different, partly independent opinion, and the average is more reliable than any of them. On top of that, a random 64-pixel crop from the 72-pixel image adds small translations, because the centring of the cutouts is not perfect.

The augmented network learns more slowly (its training loss stays higher, since the task is harder), but it overfits much less: it needs more epochs, and then it is clearly better.

The full solution

Preprocess once: crop the central 75 % of each 256-pixel image (192 pixels), resize to 96 pixels, store as uint8. At training time, resize to 72 pixels and normalise each colour channel.
Network from scratch: four blocks of two 3 x 3 convolutions with batch normalisation and ReLU, max-pooling between blocks, widths 48, 96, 192, 384, global average pooling, dropout 0.3, a linear layer to 10 classes.
Augmentation: random 64-pixel crop, then one of the 8 $D_4$ symmetries chosen at random for each batch.
Training: 40 epochs of AdamW (weight decay $5 \times 10^{-4}$ , batch 64), one-cycle learning rate up to $2 \times 10^{-3}$ , cross-entropy with label smoothing 0.05. About 12 minutes on an Apple GPU.
Prediction: average the softmax over the 8 symmetries of the central 64-pixel crop.

def d4(x, k):                          # k = 0..7
    x = torch.rot90(x, k % 4, dims=(2, 3))
    return torch.flip(x, dims=(3,)) if k >= 4 else x
probs = torch.stack([model(d4(x, k)).softmax(1) for k in range(8)]).mean(0)

Width and number of epochs were chosen on the validation split (0.863 with TTA for width 48 and 40 epochs, against 0.848 for width 32 and 25 epochs).

Confusion matrix of the final model on the test set

Where does it fail? The errors follow the astronomy. "Disturbed" galaxies are recognised only half of the time (49 %): they are irregular by definition and are often taken for loose spirals (20 %) or in-between round smooth galaxies. Tight and loose unbarred spirals are confused with each other (13 % and 5 %), since "how tightly wound" is a matter of degree. Cigar-shaped galaxies sometimes go to edge-on with bulge (11 %): a very elongated smooth galaxy looks like a disc seen from the side. Round smooth, in-between round, both edge-on classes and merging galaxies are all at 90 % or above.

Results

Accuracy on the 1 774 test images of the public Galaxy10 DECaLS split (rebuilt task); all choices made on our validation split.

approach	validation	test
CNN width 32, no augmentation, 15 epochs	0.810	0.819
same + 8-fold symmetry averaging at test time	0.833	0.846
CNN width 32, $D_4$ + crop augmentation, 25 epochs	0.844	0.850
same + 8-fold test-time averaging	0.848	0.861
CNN width 48, $D_4$ + crop, 40 epochs	0.858	0.870
same + 8-fold test-time averaging (final)	0.863	0.872

The final model reaches 87.2 % test accuracy, trained from scratch in 12 minutes. For comparison, the best public score on the camp's Kaggle leaderboard was 0.822, on different test images.

What to remember

Find the symmetries of your problem. Galaxies have no preferred orientation; the 8 rotations and flips of the square are exact, cost nothing, and multiply the data by 8.
Use the same symmetries at test time. Averaging the predictions over the 8 views gained 1 to 3 points, even for a model trained without augmentation.
Augmentation trades training speed for generalisation. The augmented model needs more epochs and a larger width, but then overfits far less.
Crop to where the information is. The galaxy is in the centre: cropping and downsampling made training 7 times cheaper without hurting accuracy.
Read the confusion matrix with domain knowledge. The remaining errors (disturbed galaxies, tight against loose spirals) are the classes where human labellers also disagree.

"""
Cyprus AI Camp 2026, "Galaxy10 Classification": 10 morphology classes from 256x256 colour cutouts, accuracy.
Rules: no pretrained model, no extra data -> a CNN trained from scratch.

The Kaggle files need a login, so the task is REBUILT from the public Galaxy10 DECaLS dataset
(huggingface.co/datasets/matthieulel/galaxy10_decals: 15 962 train / 1 774 test images, labels included).
prep step (see PREP below): each image is centre-cropped to its central 75 % (192 px) and resized to 96 px.
We keep 10 % of train as validation for every choice; the 1 774 test images are scored once at the end.

Key idea: a galaxy's class does not change when the image is rotated or mirrored, so we train with the
8 symmetries of the square (dihedral group D4) as augmentation and average the 8 predictions at test time.
Run: python solution.py --aug d4 --epochs 40 --width 48   (about 12 min on Apple MPS; final model)
"""
import argparse, glob, os, time, io
import numpy as np, torch, torch.nn as nn, torch.nn.functional as F

p = argparse.ArgumentParser()
p.add_argument("--aug", default="d4", choices=["none", "d4"]); p.add_argument("--epochs", type=int, default=40)
p.add_argument("--width", type=int, default=48); p.add_argument("--tag", default="run")
args = p.parse_args()
D = os.environ.get("DATA_DIR", "work"); torch.set_num_threads(2); torch.manual_seed(0); np.random.seed(0)
dev = "mps" if torch.backends.mps.is_available() else "cpu"

def PREP():   # run once: parquet shards from HF -> compact npz (centre crop 75 %, 96 px)
    import pandas as pd, subprocess; from PIL import Image
    base = "https://huggingface.co/datasets/matthieulel/galaxy10_decals/resolve/main/data/"
    for f in [l for l in open(f"{D}/shards.txt").read().split()]:
        subprocess.run(["curl", "-sL", "-o", f"{D}/tmp.parquet", base + f + ".parquet"], check=True)
        d = pd.read_parquet(f"{D}/tmp.parquet"); X = []
        for v in d["image"]:
            im = Image.open(io.BytesIO(v["bytes"])).convert("RGB"); w = im.size[0]; c = int(0.75 * w); l = (w - c) // 2
            X.append(np.asarray(im.crop((l, l, l + c, l + c)).resize((96, 96), Image.BILINEAR)))
        np.savez(f"{D}/{f}.npz", X=np.stack(X), y=d["label"].values); os.remove(f"{D}/tmp.parquet")

def load(pattern):
    z = [np.load(f) for f in sorted(glob.glob(f"{D}/{pattern}*.npz"))]
    return np.concatenate([a["X"] for a in z]), np.concatenate([a["y"] for a in z])
Xtr, ytr = load("train"); Xte, yte = load("test")
perm = np.random.permutation(len(Xtr)); nv = len(Xtr) // 10
Xva, yva, Xtr, ytr = Xtr[perm[:nv]], ytr[perm[:nv]], Xtr[perm[nv:]], ytr[perm[nv:]]
to_t = lambda X: torch.from_numpy(X).permute(0, 3, 1, 2).contiguous()          # uint8 NCHW, kept on CPU
Xtr, Xva, Xte = to_t(Xtr), to_t(Xva), to_t(Xte)
mean = Xtr[:2000].float().mean((0, 2, 3), keepdim=True) / 255; std = Xtr[:2000].float().std((0, 2, 3), keepdim=True) / 255
def prep(xb):                     # uint8 96 px -> normalised float 72 px on the device
    x = (xb.to(dev).float() / 255 - mean.to(dev)) / std.to(dev)
    return F.interpolate(x, size=72, mode="bilinear", align_corners=False)
def d4(x, k):                     # the k-th symmetry of the square (k = 0..7)
    x = torch.rot90(x, k % 4, dims=(2, 3))
    return torch.flip(x, dims=(3,)) if k >= 4 else x

def block(cin, cout):
    return nn.Sequential(nn.Conv2d(cin, cout, 3, padding=1, bias=False), nn.BatchNorm2d(cout), nn.ReLU(inplace=True),
                         nn.Conv2d(cout, cout, 3, padding=1, bias=False), nn.BatchNorm2d(cout), nn.ReLU(inplace=True))
class Net(nn.Module):
    def __init__(s, w):
        super().__init__()
        s.f = nn.Sequential(block(3, w), nn.MaxPool2d(2), block(w, 2 * w), nn.MaxPool2d(2),
                            block(2 * w, 4 * w), nn.MaxPool2d(2), block(4 * w, 8 * w))
        s.head = nn.Sequential(nn.Dropout(0.3), nn.Linear(8 * w, 10))
    def forward(s, x): return s.head(s.f(x).mean((2, 3)))       # global average pooling
model = Net(args.width).to(dev)
opt = torch.optim.AdamW(model.parameters(), lr=2e-3, weight_decay=5e-4)
steps = args.epochs * (len(Xtr) // 64); sched = torch.optim.lr_scheduler.OneCycleLR(opt, 2e-3, total_steps=steps)

@torch.no_grad()
def predict(X, tta):
    model.eval(); out = []
    for i in range(0, len(X), 256):
        x = prep(X[i:i + 256])[:, :, 4:68, 4:68]                   # centre 64 px crop
        ks = range(8) if tta else [0]
        out.append(torch.stack([model(d4(x, k)).softmax(1) for k in ks]).mean(0).cpu())
    model.train(); return torch.cat(out).numpy()

t0 = time.time()
for ep in range(args.epochs):
    idx = torch.randperm(len(Xtr)); tot = 0
    for i in range(0, len(Xtr) - 63, 64):
        b = idx[i:i + 64]; x = prep(Xtr[b]); y = torch.from_numpy(ytr[b.numpy()]).long().to(dev)
        if args.aug == "d4":
            ox, oy = np.random.randint(0, 9, 2); x = x[:, :, ox:ox + 64, oy:oy + 64]   # random 64 px crop
            x = d4(x, np.random.randint(8))                                          # random symmetry
        else:
            x = x[:, :, 4:68, 4:68]
        loss = F.cross_entropy(model(x), y, label_smoothing=0.05)
        opt.zero_grad(); loss.backward(); opt.step(); sched.step(); tot += loss.item()
    if ep % 5 == 4 or ep == args.epochs - 1:
        acc = (predict(Xva, False).argmax(1) == yva).mean()
        print(f"epoch {ep + 1:3d}  train loss {tot / (len(Xtr) // 64):.3f}  val acc {acc:.4f}  {(time.time() - t0) / 60:.1f} min", flush=True)

pv, pv8 = predict(Xva, False), predict(Xva, True)
pt, pt8 = predict(Xte, False), predict(Xte, True)
print(f"[{args.tag}] val acc {(pv.argmax(1) == yva).mean():.4f} | val acc with 8-fold TTA {(pv8.argmax(1) == yva).mean():.4f}")
print(f"[{args.tag}] TEST acc {(pt.argmax(1) == yte).mean():.4f} | TEST acc with 8-fold TTA {(pt8.argmax(1) == yte).mean():.4f}")
np.savez(f"{D}/pred_{args.tag}.npz", pv=pv8, pt=pt8, yva=yva, yte=yte)