NEOAI 2026 · Correction by IOAI Training

Terminal Cuties Segmentation

This page gives the full approach. Read it once you have searched.

Metric: Dice coefficient
Evaluated on: held-out validation (official test labels are private)
Our score: Dice 0.903 (400 held-out images)
Expected level: best public LB 0.921 (4th entry 0.903)

Our solution does not reach the expected level: there is still room to beat it

We solved the problem ourselves: downloaded the data, ran the experiments and measured the score below. The statement and data belong to their organisers.

The problem in brief

Photos of cats and dogs were converted into coloured ASCII art: every image is a grid of 128 × 128 characters, each drawn in its own colour inside a 10 × 10 pixel cell, giving a 1280 × 1280 picture. The characters come from a fixed palette: ten "fill" characters from light to dense (space, ., ;, c, o, P, O, ?, @, ■) and four "edge" characters (|, -, \, /) placed on contours. You must segment the pet: output a binary mask of the 1280 × 1280 image. You get 3 680 training images with masks and 500 test images, and two rules: no pretrained models, no external data. Everything has to be learned from scratch on the given images.

The data comes from the Kaggle competition archive. The test masks are private and we never submitted anything, so all scores below are measured on 400 held-out training images, with the exact pixel-level metric.

How you are scored

The Dice coefficient between the predicted mask $P$ and the true mask $M$ , at full resolution, averaged over images:

\text{Dice}(P, M) = \frac{2\,|P \cap M|}{|P| + |M|}.

What to aim for. The organisers publish no threshold. On the public leaderboard the best entry scores 0.921, the next ones 0.909, 0.906 and 0.903.

First look at the data

The organisers' baseline shrinks each image to 128 × 128 (one pixel per character), trains a small CNN, and enlarges the predicted mask back to 1280 × 1280. Before training anything, it is worth asking what the best possible mask of that kind would score. We took the true character-level labels, filled each pet character's whole 10 × 10 cell, and compared with the real mask: Dice 0.23. Even a perfect classifier would fail with this output.

The reason is visible once you zoom in:

The mask is made of ink pixels only

The true mask is not the silhouette of the pet. It is the set of ink pixels of the characters that belong to the pet: the strokes of the /, the ring of the o, the bar of the -. The black background pixels between strokes are 0 even in the middle of the dog. Only 7 % of the pixels are 1 in a typical mask, while the silhouette covers about half the image.

Two checks confirm it on a training image: every mask pixel is an ink pixel (non-black in the render), and the rule

\hat M(x) = \text{ink}(x) \;\wedge\; \text{pet}\big(\text{cell}(x)\big)

rebuilds the mask with Dice 0.994 from the character labels. The 0.6 % missing comes from a few glyphs (like \) that slightly overflow into the next cell.

A first baseline

So the task reduces to a 128 × 128 classification problem: which characters belong to the pet? The pixel mask then follows from the ink of the image itself, which we can read exactly.

A U-Net trained for a single epoch on character-level features already gives 0.79 with the ink rule. The same predictions turned into full blocks give 0.17: the output format is worth more than the model.

The key idea

Work at the level of the generating process: one decision per character, then let the image provide the exact pixels. Two consequences guide the design.

What the model sees. Instead of a blurred 128 × 128 RGB image, we describe each character by 13 numbers: the ink density in a 3 × 3 grid of sub-blocks (3, 4 and 3 pixels wide), the mean colour of its ink (3 values) and its total amount of ink. The sub-block densities tell the glyphs apart: | lights up the middle column, - the middle row, \ and / the two diagonals, and the fill characters differ by their total density. This matters because the converter put the edge characters on the contours of the photo, so the glyph shape is a direct hint of where the pet's outline is. The colour carries the content (fur versus grass versus a sofa).

What the model optimises. A character's weight in the Dice score is its amount of ink: a missed @ costs many pixels, a missed . almost none, a missed space nothing. So the loss counts only characters with ink, and the soft Dice part is weighted by ink:

\mathcal L = \text{BCE}_{\text{ink}>0} + 1 - \frac{2\sum_c n_c\, p_c\, y_c + 1}{\sum_c n_c (p_c + y_c) + 1},

where $n_c$ is the ink of character $c$ , $p_c$ the predicted probability and $y_c$ the fraction of its ink that is in the mask.

Augmenting correctly. A horizontal mirror is a natural augmentation for pets, but a mirrored image would contain mirrored glyphs, and a / becomes a \. Mirroring our features therefore means flipping the 128 × 128 grid and swapping the left and right sub-block columns of each character, so the augmented example is still a valid ASCII rendering.

The full solution

Features (about 6 minutes for all images, read directly from the zip): a 128 × 128 × 13 array per image, plus the target $y_c$ per character.
U-Net from scratch: 5 levels, widths 32 to 256, batch normalisation, 5.1 M parameters, trained 30 epochs (batches of 8, AdamW, one-cycle learning rate up to $2\times10^{-3}$ ) with the loss above and the glyph-aware mirror. About 30 seconds per epoch on the Apple GPU.
Prediction: average the probabilities of the image and of its mirror (test-time augmentation), threshold at 0.5.
Back to pixels with the ink rule:

def to_pixels(cells, im):
    return (im.max(2) > 0) & np.kron(cells, np.ones((10, 10), bool))   # ink AND pet character

Encode each mask in column-major run-length encoding, as in the organisers' baseline.

Results

Mean pixel Dice at 1280 × 1280 on 400 held-out training images (the test masks are private).

approach	Dice
perfect character labels, full 10 × 10 blocks (best possible "baseline-style" output)	0.233
U-Net 1 epoch, width 16, full blocks	0.171
U-Net 30 epochs, width 32, full blocks	0.197
U-Net 1 epoch, width 16, ink rule	0.792
U-Net 30 epochs, width 32, ink rule (final)	0.903
perfect character labels, ink rule (upper bound)	0.994

The final model reaches 0.903 on held-out images, the level of the fourth public-leaderboard entry and 2 points below the best one (0.921). The gap to the upper bound (0.994) is now purely about deciding which characters are pet, mostly near the outline and in dark regions where fur and background look alike in ASCII.

What to remember

Understand the target before choosing the model. Ten minutes spent looking at one mask at full zoom changed the best achievable score from 0.23 to 0.99. When data is synthetic, find the rule that generated it.
Predict at the scale of the decision. Each character is either pet or not; predicting 1.6 million pixels independently would waste capacity, and predicting blocks loses the exact shape. One decision per character plus an exact, deterministic expansion gets the best of both.
Weight the loss like the metric. The Dice score counts pixels, so characters with more ink matter more; an ink-weighted soft Dice aligns training with evaluation.
Augmentations must keep the data valid. Mirroring ASCII art turns / into \; the feature transform has to follow.
From scratch is fine when inputs are simple. With a compact, meaningful encoding (13 numbers per character), a U-Net without pretraining reaches the top of the leaderboard in 15 minutes.

Full code (solution.py)

"""Terminal Cuties Segmentation (NEOAI 2026, day 1, CV).

Each 1280x1280 image is a 128x128 grid of coloured ASCII characters (10x10 px each), and
the target mask is made of the INK pixels of the characters that belong to the pet.
So the problem is: classify each of the 128x128 characters as pet / background,
then the mask is   ink(pixel) AND pet(character of the pixel).
  1. Read each render into 13 numbers per character: ink density in a 3x3 grid of
     sub-blocks (glyph shape: '|', '-', '/', '\' outline the contours), mean ink colour,
     total ink.
  2. Train a U-Net from scratch (no pretrained weights, as the rules require) on the
     128x128x13 grids; target = fraction of the character's ink that is in the mask.
  3. Predict with flip test-time augmentation, threshold 0.5, expand to pixels with the
     ink rule, encode as RLE.

Data: kaggle competition "neoai-2026-day-1-cv" (the zip is read directly, never extracted).
Usage: python solution.py   -> held-out pixel Dice (400 images) + submission.csv.gz (not submitted)
Runtime: ~6 min feature extraction + ~18 min training on Apple MPS; < 2 GB memory.
"""
import io, time, zipfile, numpy as np, pandas as pd, torch, torch.nn as nn, torch.nn.functional as F
from PIL import Image

torch.set_num_threads(2); torch.manual_seed(0); np.random.seed(0)
DEV = 'mps' if torch.backends.mps.is_available() else 'cpu'
if DEV == 'mps':
    torch.mps.set_per_process_memory_fraction(0.1)
ZIP, EPOCHS, BS, WIDTH = 'neoai-2026-day-1-cv.zip', 30, 8, 32
EDGES = np.array([0, 3, 7, 10])                                   # 3x3 sub-blocks of a 10x10 cell
FLIP = [i * 3 + (2 - j) for i in range(3) for j in range(3)] + [9, 10, 11, 12]   # channel order after a mirror


# ----------------------------------------------------------------------------- features
def cell_features(im):
    im = im.astype(np.float32); b = (im.max(2) > 0).reshape(128, 10, 128, 10)
    sub = [b[:, EDGES[i]:EDGES[i + 1], :, EDGES[j]:EDGES[j + 1]].mean((1, 3)) for i in range(3) for j in range(3)]
    n = b.sum((1, 3))
    col = im.reshape(128, 10, 128, 10, 3).sum((1, 3)) / np.maximum(n, 1)[..., None]   # mean ink colour
    f = np.concatenate([np.stack(sub, -1) * 255, col, (n / 100.0 * 255)[..., None]], -1)
    return np.clip(f, 0, 255).astype(np.uint8)                                      # (128, 128, 13)


def cell_target(ink, m):
    """Fraction of a character's ink inside the mask (glyphs may overflow into a neighbour cell)."""
    a = (ink & m).reshape(128, 10, 128, 10).sum((1, 3)); n = ink.reshape(128, 10, 128, 10).sum((1, 3))
    return np.where(n > 0, a / np.maximum(n, 1), 0).astype(np.float16)


def to_pixels(cells, im):
    return (im.max(2) > 0) & np.kron(cells, np.ones((10, 10), bool))                 # the ink rule


# ----------------------------------------------------------------------------- model
def block(i, o):
    return nn.Sequential(nn.Conv2d(i, o, 3, padding=1), nn.BatchNorm2d(o), nn.ReLU(inplace=True),
                         nn.Conv2d(o, o, 3, padding=1), nn.BatchNorm2d(o), nn.ReLU(inplace=True))


class UNet(nn.Module):
    def __init__(self, cin=13, b=WIDTH):
        super().__init__(); c = [b, 2 * b, 4 * b, 8 * b, 8 * b]
        self.down = nn.ModuleList([block(cin, c[0])] + [block(c[k], c[k + 1]) for k in range(4)])
        self.up = nn.ModuleList([nn.ConvTranspose2d(c[k + 1], c[k], 2, stride=2) for k in range(4)])
        self.dec = nn.ModuleList([block(2 * c[k], c[k]) for k in range(4)]); self.out = nn.Conv2d(c[0], 1, 1)

    def forward(self, x):
        skips = []
        for k, d in enumerate(self.down):
            x = d(x if k == 0 else F.max_pool2d(x, 2)); skips.append(x)
        for k in range(3, -1, -1):
            x = self.dec[k](torch.cat([self.up[k](x), skips[k]], 1))
        return self.out(x)[:, 0]


def to_t(xb):
    return torch.from_numpy(np.ascontiguousarray(xb)).permute(0, 3, 1, 2).contiguous().float().div(255).to(DEV)


def train(X, Y, idx):
    model = UNet().to(DEV)
    opt = torch.optim.AdamW(model.parameters(), lr=2e-3, weight_decay=1e-4)
    sch = torch.optim.lr_scheduler.OneCycleLR(opt, 2e-3, total_steps=EPOCHS * (len(idx) // BS), pct_start=0.1)
    for ep in range(EPOCHS):
        model.train(); perm = np.random.permutation(idx); t0 = time.time()
        for b in range(len(idx) // BS):
            ib = np.sort(perm[b * BS:(b + 1) * BS]); xb = X[ib].copy(); yb = Y[ib].astype(np.float32)
            fl = np.random.rand(len(ib)) < 0.5                  # mirror: flip columns AND glyph sub-blocks
            xb[fl] = xb[fl][:, :, ::-1][..., FLIP]; yb[fl] = yb[fl][:, :, ::-1]
            x = to_t(xb); y = torch.from_numpy(np.ascontiguousarray(yb)).to(DEV)
            ink = x[:, 12]; w = ink > 0                          # only characters with ink count
            logit = model(x); p = torch.sigmoid(logit)
            bce = (F.binary_cross_entropy_with_logits(logit, y, reduction='none') * w).sum() / w.sum()
            inter = (p * y * ink).sum((1, 2)); den = ((p + y) * ink).sum((1, 2))   # ink-weighted soft Dice
            loss = bce + (1 - (2 * inter + 1) / (den + 1)).mean()
            loss.backward(); opt.step(); sch.step(); opt.zero_grad()
        print(f'  epoch {ep} last loss {loss.item():.3f} ({time.time() - t0:.0f}s)', flush=True)
    return model


@torch.no_grad()
def predict(model, Xs):
    model.eval(); out = []
    for i in range(0, len(Xs), 8):
        xb = np.array(Xs[i:i + 8])
        p = torch.sigmoid(model(to_t(xb))) + torch.sigmoid(model(to_t(xb[:, :, ::-1][..., FLIP]))).flip(2)
        out.append((p / 2).cpu().numpy())
    return np.concatenate(out)


def rle(mask):                                                   # column-major Kaggle RLE, as in the baseline
    px = np.concatenate([[0], mask.flatten(order='F').astype(np.uint8), [0]])
    runs = np.where(px[1:] != px[:-1])[0] + 1; runs[1::2] -= runs[::2]
    return ' '.join(map(str, runs))


if __name__ == '__main__':
    z = zipfile.ZipFile(ZIP); names = z.namelist()
    tr = pd.read_csv(io.BytesIO(z.read([n for n in names if n.endswith('train.csv')][0])))
    te = pd.read_csv(io.BytesIO(z.read([n for n in names if n.endswith('test.csv')][0])))
    pre = [n for n in names if n.endswith(tr.img[0])][0][:-len(tr.img[0])]
    rgb = lambda p: np.array(Image.open(io.BytesIO(z.read(pre + p))).convert('RGB'))
    lab = lambda p: np.array(Image.open(io.BytesIO(z.read(pre + p)))) > 0
    X = np.zeros((len(tr), 128, 128, 13), np.uint8); Y = np.zeros((len(tr), 128, 128), np.float16)
    for i, (p, l) in enumerate(zip(tr.img, tr.label)):
        im = rgb(p); X[i] = cell_features(im); Y[i] = cell_target(im.max(2) > 0, lab(l))
    va = np.sort(np.random.RandomState(0).choice(len(tr), 400, replace=False)); fit = np.setdiff1d(np.arange(len(tr)), va)
    model = train(X, Y, fit)
    P = predict(model, X[va]); dices = []
    for n, i in enumerate(va):                                   # exact pixel Dice at 1280x1280
        pm, m = to_pixels(P[n] > 0.5, rgb(tr.img[i])), lab(tr.label[i])
        s = pm.sum() + m.sum(); dices.append(1.0 if s == 0 else 2 * (pm & m).sum() / s)
    print(f'held-out mean pixel Dice: {np.mean(dices):.4f}')
    del X, Y
    rows = []
    for _, r in te.iterrows():
        im = rgb(r.img); p = predict(model, cell_features(im)[None])[0]
        rows.append((r.id, rle(to_pixels(p > 0.5, im))))
    pd.DataFrame(rows, columns=['id', 'mask_rle']).to_csv('submission.csv.gz', index=False, compression='gzip')
    print('wrote submission.csv.gz (not submitted)')