IOAI 2024 (Burgas, Bulgaria), On-Site Round

Lost in a Hyperspace: ML Regression challenge

![](https://raw.githubusercontent.com/IOAI-official/IOAI-2024/main/On-Site-Round/Lost_in_Hyperspace/figs/Lost in Hyperspace Fig 1.png)

Story Background

Congratulations on your promotion to Principal Engineering Detective! Your impressive work on the previous task has earned you this exciting new challenge. Now, you are entrusted with the ancient and mesmerizing Glowing Hypercubes, which share some intriguing characteristics with the "Pulse of the Machine" widgets from your last mission (refer to Important Tips for details). Your mission is to unravel the mysteries of these Glowing Hypercubes by predicting three vital properties using the provided data.

Objective and Limitations

Your ultimate goal is to effectively predicts three properties of the Glowing Hypercubes
Every Glowing Hypercube is represented by the (5 x 5 x 5 x 6) array with lots of symmetries and unique properties (see Important tips section for details)
You need to engineer a small number of features from the Glowing Hypercube data, since efficient factory procedures allow you to only use Linear Regression as a model, with no hyperparameters change allowed. You are also limited by 300 features for each task.
Your success will be measured by Root Mean Square Error metric for each feature independently and is translated into the score on the leaderboard.
Note that different features have different weights in the final score. See SCALING_WEIGHTS variable for details. After scaling, to make a single score number, we will average normalized RMSEs for each property.
Your solution for each task should not exceed 5 minutes for feature generation, training, and inference on the standard Colab non-GPU instance.
Share the ml_feature_0.txt, ml_feature_1.txt, and ml_feature_2.txt files with us, and don't forget to supply your Google Colab as well

Important Tips

![](https://raw.githubusercontent.com/IOAI-official/IOAI-2024/main/On-Site-Round/Lost_in_Hyperspace/figs/Lost in Hyperspace Fig 2.png)

Linear Regression documentation
- https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html
Handy Numpy functions:
Root Mean Square Error
- https://en.wikipedia.org/wiki/Root_mean_square_deviation

Clarification of the methods' usage

Methods' usage limitations mostly resemble the ones for the home task, namely:

Mind the time limits. These are separate for each forecasting model, and suggest non-GPU instance. This time includes feature generation, training, and inference on the validation/test set. Data analysis, feature search and selection are not subject to the time limitation
Supervised neural networks (and any supervised models: LDA, boosting trees, etc) are not allowed as a feature extractor. Usage of simpler supervised models (e.g., ensembles on trees, linear regression) for the feature selection is allowed, given the model not being used as a feature extractor. Unsupervised learning is allowed (including autoencoders).
Usage of pretrained models or auto ML solutions is not allowed. Libraries that automatically sort through various approaches (including unsupervised ones) for the users are not allowed as well.
Given the time constraints, Colab notebooks should be as reproducible as possible. In case of doubt (abuse of the time limits, data usage, etc), Jury have the right to use the models/answers generated by the notebook, and pick the answers with the lower score.
Yes, different models can use different feature sets.
No, you cannot use the validation data for training.

If you are not sure, please ask Jury!

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt

SCALING_WEIGHTS = [100/15, 100/8, 100/100]

data = pd.read_pickle('./training_set/ml_data_onsite_start.pickle')
for key in data.keys():
  print(key)

for key in data['X'].keys():
  print(key)

for key in data['y'].keys():
  print(key)

X_train = data['X']['train']
y_train = data['y']['train']

X_val = data['X']['val']
y_val = data['y']['val']

X_test = data['X']['live_test']

X_train.shape, y_train.shape, X_val.shape, y_val.shape, X_test.shape

def vis(arr):
  plt.figure(figsize=(8, 8))

  cnt = 1
  for z in range(5):
    for q in range(6):
      plt.subplot(5, 6, cnt)
      plt.imshow(arr[:, :, z, q], vmin=-40, vmax=40, cmap='hsv')
      plt.grid()
      plt.axis('off')
      cnt += 1
  plt.tight_layout()

vis(X_train[0])

Functions for result evaluation / writing predictions

Do not change it!

def test_solution(X_train, y_train, X_val, y_val, feature_num=0):
    assert X_train.shape[-1] <= 300, "Too many features! Should be less than 300"
    assert X_val.shape[-1] <= 300, "Too many features! Should be less than 300"

    model =  LinearRegression().fit(
        X_train,
        y_train[:, feature_num]
    )
    predictions = model.predict(X_val)
    rmse = mean_squared_error(
        predictions,
        y_val[:, feature_num]
    )**.5
    normalized_rmse = rmse * SCALING_WEIGHTS[feature_num]
    print(f"Property #{feature_num}:    raw RMSE={rmse:.6f}")
    print(f"Property #{feature_num}: scaled RMSE={normalized_rmse:.6f}")
    return round(normalized_rmse, 6)

Let's try a baseline solution

def dummy_feature_extractor(X):
    X_new = X.reshape((X.shape[0], -1)) # ravel
    X_new = X_new[:, :300] # pick first 300 features
    return X_new

 dummy_feature_extractor(X_train).shape

%%time
total_score = 0
for feature_number in range(3):
  total_score += test_solution(
      dummy_feature_extractor(X_train),
      y_train,
      dummy_feature_extractor(X_val),
      y_val,
      feature_num=feature_number
  )
  print()
total_score /= 3
print('='*16)
print(f"Total score = {total_score:.6f}")

How to prepare the answer files

def generate_predictions(X_train, y_train, X_test, feature_num=0):
    assert X_train.shape[-1] <= 300
    assert X_test.shape[-1] <= 300

    model =  LinearRegression().fit(
        X_train,
        y_train[:, feature_num]
    )
    predictions = model.predict(X_test)
    return predictions


## Generate solutions and write to the file
combined = {'ID': np.arange(X_test.shape[0])}

for feature_number in range(3):
    predictions = generate_predictions(
        dummy_feature_extractor(X_train),
        y_train,
        dummy_feature_extractor(X_test),
        feature_num=feature_number
    )

    combined[f'y{feature_number+1}'] = predictions

pd.DataFrame(combined).to_csv('predictions.csv', index=False)

# load the test dataset
loaded = pd.read_pickle("./Solution/test_set/ml_data_onsite_final_test.pickle")
X_test_final = loaded['X']['final_test']


# make final predictions
combined = {'ID': np.arange(X_test_final.shape[0])}

for feature_number in range(3):
    predictions = generate_predictions(
        dummy_feature_extractor(X_train),
        y_train,
        dummy_feature_extractor(X_test_final),
        feature_num=feature_number
    )

    combined[f'y{feature_number+1}'] = predictions

pd.DataFrame(combined).to_csv('final_predictions.csv', index=False)