Complete pipeline

The junior data scientist: pick the right tool, chain the steps cleanly, deliver a report

In this chapter, you will learn to

• Chain every step of a project, from raw data to a report
• Choose the right tool for the problem

Tools: pandas, scikit-learn

5 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   In real life, nobody hands you a clean table and a model to train: you get a problem, raw data, and people expect a reliable, explained result. - The model is only a small piece. In a now famous 2015 paper…

2. The Right Tool

   sklearn allowed. Your boss asks you: "Why this model and not another one?" Same dataset as the previous exercise (already cleaned this time). Compare at least 3 different algorithms and submit the predictions…

3. Lesson: preparing each column with scikit-learn

   At level 1, you filled holes with fillna and turned categories into 0/1 columns with get_dummies. On a real project, these moves have a flaw: nothing guarantees that the test set will be processed exactly like…

4. The Complete Pipeline

   Last mission of the week: a client sends raw data. You have to deliver reliable predictions, end to end. The dataset is a mix of everything you have seen: - Missing values - Categorical columns - Different scales -…

5. The Report

   Friday evening. Your boss wants an error report before the weekend. Images are being classified into 4 categories: chat, chien, oiseau, poisson (4 possible classes). You are given the true labels (y_true)…