Supervised models 2

Medical diagnosis, the bouncer, the mail sorter: KNN, margins, trees, forests

In this chapter, you will learn to

• Classify by similarity to neighbours (k-nearest neighbours)
• Separate groups with a boundary, then with a series of questions (trees, forests)
• Choose the right model for the shape of the data

Tools: NumPy, scikit-learn

13 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   There is no single best model: each family draws its boundaries in its own way, and the right choice depends on the data. - Search by similarity. When Pinterest or Google Lens show you "similar" images, they…

2. The new patient

   You work in a hospital. Each patient is described by 2 biological measurements (symptom A and symptom B). Some are sick (1), others are healthy (0). The known data is in X_train (100×2 matrix) and labels (vector…

3. How many neighbors?

   Your strategy from the previous exercise works, but a question arises: how many similar patients should you consult? You now have X_train, y_train (training data; beware: as in a real medical record, some of the…

4. Lesson: the curse of dimensionality

   k-nearest neighbors: the price of "compare with everything" The method you are using has a name: k-nearest neighbors (k-NN). There is no training phase: all the information stays in the stored examples, and…

5. The deceptive scale

   The medical department adds a new measurement: the patient's annual income (in euros). The data now have 2 columns: age (0-100) and income (0-100,000). You have X_train (300, 2) and y_train (300,) to learn from…

6. Benign or malignant tumor

   Same hospital, another department: oncology hands you a real dataset. The Breast Cancer Wisconsin Dataset is loaded for you: 569 tumors described by 30 measurements (radius, texture, perimeter, area...). The dataset…

7. The best barrier

   You are the bouncer at a festival. Two groups, the VIPs and the regulars, are clearly separated on the site map. Three barriers (three straight lines) separate the known festival-goers without a single…

8. The bouncer and the rings

   New festival, new layout. The VIPs (class 0) stand at the center, the regulars (class 1) form a ring all around them. Seen from above, the two groups draw two concentric rings. Each person is at…

9. The overzealous bouncer

   The guard now knows how to bend the boundary to separate the groups even where a straight line fails. One tricky setting remains: how far to bend it? Tonight, the guard knows the status of every person present: the…

10. The mail sorter

    First day at the Rivebourg postal sorting center. The depot manager, Madame Lenoir, puts you in charge of the arrivals dock: every piece of mail that comes through must end up in one of three bins, "lettre"…

11. The most useful question

    Your question system is up and running, and a colleague who started this morning, Tarek, watches you at the dock. He wants to get going, but he stumbles on the very first decision: "What do I start with? The weight? The…

12. Lesson: reading a trained decision tree

    A cascade of nested questions has a name: a decision tree. The vocabulary: - an internal node asks a binary question about ONE feature: $x_j \leq s$? (feature $j$, threshold $s$); - depending on the answer…

13. The sorting committee

    A few weeks later, question-based sorting is in service, but on a new stream it disappoints. Each piece of mail here carries 2 measurements (matrix X_train) and a category, 0 or 1 (y_train), and two troubles pile…