Supervised models 1

The crêpe seller, then admissions: linear regression, log/exp, logistic classification

In this chapter, you will learn to

• Make a machine predict a number (linear regression)
• Make it predict a category, yes or no (logistic regression)
• Read what the model learned and notice when it goes wrong

Tools: NumPy, scikit-learn

16 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   Forecasting is the first thing we ask a machine to do: how much, and yes or no. - Electricity. In France, winter electricity demand climbs when it gets cold, because of electric heating. RTE, the grid operator…

2. Crêpe sales

   A crêpe vendor has recorded his sales: | Temperature (°C) | Crêpes sold | |---|---| | 5 | 120 | | 10 | 155 | | 14 | 190 | | 18 | 210 | | 22 | 260 | | 27 | 300 | | 30 | 325 | These readings are already loaded into temp…

3. Who has the best line?

   Three students each propose values of $a$ and $b$ for the crêpe line: - Alice: $a = 8, b = 80$ - Bob: $a = 10, b = 50$ - Charlie: $a = 7.5, b = 90$ Invent a single number that measures "how bad a line is". It will…

4. Line-o-matic

   Comparing three lines is nice. But we don't want to test lines one at a time by hand. Build a machine that finds, all by itself, the $a$ and $b$ that make the error as small as possible. The error is the one from…

5. Points that lie

   Recorded data sometimes contains aberrant points: a data-entry mistake, an exceptional event... We want the line that sticks to the majority of the points, without being dragged off course by those few wild…

6. Lesson: scikit-learn, your first models

   Good news: the machine that searches for the best line already exists, ready-made (you will build it yourself at level 3). The free library scikit-learn contains…

7. Predicting house prices

   You have built everything by hand. Now let us tackle a real dataset. An example real-estate dataset is loaded for you: 13 variables (crime, lot area, number of rooms, age of the building, distance to the city…

8. The runaway model

   Your agency is expanding into a housing development where the properties are all very much alike. For each house, some sixty attributes are recorded, but many measure almost the same thing (living area, floor area, area…

9. Too many attributes

   You move on to another, more recent area. This time the agency cast a wide net: 80 attributes per property (orientation, distance to each shop, neighborhood indices, assorted amenity scores), for only 50 sales…

10. The mold

    A microbiology lab is monitoring a mold growing on culture dishes. At regular time intervals, a sensor measures the area covered (in mm²). Here are the readings from one dish, from oldest to most recent…

11. The tipping point

    The same mold, this time observed continuously (it grows at every instant, not in steps). You know: - start: the area at time 0, - facteur: the coefficient by which the area is multiplied at each unit of time. The…

12. The best bettor

    Every day, several bettors announce, for each match on the schedule, the probability that the home team wins. You test their reliability with a game: at each match, your fortune is multiplied by the probability…

13. Admitted or rejected

    You know how to fit a line to data. Let's try it on a new problem. A high school has the following applications: | Application grade | Admitted? | |---|---| | 4 | 0 (no) | | 6 | 0 | | 7 | 0 | | 8 | 0 | | 9 | 1 (yes) | |…

14. The mystery function

    The line will not do: it leaves $[0, 1]$. We need to find a function $f$ that "squashes" any real number into the interval $[0, 1]$. Your function f(x) must satisfy all of these properties: 1. When $x$ is very…

15. When does it get it wrong?

    You have a function that turns a line's output into a probability. But how do you measure whether those probabilities are any good? A model makes these predictions: | Student | Predicted P(admitted) | Actually admitted?…

16. Lesson: reading a trained logistic regression

    A logistic regression does more than predict: it is a readable model. It computes $p = \sigma(w \cdot x + b)$, the estimated probability of class 1. Inverting the sigmoid gives the log-odds…