NumPy & vectors

The weather analyst, then the recommendation engine: arrays, masks, dot product, cosine

In this chapter, you will learn to

• Compute on whole arrays without writing loops
• Select data with masks
• Measure how similar two items are (dot product, cosine), the basis of recommendations

Tools: NumPy

15 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   Real-world data are arrays of numbers, and they are huge. - Weather. A forecast model cuts the atmosphere into millions of cells and updates each one every few minutes. Nobody writes a loop over millions of…

2. Lesson: first steps with NumPy

   In this chapter, you will discover NumPy, Python's numerical computing library. It is used to manipulate arrays of numbers, in one or more dimensions (a series of temperatures, an image in pixels, a grid of…

3. Lesson: NumPy in two dimensions

   Grades (one row per student, one column per subject), an image (rows and columns of pixels), a dataset (one row per example, one column per measurement): almost everything you will handle is a two-dimensional array.…

4. Station too slow

   You work for a network of 2000 weather stations. Each one has recorded a temperature per day for 1000 days: two million values, stored in a Python list data[station][jour]. You are asked for the average…

5. Heatwave alerts

   Your boss bursts into the office in a panic: "The press wants a heatwave report! I need these numbers in 10 minutes!" You have the array temperatures of shape (100 cities, 365 days) already loaded in numpy. Find and…

6. Ghost sensors

   Bad news: some of the station's sensors are going haywire. When a sensor breaks down, it records -999.0 instead of the true temperature. Your colleague explains: "We can't throw those days away. The protocol says…

7. Overall ranking

   The school principal gives you the grades of 200 students in 8 subjects (array grades, shape 200×8), the coefficients of each subject (array coeffs, shape 8), and warns you: some grades are missing…

8. Booby-trapped NumPy

   A junior data scientist wrote a NumPy program to analyze sales data. The program has 4 bugs tied to classic NumPy pitfalls. The program is supposed to: 1. Load the monthly sales of 5 stores (5×12 matrix) 2. Save a…

9. Shared genres

   You work for a streaming service. Each movie is described by the genres it belongs to, encoded as 0/1: [action, romance, comédie, horreur, sci-fi, thriller, drame, animation] For example, an action/sci-fi/thriller…

10. Blockbuster trap

    The streaming service now describes each movie with a rating per genre, from 0 to 10, instead of 0s and 1s. You keep the dot product from the previous exercise as your resemblance measure, but a colleague notices a…

11. Recovering the recipe

    A perfumer creates fragrances by blending base ingredients. Each ingredient has a scent profile (a vector of features). You have: - ingredients: a matrix (N ingredients × D features) giving each ingredient's…

12. Recommendation engine

    Your streaming service has grown: 10,000 movies, each described by 50 features. You are given two variables: - catalogue: a NumPy array (10000, 50), one movie per row; - queries: a list of 5 target…

13. Lesson: the vector and the dot product

    A vector is an ordered list of numbers, written $\vec{u} = (u_1, \dots, u_n)$; each number is a component. A movie described by its genres is a vector. The basic operations: - Addition (component by…

14. Lesson: norm and cosine similarity

    The norm (the "length") of a vector: $\|\vec{v}\| = \sqrt{\sum_i v_i^2}$. For example $\|(3, 4)\| = \sqrt{9 + 16} = 5$. In NumPy: np.linalg.norm(v). The raw dot product grows with the size of the vectors (the…

15. Lesson: linear combinations and systems

    Writing a vector as $p_1 \vec{u_1} + \dots + p_N \vec{u_N}$ is a linear combination of the $\vec{u_i}$. Recovering the coefficients $p_i$ amounts to solving a system of linear equations $A\,p = b$. In NumPy…