Linear algebra for ML

The social network and the seismologist: products, inverses, eigenvectors, PCA

In this chapter, you will learn to

• See a matrix as a transformation and as a network of relations
• Understand eigenvectors and eigenvalues, and what they are for
• Reduce the dimension of data while keeping what matters (PCA)

Tools: NumPy

11 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   A matrix is both a table of relationships and a machine that reshapes space, and both faces show up everywhere. - Google's ranking. In its early days, Google ranked web pages with PageRank (Brin and Page, 1998)…

2. Who's popular?

   You are analyzing a small social network of 5 people: Alice (0), Bob (1), Clara (2), David (3) and Eva (4). The data lives in a 5x5 array called network. The cell network[i][j] is 1 if person i follows person j…

3. Reciprocal network

   Your social network again. A friendship is a reciprocal link: person i follows j AND j follows i. How many friendships (pairs of people who follow each other) does the network contain? Be careful, the…

4. A friend of a friend

   Still in your social network, a new question: if Alice follows Bob, and Bob follows Clara, then Alice can indirectly reach Clara in 2 steps. Write a function indirect_connections(network) that, from the follow…

5. The bag of chips

   A casino uses three types of chips, whose values (in grams) are unknown. You are handed three bags. For each bag, you know the number of chips of each type it contains, and its total weight. Recover the…

6. The secret message

   The social network's administrator encrypted an alert message using a transformation. He took the message as a vector of numbers and transformed it by combining it with a secret key (a 3x3 array). Here is what you know…

7. The oscillating sensor

   You are a seismologist. A building is fitted with sensors measuring vibrations in 2 directions (x, y). The measurements are available in data, a matrix of shape (200, 2). If you plotted the points, you would see an…

8. Ten sensors, two secrets

   The building now has 10 sensors. The data is in data, a (500, 10) matrix. Your manager suspects that despite the 10 sensors, the building really only vibrates in a few directions. They want to know: which…

9. Fifty columns too many

   You are a data scientist at a bank. You are given a client dataset with 50 variables (matrix data, 300×50). Your manager is grumbling: "50 columns is unmanageable. A lot of them must be redundant: height and weight…

10. The 2D report

    Your manager wants a 2D chart of the clients, but the data has 50 dimensions. Project the data from 50D down to 2D while keeping the maximum amount of information (of variability). The 2 projection directions…

11. The right tradeoff

    Your 2D projection looks nice, but your manager is worried: "We lost information going from 50D to 2D. Exactly how much? And what if we kept 5D? 10D?" Write explained_variance(data, dims) which, for each number of…