Text & embeddings
Spam, TF-IDF, review classification, then meaning spaces (analogies, embeddings)
In this chapter, you will learn to
- Turn text into numbers (bag of words, TF-IDF)
- Classify texts
- Represent the meaning of words as vectors (embeddings)
Tools: scikit-learn, Hugging Face Transformers
10 exercises
Each exercise is graded automatically, in your browser. The full statements are available once you have an account.
What this is for in real life
Much of what we write online is read by a machine before any human sees it. - Spam filters. Gmail, Outlook and the rest check every email before deciding whether it deserves a place in your inbox. -…
The spam detector
First day on the data team at Cinéphage, the film review site. You are handed a problem everyone dreads: the moderation queue is overflowing, and the moderators spend their days deleting messages that have nothing to do…
The automatic critic
Cinéphage now displays, next to each film, an audience mood gauge: how many positive opinions, how many negative ones. Until now, a moderator patiently classified each review by hand as "positive" or "negative".…
The first filter
You have been hired as a moderator on an online forum. 500 comments arrive every hour. Reading everything manually is impossible. You are given a set of labeled comments (
comments: a list of texts,labels: 0 =…The words that count
Your moderation filter works, but it makes mistakes. Analysing the hard cases, you spot a problem: in the sentence "le produit est vraiment horrible", the word "le" carries as much weight as "horrible" in the average of…
The zero-shot translator
Plot twist: the forum goes bilingual (French and English). Your boss asks you to moderate the English comments too, but you have no labeled English comments at all. Fortunately, you have multilingual word…
Waking a giant
You have classified text with your own features (the words that matter). Here comes the heavy artillery: BERT, a language model pre-trained by Google on billions of words. You have neither the tens of thousands of…
The magic dictionary
You are a translator and you discover an old dictionary in which each word is represented by a list of numbers (a "vector"). The word vectors are in the dictionary
embeddings(key = word, value = numpy array).…The revealing context
You have no ready-made dictionary of vectors. But you do have a corpus of sentences. Your intuition: two words that often appear in the same contexts must have similar meanings ("chat" and "chien" appear in similar…
The magic compression
The co-occurrence vectors you built are enormous: each word is represented by a vector of size |vocabulary|. For a real corpus, that can be 50 000 dimensions! Storing and comparing vectors that large is impractical.…