Here, storm means Ororo, not the weather

A word's TFTerm frequency: how often a word appears in a document, divided by the number of words in that document. on a page is its count divided by the page's length. Its DFDocument frequency: the number of documents a word appears in at least once. is how many of the 303 pages use it, and its IDFInverse document frequency: a weight that is high for a word few documents use and 0 for a word every document uses. is \ln \tfrac{303}{\text{DF}}. TF-IDF multiplies the two. He is on 291 pages, so its IDF is 0.04, while she is on 191, so \htmlClass{x-ln}{\ln} \tfrac{\htmlClass{x-n-docs}{303}}{\htmlClass{x-df}{191}} = \htmlClass{x-idf-value}{0.46}. Put together, they make a document-term matrixA table with one row per document and one column per word. Each cell says how much that word counts in that document. with one row per page and one column per word, where each cell holds a word's TF-IDF on that page.

CosineCosine similarity: how alike two lists of numbers are by their direction alone, ignoring their size. It's 1 when they point the same way and 0 when they share nothing. divides the dot product of two vectors by the product of their lengths. That gives the cosine of the angle between them, which is 1 when they point the same way and 0 when they share no word. Written out word by word, it's a sum with one term per word, so we can show which words carry it. Storm and the Human Torch have a raw-count cosine of 0.845, and the gives 0.396 of it. Under TF-IDF it's 0.297, and storm, Johnny Storm's surname, gives 0.234.

Wikidata labels 52 pages as women and 145 as men. The women's pages hold 197,637 words and the men's 397,709. Scattertext places each of the 3,188 words seen 20 times or more by its frequency rank in each group, and gives it a z-scoreHow far a value lies from what chance would give, measured in standard deviations, the typical size of chance's swings. The farther from 0, the less likely chance alone produced it. for how far it leans, in standard deviations. Anything beyond ±1.96 counts as a lean.

A topic modelA method that finds groups of words that tend to appear together across many documents and treats each group as a topic. looks for groups of words that recur together across documents, and LDA is one of them. It treats each topic as a distribution over words and each page as a mixture of topics. We fitted it with 5 to 12 topics, twice each, with seeds 0 and 1, on the 5,824 words left after removing names, stopwords, words under three letters, and words on fewer than 5 or more than 151 pages.

A word-context matrixA table with one row per word and one column per neighbouring word. Each cell counts how often the two appear close together. has a row for each target word and a column for each context word. A cell counts how often the context word sits within the target's windowThe context window: the words counted as a word's neighbours, a fixed number of words on either side of it. on one page. We count windows of 2, 5 and 10 words on either side.

PMI is the natural log of how often a word and a context word meet, divided by how often they would meet if they were independent. A score above 0 means the pair meets more often than independence predicts. Below 0, it meets less often. PPMI sets the negatives to 0.

Skip-gram predicts the context from the word, and CBOWContinuous bag of words: a Word2Vec model that learns each word's numbers by predicting the word from the words around it. predicts the word from its context. From Storm's opening words, skip-gram makes pairs such as (storm, is) and (storm, a), while CBOW makes one example, (is, a, superhero, appearing, in) → storm. Both train with negative samplingA training shortcut. Instead of scoring every word in the vocabulary, the model learns to tell a real neighbour from a few randomly drawn words.. Each real pair is set against 5 random words, drawn more often the more common they are. Both are also staticA static word embedding gives each word one fixed list of numbers, learned from a corpus, whatever sentence the word appears in., with one vector per word whatever the sentence, so a word with two senses gets one vector for both. On these pages storm is the character, not the weather, and its nearest words are cyclops, panther, pryde and ororo. PPMI rows are a sparse static embedding of the same kind.

GloVe fits vectors to a whole corpus's word co-occurrence counts at once instead of predicting window by window. We took its 50-dimensional vectors, trained on Wikipedia 2014 and Gigaword 5, and compared each word's ten nearest with skip-gram's on the 303 pages, both ranked over the 3,645 words the two models know.