Week 5: Assembling the CLI, Tuning Temperature, and Feeling the Speed Wall

⏱️ This is the week the whole "pure Python, no numpy" constraint stops being an inconvenience and starts being the point.

🎯 Learning objectives

By the end of this week you can:

• Assemble every piece from Weeks 1–4 into one end-to-end command-line text generator.
• Add a temperature parameter that controls how random vs. predictable the generated text is.
• Time your pure-Python pipeline on a larger corpus and explain, from firsthand measurement, why this motivates Section 2.
• Reason informally about how your pipeline's running time scales with input size.

Lesson

Assembling the pipeline

Every function from Weeks 1–4 composes into one pipeline — load, tokenize, count, convert to probabilities, generate:

import csv
import random

def load_corpus(path):
    with open(path, newline="") as f:
        return [row["sentence"] for row in csv.DictReader(f)]

def tokenize(sentence):
    return sentence.lower().split()

def bigrams(tokens):
    return [(tokens[i], tokens[i + 1]) for i in range(len(tokens) - 1)]

def build_bigram_probabilities(sentences):
    counts_table = {}
    for sentence in sentences:
        for first, second in bigrams(tokenize(sentence)):
            counts_table.setdefault(first, {})
            counts_table[first][second] = counts_table[first].get(second, 0) + 1
    probs_table = {}
    for word, next_counts in counts_table.items():
        total = sum(next_counts.values())
        probs_table[word] = {w: c / total for w, c in next_counts.items()}
    return probs_table

def generate_text(probs_table, start_word, max_words=10):
    words, current = [start_word], start_word
    for _ in range(max_words - 1):
        if current not in probs_table:
            break
        next_words = list(probs_table[current].keys())
        weights = list(probs_table[current].values())
        current = random.choices(next_words, weights=weights, k=1)[0]
        words.append(current)
    return " ".join(words)

if __name__ == "__main__":
    corpus = load_corpus("slm-corpus.csv")
    probs_table = build_bigram_probabilities(corpus)
    print(generate_text(probs_table, "the", max_words=8))

dict.setdefault(key, {}) in build_bigram_probabilities is a small shortcut for the if key not in table: table[key] = {} pattern from Week 3 — it does the same thing in one line. if __name__ == "__main__": is the standard way to mark "run this when the file is executed directly" — you'll see this convention constantly once you install Python locally in the Capstone Bonus.

Temperature: tuning randomness

Temperature is a single knob that stretches or flattens a probability distribution before sampling from it, without changing which outcomes are possible — only how close to uniform (flat) or how close to greedy (peaked) the sampling behaves:

$$P_T(w) = \frac{P(w)^{1/T}}{\sum_{w'} P(w')^{1/T}}$$

• $T = 1$: unchanged — normal sampling from the original distribution.
• $T < 1$ (e.g. $0.5$): sharpens the distribution — high-probability words become even more likely, pushing generation toward the greedy behavior from last week.
• $T > 1$ (e.g. $2.0$): flattens the distribution — low-probability words get relatively more likely, producing more surprising, more chaotic text.

Why exponentiation, rather than just scaling the probabilities directly? Because raising each $P(w)$ to a power $\frac{1}{T}$ affects large and small probabilities unevenly in exactly the direction you want: for $T < 1$ (so $\frac{1}{T} > 1$), a probability like $0.5$ raised to a power greater than 1 shrinks, but a smaller probability like $0.05$ shrinks by a much larger relative amount — widening the gap between likely and unlikely words. The renormalization step (dividing by the new sum) is what turns this reshaped set of numbers back into a valid probability distribution that sums to 1, the same "sums to 1" property from Week 2.

def apply_temperature(probs, temperature):
    adjusted = {w: p ** (1 / temperature) for w, p in probs.items()}
    total = sum(adjusted.values())
    return {w: v / total for w, v in adjusted.items()}

def sample_next(word, probs_table, temperature=1.0):
    if word not in probs_table:
        return None
    probs = probs_table[word]
    if temperature != 1.0:
        probs = apply_temperature(probs, temperature)
    return random.choices(list(probs.keys()), weights=list(probs.values()), k=1)[0]

This is the exact same "temperature" concept you'll see referenced in real large language model APIs — you're implementing a simplified version of a real, widely-used control knob.

Timing it: feeling the speed wall

Here's the exercise this whole track has been building toward. Time your pipeline on a larger corpus — say, a version of the corpus repeated 200 times to simulate more data (corpus * 200, using list multiplication) — using Python's built-in time module:

import time

big_corpus = corpus * 200   # simulate a much larger dataset
start = time.time()
big_probs_table = build_bigram_probabilities(big_corpus)
elapsed = time.time() - start
print(f"Built bigram table for {len(big_corpus)} sentences in {elapsed:.3f} seconds")

Run this in the playground and note your actual number. It's probably still fast at this size — but watch what happens as you multiply the corpus size again (* 1000, * 5000): the nested-loop, dict-of-dicts approach from Week 3 does real work for every single word occurrence, with no shortcuts. There's no way to "vectorize" a plain Python for loop the way specialized numeric libraries can.

What the timing actually shows

Every extra sentence in the corpus adds a roughly constant amount of work: tokenize it, form its bigrams, update a handful of dict entries. Doubling the number of sentences roughly doubles the number of word occurrences to process, which roughly doubles the running time — a linear relationship between input size and time, informally $O(n)$ where $n$ is the total word count. That's actually a good scaling behavior, not a bad one — the real issue isn't that this algorithm scales poorly in theory, it's that each individual step (a dict lookup, a function call, a loop iteration) costs more in plain, interpreted Python than the equivalent step would cost in the compiled, vectorized code underneath numpy/pandas. Illustrative numbers (yours will differ by machine, but the shape should look similar):

Corpus repeats	Approx. sentences	Approx. time
1×	20	a few milliseconds
200×	4,000	still well under a second
5,000×	100,000	now clearly noticeable

That gap — between what plain Python can do comfortably and what real-sized text/data problems need — is exactly what Section 2 exists to close. Pandas and numpy don't just make code shorter; they make operations like this run orders of magnitude faster, by pushing the actual looping down into optimized, compiled code instead of Python's own interpreter loop.

⚠️ Common pitfalls

• Timing something that includes one-time setup costs. If your timing code accidentally includes load_corpus's file-reading time alongside build_bigram_probabilities's actual computation, you're measuring two different things at once — time only the specific step you care about.
• Comparing timings across wildly different machine loads. Running other heavy programs at the same time can skew a timing measurement; if a result looks surprising, re-run it a couple of times.
• Assuming linear scaling means "no problem at any size." $O(n)$ is good, but a large enough constant factor per operation still adds up — this week's whole point is that the constant factor per operation is what plain Python loses on compared to vectorized code, even with the same big-picture scaling.

🧩 Challenges

🧩

Extend the if __name__ == "__main__": block so it asks the user (via input()) for a start word and a max word count, instead of hardcoding them.

🧩

Generate text at temperature=0.5 and temperature=2.0 with the same start word, several times each. Describe the qualitative difference you observe.

🧩

Time build_bigram_probabilities at increasing corpus sizes (corpus * 1, * 50, * 200, * 1000) and print a small table of size vs. elapsed time. Does the growth look linear, or worse than linear?

🧩

Why does sample_next special-case temperature != 1.0 instead of always calling apply_temperature? What does apply_temperature actually compute when temperature == 1.0?

🧩

Using your Challenge 3 timings, compute the ratio of time at 1000x to time at 50x. Compare that ratio to the ratio of corpus sizes (1000/50 = 20). Does the time ratio roughly match, confirming linear scaling?

🧩

Write a function generate_batch(probs_table, start_word, temperatures, max_words=10) that generates one sentence per temperature value in a given list (e.g. [0.5, 1.0, 1.5, 2.0]) and returns all of them together, so you can compare the effect of temperature side by side in one call.

🤔 Socratic Questions

• You just personally measured plain-Python's performance on a text-processing task. Do you think the slowdown you saw is mainly about Python's for loop, the dict operations, or something else? What would you need to test to isolate the cause?
• Temperature reshapes a probability distribution but never changes which outcomes are possible (a word with probability 0 stays at probability 0 no matter the temperature). Why is that an important property for a "randomness knob" to have?
• Section 2 begins with a demo showing the same kind of word-counting task running dramatically faster with numpy/pandas than the pure-Python version you just built. Based on what you now know about how your build_bigram_probabilities actually works (nested loops, dict lookups), what do you predict a vectorized version would need to do differently to be faster?
• This week's scaling was linear ($O(n)$) in total word count — not the worst possible scaling behavior. Can you think of a step in this whole 5-week pipeline that could have been much worse (e.g. quadratic) if it had been written a different, more naive way?

✅ Weekly quiz

✅ Weekly quiz

1. A temperature below 1.0 (e.g. 0.5) makes generated text:

More random/chaoticMore predictable/repetitive, closer to greedyExactly identical to temperature 1.0Impossible to generate (raises an error)

2. What does dict.setdefault(key, {}) do if key is already present?

Overwrites the existing value with {}Raises a KeyErrorLeaves the existing value unchangedDeletes the key

3. This week's timing exercise on a larger corpus is meant to demonstrate:

That Python is always too slow to be usefulA concrete, felt motivation for why vectorized tools like numpy/pandas existThat bigram models are faster than unigram modelsHow to fix slow code using multithreading

4. if __name__ == "__main__": is used to:

Declare the main class of the programMark code that should run only when the file is executed directlyImport the standard librarySpeed up the script

5. Roughly doubling the corpus size and seeing the build time roughly double is an example of:

Constant time scalingLinear time scalingQuadratic time scalingNo relationship at all

🎁 Bonus: packaging the model as a class

🔒 Available once you pass this week's quiz.

Mark this week complete

---

🎉 You've finished Python 101. You built a working (if simple) language model using nothing but the Python standard library — and you personally measured where it starts to strain. That's exactly the door Section 2, Pandas & Data Analysis, opens next.