PaperLab case study | Garfield Lim

Problem

Attention Is All You Need is the paper most AI work now rests on. It explains attention in a few lines of matrix notation. Most people who read it cannot picture what those lines do to a sentence.

The usual fixes are static diagrams or videos. They show one example someone else picked. You cannot type your own sentence and watch the numbers move.

I wanted a page where a student, or an interviewer, could do exactly that in a browser tab with nothing to install.

Approach

PaperLab runs one real Transformer layer on small seeded weights. It is not a trained model, and the page says so on screen. The math is the paper's math. The weights are toys.

You type a sentence of up to 16 tokens. Then you step through four stages: tokens, embeddings with positional encoding, attention, and the feed-forward block. Each stage shows the formula, a short plain explanation, a link to the paper section, and the actual matrices.

The model is sized to fit on screen: d_model = 16, 4 heads with d_k = 4, and d_ff = 32. Attention gets a heatmap per head, head toggles, a causal mask switch, and click-a-token to see what it attends to.

Architecture

All math lives in one module with no DOM or React dependency, so the tests cover it directly. Redrawn from the diagram in the project README.

Decisions

Chose a pure math module over math inside componentsbecause a function with no DOM can be tested against the paper's formulas, and the UI cannot drift from the numbers it shows.
Chose seeded toy weights over a trained modelbecause the page needs no download, no backend and no API key. The same sentence always gives the same output, so a test can pin it.
Chose HTML and Unicode formulas over KaTeXbecause a maths renderer would have been the largest thing in the bundle, for a handful of short formulas.
Chose a word-level tokenizer over subword BPEbecause whole words read clearly in a heatmap row. The page states the simplification next to the tokens.
Chose computed text colour over a fixed palette for heatmap cellsbecause cell values sit on every shade of the ramp, and a fixed colour fails contrast somewhere. ramp.ts picks it per cell for both themes.

Results

19 testsnpm test runs 19 Vitest tests on the math: softmax rows sum to 1 within 1e-6, attention shapes, causal masking, positional encoding against the paper formula, layer norm, tokenizer edges and determinism.
58 KBInitial JS and CSS, gzipped: 53.5 KB JS plus 4.4 KB CSS, from the production build. Self-hosted fonts add about 159 KB.
96 / 100 / 100 / 100Lighthouse mobile preset on the build served by vite preview: performance, accessibility, best practices, SEO.
0 violationsaxe-core in light and dark, on all four steps, at 360 and 1280 px, colour contrast included. No horizontal scroll at 360 px.

Measured on one machine against a local build, not the live site.

Screens

PaperLab input panel with a typed sentence, preset sentences, the token counter and the four-step stepper. — The input. A sentence goes in, the token counter shows the 16-token cap, and the stepper walks the four stages.

Two heatmaps: token embeddings and the sinusoidal positional encoding, one row per token and 16 columns. — Step 2. Embeddings plus the sinusoidal position wave from section 3.5 of the paper.

The feed-forward step with its formulas and the first matrix after attention and layer norm. — Step 4. The feed-forward block, the residual and layer norm, with the formula above the numbers.

What I'd do next

Re-run Lighthouse and axe against the live Pages URL, not only vite preview.
Add a second layer, so the residual stream can be followed from one block into the next.
Offer a small subword vocabulary beside the word tokenizer, to show why real models split words.