FWPRec Experiments
All experiments
MIND-small · seed 42 · RTX 4080 SUPER · no training

Where the notepad starts beating attention

To score your next article, an attention model has to hold on to every past click it is still allowed to look at. Keep-the-pad holds one small notepad instead, the same size whether you have clicked ten times or five hundred. That means there is a history length where attention's stored clicks get more expensive than the notepad, and nobody here had measured it. So we did: we stopped the clock on bytes and milliseconds for six models across seven history lengths. SASRec overtakes the dense notepad at 33 clicks; HSTU at 26. Against the smaller rank-16 notepad it is 17 and 13. Nothing was trained — these numbers come from the shape of the models, not from what they learned.

Takeaway

Attention's per-user memory grows by about a kilobyte for every click. The notepad's does not grow at all. They cross between 13 and 33 clicks, which is well inside a normal news-reading history. But a cached GRU keeps only 256 bytes — 129× less than the dense notepad — and it never stops being cheaper. The honest claim is about attention, not about every baseline.

Bytes kept per user, by history length

Per-user state in bytes against user-history length Log-log chart. FWPRec dense (33,024 bytes), FWPRec rank-16 (16,656 bytes) and GRU4Rec (256 bytes) are flat lines: their per-user state does not change with history length. SASRec, SASRecProfile and HSTU rise in a straight line because their key/value cache grows by 1,024 and 1,284 bytes per click. SASRec crosses FWPRec dense at 33 clicks and FWPRec rank-16 at 17 clicks; HSTU crosses them at 26 and 13 clicks. every crossover falls in here · 13–33 clicks 256 B 1 KB 4 KB 16 KB 64 KB 256 KB 1 MB 10 25 50 100 200 350 500 clicks in the user's history bytes kept per user FWPRec dense · 10 clicks · 33,024 bytes FWPRec dense · 25 clicks · 33,024 bytes FWPRec dense · 50 clicks · 33,024 bytes FWPRec dense · 100 clicks · 33,024 bytes FWPRec dense · 200 clicks · 33,024 bytes FWPRec dense · 350 clicks · 33,024 bytes FWPRec dense · 500 clicks · 33,024 bytes FWPRec dense SASRec / SASRecProfile · 10 clicks · 10,240 bytes SASRec / SASRecProfile · 25 clicks · 25,600 bytes SASRec / SASRecProfile · 50 clicks · 51,200 bytes SASRec / SASRecProfile · 100 clicks · 102,400 bytes SASRec / SASRecProfile · 200 clicks · 204,800 bytes SASRec / SASRecProfile · 350 clicks · 358,400 bytes SASRec / SASRecProfile · 500 clicks · 512,000 bytes SASRec / SASRecProfile GRU4Rec · 10 clicks · 256 bytes GRU4Rec · 25 clicks · 256 bytes GRU4Rec · 50 clicks · 256 bytes GRU4Rec · 100 clicks · 256 bytes GRU4Rec · 200 clicks · 256 bytes GRU4Rec · 350 clicks · 256 bytes GRU4Rec · 500 clicks · 256 bytes GRU4Rec HSTU · 10 clicks · 12,840 bytes HSTU · 25 clicks · 32,100 bytes HSTU · 50 clicks · 64,200 bytes HSTU · 100 clicks · 128,400 bytes HSTU · 200 clicks · 256,800 bytes HSTU · 350 clicks · 449,400 bytes HSTU · 500 clicks · 642,000 bytes HSTU FWPRec rank-16 · 10 clicks · 16,656 bytes FWPRec rank-16 · 25 clicks · 16,656 bytes FWPRec rank-16 · 50 clicks · 16,656 bytes FWPRec rank-16 · 100 clicks · 16,656 bytes FWPRec rank-16 · 200 clicks · 16,656 bytes FWPRec rank-16 · 350 clicks · 16,656 bytes FWPRec rank-16 · 500 clicks · 16,656 bytes FWPRec rank-16 33 clicks · SASRec passes FWPRec dense 26 clicks · HSTU passes FWPRec dense 17 clicks · SASRec passes FWPRec rank-16 13 clicks · HSTU passes FWPRec rank-16 FWPRec dense SASRec / SASRecProfile GRU4Rec HSTU FWPRec rank-16

Both axes are logarithmic, so a flat line means "does not grow" and a straight rising line means "grows in step with the history". The three flat lines are the models that summarise your history into a fixed amount of state. The two rising lines are the attention models, which keep the summary of each past click — the key/value cache — so they can attend to it again next time. The open circles are the crossings.

Where each attention model overtakes each fixed-size model

Grows with historyFixed sizeFirst length it costs moreExact crossing
SASRecFWPRec dense3332.25
SASRecFWPRec rank-161716.27
SASRecGRU4Rec10.25
SASRecProfileFWPRec dense3332.25
SASRecProfileFWPRec rank-161716.27
SASRecProfileGRU4Rec10.25
HSTUFWPRec dense2625.72
HSTUFWPRec rank-161312.97
HSTUGRU4Rec10.20

SASRec and SASRecProfile share a backbone — two layers, 64 dimensions, two heads — so their caches come out byte-for-byte identical and they cross at the same place. HSTU crosses earlier because it stores a wider value vector per click plus each click's timestamp, which its relative-time bias needs.

Full memory sweep · bytes per user

ModelPer extra click102550100200350500
FWPRec dense033,02433,02433,02433,02433,02433,02433,024
FWPRec rank-16016,65616,65616,65616,65616,65616,65616,656
GRU4Rec0256256256256256256256
SASRec+1,02410,24025,60051,200102,400204,800358,400512,000
SASRecProfile+1,02410,24025,60051,200102,400204,800358,400512,000
HSTU+1,28412,84032,10064,200128,400256,800449,400642,000

At 500 clicks SASRec is holding 512,000 bytes per user and HSTU 642,000 — 15.5× and 19.4× the dense notepad. You can argue that a real deployment would cut the history off rather than let the cache run away, and that is fair. But the window these baselines were actually trained with is 50 clicks, and even truncated there SASRec keeps 51,200 bytes — 1.55× the dense notepad and 3.07× the rank-16 one, while having thrown away everything older. The notepad's 33,024 bytes still summarise the whole history.

The part that does not favour us: speed

ModelIngest 1 click @10Ingest 1 click @500Score 50 @500Total @500p95 total @500
FWPRec dense0.6460.6490.1820.8330.854
FWPRec rank-160.6870.6910.1940.8870.905
GRU4Rec0.0700.0680.0410.1090.116
SASRec0.2900.2910.0700.3620.371
SASRecProfile0.2960.2980.1050.4050.416
HSTU0.4710.4950.0480.5440.561

Milliseconds, median of 200 timed steps after 25 warm-up steps, one user at a time. Keep-the-pad is the slowest model here at every history length — 2.3× slower per step than a cached SASRec and 7.7× slower than GRU4Rec. Writing to the notepad is a small matrix update, and a matrix update costs more than appending one row to a cache. The memory story and the speed story point in opposite directions, and both are in the table.

One more honest detail: at these sizes the GPU spends most of each step on fixed per-call overhead rather than on arithmetic, so SASRec's step time is flat from 10 clicks to 500 even though it is attending over fifty times more of them. HSTU's creeps up about 5% over the same range. Do not read this table as evidence that attention is free at long histories — read it as evidence that a single user's step is too small to saturate this GPU.

The comparison we are not hiding

Holds

Against attention, keep-the-pad's memory wins past 33 clicks (SASRec) and 26 clicks (HSTU), and the gap widens without limit because one side grows and the other does not.

Does not hold

Against a cached GRU, keep-the-pad loses on memory at every length: 256 bytes versus 33,024, a factor of 129. GRU4Rec's state is constant too, so this never turns around.

Does not hold

Keep-the-pad is also the slowest per step of the six, by roughly 2.3× against SASRec.

Scope

This measures bytes and milliseconds, not ranking quality. It says nothing about whether any of these models recommends well. That argument lives in the other pages.

Which weights we used

ModelWeightsNote
FWPRec densereal MIND checkpointseed 42, keep-the-pad
FWPRec rank-16real MIND checkpointseed 42, rank-16 notepad
GRU4Recreal MIND checkpointseed 42, matched baselines
SASRecProfilereal MIND checkpoint, position table widenedseed 42, matched baselines
SASRecfresh weights, real configno MIND SASRec was ever trained here
HSTUreal MIND checkpoint, position table widenedseed 42, matched baselines

What a model stores per user, and how long one step takes, are set by its shape and its settings — how many layers, how wide, what the memory is made of. They are not set by the values it learned during training. So a missing checkpoint is not a hole in this result. We still used the real ones wherever they exist, because it keeps the provenance clean. Two of them were trained with a 50-click window, so their position tables were copied into wider ones to reach 500; the extra rows are fresh. SASRec has no trained MIND run in this repo at all, so it was built from the repo's SASRec config with the same table sizes as the trained baselines.

How we know the cache is real

The shared SASRec code in this repo has no step-by-step path — it re-reads the whole history every time. The earlier cost audit reported a blank for its per-user memory for exactly that reason. So this lab had to write the missing incremental path: keep each layer's keys and values, add one column per new click, attend against the stored columns instead of recomputing them.

A cache you wrote yourself is easy to fool, so the test suite checks it against the shared code: run the cached path click by click, run the original batch path on the same history, and require the final scores to agree. They do, for SASRec, SASRecProfile and HSTU. The bytes on the chart are the bytes that path actually holds.

The same test suite pins this package against the earlier cost audit: it rebuilds the three fixed-size states here and asserts they still come out at 33,024, 16,640 and 256 bytes — the numbers that audit published — so the copied measurement code cannot quietly drift away from it. (The dense notepad's live streaming state is 33,024 bytes; the rank-16 one is 16,656, which is the published 16,640 plus two 8-byte slot counters the earlier audit's snapshot did not carry.)

Isolated suite experiments/isolated/mind_kv_crossover_lab on damlr-w03 (RTX 4080 SUPER, driver 580.126.20, torch 2.13.0+cu130). Inference only — nothing was trained, and no ranking metric was computed. One user at a time, 50 candidates, seed 42; median and 95th percentile over 200 timed steps after 25 warm-up steps. History lengths 10 to 500 are synthetic click streams, because MIND-small's own histories are capped at 50; the quantities being measured depend on model shape, not on which items appear. Bytes count tensor storage only, not Python object overhead or allocator slack. One machine, one seed, one batch size. Do not read the latency column as a throughput claim — a real server batches users, and this did not.