Experiments
What we ran, in plain words, with the numbers behind each call.
The idea
Not a run. Anchor, sticky-note memory, what we test, and what we do not claim.
Read the ideaHow the model works
Not a run. Snapshot, notepad, and the write network Adam actually trains.
Read the methodImportant things to note
Not a run. Mixed results, the next steps in order, the baselines, and which datasets can support a claim.
Read the notesRuns
How does the notepad compare to the field?
We trained nineteen models on three datasets, twice each, under one identical training budget. The notepad lands second on hit rate on MovieLens-1M, mid-pack on KuaiRand, and eleventh on Amazon. The bounded rank-16 version matches the full one everywhere, so the smaller memory costs nothing.
Where the notepad starts beating attention
An attention model has to keep every past click in memory; keep-the-pad keeps one fixed-size notepad. We measured both on MIND-small. SASRec's stored clicks cost more than the dense notepad from 33 clicks on, HSTU from 26. But GRU4Rec uses 129 times less than either, and always will.
OpenDoes our SASRec actually work?
Before spending real GPU time training baselines, we checked whether our own from-scratch SASRec actually works. We rebuilt the original paper's evaluation setup on MovieLens and tested it three times. It matched the published numbers and a well-known independent copy of the same model. The implementation is sound.
OpenCredit vs actually deleting the click
We deleted each past click one at a time and re-scored, 22 million click-candidate pairs in total. The notepad's claimed credit correlates 0.48 with the real effect and finds the influential clicks 12 times better than chance. Magnitudes are not counterfactual. One seed.
OpenBlamed clicks vs random clicks
The notepad blames past clicks for each ranking score. We deleted the top-3 and top-5 most-blamed clicks on 15,657 MIND-small validation impressions, then deleted the same number at random. Blamed clicks drop the score about 12 times more. One seed. Validation only.
OpenAnchor size sensitivity
Does it matter how many early clicks build the anchor? We tried caps of 10, 25, and 50 on MIND-small. Val NDCG@10 rises from 0.390 to 0.395 — a spread of 0.005. Cap-50 matches uncapped keep-the-pad. Ten early clicks already build a useful anchor.
OpenKeep the notepad vs sequence models
On the same MIND-small validation split as keep-the-pad, we trained SASRec, GRU4Rec, and HSTU at three seeds. Keep-the-pad was not retrained. It beat every model on every seed. HSTU bounced the most. Validation only. Do not mix with published test scores.
OpenKeep the notepad, 3 seeds
We re-ran keep-the-pad versus wipe-and-rebuild at two new training seeds on the same MIND-small split. All three seeds agree: keeping the notepad beats wiping it. For ~90% of users, the top recommendation was the same whether you kept or wiped the pad — the two strategies are nearly equivalent in what they recommend, but keep-the-pad scores slightly higher overall.
OpenRank-16 notepad on MIND
Factoring the notepad to rank 16 cuts per-user memory from 33 KB to 16.7 KB at a cost of 0.001 NDCG@10. Keep still beats wipe. Agreement with a full rebuild drops to 85.5%. Speed is unchanged because the rank-16 path is not yet optimised.
OpenMIND keep the notepad, confirmation
We reran the keep-the-pad test on a university RTX 4080 SUPER. Same MIND-small setup, one seed, validation only. Keeping the notepad again beat wiping it. This time the run finished: a checkpoint was written, and the keep-versus-wipe score check completed. Test split stayed sealed.
OpenMIND keep the notepad
On MIND-small we trained the snapshot-plus-notepad model two ways: wipe the notepad and rebuild it from the last twenty clicks, or keep it across visits like a live app. Keeping the pad ranked better on validation. One seed. Training stopped early after the score peaked.
OpenMIND snapshot and notepad
On MIND-small news ranking we freeze each user's first training clicks as a stable snapshot, then let a small notepad of fast weights adapt on later clicks. Across three model seeds, snapshot plus notepad beat a moving history mean. Validation only. The held-out test split stayed sealed.
OpenAmazon snapshot and notepad
On Amazon reviews we used the same four-row design with frozen product embeddings. The notepad still added a large ranking gain. A moving profile beat the earliest-product snapshot when memory was on. One seed only. A stale shopping snapshot may be the wrong slow state for this task.
Open