FWPRec Method
Experiments
Not a run

How the model works

Adam does not train the news vectors, and it does not train the snapshot. It only trains a small write network that decides how each click is written into a per-user notepad. The notepad itself is runtime state, not a weight.

34,908Trainable parameters
10.2MTotal parameters, mostly frozen articles
64×64Each notepad matrix

Three pieces

Frozen

Snapshot

Mean of the user’s first training clicks, in frozen 64-d Qwen article space. Looked up once and held fixed for the whole run.

State

Notepad

Two dense 64×64 matrices (like / dislike). Starts empty. Written during the forward pass. Not stored as model weights.

Trained

Write network

Shared trunk plus key, value, and two gates. Also action embeddings and two memory-gain scalars. About 35k numbers.

One visit

Score is a dot product. The snapshot is the slow taste. The notepad is the short-term correction. Writes are keyed off the frozen snapshot, not off the current corrected query.

Snapshot frozen 64-d mean Notepad M+ and M−, 64×64 Read q = snapshot + pad Score q · article loss Write network — this is what Adam sees click + article + time → key, value, gates writes into pad

Write network

Shared trunk, four heads. Input is 140 numbers: frozen snapshot (64), frozen article (64), trained action embedding (8), click sign (1), and three time features from the gap since the last event. Hidden size 128. Residual keys and values, then a delta-rule write.

input · 140-d concat Snapshot 64 · frozen Article 64 · frozen Qwen Action embed 8 · like / skip / dislike Feedback 1 · click sign Time 3 · log gap, two decays Concat 140 Shared trunk Linear 140 → 128 GELU 18,048 params Key head Linear 128 → 64 Value head Linear 128 → 64 Like gate Linear 128 → 1, sigmoid Dislike gate Linear 128 → 1, sigmoid Then write the notepad, not a weight k = normalize(snapshot + 0.1 Δk)  v = normalize(article + 0.1 Δv) M+ ← 0.99 M+ + (like · (v − M+ k)) ⊗ k  M− ← 0.9 M− + (dislike · (v − M− k)) ⊗ k

Key becomes normalize(snapshot + 0.1 Δk). Value becomes normalize(article + 0.1 Δv). Like-gate scales the write into M+; dislike-gate scales the write into M−. Decay is 0.99 / 0.9 and is not learned.

Keep the pad versus wipe

Same snapshot, same write network, same articles. The only training difference is whether the notepad lives across visits. That contrast is a run, not this page.

Keep

Keep the notepad

Walk each user in time. Write only new clicks. Score the current slate. Keep the matrices. Every four visits: one backward pass, then cut the graph but leave the values. Never wipe.

Control

Wipe and rebuild

At every score, start a blank pad, replay the last 20 events, score, throw the pad away. Trains the same write rule, but only under that rebuild-from-scratch habit.

See the keep-the-pad run

Pieces

PieceRoleUpdated by Adam?
Qwen article vectors64-d frozen news embeddingsNo
SnapshotMean of first training clicksNo
Notepad M⁺, M⁻Per-user 64×64 memory, written on the flyNo (state)
Write networkTurns a click into a notepad writeYes
Gains + action embeddingsHow strongly likes/dislikes move the queryYes

Trainable count 34,908 of 10,229,916 is from the Modal keep-the-pad job on MIND-small, seed 42, with write-time features. This page is the layout. Scores live on the experiment pages.