Table 2 campaign · seeds 42/43/44 · updated 2026-09-23 11:55 UTC

The paper's table, run to spec

5 datasets
10 models
frozen Qwen
3 seeds
held-out test

This run treats the paper's own text as law and fills its main comparison table from nothing: no number, checkpoint, or shortcut from any earlier campaign. Every model reads the same frozen 64-dimensional text vectors for items, has its learned per-item bias removed, sees identical splits and candidates, and is picked by validation score before touching the test set once. Where the paper names training settings we use them verbatim; where it is silent, each model earned its learning rate through the same three-way validation screen. The one licensed change: MIND-large stands in for MIND-small, run last.

So far

Four of five columns are final, zero failed runs. The notepad (FWPRec) leads Amazon and EB-NeRD outright, sits mid-table on MovieLens-1M where plain item co-occurrence still rules, and is in a photo finish with SASRec on KuaiRec while holding the best AUC there. MIND-large is training now.

The deal every model gets

Same frozen Qwen3-Embedding-0.6B item vectors (64-d, length-normalized) · no learned item bias · same chronological splits, histories, candidates and labels (construction seed 42) · positive-only histories for sequence models, with the signed stream reaching only the notepad's memory · best-validation checkpoint, then one pass over test · means over training seeds 42/43/44. Popularity and ItemKNN use no item vectors at all — they are the representation-independent floor, run once. BPRMF with a frozen item table trains only its user vectors, a real capacity cut, noted here so the row is read fairly.

MIND-large

News clicks at full scale (~376k test impressions). The paper's MIND-small settings applied unchanged, per the owner's substitution instruction.

⏳ In progress — 3 of 10 model rows final. Numbers shown are complete three-seed means; missing rows are still training.

Model	AUC	MRR	nDCG@5	nDCG@10	seeds
Popularity	0.4941	0.2527	0.2253	0.2910	1/1
ItemKNN	0.5034	0.2643	0.2366	0.3028	1/1
BPRMF	0.5771 ±0.0005	0.3223 ±0.0003	0.2974 ±0.0003	0.3635 ±0.0004	3/3
GRU4Rec	0.6176	0.3499	0.3278	0.3960	1/3 ⏳
SASRec	0.5710	0.3138	0.2862	0.3584	1/3 ⏳
DuoRec	0.6152 ±0.0001	0.3554 ±0.0017	0.3327 ±0.0013	0.3983 ±0.0011	2/3 ⏳
LRURec	·	·	·	·	running
Mamba4Rec	0.6197 ±0.0028	0.3536 ±0.0010	0.3320 ±0.0015	0.3999 ±0.0017	2/3 ⏳
HSTU	0.6263	0.3592	0.3377	0.4050	1/3 ⏳
FWPRec (Ours)	·	·	·	·	running

EB-NeRD

Danish news clicks. The officially labeled validation partition serves as the held-out test, so this column is test numbers like every other.

Model	AUC	MRR	nDCG@5	nDCG@10	seeds
Popularity	0.4436	0.2635	0.2893	0.3854	1/1
ItemKNN	0.4894	0.3014	0.3330	0.4197	1/1
BPRMF	0.5623 ±0.0007	0.3612 ±0.0005	0.3997 ±0.0005	0.4749 ±0.0005	3/3
GRU4Rec	0.5633 ±0.0008	0.3586 ±0.0005	0.3976 ±0.0007	0.4729 ±0.0007	3/3
SASRec	0.5328 ±0.0074	0.3388 ±0.0064	0.3747 ±0.0069	0.4544 ±0.0059	3/3
DuoRec	0.5424 ±0.0066	0.3422 ±0.0057	0.3792 ±0.0065	0.4577 ±0.0052	3/3
LRURec	0.5635 ±0.0010	0.3586 ±0.0006	0.3976 ±0.0008	0.4731 ±0.0008	3/3
Mamba4Rec	0.5640 ±0.0016	0.3594 ±0.0012	0.3984 ±0.0012	0.4736 ±0.0012	3/3
HSTU	0.5620 ±0.0005	0.3572 ±0.0009	0.3963 ±0.0008	0.4717 ±0.0007	3/3
FWPRec (Ours)	0.5699 ±0.0044	0.3642 ±0.0031	0.4043 ±0.0037	0.4777 ±0.0031	3/3

Amazon Video Games

Product reviews with star ratings. Test set: 19,773 impressions, the paper's exact number.

Model	AUC	MRR	nDCG@5	nDCG@10	seeds
Popularity	0.3126	0.0707	0.0598	0.0745	1/1
ItemKNN	0.5320	0.0965	0.0789	0.0953	1/1
BPRMF	0.5572 ±0.0013	0.0761 ±0.0007	0.0553 ±0.0008	0.0769 ±0.0011	3/3
GRU4Rec	0.7957 ±0.0027	0.1910 ±0.0031	0.1790 ±0.0034	0.2292 ±0.0036	3/3
SASRec	0.7718 ±0.0009	0.1823 ±0.0011	0.1713 ±0.0017	0.2153 ±0.0018	3/3
DuoRec	0.7793 ±0.0028	0.1621 ±0.0021	0.1456 ±0.0023	0.1935 ±0.0036	3/3
LRURec	0.8114 ±0.0017	0.2145 ±0.0021	0.2075 ±0.0022	0.2566 ±0.0026	3/3
Mamba4Rec	0.8055 ±0.0017	0.2046 ±0.0002	0.1951 ±0.0005	0.2442 ±0.0007	3/3
HSTU	0.8146 ±0.0038	0.2247 ±0.0026	0.2184 ±0.0032	0.2674 ±0.0031	3/3
FWPRec (Ours)	0.8233 ±0.0020	0.2390 ±0.0020	0.2342 ±0.0028	0.2825 ±0.0023	3/3

MovieLens-1M

Movie ratings, split 80/10/10 inside each user's own timeline (the paper's wording, implemented fresh for this run).

Model	AUC	MRR	nDCG@5	nDCG@10	seeds
Popularity	0.8286	0.2063	0.1948	0.2478	1/1
ItemKNN	0.8557	0.2946	0.3035	0.3640	1/1
BPRMF	0.6631 ±0.0000	0.1043 ±0.0001	0.0826 ±0.0000	0.1150 ±0.0001	3/3
GRU4Rec	0.7997 ±0.0004	0.2372 ±0.0004	0.2356 ±0.0004	0.2833 ±0.0006	3/3
SASRec	0.7909 ±0.0007	0.2408 ±0.0005	0.2392 ±0.0005	0.2848 ±0.0008	3/3
DuoRec	0.7988 ±0.0003	0.2560 ±0.0021	0.2565 ±0.0024	0.3015 ±0.0020	3/3
LRURec	0.8112 ±0.0003	0.2816 ±0.0009	0.2842 ±0.0009	0.3278 ±0.0010	3/3
Mamba4Rec	0.8102 ±0.0011	0.2803 ±0.0010	0.2828 ±0.0009	0.3264 ±0.0013	3/3
HSTU	0.8159 ±0.0008	0.2914 ±0.0012	0.2949 ±0.0017	0.3378 ±0.0012	3/3
FWPRec (Ours)	0.7993 ±0.0021	0.2360 ±0.0020	0.2334 ±0.0022	0.2804 ±0.0023	3/3

KuaiRec (small_chronological)

Short-video watch logs, bounded small-matrix protocol; video captions provide the shared text vectors.

Model	AUC	MRR	nDCG@5	nDCG@10	seeds
Popularity	0.1419	0.0195	0.0059	0.0101	1/1
ItemKNN	0.3615	0.0189	0.0006	0.0015	1/1
BPRMF	0.5567 ±0.0008	0.0670 ±0.0006	0.0444 ±0.0007	0.0645 ±0.0007	3/3
GRU4Rec	0.5864 ±0.0044	0.0784 ±0.0034	0.0567 ±0.0044	0.0801 ±0.0049	3/3
SASRec	0.5757 ±0.0093	0.0935 ±0.0037	0.0723 ±0.0040	0.0955 ±0.0048	3/3
DuoRec	0.5294 ±0.0060	0.0628 ±0.0016	0.0407 ±0.0018	0.0583 ±0.0020	3/3
LRURec	0.5926 ±0.0045	0.0821 ±0.0002	0.0611 ±0.0006	0.0860 ±0.0010	3/3
Mamba4Rec	0.5824 ±0.0038	0.0765 ±0.0024	0.0550 ±0.0031	0.0796 ±0.0032	3/3
HSTU	0.5811 ±0.0113	0.0785 ±0.0052	0.0571 ±0.0056	0.0801 ±0.0065	3/3
FWPRec (Ours)	0.6106 ±0.0094	0.0889 ±0.0028	0.0678 ±0.0025	0.0941 ±0.0029	3/3

Where the learning rates came from

MIND-large and Amazon: the paper's stated values (default 10⁻³; 2·10⁻³ for GRU4Rec, DuoRec, LRURec, HSTU on MIND and for LRURec on Amazon). EB-NeRD, MovieLens-1M, KuaiRec: the paper names no settings, so every neural model ran a three-way screen (5·10⁻⁴, 10⁻³, 2·10⁻³) at seed 42, judged on validation nDCG@10 only — 72 screen runs whose winners are frozen in a checked-in selection file. Screens structurally cannot touch the test split.