FWPRec Experiments
All experiments
MIND-small · seeds 42, 43, 44 · RTX 4080 SUPER

Keep the notepad, 3 seeds

Single-seed evidence is development. Three seeds make it confirmed. We re-ran the keep-the-pad versus wipe-and-rebuild protocol at two new training seeds (43 and 44) on the same MIND-small split as seed 42. Dataset split, snapshot, and notepad config were unchanged. Both new seeds kept the notepad ahead of wiping it. The top-1 ranking after keeping the pad matched a fresh rebuild on roughly 90% of scored pairs.

Takeaway

Keep beats wipe on all three seeds. Best NDCG@10: 0.3951, 0.3962, 0.3939 vs wipe 0.3885, 0.3894, 0.3897.

Best keep vs wipe · NDCG@10

Wipe, seed 42
0.3885
Keep, seed 42
0.3951
Wipe, seed 43
0.3894
Keep, seed 43
0.3962
Wipe, seed 44
0.3897
Keep, seed 44
0.3939

Summary · all seeds

SeedWipe NDCG@10Keep NDCG@10Best epochTop-1 agree
420.38850.3951490.5%
430.38940.3962689.5%
440.38970.3939390.5%

Seed 43 · keep NDCG@10 by epoch

EpochKeep the padSame weights, rebuild eval
10.38420.3817
20.39210.3887
30.38890.3862
40.39200.3886
50.39560.3913
60.39620.3930
70.39370.3902
80.39470.3922
90.39240.3902

Seed 44 · keep NDCG@10 by epoch

EpochKeep the padSame weights, rebuild eval
10.39150.3876
20.39030.3866
30.39390.3892
40.39270.3890
50.39340.3891
60.39360.3898

Agreement · keep vs fresh rebuild

SeedPairsTop-1 matchLargest score gap
4220090.5%2.55
4320089.5%2.73
4420090.5%2.19

Isolated trainer copy; seed-42 files untouched. Dataset seed 42 for all three runs (same split). Only training seed changes. RTX 4080 SUPER, torch 2.13, CUDA 13. Seeds 43 and 44 ran in parallel on the same GPU. Wipe ran 10 epochs for all seeds. Keep early-stopped with patience 3. Validation only. Test split sealed.