The same six memories on MIND news
Same models, grid, seeds and reporting as the second Amazon study, on MIND-large. Development data is the 10% of training behaviours the loader holds out; its users were split into tuning and held-out halves, and the labelled dev split (the test set) was never scored. One shared cold-start profile was trained once and shipped to every machine with its hash. Each model was tuned on the 3 × 3 grid, retrained on seeds 42 to 49 and scored on 111,875 held-out impressions from 85,505 users. Exploration: no win rule, results described rather than judged.
On MIND the merged network's lead over the grid is +0.0000 (5 of 8 seeds ahead), and plain recency averages come within 0.026 of the learned memories, against 0.09 on Amazon. Two separate networks: 0.3784.
Held-out users · mean NDCG@10 over 8 seeds
Higher is better. Bars start at zero, so small gaps look small.
Per seed
| Seed | Grid memory (FWPRec) | Network, normalized | Network, additive | Two networks | Window mean, no memory | Signed EMA, no memory |
|---|---|---|---|---|---|---|
| Seed 42 | 0.37964 | 0.37803 | 0.37453 | 0.37885 | 0.34661 | 0.35210 |
| Seed 43 | 0.37768 | 0.37849 | 0.37319 | 0.37698 | 0.34648 | 0.35217 |
| Seed 44 | 0.37879 | 0.37911 | 0.37217 | 0.37803 | 0.34647 | 0.35204 |
| Seed 45 | 0.37952 | 0.37692 | 0.37228 | 0.37609 | 0.34650 | 0.35221 |
| Seed 46 | 0.37757 | 0.37851 | 0.37198 | 0.37744 | 0.34647 | 0.35209 |
| Seed 47 | 0.37735 | 0.37936 | 0.37412 | 0.37980 | 0.34640 | 0.35208 |
| Seed 48 | 0.37845 | 0.37830 | 0.37597 | 0.37951 | 0.34646 | 0.35210 |
| Seed 49 | 0.37787 | 0.37845 | 0.37536 | 0.38034 | 0.34648 | 0.35208 |
| Mean | 0.37836 | 0.37840 | 0.37370 | 0.37838 | 0.34648 | 0.35211 |
| Spread (SD) | 0.00089 | 0.00074 | 0.00153 | 0.00149 | 0.00006 | 0.00005 |
| Seeds ahead of the grid | - | 5 of 8 | 0 of 8 | 3 of 8 | 0 of 8 | 0 of 8 |
Lead over the grid memory, per seed
Same impressions scored by both models; brackets are 95% intervals from 2,000 user-level bootstrap samples.
| Model | Seed 42 | Seed 43 | Seed 44 | Seed 45 | Seed 46 | Seed 47 | Seed 48 | Seed 49 |
|---|---|---|---|---|---|---|---|---|
| Network, normalized | -0.0016 [-0.0025, -0.0007] | +0.0008 [-0.0001, +0.0017] | +0.0003 [-0.0006, +0.0012] | -0.0026 [-0.0035, -0.0017] | +0.0009 [+0.0000, +0.0018] | +0.0020 [+0.0011, +0.0029] | -0.0001 [-0.0011, +0.0007] | +0.0006 [-0.0004, +0.0015] |
| Network, additive | -0.0051 [-0.0063, -0.0040] | -0.0045 [-0.0056, -0.0034] | -0.0066 [-0.0077, -0.0055] | -0.0072 [-0.0084, -0.0061] | -0.0056 [-0.0068, -0.0045] | -0.0032 [-0.0043, -0.0022] | -0.0025 [-0.0036, -0.0014] | -0.0025 [-0.0036, -0.0014] |
| Two networks | -0.0008 [-0.0017, +0.0001] | -0.0007 [-0.0016, +0.0002] | -0.0008 [-0.0016, +0.0001] | -0.0034 [-0.0043, -0.0025] | -0.0001 [-0.0010, +0.0007] | +0.0025 [+0.0016, +0.0033] | +0.0011 [+0.0002, +0.0019] | +0.0025 [+0.0016, +0.0033] |
| Window mean, no memory | -0.0330 [-0.0346, -0.0315] | -0.0312 [-0.0327, -0.0297] | -0.0323 [-0.0339, -0.0308] | -0.0330 [-0.0345, -0.0315] | -0.0311 [-0.0326, -0.0295] | -0.0309 [-0.0325, -0.0294] | -0.0320 [-0.0335, -0.0305] | -0.0314 [-0.0329, -0.0299] |
| Signed EMA, no memory | -0.0275 [-0.0290, -0.0261] | -0.0255 [-0.0269, -0.0241] | -0.0267 [-0.0282, -0.0253] | -0.0273 [-0.0287, -0.0259] | -0.0255 [-0.0269, -0.0240] | -0.0253 [-0.0267, -0.0239] | -0.0263 [-0.0278, -0.0249] | -0.0258 [-0.0272, -0.0243] |
All metrics, mean over seeds
Training seconds are only comparable within a host; w06 has a faster GPU.
| Model | AUC | MRR | NDCG@5 | NDCG@10 | ≤20 events (n=18,482) | >20 events (n=93,393) | Private bytes | Host | Train s |
|---|---|---|---|---|---|---|---|---|---|
| Grid memory (FWPRec) | 0.6149 | 0.3400 | 0.3156 | 0.3784 | 0.4321 | 0.3677 | 32,768 | w02 | 583 |
| Network, normalized | 0.6152 | 0.3402 | 0.3156 | 0.3784 | 0.4330 | 0.3676 | 16,512 | w03 | 426 |
| Network, additive | 0.6087 | 0.3372 | 0.3107 | 0.3737 | 0.4307 | 0.3624 | 16,512 | w06 | 624 |
| Two networks | 0.6156 | 0.3396 | 0.3156 | 0.3784 | 0.4334 | 0.3675 | 32,768 | w06 | 879 |
| Window mean, no memory | 0.5754 | 0.3078 | 0.2828 | 0.3465 | 0.4045 | 0.3350 | 0 | w03 | 164 |
| Signed EMA, no memory | 0.5825 | 0.3135 | 0.2890 | 0.3521 | 0.4125 | 0.3402 | 0 | w02 | 144 |
| Static profile, no adaptation | 0.5630 | 0.2989 | 0.2733 | 0.3367 | 0.3692 | 0.3303 | 0 | - | - |
Chosen settings
5 of 6 models chose temperature 0.4, the top of the grid, so a wider grid might score higher for all of them.
| Model | Learning rate | Temperature | Epoch | Tuning NDCG@10 |
|---|---|---|---|---|
| Grid memory (FWPRec) | 0.003 | 0.4 | 8 | 0.3783 |
| Network, normalized | 0.003 | 0.4 | 5 | 0.3780 |
| Network, additive | 0.003 | 0.2 | 10 | 0.3741 |
| Two networks | 0.003 | 0.4 | 10 | 0.3785 |
| Window mean, no memory | 0.003 | 0.4 | 3 | 0.3471 |
| Signed EMA, no memory | 0.003 | 0.4 | 3 | 0.3524 |
Noise check
The normalized network trained twice with seed 42 and identical settings scored 0.37547 and 0.37845 (gap 0.00298).
Development data only; exploratory. No test split scored. Hosts: w02 grid + signed EMA, w03 normalized network + window mean + noise check, w06 additive network + two networks. 54 tuning runs, 2 noise-check runs, 48 retraining runs, one anchor run. Files: experiments/isolated/mind_memory_exploration/ on branch DPFWM-HP. Status: Complete.