Outcome Prediction Quiz: Capacity
How to answer
This quiz has six questions. Predict the outcomes using your A5 experiments. Evidence and reasoning can recover up to 75% of missed points. Enter your choices and explanations below; the original four-page answer sheet is included in the PDF.
Reference recipe
Use A5 defaults: d8 ($D=8,W=512$), vocabulary 4096, context 1024, 600,000 training sequences (614.4M tokens), batch 64, LR .003 and seed 42. Loss comparisons use the full default validation set; lower is better. Router diagnostics follow A5’s eight-sequence, 128-position example.
All LRs below are peak values: use 1% warmup followed by linear decay. Each part states its peak LR or tuning procedure. LR searches keep seed 42, data order and the full token budget fixed, and select by final validation loss. Parameter counts include embeddings and the LM head.
Code reference
Each diff applies independently to the released A5 starter at commit d240711. New recipe functions use A5’s existing model classes and trainer. The original Appendix A is also in the PDF.
Your result
0 / 90
This score covers answer choices only. Rationale credit requires human evaluation.
View answer slides in PDF form