Outcome Prediction Quiz: Capacity

40 minutes · 90 points · 6 questions

How to answer

This quiz has six questions. Predict the outcomes using your A5 experiments. Evidence and reasoning can recover up to 75% of missed points. Enter your choices and explanations below; the original four-page answer sheet is included in the PDF.

Reference recipe

Use A5 defaults: d8 ($D=8,W=512$), vocabulary 4096, context 1024, 600,000 training sequences (614.4M tokens), batch 64, LR .003 and seed 42. Loss comparisons use the full default validation set; lower is better. Router diagnostics follow A5’s eight-sequence, 128-position example.

All LRs below are peak values: use 1% warmup followed by linear decay. Each part states its peak LR or tuning procedure. LR searches keep seed 42, data order and the full token budget fixed, and select by final validation loss. Parameter counts include embeddings and the LM head.

Code reference

Each diff applies independently to the released A5 starter at commit d240711. New recipe functions use A5’s existing model classes and trainer. The original Appendix A is also in the PDF.

Submitting reveals every answer and explanation. Your work is graded locally in this browser and is not sent to the course staff.