THE GITHTML FIELD GUIDE
Review performance claims without being fooled by one fast run
Evaluate a performance claim by checking what was measured, under which conditions, and how the observations varied. Compare equivalent workloads and examine the distribution rather than one favorable run. Separate measured improvement from a proposed explanation, and ask whether the reported change matters to the user experience being evaluated.
Start with the exact interval
An illustrative claim that exports are faster could refer to request acceptance, queue delay, processing, or time until the file is ready to download. Ask for the start and end event before discussing percentages. Record the code revision, environment, input characteristics, and observation window. Google's monitoring guidance highlights the importance of meaningful latency observations; your review should connect the measurement to a specific task. A server-side timer may not include client-side waiting, so avoid treating it as a complete account of what a user experiences.
Compare conditions and variation
Check whether baseline and candidate used comparable hardware, configuration, concurrency, data, and cache state. Look for warmup differences, excluded failures, and changes in the number of observations. An illustrative review note might ask for separate reporting of successful and failed export attempts because omitting slow failures could change the interpretation. Do not invent thresholds or sample sizes as universal rules. The necessary evidence depends on variability and the decision's consequence. Ask for the observed spread and tail behavior when an average would hide important delays.
- Definition: verify the start event, end event, unit, and affected users.
- Comparability: inspect workload, environment, failures, and collection windows.
- Interpretation: separate the measured difference from its suspected cause.
Write a decision proportionate to evidence
A useful result can be accept, reject, or collect more evidence. If the measurement is only a small local experiment, describe it that way rather than projecting a production outcome. Ask what additional observation would change the decision and whether the follow-up can be performed safely. In an HTML review packet, pair charts with text describing axes, units, and conditions. Keep the conclusion near the evidence rather than using a large percentage headline that survives after caveats are cropped from a screenshot.
Sources and further reading
AI-assisted writing with source-linked guidance and illustrative examples. Read our editorial approach or report a correction.