The per-device OS pin (CHART_REGIME) keeps an OS update from silently
re-baselining a trend, but its failure mode was silent too: after an update
every new point fails the pin and the chart freezes at the last old-regime
build with nothing saying so. Print a warning when newer-than-charted points
are being dropped, so the pin gets rolled forward instead.
Assisted-by: Claude Opus 5 (Anthropic)
run_environment.csv is keyed by (build, device) but the env lookup was keyed by
build alone — the Redmi rows overwrote the A36's and the gate scorecard lost its
2.38.2 baseline on every surface. Same hole in the per-surface charts: a surface
naming no device charted all devices (21 Redmi points on 8 A36 charts). Both now
scope to one phone; unnamed = the gate phone.
Adds a low-end scorecard, an A36-vs-Redmi comparison and per-surface Redmi trends
under docs/android/lowend/, published apart from the gate — the gate stays one
reference phone by design.
Gate verdicts are also withheld when only one side is below the measurement floor
(2.38.2 read Buy and Home at ~0.1s, so a normal 0.6s printed as "+0.5s slower").
Assisted-by: Claude Opus 5 (Anthropic)
The scorecard had shown no-baseline on most surfaces since the A36 took
the June security patch (CZE3 -> CZF1): the same-OS guard correctly
refused the 2.38.0 baseline measured on the old OS. 2.38.2 is measured
on the current OS, so the guard passes again and per-surface verdicts
return - including the Wallet Buy +0.59s regression (#21698), which the
no-baseline gap had been hiding from this card.
Chart reference line, normal-range band, and labels move 2.38.0 -> 2.38.2.
Assisted-by: Claude <noreply@anthropic.com>
The 2.38.2 patch build (3ef171) was measured on the gate device on the
current OS regime, so it joins 2.37.1 and 2.38.0 as a pinned left-column
baseline on every Android chart that has data for it. The 2.38.0
reference line and +/-15% band are unchanged.
Assisted-by: Claude <noreply@anthropic.com>
New per-build series from the Moto pinned settled account: committed PSS per
process (median of 3 dumpsys reads) and mean CPU% per process over a 5-min
idle window. Chart code gains a backward-compatible 'series' field; all 40
existing charts verified byte-identical before/after.
Assisted-by: Claude
- 2.38.0 reference line solid + lighter so it reads distinct from the dashed zone lines
- ok zone (0.5-1.0s) amber -> gold so it no longer looks like the red slow zone
- normal-range band 8% -> 15% (matches the drift threshold)
- every response chart shows the speed zones; fast surfaces scale to their own data
- zone/threshold legend moved below the plot; first-open % annotated below the point
Assisted-by: Claude Opus 4.8 (Anthropic)
Render the four frozen 2.38 charts (Market, Wallet Send, first-vs-returning,
Communities directory load) in the current style so the release page matches the
live pages. The three nav-scale response charts get speed zones — Market visibly
crosses into slow, Send drops back to ok; the tens-of-seconds directory load does
not, since zones now draw only on nav-scale charts (the 0.5/1.0s bands are a
vestigial sliver on a network-load axis).
Assisted-by: Claude Opus 4.8 (Anthropic)
Every per-surface chart now shows the fast/ok/slow speed zones (clipped to its
y-range). Sub-action / sub-screen charts also get a dotted 2.38.0 +/-8% normal-range
channel as their drift reference; the nav charts stay clean (zones + the 2.38.0 line
are enough). One rule across the whole set instead of two formats.
Assisted-by: Claude Opus 4.8 (Anthropic)
Per-surface charts: shade fast/ok/slow speed zones on the nav charts, overlay a
5-build rolling average, and anchor the non-nav normal-range band to 2.38.0
(was latest +/-15%, which by construction could never show drift).
Scorecard: lead with the change vs the last release in seconds (parity / slower /
faster) plus an absolute speed band; drop the recent-trend column (drift now lives
in the charts) and the at-floor jargon. Tighter 4-column layout.
Assisted-by: Claude Opus 4.8 (Anthropic)
Plot points by build position, not date, so the pinned release baselines
(measured later than the first nightlies) no longer draw a backward segment
from the baseline column to the first nightly.
For the five navigation charts, drop the 2.0s target and mark the UX bands
instead: a nav tap reads as fast below 0.5s and slow above 1.0s.
Assisted-by: Claude Opus 4.8 (Anthropic)
Two baselines (2.37.1/2.38.0 on Android 16) pinned as fixed left columns with a separator + a dashed last-release reference line; first-open drawn as a second line per nav tab, annotated with its % over the repeat open (suppressed when the repeat is sub-floor; Wallet skipped as its first-open is the warm landing screen).
Assisted-by: Claude Opus 4.8 (Anthropic)
Drop legacy-OS points + the OS divider (one comparable regime); rolling window 12->30; angled date+name x-labels with the commit hash dropped, so build data is readable per Volo feedback.
Assisted-by: Claude Opus 4.8 (Anthropic)
A device OS update (now Android 16 / One UI 8) shifted the perf baseline, so the
same APK reads differently before and after. Re-measured 2.37.1 and the shipped
2.38.0 on the new OS (suffixed builds 760417N / 5f66deN, so the old-OS rows are
preserved) to separate the OS effect from the app:
- Wallet: +13% on BOTH releases identically (1.03 -> 1.17) — a pure OS effect,
not an app change.
- Market: the ~2x regression seen on the previous OS DISAPPEARS on the new OS —
2.38 (0.62 s) is at parity with 2.37.1 (0.59 s). The regression was real but
OS-conditional.
Changes:
- Trend charts draw a "device OS update" divider (derived from run_environment.csv)
so a step caused by the device software is not misread as an app change.
- The scorecard now compares each surface against the 2.37.1 baseline measured on
the same device OS (Market therefore reads parity on the current OS; Send/Swap
keep their old-OS comparison, still same-OS, ~44% faster).
- run_environment.csv records the OS for the re-baseline builds.
Assisted-by: Claude Opus 4.8 (Anthropic)
A line missing some builds was plotted at per-line positions 0..n, which on a multi-line/parametrized chart would put its values under the wrong build labels. Map each point to the shared build order instead. No change to the current single-line charts (verified byte-identical).
Assisted-by: Claude Opus 4.8 (Anthropic)
Adds an Android response-time dashboard alongside the existing desktop charts,
without touching the desktop path.
- scripts/benchmark_mobile.py: self-contained matplotlib/seaborn charting for the
mobile surfaces. Kept separate from benchmark.py (desktop, plotly) so the two
share no dependency, because the mobile charts are generated on a Raspberry Pi,
which cannot build plotly's kaleido image-export backend.
- scripts/tests_config_android.toml: the mobile surface definitions, separate from
tests_config.toml so the desktop chart and README pipeline never sees them.
- requirements-mobile.txt: mobile-only dependencies (matplotlib, seaborn).
- data/android/ and docs/android/: the dataset, the rendered charts, and the
dashboard README (nav response times with a normal-range band, wallet actions,
Settings sub-screens, first-open vs returning, cold opens, communities). Build
labels carry an optional exclude column for hiding a pre-final build from the
trend while keeping its raw rows.
benchmark.py, tests_config.toml and requirements.txt are unchanged from master,
so the desktop charts are unaffected.
Assisted-by: Claude Opus 4.8 (Anthropic)
glitchminer