Skip to content

Change Throughput Diagnostic Prompt

Maturity: experimental — worth exploring; promising but not yet proven across enough contexts.

When to use: Deliberately invoke in Agent mode against a git repository to produce a point-in-time read on Lead Time For Changes (LTFC) and Deployment Frequency (DF), plus the contributor mix behind them. Re-run periodically (e.g. monthly) to track delivery throughput and set or check a per-active-contributor benchmark.

The agent needs read access only; the bundled scripts run entirely against the local git clone — no API key, no network, no writes to the target repo are required unless you ask it to save the report.

If the user asks to "run this here" (or equivalent), still run the scope gate below. You may proceed immediately only when window, target repo/branch, scope, and deploy trigger are already explicit in the request. Otherwise ask concise questions first and wait for answers.

Bundled scripts: ../scripts/ — a small, tested Python engine (metrics.py + test_metrics.py) and a git-only collector (collect.py). Stdlib only; no pip install. See ../scripts/README.md.

What this diagnostic covers

Every run produces evidence across three headline views:

View What it answers
Throughput LTFC (first commit → merge to trunk) and DF (merges to trunk per week / work-day), split into meaningful vs micro changes
Contributor mix Who is genuinely active vs occasional/part-time; how concentrated delivery is (bus-factor)
Benchmark check Actual vs a target of 1 good change per N active contributors per work-day (default N=2)

Why this is trustworthy (and git-only)

  • LTFC uses git author dates — immune to rebase (rebase rewrites committer dates only). Verify author≈committer dates to confirm no history rewriting distorts the measure.
  • DF counts merge commits to the trunk. This is valid only if a merge to the trunk deploys. Confirm the deploy trigger for the target project before calling merge frequency "deployment frequency" — if deploys are scheduled or manual, DF is gated by that pipeline, not by merges.
  • Cycle time (PR created_on → merge) is intentionally out of scope here: it requires a forge API and a user-supplied token. The optional API path is documented in scripts/collect.py if review-latency metrics are ever needed.

Scope gate — confirm before running (mandatory)

Do not assume scope. Before running any history commands, confirm two things with the user. If either is not explicit from the request or workspace, ask and wait for an explicit answer.

  1. Time window. Which period should the diagnostic cover? Offer concrete options:
  2. Last 30 days · last 90 days · a named month · an explicit --since YYYY-MM-DD [--until YYYY-MM-DD]
  3. If the user wants a trend, run two adjacent windows (e.g. prior 30d and current 30d) so concentration/throughput shifts are visible.
  4. Target project / repo and trunk branch. Which repository is being measured, and what is its deploy trunk? Default trunk is main. In a monorepo, confirm whether the read is for the whole repo or scoped to one domain/folder. If the workspace clearly is the target repo with a main branch, say so and proceed.

Also confirm the deploy trigger (see above) so the DF number is labelled correctly.

When interactive option tools are available, use them first. Otherwise ask in plain chat and wait for an explicit choice.

Execution preference for local machines:

  • Prefer the workspace Python interpreter / active venv rather than plain python.
  • If scripts are not copied into the target repo, run them by absolute path from this skill's scripts/ folder.
  • Prefer writing intermediate JSON output to analysis/tmp/ in the target repo so the run is inspectable and repeatable.

Prompt

Produce a Change Throughput Diagnostic for this repository, focused on Lead Time For Changes (LTFC) and Deployment Frequency (DF). Work entirely from the local git clone.

First, settle scope (see Scope gate). Confirm the time window, the target repo/branch (and whether it is whole-repo or scoped to one domain), and the deploy trigger. If any of these is unclear from my request or the open workspace, ask me and wait. When interactive option tools are available, use them first. If scope is explicit, restate it in one line, then proceed.

1. Run the bundled engine

Use the scripts in this skill folder (copy them into the target repo or run by path):

# from the scripts folder (should be all green before trusting output)
python -m unittest test_metrics.py

python collect.py --since <YYYY-MM-DD> --until <YYYY-MM-DD> --branch <trunk>

# run by path from the target repo
python /path/to/library/skills/change-throughput/scripts/collect.py --since <YYYY-MM-DD> --until <YYYY-MM-DD> --branch <trunk> --json analysis/tmp/throughput-current.json

Report the engine output: PRs / meaningful / micro counts, LTFC (median, p75, max), DF (all and meaningful, per week and per work-day), the per-contributor table, the genuinely-active set, and the benchmark line.

2. Validate the measurement (cross-checks)

Before trusting the numbers, confirm the method holds for this repo: - No squash/rebase distortion: check that merges to trunk are true two-parent merge commits and that commit author dates == committer dates (a non-zero gap signals rebasing, which would compress LTFC). State the result. - Merge == deploy: restate the confirmed deploy trigger; if merges do not deploy, relabel DF as "integration frequency" and note what actually gates deployment. - Micro vs meaningful: a "micro" change is files<=1 and churn<=5 and commits<=1 (trivial one-liner). Note how many changes this excludes and whether loosening the threshold would move the headline.

3. Contributor mix and concentration

From the per-contributor table: - Classify each contributor as consistent core, rotating/regular, or occasional (genuinely-active default = >=3 PRs OR >=5 active days in the window). - Report top-N concentration (e.g. share of PRs held by the top 3) and flag bus-factor risk where a domain or the bulk of throughput sits with one or two people. - If you ran two windows, note who is stable across both vs who appeared/dropped off — the honest "genuinely active" count is those present in both windows. - Alias note: metrics.py carries a project-specific AUTHOR_ALIASES map (people who commit under both a name and a username/email). Extend it for the target project so one person is not counted as several; report the alias-merged headcount.

4. Area / domain breakdown (monorepo only)

If the repo is a monorepo, tabulate which top-level domains the core contributors actually touched in the window, and how many of the repo's total domains saw no core activity. This scopes the benchmark honestly: a per-active-contributor rate describes the actively-developed slice, not the dormant remainder. Pair throughput with a domain- ownership view so single-owner domains stay visible.

5. Benchmark check

State the benchmark — 1 good (meaningful) change per 2 genuinely-active contributors per work-day (tunable via --contributors-per-change semantics in benchmark()), the implied target for the window, the actual, and attainment %. Recommend the active- contributor denominator to anchor on (prefer the count stable across adjacent windows), and explicitly scope the benchmark to the active product lines rather than the whole monorepo.

6. Summary

Open with an At a glance block: window, branch, LTFC median/p75, DF (meaningful)/week, genuinely-active count, benchmark attainment. Then the sections above. Use tables for all quantitative data and cite actual numbers, not generalities. Flag any process observations (e.g. near-zero review latency straight to prod for large changes).

Output format: a single markdown document. If I ask you to save it, write to docs/change-throughput-YYYY-MM.md in the target repo using the window's year and month, and compare against any prior report there under section Progress since last run.

Tips for reviewers

  • Scope first, always. The window and target project/branch change the numbers materially. Never run history commands before the scope gate is satisfied.
  • Two windows beat one. A single window hides whether the active-contributor count is stable. Run the prior period too before locking a benchmark denominator.
  • Trust the source, not the headline. LTFC from git author dates and DF from merge counts are rewrite-immune; cycle time is not measurable without an API token and adds little here.
  • Tune the definitions in metrics.py, not in prose: micro thresholds, active-contributor thresholds (min_prs, min_active_days, require_both), and the benchmark rate (contributors_per_change) are all parameters with unit tests.
  • Keep aliases current. Update AUTHOR_ALIASES per project so headcount is honest.
  • Optional API path: to add cycle time / review latency, see the documented optional forge API notes in scripts/collect.py — it requires a user-supplied personal token.