Run a closed test from the terminal: create a cohort, invite people, see where they stand, chase whoever needs chasing, and deploy the daily pass that does it on a schedule.
The dashboard is the paid tier; this is not. Every operation the control-plane routes expose has a command here, each non-interactive and each with --json.
This is the command surface of the testers capability; provisioning Testers stands the inbox up first.
Synopsis
pithy testers provision [--env <env>] [--json]
pithy testers deprovision [--env <env>] [--json]
pithy testers create <name> [--env <env>] [--target-size <n>] [--window-days <n>] [--max-roster <n>] [--platform <android|ios>] [--store-url <url>] [--json]
pithy testers list [--env <env>] [--json]
pithy testers invite <cohort> --email <address> [--email <address>…] [--name <name>] [--env <env>] [--json]
pithy testers pending <cohort> [--env <env>] [--json]
pithy testers roster <cohort> [--env <env>] [--json]
pithy testers status <cohort> [--env <env>] [--trend-days <n>] [--json]
pithy testers remove <cohort> --email <address> [--reason <text>] [--env <env>] [--json]
pithy testers close <cohort> [--env <env>] [--json]
pithy testers run [--cohort <cohort>] [--env <env>] [--skip-nudges] [--json]Which operations need a Cloudflare account. provision and deprovision always do — they deploy and delete a Worker. The nine roster subcommands read and write the database directly, so at the default --env dev they run entirely locally through Miniflare.
The roster commands talk to the database, not to the Worker over HTTP. The control-plane routes exist for a management client holding a credential; a developer at a terminal in their own repo already has the database, and minting a token to read their own roster would be ceremony with no security benefit.
Flags
--env is two different sets, deliberately. The roster subcommands default to dev and accept any environment, because they talk to a local D1 and dev is where you use them. provision and deprovision deploy a Worker to a Cloudflare account, where there is no dev to deploy to, so they take a managed environment only — and omitting the flag means every one.
provision and deprovision
| Flag | Default | Purpose |
|---|---|---|
--env <env> | every managed environment | Act on one environment only. --env dev is refused with the reason |
--json | false | Machine-readable output |
create <name>
| Flag | Default | Purpose |
|---|---|---|
<name> | — | Required positional. A label for the cohort, e.g. closed-test |
--target-size <n> | 12, Play’s floor | Testers required simultaneously |
--window-days <n> | 14, Play’s window | Continuous days required |
--max-roster <n> | 100 | Roster cap |
| `–platform <android | ios>` | android |
--store-url <url> | the capability default | The store’s own opt-in link |
--env <env>, --json | dev, false | As everywhere |
The three numeric flags must each be a whole number above zero, and --platform must be one of the two. Both are checked at the flag rather than at the schema, so the answer names what you typed.
invite <cohort>
| Flag | Default | Purpose |
|---|---|---|
<cohort> | — | Required positional. The cohort name or id |
--email <address> | — | Required. Address to invite. Repeat the flag for several |
--name <name> | — | A display name for the roster, applied to every address in this run |
invite adds people to the roster and sends nothing. pithy testers run is what asks them whether they will test.
status <cohort>, remove <cohort>, run
| Flag | Applies to | Default | Purpose |
|---|---|---|---|
--trend-days <n> | status | 30 | How many daily snapshots to include in the trend series |
--email <address> | remove | — | Required. The tester’s address |
--reason <text> | remove | — | A short note recorded on the event |
--cohort <cohort> | run | every open cohort | Run one cohort only |
--skip-nudges | run | false | Advance state and record the day, but send nothing |
What it does
provision deploys the daily-pass Workflow worker for each requested environment and then writes its TESTERS_DAILY binding into the project’s wrangler.jsonc. pithy add testers cannot write that binding — wrangler requires both a name and a class_name, and the deployed name is per environment. Preflight runs once before the first deploy, so failing means failing before one environment is half provisioned.
Each environment’s deploy resolves two ids first: the app DB id from that environment’s stanza, and this project’s shared email-suppression database — the daily pass reads it to reconcile bounced addresses, so pithy email provision has to have run.
The sending identity is read off the composed email capability rather than asked for. When no email capability is composed, the host is still deployed and the pass records the day but sends nothing — said at provision time rather than discovered on the first silent morning. The pass runs daily at 05:00 UTC.
deprovision deletes the host and nothing else. The cohorts, the roster and the whole snapshot series are rows in your own D1 and are not this command’s to remove.
create creates a cohort, freezing the target, window, roster cap, platform and reset policy onto the row. They are stored rather than read from config at query time: raising targetSize from twelve to fifteen later would otherwise retroactively rewrite whether last Tuesday counted.
list, roster and status read. All three degrade rather than fail when the activity read cannot run — auth tables absent, or a transient D1 error — and say so on stderr at warn, which shows without --debug and leaves the --json contract on stdout untouched. Without that line, a whole roster reading never signed in is indistinguishable from a cohort nobody ever used.
invite adds addresses to the roster, refusing one already on it in a live state rather than quietly resetting the date the streak is measured from. A previously removed member is revived, keeping their id so their history stays attached to one person. A tester who withdrew is not revived at all.
pending prints the addresses that have agreed to test and are waiting to be added to the store’s own tester list. This is the one step Pithy cannot do for you: the Play Developer API has no way to add an address to an email list, so you paste these into the console. Until that is done the store link answers App not available, which is why the second email waits.
remove takes a tester off the roster. close closes a cohort: its roster and trend stay readable, and nothing further is sent.
run runs the daily pass now — advance state, chase whoever is due, record the day. Sends are real: the pass writes a row into the email job table and the email worker delivers it, so the CLI needs no sending domain of its own. The row stays pending until that worker’s every-minute scheduler picks it up, which costs up to a minute and loses nothing.
--json
One line, one object, one shape per subcommand.
testers provision and testers deprovision
| Key | Type | Meaning |
|---|---|---|
command | string | The subcommand that produced this line |
results | array | One entry per environment acted on |
results[].env | string | The environment this entry describes |
results[].worker | string | The deployed daily-pass worker name. On teardown, absent is success — it is idempotent |
sends | boolean | provision only. Whether an email capability is composed. false means the pass will record the day and send nothing |
testers create and testers close
Both emit the whole cohort row, under the same key and the same shape.
| Key | Type | Meaning |
|---|---|---|
command | string | The subcommand that produced this line |
cohort.id | string | The cohort’s UUID. Text rather than a sequential id, which would leak how many test programs a project has run |
cohort.name | string | The human label |
cohort.targetPlatform | "android" or "ios" | Which store’s program this cohort serves |
cohort.targetSize | integer | How many testers must be opted in simultaneously. Frozen on the row |
cohort.windowDays | integer | How many continuous days the target must hold. Frozen on the row |
cohort.maxRosterSize | integer | The most members this roster may hold |
cohort.storeOptInUrl | string or null | The store’s own opt-in page. null until set — and until it is, nobody can actually join the test |
cohort.resetPolicy | "reset" or "pause" | Pithy’s assumption about what a dip below target does to the streak. Google documents neither behavior |
cohort.closedAt | ISO-8601 or null | When the cohort was closed |
cohort.createdAt | ISO-8601 | The zero point of its day axis |
cohort.updatedAt | ISO-8601 | When the row was last written |
testers list
| Key | Type | Meaning |
|---|---|---|
cohorts[].estimatedOptedIn | integer | Pithy’s estimate of the currently opted-in count |
cohorts[].estimatedHeldDays | integer | The estimated unbroken at-target run ending today |
cohorts[].targetSize, .windowDays | integer | The target and window in force for this cohort |
cohorts[].closed | boolean | Whether the cohort has been closed |
testers invite, pending, remove
| Key | Type | Meaning |
|---|---|---|
cohort | string | The resolved cohort id, whether you passed a name or an id |
invited[].id | string | invite. The member id. Stable across a revival, so history stays attached to one person |
invited[].email | string | invite. The invited address, normalized |
invited[].created | boolean | invite. false when the row already existed and was revived. Re-inviting is not an error, so this is how the two are told apart |
emails | string[] | pending. Every tester who has agreed to test and is waiting for the store’s tester list |
member | object | remove. The removed tester’s id and email |
testers run
| Key | Type | Meaning |
|---|---|---|
results[].cohortId | string | The cohort this entry describes |
results[].snapshotOn | string | The UTC day the pass recorded |
results[].nudged | object | Nudges enqueued by kind — confirm, store, inactive, closing. Enqueued, not delivered: that is the email worker’s job |
results[].estimatedOptedInCount | integer | Pithy’s estimate after the pass |
results[].estimatedHeldDays | integer | The estimated unbroken at-target run ending on this day |
results[].trendDirection | "improving", "steady", "declining" or "unknown" | The direction as of this pass |
results[].pruned | integer | Snapshots pruned by the retention policy in this pass |
results[].nudgesSkipped | "no_base_url" | Why nothing was sent, when sending was on and no link could be built. Absent otherwise, including under --skip-nudges — a pass that mailed nobody because it could not build a link looks identical to a pass where nobody was due, and those are very different states to be in on day 9 |
testers roster and testers status
Both emit one cohort view under cohort, and the view is the same shape the control plane serves a dashboard. The two differ in exactly two ways: roster includes cohort.members and loads no snapshots; status omits cohort.members entirely and loads the last --trend-days of them.
That difference reaches cohort.trend. Direction and every delta come off the most recent stored snapshot, so under roster — which loads none — the direction is "unknown", the deltas are null and the series is empty. Read the trend from status. fragile is the one field that still answers under roster: with no snapshot to read it is computed live from the clock.
The view carries five blocks beside the cohort’s own fields:
| Block | What it holds |
|---|---|
cohort.roster | Current composition: size, headroom to the cap, and a count per state — invited, accepted, optedIn, lapsed, unreachable, neverLinked. Headroom counts against live states only, so a lapsed member does not occupy a slot |
cohort.estimatedClock | Pithy’s estimate, named estimated* field by field so it cannot be mistaken for Google’s. Carries source: "pithy_estimate" as a literal discriminator, and dayBoundary: "UTC" — Google’s boundary is undocumented and may differ |
cohort.activity | The observed half, read from the auth tables. source: "observed" — this block is fact rather than estimate. Buckets testers by how long they have been quiet, with 8–13 days the strongest silent-uninstall signal |
cohort.projection | Pithy’s forecast. basis says why the forecast reads as it does and why any null in it is null; calibration: "default" says out loud that the survival priors are numbers Pithy chose rather than values fitted to your data; method is named and versioned, because a chart must not splice two methods into one line |
cohort.trend | Direction, four deltas, and the daily series |
Testers reference has the full field list.
Errors
The capability is not configured, or will not load — classified rather than assumed, exactly as the other provisioning commands classify it.
--env dev on provision or deprovision. Refused with the reason: there is no dev to deploy to.
A numeric flag that is not a whole number above zero, or a --platform outside the two. Checked at the flag, so the answer names what you typed.
A cohort that does not resolve. By name or by id, and the refusal says which were tried.
An address already on the roster in a live state. Refused rather than quietly resetting the date the streak is measured from.
Cloudflare credentials are missing, on the two commands that deploy, and on any roster command pointed at a deployed environment.
The project has no name. Every deployed name leads with it.
Examples
pithy testers create closed-test --store-url https://play.google.com/apps/testing/com.example.app
pithy testers invite closed-test --email ada@example.com --email grace@example.com
pithy testers run --cohort closed-test
pithy testers pending closed-test
pithy testers status closed-test --trend-days 14Then, once the daily pass should run without you:
pithy testers provision --env prod