Pilot-result statistical interpretation

Zero-Failure Pilot Builds: What the Sample Can Actually Establish

Interpret an all-pass prototype sample using an exact one-sided binomial bound, a fixed failure definition and an explicit population rather than claiming zero production risk.

Send Drawings6 min read
High-resolution industrial engineering scene showing resistor lot sampling for thick-film ceramic circuit review.
Engineering illustration; not a product photograph or a test result.
On this page

An all-pass pilot is useful evidence, but zero observed failures is not the same as a zero failure probability. The statistical strength depends on how many independent units were evaluated, what counted as failure and which population the trial represents. A transparent bound lets the engineering team see what the pilot establishes and what additional evidence a stricter requirement would need.

Key design decisions

  • Define failure per unit and freeze the sample plan before observing outcomes.
  • Use an exact one-sided bound for a zero-failure count instead of a normal approximation that collapses to zero.
  • Separate repeated readings, repeated cycles and separate production units.
  • Keep the statistical population and tested exposure attached to the resulting bound.

1. Convert the requirement into one unambiguous unit outcome

Specify which checks a pilot unit must complete to count as passing. If the requirement includes several electrical and physical characteristics, define whether failure means missing any one of them or a named individual characteristic. Those choices produce different probabilities and must not be mixed after the results are known.

Use a fixed measurement stage, threshold and retest rule. A unit that first failed and passed after rework is not an original first-pass success. An unreadable result or a unit removed before the complete sequence needs an explicit disposition, not automatic inclusion among the passing units. Retain the event history even when a separate repaired-population result is useful.

2. Derive the upper bound from the chance of seeing no failures

For n independent units sharing an unknown probability p of the defined failure, the probability of zero failures is (1 − p) raised to n. To find the exact one-sided upper confidence bound after observing zero failures, set that probability equal to the selected tail probability α and solve for p.

The resulting bound is not the probability that the particular lot is good, and it is not a posterior probability assigned to p. Its confidence level describes the long-run coverage of the stated statistical procedure under its assumptions. State explicitly that the result is one-sided; a two-sided interval allocates the tail probability differently and gives a different upper endpoint.

P(X = 0 | p) = (1 − p)^n; pU = 1 − α^(1/n)

  • X is the number of units with the predeclared failure in n completed independent trials.
  • p is the common failure probability for the represented population and condition.
  • pU is the upper one-sided confidence bound; confidence level is 1 − α.

Fixed sample size, independent identically distributed binary outcomes, complete observations and a predeclared failure definition. A finite-lot sample without replacement or a dependent production cluster requires its appropriate model.

3. Calculate what an all-pass small pilot leaves unresolved

For an assumed pilot of 20 independent units with zero failures, a 95% one-sided confidence level uses α = 0.05. The upper bound is approximately 0.1391, or 13.91%. That is much wider than the observed zero-percent sample proportion. It shows the limited information available from twenty successes about a small underlying failure probability.

With 60 independent units and zero failures, the corresponding bound is approximately 4.870%; with 100 it is approximately 2.951%. These are hypothetical sample-size calculations, not reported factory results. A normal approximation based directly on the observed proportion can produce a zero-width interval at zero failures, which misses the unresolved probability entirely. Use the selected exact method and retain its assumptions.

4. Work backward from a required upper bound

If the purpose is a planned zero-failure demonstration against p0, choose a sample size satisfying (1 − p0)^n ≤ α. Equivalently, n must be at least ln(α)/ln(1 − p0), rounded upward. The calculation assumes the complete planned sample finishes with no failures; it does not guarantee that outcome.

For an assumed five-percent upper-bound target at 95% one-sided confidence, the smallest integer is 59 units. A one-percent target requires 299 units under the same model. These quantities express the requested evidence strength, not a universal prototype build quantity or an acceptance-sampling rule. If failures occur, retain them and use the preplanned analysis; do not continue invoking the zero-failure formula.

5. Count the opportunities represented by the physical experiment

Twenty resistance readings on one printed circuit are not twenty independent circuit outcomes. Nor are twenty repeated exposures of one unit necessarily equivalent to twenty separate units completing one exposure. Repetition can improve knowledge of measurement stability or a particular unit's behavior without supplying the same population information.

Shared panel history, paste lot, fixture and operator can also create dependence. Preserve those identities and use the pilot's hierarchical sampling analysis to assess what can reasonably be treated as independent. Do not fix dependence by inventing an effective sample size from the number of spreadsheet rows. If only one build condition is represented, limit the conclusion to that condition instead of extending it to every future material lot.

6. Keep stopping rules and exposure scope fixed

A fixed-size confidence calculation is not automatically valid for a rule that repeatedly checks the result and stops as soon as a favorable bound appears. Sequential monitoring needs a procedure designed for that stopping rule. Similarly, restarting a count after an inconvenient failure discards information and changes the experiment.

For an exposure test, each unit's pass must refer to the same defined endpoint or an explicitly modeled observation scheme. Ten units completing a short sequence cannot simply be added to ten completing a longer sequence to claim twenty passes at the longer endpoint. A bound on failure by one exposure duration does not establish a constant failure rate, mean time between failures or untested service life.

7. Choose the correct interpretation for the available evidence

Use the calculation only when its sample and outcome assumptions match the record. Other useful engineering results can remain valid without being forced into a binomial demonstration.

All-pass pilot results and their proper interpretation
RecordValid next analysisDo not claim
Zero failures in a fixed independent sampleExact one-sided bound for the defined populationZero failure probability
Many repeats on a few unitsMeasurement or within-unit repeatability assessmentA large independent unit sample
Sample spread across correlated panelsHierarchical model or justified independence reviewIndependent trials from every reading
Some units reworked before final passSeparate first-pass and repaired-population outcomesAll units passed on the original route
Different completed exposure durationsEndpoint-specific or suitable censored-data analysisAll units survived the longest duration
Testing stops when a desired result appearsPredeclared sequential procedureUnchanged fixed-size confidence coverage

8. Use the bound alongside the actual transfer decision

The bound answers how much a particular all-pass sample constrains one probability under its model. It does not explain a failure mechanism, measure dimensional process capability or decide whether an incoming lot should be accepted. Those tasks need their own response data and decision plans.

For a transfer review, report the observed numerator and denominator, the exact method, confidence level, population, test endpoint and exclusions together. Compare the resulting bound with the agreed evidence objective. If it is too weak, choose the next build to add genuinely relevant independent information rather than merely increasing the number of repeated measurements. Keep the final manufacturing decision linked to both statistical evidence and the demonstrated process configuration.

Send the pilot outcome definition and sample record

Provide the actual independent units and completed checks needed to interpret an all-pass result.

  • Predeclared per-unit failure definition and the stage at which it is applied
  • Fixed planned sample size, actual completed units and original first-pass outcomes
  • Lot, panel and unit identities with repeated measurements separately labeled
  • Required upper probability bound and one-sided confidence level
  • Exposure endpoint, early removals, missing records and permitted retest or rework rules
  • Whether the decision concerns process transfer, a finite incoming lot or a lifetime endpoint

The drawing-upload form loads as you reach this section.