Printed Resistor Acceptance: Rounding Cannot Decide a Borderline Result

Measurement purpose

Verify that software and reports implement the approved resistor acceptance rule consistently, without adding or removing margin through rounding.

Specimens and conditions

Controlled requirement

Identify drawing revision, nominal resistance, absolute or percentage limits, reference temperature and whether endpoints are inclusive.

Valid observation

Use a measurement with known units, contact definition, acquisition identity and uncertainty information. Invalid or stale readings are not numerical borderline cases.

Equipment and records required

• Decision implementation: Inspect the actual calculation path, numeric representation, unit conversion and stored precision used before report generation.
• Boundary test dataset: Prepare synthetic values immediately inside, exactly on and immediately outside both limits, plus invalid data and equivalent alternate-unit values.

Method sequence

1. Resolve the rule

   Translate the approved tolerance and uncertainty treatment into explicit inequalities.

   Record: Requirement revision, limits, endpoint convention and responsible approval.

2. Exercise numerical boundaries

   Evaluate synthetic cases through the same production decision code and compare with independent arithmetic.

   Record: Inputs, expected outcomes and actual outcomes before display formatting.

3. Verify reporting parity

   Export the same results in available report formats without recomputing acceptance from rounded text.

   Record: Stored result, displayed value, outcome and rule version.

Decision and uncertainty

A result is accepted only under the pre-agreed numerical and uncertainty rule. Formatting must not alter the computed outcome or suppress an unresolved case.

Rounding is not a substitute for measurement uncertainty. A guard band changes the acceptance interval only when its purpose and magnitude are authorized.

The customer specification and authorized quality function control the decision rule; report designers cannot redefine it by choosing decimal places.

Traceable outputs

Start with a pair that rounds to the same display

Assume, for illustration, a 1,000-ohm resistor with an inclusive upper limit of 1,010 ohms and no additional guard band in the stated rule. Measurements of 1,009.96 and 1,010.04 ohms both display as 1,010.0 ohms when rounded to one decimal place. The first is within the interval and the second is outside it under that simple rule.

A displayed number alone cannot reproduce that distinction. Keep the unrounded result and the original outcome together. The report may show additional digits, a suitable explanation or an uncertainty statement, but it must not quietly convert the second measurement into a passing result because the formatted strings match.

Calculate percentage limits from the nominal value

For symmetric percentage tolerance, compute the lower and upper specification limits from the nominal value using the approved percentage convention. Do not repeatedly apply the percentage to the measured value. For an assumed 1,000-ohm nominal and one-percent tolerance, the specification interval is 990 to 1,010 ohms.

Some networks instead have asymmetric limits, ratio requirements or temperature-dependent acceptance tables. Store these directly with their units and conditions. A universal plus-or-minus field is not sufficient for every printed resistor specification, and an operator should not reconstruct the intended rule from the number of digits printed beside the nominal value.

Keep a guard band separate from rounding

Suppose an agreed illustrative rule narrows the preceding interval by a guard band of 0.20 ohm at each end. The acceptance interval becomes 990.20 to 1,009.80 ohms. A measurement of 1,009.96 ohms is inside the specification interval but outside this acceptance interval. That is a consequence of the agreed rule, not a change in the physical resistor.

Do not automatically use expanded uncertainty as the guard band. Different decision rules allocate different risks and may include conditional or indeterminate outcomes. The numerical example only shows where a chosen guard band enters the comparison. Its selection requires the authorized requirement and an appropriate uncertainty model.

Verify exact edges rather than testing only nominal parts

A test dataset centered on the nominal value rarely catches inequality errors. Include exact endpoints and deliberately close neighbors. Use sufficient internal precision to represent the test inputs, and document any decimal-to-binary conversion policy in the implementation.

Do not parse the printed report back into the decision

Keep numerical values and units separate from display strings. A string containing a comma may mean a thousands separator or a decimal separator depending on locale. Converting it back to a number without an explicit convention can produce a much larger error than ordinary rounding.

Use a tested decimal representation or an explicitly defined internal scaling and comparison policy where appropriate. Do not solve floating-point edge behavior by adding an arbitrary epsilon to every tolerance. That silently widens acceptance. Any numerical accommodation must preserve the authorized physical interval and have boundary tests demonstrating its effect.

Keep trim targets distinct from outgoing acceptance

A laser-trim station may stop before the final acceptance boundary to allow for endpoint overshoot or subsequent settling. That process target is not necessarily the customer's upper specification limit. Store the target, process stop criterion and outgoing acceptance interval as different quantities.

If displayed rounding is used in the trim feedback loop, multiple distinct measurements can appear to reach the same target. Increasing report precision afterward does not recover the lost feedback information. Review the actual instrument-to-controller data path, including filters, update timing and any conversion performed by the trim software.

Preserve a changed decision instead of overwriting history

When a rule changes, retain the old result and the reason for any authorized re-evaluation. A new drawing revision, corrected unit conversion or revised uncertainty treatment can justify a new decision, but it does not mean the original measurement was taken under the new rule.

Keep the measurement identifier constant while adding a separately versioned evaluation. This allows a customer to distinguish fresh remeasurement from recalculation of existing data. It also prevents a spreadsheet refresh from rewriting earlier lot-release evidence without a visible change history.

Specify enough precision for a reproducible handoff

Agree the internal retained precision, customer display convention and uncertainty presentation before comparing laboratories. More displayed digits do not imply greater accuracy; fewer digits should not make acceptance impossible to explain. The report should communicate both the result and the rule used, while the underlying record preserves sufficient precision for reconstruction.

For a new thick-film drawing, provide representative boundary cases with the required outcome. These small examples are often clearer than a long description of formatting preferences. Validate them in the actual test software and export formats so the specification, measured evidence and customer report remain aligned.