Meteorological Instrument Performance Characteristics

Instrument Performance Characteristics » Precision and Resolution

The random error is related to the precision of the instrument and is also used to determine instrument resolution. The precision of the instrument determines the resolution, and various multiples of the precision are often quoted as the resolution, so it is necessary to consider which convention is used. If the precision of the measurement is given by a value p, you can conclude with about 50% confidence that the measurements are different if they are separated by p. At 2p, you can make the same conclusion with about 2/3 confidence [corresponding to one-standard-deviation confidence], and for 95% confidence, a separation of about 3p is needed.

The term “resolution” has many definitions, some conflicting. This lesson seeks to clarify why characterizing the resolution is difficult, and it will explain some of the varying usages of this term.