Even More Powder Measure data, this time with Math!

This data is different as it is entirely numerically simulated.

I used the random number generator (R) in Excel to simulate some loading results given a maximum weight (Wmax) and minimum weight (Wmin)

(1) R(i) = A random value with linear probability between -0.5 and 0.5; done at time i
(2) Waverage = (Wmax+Wmin)/2
(3) Wdelta = Wmax-Wmin
(4) W(i) = R(i)*Wdelta+Waverage

W(i) is calculated a number of times (n).

Then each value of W(1) to W(n) is "measured".
(5) Rez = Precision of measuring IE 1 for 0.1 grains, 2 for 0.01 grains.
(6) W(i)mrd = round(W(i),Rez)

As one can see equation 6 is the result of rounding the actual value based on the accuracy of the measuring device. Thus W(1)mrd to W(n)mrd is calculated.

Statistics can then be calculated from that list of data.

Example 1 Given: Rez = 1, Wmin = 4.999, Wmax = 5.101, n=200

This means that 200 measurements are taken on a value that ranges between about 5.0 grains and 5.1 grains by a device that can only measure to 0.1 grains. Thus there's only two values W(i)mrd can be.

The results are
Minimum of Set = 5.0
Nominal Average = 5.05
Average of Set = 5.048
Maximum of Set = 5.1
Stdev of Set = 0.050085354
Nominal Stdev = 0.0294
Max-Min = 0.1

The nominal values are the average and standard deviation that would be expected from an idealized infinite data set and no rounding from "measurement" (n & Rez = infinity).

One can see that the standard deviation is quite a bit higher than expected. Note should be taken that since the data is generated randomly the average and standard deviations calculated from the set date will change and vary each time the data is generated.

Let's try with a "scale" that measures a bit better.

Example 2 Given: Rez = 2, Wmin = 4.999, Wmax = 5.101, n=200

This means that 200 measurements are taken on a value that ranges between about 5.0 grains and 5.1 grains by a device that now can measure to 0.01 grains.

The results are
Minimum of Set = 5.0
Nominal Average = 5.05
Average of Set = 5.0497
Maximum of Set = 5.1
Stdev of Set = 0.029517221
Nominal Stdev = 0.0294
Max-Min = 0.1

The only change of note is that the standard deviation is now much closer to the nominal value. In fact depending on the random results the value bounces above and below the nominal.
The range is about 0.030 to 0.028 grains.

Thus a higher resolution in measurement results in a more accurate picture of the data.

I also ran the same test again with the Rez being set to the floating point value Excel uses for internal rounding. This did not have much influence on the Standard Deviation variance. I suspect that the effect was now from having n being too low.

Another interesting effect can be seen for this example
Example 3 Given: Rez = 1, Wmin = 5.049, Wmax = 5.151, n=200

Wdelta is the same as before but now the average weight (Wave) is 5.1 grains. Now there's three values W(i)mrd can be: 5.0, 5.1, 5.2.

The results are
Minimum of Set = 5.0
Nominal Average = 5.1
Average of Set = 5.101
Maximum of Set = 5.2
Stdev of Set = 0.014178
Nominal Stdev = 0.0294
Max-Min = 0.2

Two things are obvious. Despite measuring only to the tenth of a grain like the first example and having the exact same Wdelta, the standard deviation of this set is far lower.

And secondly, the standard deviation of the data is lower than even the nominal standard deviation for this set of data.

This, like the previous example, is entirely due to rounding. While in reality the Weights are varying evenly in a range of +/- 0.05 grains. The vast majority of the data points either round up to or down to 5.1 grains with only a bare handful that round to 5.0 or 5.2

As expected when this is tried:
Example 4 Given: Rez = 2, Wmin = 5.049, Wmax = 5.151, n=200

The results are
Minimum of Set = 5.05
Nominal Average = 5.1
Average of Set = 5.10105
Maximum of Set = 5.15
Stdev of Set = 0.030579
Nominal Stdev = 0.0294
Max-Min = 0.2

The same values for the set standard deviation are seen as in the second example.

This is important to keep in mind. If you are trying to measure data with a variance that falls on the same order of magnitude of as your measurement precision then you could be getting results that seem far more accurate than they really are.

Contrast the results of example 1 and example 3. T he exact same powder range (Wdelta) and exact same measurement precision yield dramatically different standard deviations.

In the latter one would be inclined to believe his data was far more accurate than the former. Despite both cases having the exact same variance.

This can be seen when the range is expanded: Wdelta = 0.2 grains.
In this case the nominal standard deviation is 0.58 grains.

As expected when the resolution is upped to 0.01 grains the nominal standard deviations do match well.

And again when dealing with a resolution of just 0.1 grains one gets one standard deviation well above the nominal (~0.066 grains) and another well below (0.051 grains).

However as Wdelta increases the results get better. This can be expected as if Wdelta = 1 grain then a resolution of 0.1 grains would be just like a Wdelta = 0.1 grains with a resolution of 0.01 grains.

In both cases there are 11 possible values that the weight could round too.

So in short, be careful with your statistics. Even if your instrument is giving you the right value, rounding may be making you think your throws are far more accurate than they really are.


Standard deviations are handy but run into some real limitations when you start to get down towards +/- 0.1 grains.

Unless that is you are measuring at a higher precision than that.