Shotgun-Insight How it works

Save money & improve scores - real examples

Skeet shells . . .

Tests were carried-out using a Miroku 6000 skeet gun. This is probably 20 years old but has seen little use. However, see the later comments about the chokes.

Olympic Skeet Shells.

These results show how Hull ProSkeet (a medium price shell marketed for skeet) compares with a 'general' 24g No9 - Winchester X3 that is quite low cost. Before analysing the patterns the average pellet count of three cartridges was found to be ProSkeet = 473.3 and WinX3 = 555. The Eley Shooter's Diary gives 492 pellets for a 24g load so the ProSkeet are slightly over-size and the Winchester slightly under. This makes it hard to compare like with like because the larger number of pellets will always be able to cover a larger area but the pattern plate does not record striking energy which is where larger pellets have the advantage. Having said that, a gap in the pattern is a gap in the pattern and it doesn't matter what the energy of the pellet is if they all miss the clay!

The centre-peg for skeet is 63' (21 yards) away. At this distance a skeet gun may have a very wide spread. In order to capture almost all of the shot on the pattern plate, the target was shot at 17yds and then the results adjusted to simulate 21 yards.

The two screen shots shown above are of the same data, but scaled to simulate different distances.

As already suggested, looking at a single pattern reveals very little. The screen shots below show the average of 10-shots for each cartridge.

ProSkeet 10-shell average (21yards)

Winchester X3 10-shell average (21 yards)

The '75% Diameter' figure shows both shells have a very similar spread and shot to shot variation.

As can also be seen, neither shell gives guaranteed coverage in the inner 10" diameter area. This is not so good for a skeet gun and shell when a top shot might regularly expect to get 100ex100.

To simulate a tighter choke, the analysis distance was reduced to 19 yards and then 17 yards. Even at 17 yards neither shell gave 100% coverage of the central area every time (unlike the sporting shells with the B25). This may be because at some time in its life the gun appears to have had 'spreader' chokes (rifling) and although they have been largely removed there are still traces left in the bore. Shown below are the average probabilities of a hit when scaled to 19 yards.

ProSkeet 10-shell average (19 yards)

Winchester X3 10-shell average (19 yards)

These are probably better results for skeet and suggest that the gun could benefit from some choke. Note that the Winchester shell gives a higher probability of hitting the clay whichever method is used to weight the 30" circle (see Terminology for explanations of the different rating regimes). This of course assumes that the smaller shot breaks the clay! Given that the Winchester is significantly cheaper it makes the best buy.

28g No.9 Shells.

These are just indicative results using only three fibre-wad and three plas-wad 28g No.9 to allow a quick comparison. With a sample size of three not too much should be drawn from the results. However, it can be seen that neither shell does a convincing job of covering the centre of the target. Having found that the Hull DTL300 trap loads group a lot tighter than other loads in the Browning the next step will be to try the Hull DTL300 with No.8 shot to see if it tightens-up the spread a little in the Miroku skeet gun.

ProFibre 28g No.9 at 21 yards (3-shell average)

ProOne 28g No.9 at 21 yards (3-shell average)

As can be seen, the fibre-wad gives almost the same 75% diameter spread as the plas-wad 24g loads. The plastic-wad 28g proOne gives a significantly tighter pattern compared to all the other shells suggesting that if this is confirmed by a larger sample size this would be a cartridge better suited to this gun. However, the relatively low pellet count (for a 28g No.9) probably means it will not match the pellet coverage of the Winchester 24g load.

Both 28g shells contain almost the same number of pellets so direct comparisons between the probability of a hit target are valid. It will be noted that the plastic wad variant gives a lower variance when considering the probability of a hit, but the fibre-wad gives the better consistency when looking at the variance in the spread. This is why the hit probability figures are included. In the end, the most important thing is consistently putting pellets on the target.

The variance in the fibre-wad hit probabilities is mainly due to one of the patterns (not shown explicitly here) having a very poor coverage in the centre area. Some people suggest that fibre wads can disrupt the pattern giving a hole in the centre of the pattern. Further investigation would be needed to confirm this, but of all the shells tested this one pattern gave the worst coverage in the centre of the pattern by a large margin.

Three targets is not enough to draw too many conclusions, other than: The skeet gun would work better with either of these shells if it had some more choke. If this cannot be done the ProOne with the tighter pattern is the better shell for a moderate to good shot, or one who is a bit slow resulting in the second clay on stand 4 being more than 21 yards away.

See skeet shells ongoing summary for the latest status of testing for skeet shells!

Sporting shells . . .

Tests were carried-out using a B25 with 1/2 choke (Teague long choke) at 25 yards distance. Again, this distance was chosen to ensure almost all of the pellets were captured.

28g Shells.

The shells tested were:

Express High Velocity 28g 7.5 - a low cost shell at about £86/1000 (in the UK);
Hull ProOne 28g 7.5 - a medium price sporting shell at about £105/1000;
Hull ProOne DTL300 28g 7.5 - these are the same price as the ProOne but a lot milder on the shoulder and marketed as a, surprise, surprise, Down-The-Line shell;
Express World-Cup 28g 7.5 - these are relatively pricey at about £116/1000 these have the trappings of a quality shell with a relatively high head of 16mm and high antimony content of 7%.

For the sake of thoroughness the average pellet count of each cartridge was found to allow traditional measures of pattern efficiency to be calculated if needed. This produced the first surprise. According to the Eley Shooter's Diary a 28g load with No.7.5 (English) should contain approximately 400 pellets. The three-shell average was found to be 417.7, 396, 398.7 and 291 for the Express HV, ProOne, DTL300 and World-Cup respectively. The World-Cup shot size was actually nearer to 6/6.5 shot - this makes direct comparisons regarding pellet coverage pointless because they are effectively different shells best suited to different distance and choke scenarios. However, consistency in terms of variation in the spread can be compared.

The second big surprise came regarding the Express HV shells. The first four targets were shot about four weeks before the last six and seemed to be completely different shells! The first batch had ~417.7 pellets as counted by hand, the second batch had ~350 pellets as captured by the pattern plate. Again referring to the Eley Shooter's Diary, ~350 pellets is the load expected of a number 7 sized shot load. Because of this the two groups of results have been kept separate.

Express HV (4 shell average with 417.7 pellets per shell)

Express HV (6 shell average with ~350 pellets per shell)

Hull ProOne (10 shell average)

Hull ProOne DTL300 (10 shell average)

Express World Cup (5 shell average)

Some salient points from the above data:

ProOne DTL300 gives a significantly tighter pattern. Referring again to the Eley Shooter's Diary and expanding the data to 40 yards, the DTL300 places ~70% of the shot inside the 30" circle which is approximately full choke. This could be useful for shooters with fixed choke looking for a shell that can throw a reliable pattern further. Note how the tighter pattern contributes to the almost 100% hit probability inside the 10" diameter area - just what trap shooters need!
The Express HV and Hull ProOne are very similar but since Express HV costs considerably less this is the better value shell.
The first batch of Express HV (the 4-shell group) gave the best and most repeatable coverage using the hit probability figures. It is a shame there wasn't a 10-shot group to confirm this and a double shame that the cause of this was the gross change in the pellet load by the manufacturer so maybe one cannot even get the first ~400 pellet shell anymore!
Express World Cup simply didn't have the pellet count to give the hit probabilities of the other shells. The pattern was tighter than all but the DTL300 but the variance in the spread was the worst of the group. In this gun this certainly isn't worth the premium over the other shells.

24g Shells.

Two 24g 7.5 loads were also tried:

Hull Chevron - low recoil and low cost shell at approximately £84/1000;
Gamebore Patriot F2 - the most expensive 24g cartridge in the shop at approximately £122/1000!

Those who are recoil sensitive might like to consider 24g loads for sporting. Some people attach mythical powers to 24g loads due to the shorter shot column. Now this can be investigated.

Again, the average pellet content was counted by hand from three shells. And again, the 'premium' shell used oversize shot. The Eley Diary suggests ~335 pellets in a 24g 7.5 shell. The Hull Chevron yielded 331.7 pellets but the Gamebore Patriot only 260.7. This was a shame because the Gamebore had the most beautiful shot of all the shells tested. The pellets were polished, seemed "non-stick" and were noticeably better formed. They were almost like ball-bearings! As will be seen the performance was good and in due course No.8s will have to be tested to see how the shell performs with smaller pellets. This is important because it could simply be the larger shot deforms less and this gives tighter and more consistent patterns rather than the powder/wad/shot combination being particularly suited to the gun. The Patriot also had a lovely recoil action - crisp but not heavy.

Hull Chevron (10 shell average)

Gamebore Patriot F2 (10 shell average)

The above results show that the Patriot load groups significantly tighter but there is not much difference in repeatability of spread (as a percentage of the average spread). Beyond that, it's difficult to compare the two other than to say that at distances where true 7.5 shot will reliably break a clay the Chevron load will have a higher likelihood of a kill largely due to the higher pellet count. In fact, bearing in mind the difference in cost the Chevron looks really good.

See sporter shells ongoing sumary for the latest status of testing for sporting shells!

Calibration Target . . .

A calibration target can be used to confirm the measurement process. Shown below is a screen-grab of a hand prepared target with seven points each separated by 5" aligned horizontally and seven points aligned vertically (also spaced by 5") with the centre point of each line coincident.

Calibration target with scale markers set on +/15" points.

Scale markers positioned on +/-5" points.

Scale markers positioned on +/-15" points.

Vertical scale markers at top right hand and bottom right hand corner of target (43.25" apart). Horizontal scale markers on extreme edges of the target along centre (39.5 " apart).

The true value for the spread (calculated by spreadsheet) is 7.64". There is a small change in the reported results dependent on where the scale markers are placed. This is due to the distortion caused by the digital camera. This is worst at the extreme edges/corners of the target. The last of the 'Pattern statistics' tables shown above is for the case when the right hand side of the paper is used as the vertical scale. The camera 'pinch' at the corners of the target is the worst case. The reported vertical spread of 7.82" is 0.18" (2.35%) too high. This is tolerable in the context of shotgun patterns. Furthermore, since the typical user will be comparing one shell against another, if the scale points are taken from the same position on each target the relative error will be much smaller. Alternatively, scale markers could be added to the target in-board of the edge of the paper to improve the absolute accuracy - that is if knowing the pattern to better than 0.2" really is a requirement!

Terminology . . . .

More data is not necessarily good data! Having got the ability to measure the pellet placement, what indicates that one shell works better than another? First, one needs to understand how the pellets are distributed - the Normal Distribution; second, one needs to understand the outputs from Shotgun-Insight and finally one needs to appreciate statistical significance.

The following terms used in the average results panels shown in the earlier screen shots will also be explained:

Spread x/y/75% Diameter; 'sigma', +/-, shot to shot variation'; Probability of hit 0-10" 0-20" 0-30"; Centre weighted sum; Centre weighted norm;

Brief overview of the Normal Distribution.

The Normal distribution features strongly in this analysis of shotgun patterns. The picture below illustrates its main features:

The vertical axis of the graph shows the relative probability of an 'event' happening versus its distance from the mean (average) which in this example is shown as zero. It will be seen that the likelihood of an event occurring is greatest nearest the mean. The distance +/- "1-sigma" accounts for approximately 68% of events, the distance +/- 2-sigma for 95% of events and the +/- 3-sigma range accounts for 99.7% of all events. This type of distribution model fits many naturally occurring observations, for example, heights of people, test scores etc. Oberfell and Thompson also found that the average pattern distribution from a shotgun could be modelled by the Normal distribution. It is important to note that the Normal distribution does not define the pellet distribution, it is just a model that fits the observed distributions quite well. The closeness of the pellet distributions (shown by the histograms of Shotgun-Insight in the Average Results Panel and Analysis Graphic) and the ideal Normal distribution should be convincing enough that the Normal distribution is a useful tool for characterising the pellet distributions.

The value 'sigma' is used to indicate the width of the distribution. The value of sigma can be used to compare one shell against another for a given choke to see which shell delivers the tightest pattern and hence which will be able to carry a useful pattern the longest distance. Alternatively, skeet shooters or fixed choke users may be interested in obtaining the widest spread possible.

To demonstrate that the shell and gun combination work well together, the patterns should be as repeatable as possible in much the same way as a rifle shooter looks for minimum variation shot to shot. The big difference is that rather than measuring a single distance (the maximum spread of a group of rifle bullets), the shotgunner's measure of repeatability is how all the pellets behave together. One way of doing this is to fit all the pellet positions to a model and then look at how the parameters of the model vary. In this case the Normal distribution is employed as the model.

A measure of repeatability of the pattern is given by how the spread - sigma - varies from shot to shot. Perhaps surprisingly to the uninitiated, the distribution of the values of the spread (or sigma) is itself a Normal! (If you are interested do a Google search for the phrase 'Central Limit Theorem' with a few words like 'explanation' included) Thus we can have a 'sigma' value for the variation of the spread of the pellet distributions as measured shot to shot! However, even when using perfect computer generated data there is some difference between successive measurements. Considering the flipping of a coin, although the long term average will be 50:50 heads to tails, a sample of 10 flips may yield something quite different. Statistical significance is a term used to describe how confident one can be that the measured result is genuine and not a statistical outlier.

Output from Shotgun-Insight.

Spread x/y: This is the 'sigma' value of a Normal distribution made to fit the real pellet distribution as closely as possible. From the above it is seen that a spread of 5" means approximately 68% of the shot falls within +/-5". The pellet spread is described in terms of horizontal and vertical spreads just in case there is a systematic problem with the gun and the spread is significantly larger in one dimension than the other. The '75% shot diameter' converts the spread x/y values into a diameter that on average would contain 75% of the shot. This is a convenient single number that can be used to summarise the spread.

Probability of hit (0-10", 0-20", 0-30"): This is the likelihood that the clay would be hit by at least one pellet. 0-10" considers only a 10" diameter circle about the centre of the pattern. A consistently good shot who centres most of his shots would want to see as high a likelihood of a hit in this region as possible, perhaps at the expense of a poorer probability of a hit further away from the centre. 0-20" and 0-30" are the same but consider larger diameters. A poor shot or someone shooting game that follows a very unpredictable flight might be more interested in a good coverage across the whole 30" diameter circle and be prepared to trade some coverage in the centre 10" to achieve this. The spread in the average values of each of these figures is also given. Obviously 100% with a sigma of 0.0 is the best one can get! An average hit probability of 95% with a sigma of 1% is better than one with an average of 95% but a larger sigma.

Centre weighted sum: The 0-30" probability of a hit treats all areas of the 30" diameter circle equally. An area with no pellets has the same affect on the hit probability figure regardless of whether the pellet free area occurred in the centre or on the edge of the pattern. Instinctively, one can appreciate that this is not right. If the shooter centres the pattern on the target he expects the pattern to do the job. It is not unreasonable to think that the centre of the pattern is 'worth more' than the edge. The 'Centre weighted sum' takes the 0-10" hit probability, the 10-20" hit probability (not shown individually in the results screens) and the 20-30" hit probability (also not shown explicitly) and gives the simple average of the three. Because the centre 0-10" area is one third of the 10-20" area and one fifth of the 20-30" area the simple averaging of the hit probabilities weights the centre much more than with the 0-30" figure. An alternative way of interpreting this number is that for a shooter who places the target in the centre 10" of the pattern 1/3 of the time, the 10-20" region 1/3 of the time and the outer 20-30" area of the pattern 1/3 of the time, the 'Centre weighted sum' shows the average effectiveness of the shell for this shooter.

Centre weighted norm: This weights the centre even more heavily than the centre weighted sum. The centre is weighted at ~68%, the 10-20" annulus at ~27% and the outer annulus at ~5%. The rationale for this is: If one thinks of a rifle shooter shooting at a bulls-eye target, the distribution of shots about the centre of the target will be approximated by a Normal distribution. That is, the natural variation of the shooter also leads to a Normal type distribution of the shot. Assuming the shot-gunner is the same, the relative weighting of the shot pattern should reflect the likelihood that the clay/bird falls in that part of the pattern and this relative weighting is the Normal distribution (again!). I assume that the shooter can keep 99.7% of his shots within the 30" circle (a tall order but probably not far off for good trap and skeet shooters). For good shooters of these disciplines who get high scores the 'Centre weighted norm' shows how the shell supports them. For these disciplines where high scores are common and needed to win, the centre of the pattern is obviously very important - hence the 68% weighting!

'sigma', '+/-' or 'shot to shot variation': For all of the averages the standard deviation or 'sigma' is calculated. Once again it is assumed that the distribution of the values being averaged follow a Normal. An average hit figure 90% with sigma of 2% means the average of the shells is 90% and ~68% of the shells fall within +/- 2% of this. Clearly an average of 90% with a sigma (or shot to shot variation) of 1% is better than an average of 90% with a sigma of 5%. The +/- or sigma value for the averages gives an indication of how repeatable the performance is.

Statistical significance.

Firstly, statistical significance is a tricky concept to get one's mind around. It's not needed for basic analysis but will be needed to decide finer points such as whether one choke design is better than another.

Consider the flipping of a coin. We all know that in the long run over many 100s or 1000s of flips, half the time it will land heads-up and half the time tails-up. However, for a smaller sample, say 10-flips, a quite different ratio of heads to tails might be obtained. Further 10-flip samples will give different ratios. It does not mean there is anything wrong with the coin, flipping or our counting, its just down to chance exactly what we get with 10-flips. In the long run if all the 10-flip samples were averaged, they too would give 50:50 heads to tails. In the short term there will be some variation about this.

Now consider the spread of a shot-shell. We know that in the long-run the distribution of pellets about the centre of the pattern can be described by the Normal distribution with a width 'sigma'. In the short term our estimates of this long term sigma value will tend to vary a bit. Again, it does not mean anything is wrong, it is just what happens when a small sample is used to estimate a long-term trend.

In the 'Average cartridge results' display Shotgun-Insight reports the average spread of the shells and also the 'shot-to-shot' variation about this average. The variation is expressed as the '1-sigma' value. For example, if a sample of shells had an average spread of 5.7" with a 1-sigma value of 0.45", this means 68% of the shells should have a spread within +/-0.45" of 5.7".

As more shells are tested the estimate of the average spread becomes more reliable. Intuitively this is obvious since averaging more shells will tend to reduce the effect of a 'one-off' bad shell.

Now there is a little bit of maths to quantify the benefit of testing several shells: The 1-sigma uncertainty in the average is given by the shot-to-shot 1-sigma value divided by the square root of the number of shells tested. If nine shells were tested for the example given above, the average would be estimated as 5.7" +/- 0.15" (1-sigma). This means that for each subsequent group of nine shells tested 68% should have an average within 5.7" +/-0.15". If we averaged 100 shells, the estimate of the average spread would be 5.7" +/-0.045".

It is not proposed for one minute that anybody average 100 shells. But the key point is if you want to have a very accurate estimate of the long term width of the spread of a pattern you need to average a (possibly large) number of shells.

In practice you probably do not need a very accurate estimate of the long term spread. If you find that two different shells have a very similar spread based on averaging a few of each, you will (correctly) decide that the difference for all practical purposes is irrelevant.

Statistics can help when you want to decide if two different shells are 'genuinely' different given the data available. Recall that for a given event or group under study the '+/- 1-sigma' value accounts for 68% of the occurrences. The '+/- 2-sigma' value accounts for 95% of occurrences. If we take the 2-sigma value, we can say that an event occurring just beyond this limit has a 5% chance of belonging to the first family of samples, or we are 95% confident it does not. This is known as the 2-sigma or 95% confidence limit. Similarly there is the 3-sigma or 99% confidence limit. This means that if the average of one group of shells is 2-sigma (sigma found by using the shot to shot sigma divided by the square root of the number of samples) away from another average, we can be 95% confident that the difference is genuine and can be repeated.

This is statistical significance. You may have heard of it in the news, now you know how to calculate it.

These are powerful concepts but do not worry if it seems a bit opaque! Many an engineering student has been rent asunder by statistics. For the most part, simply reading the average spread and '1-sigma' values from Shotgun-Insight along with some practical sense will suffice.

References.

George G. Oberfell & Charles E. Thompson. The Mysteries of Shotgun Patterns. Oklahoma State University Press, 1957.

Eley Shooter's Diary. Eley Hawk Limited. www.eleyhawkltd.com