One challenge of a scheme owner is to determine the most appropriate balance between the intensity of the audit and cost. In short, how many examples must an auditor check to ensure that there is conformity? Keep in mind that every additional thing an auditor is required to do will cost the client more (and therefor everyone else in the value chain).
To be blunt, finding a non-conformity is a bit like looking for your lost keys...once you find them you can stop looking. The only difference is that the auditor has no knowledge of how many 'keys' he is looking for.
If the auditor is auditing a large forestry operation and she finds a non-conformity at the first harvest site she will continue to check the rest of the sites to be audited to determine whether or not the non-conformity she found is a single error or if it occurs at each site. One feller buncher operating too close to a stream may be a minor issue (i.e. an operator that is having a bad day) but all feller bunchers operating too close to many streams can be a major issue (i.e. the company told them to do it).
OK, that sounds simple but how many sites should the auditor select for inspection in the first place? Should the auditor inspect all sites, half of them, or just one?
The "Square Root Rule"
It is a common practice for auditors to choose the number of sites based on the "square root rule"; this simply means that the number of sites to be audited is the same as the square root of the total number of possible sites (NOTE that this also applies to the number of files, employees to interview or other sets of things to be checked). If there are 16 sites, for example, then 4 should be checked. If the square root does not give you a whole number then round up the number of sites to the nearest home number. If there are 10 sites (the square root of 10 is 3.16...) the number of sites to be checked should be still be 4.
This is all well and good and the square root rule is easy to follow and easy to calculate (just about every calculator has a built in square root function). The square root rule also gives you a small number of sites to check so that the cost of an audit can be kept low (fewer sites to check means less auditor time equals a lower audit fee). But, what about the reliability of the sample?
Using Statistics to Determine Sample Size
If we change hats and look at the question of sample size using the mind of the statistician we may wish to question the reliability of the square root rule.
If your scheme decides that you want to have a sample that gives you, for example, 95% confidence, plus or minus 5%, then your sample requirement will look much different.
As you can see, the number of sites necessary to audit to achieve the statistical confidence that is like that of academic research is far greater than applying the square root rule. The smaller the sample set the less statistical confidence that can be obtained.
To calculate the number of sites, files, employees or other things to audit for a specific number there are many online tools to do it for you (i.e. http://www.surveysystem.com/sscalc.htm).
The Real Issue
The primary question that you as the scheme owner should consider is: "What level of certainty do I need?"
The answer for this is in understanding the industry sectors that your scheme involves. What do they need? What do their customers need? What do the users of the products and services that they produce need?
What is the trade-off between cost and accuracy - i.e. how much is enough?
It is important to understand what happens in sample selection, both how the sample size is determined and how that number of samples is selected. More is not always better. One way to look a is is to see the best as meeting the needs of your users both in terms of accuracy and cost.