One of the biggest challenges for new and growing schemes is dealing with data. Early on in your operations there is often strong pressure to save money and postpone putting resources into a robust data management system that can grow with the scheme and provide a rich source of data for use in managing the scheme and understanding the impact your scheme is having.

Before thinking about how to start, let's look at the uses for the data.

For certification schemes there are two clear, high priorities for collecting data:

Gathering data to be used in the ongoing management of the scheme. This includes the operation of the business and the performance of your organization (i.e. financial management, numbers of certificates, results of audits, etc...) and all other related organizations that deliver your service (i.e. certification bodies, accreditation bodies, registered consultants, registered auditors, etc...).
Gathering data to be used to understand the impact your scheme is having. That is, how much, if any, is your scheme producing the changes that your scheme is seeking to make. This is to answer the question: "Is the environment or society better off because of your scheme?"

To address the first point, data is crucial to managing your scheme. Imagine trying to manage your scheme with no financial information, you would have no way of knowing whether or not you can make payroll each month or pay your bills. Business management requires data. Business managers often speak about KPIs (Key Performance Indicators). These are the key bits that a manager needs to know in order to determine if the organization is achieving its operational goals.

For business management the sequence of what can be done with raw data looks something like this:

Data - Information - Knowledge - Insight

Data is the raw set of numbers or other points that are gathered. As you process that data you can produce information. Using the information you can begin to ask questions, explore what is happening and gain knowledge about your operations. From that knowledge you can gain insight that you can then use in making operational, tactical and strategic decisions about how to be more successful. For more information on this take a look at ISO 14031, Environmental Performance Evaluation - Guidelines (there are also loads of business books on KPIs). In this area it is important to never confuse raw data with knowledge or especially with insight.

For the process of understanding the impact of your scheme the process looks something like:

Outputs - Outcomes - Impacts

Outputs are the basic statistics about your program (i.e. how many audits were conducted, how many people attended training seminars, how many products are certified, etc...). Outcomes are the second or third order changes that may result from the outputs (i.e. how many people who took your training shared that information with other groups, how many companies sought certification because someone else got certified, etc...). Impacts are the changes that you want your scheme to achieve (i.e. the water is less polluted, are more girls are going to school, is farm income rising, etc...). For more information on this look at ISEAL Alliance's Impacts Code. The full progression is important because while you may be achieving high levels of outputs these are not necessarily a guarantee that you are ultimately having your desired impacts.

In brief, you can run your scheme with limited data only for a short while. Eventually you are going to need robust and reliable data that you can use as a basis for making decisions. One day you will get to the point that you need the data, at this point it is too late to start collecting it. Data collection is crucial and must be started before the board, CEO or a new strategy consultant asks for it.

The big challenge in starting the collection of reliable data is knowing what to collect. To design a good system for collecting data it is important that you think deeply about what you are trying to do and what you are trying to achieve. Your data collection should be focused on what you need to know about, understand and evaluate.

My last point is to encourage you to be open to the possibility that what you learn from the data you collect may challenge some (or even all) of the fundamentals of your scheme. You may be extremely successful in getting companies or products certified, that does not mean however that you are automatically achieving your desired impacts. Use the data to understand, and if necessary, change how you and your partner organizations do your work.

No matter where you are in the process of managing your scheme, but especially when you are just starting out, it is important to start early and to think deeply about data.

ISO defines a standard as: A document established by consensus and approved by a recognized body that provides for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context. (from ISO/IEC Guide 2:1996, definition 3.2)

While this is a good technical definition it does not cover all of the key points that are important. There are a few things that I would add.

First of all a standard should be written for the user, that is it should clearly describe what someone who is going to use the standard needs to do in order to conform to it.

Writing a clear document can be best done when you know for whom you are writing and when you write to that audience. A scientific paper written for a journal is written for an audience of scientists not politicians. A policy brief can be written for politicians but would not contain the detail or extended explanation of the research methodology that scientists need.

A standard should include clear statements for each requirement of:

what must be achieved,
how it is measured and
the level of that measure that must be met.

This clarity is important for two reasons: 1) the user can understand what is expected and 2) when audits take place in many locations by different auditors we know that the same specified requirements are being examined.

For the user's sake the standard should include everything that the user needs to do and nothing more.

A standard that says, for example, that 'logging shall not be done on stream banks' sounds like a good idea but this language is unclear and the user will not know what that means in each case. For example should the distance from the stream be 1m, 5m, 25m or more? Does this distance apply to every class and type of stream? Does this apply to rivers? Where do you measure the stream bank from, is it the high water mark, where the water is on the day the auditor arrives or some other point? Should the cut-bank of a stream be treated the same as all other banks? Does the steepness of the stream bank matter? Does this apply to a watercourse that is seasonal?

A standard should be written in language that is easily understood by users. For example, a standard for cryptography will most likely be written in highly technical language of the sort that mathematicians will understand but us mere mortals will not be able to follow. While a standard for growing tomatoes will use terminology that is known to farmers and farm managers and may not make much sense to mathematicians (except for the small subset of mathematician-farmers out there).

Finally, a standard should not include requirements that apply to others, including the auditor. Instructions to the auditor about how to conduct an audit should be in a separate document written just for auditors.