Referencing other Standards in your Standard

When referencing a standard written or published by another organization, the first issue to keep in mind is that of copyright. Quoting a standard owned by another organization may make them angry. Some standards organizations are essentially publishing houses, this is especially true of ISO. They make their money by selling standards that have been written by volunteers from other organizations. Including somebody else’ copywritten text in your standard can lead to a world of hurt.

Public access to standards has become a very big deal in the US and has led to years of court battles due to the practice of governments adopting voluntary standards as regulation. This means that the standard becomes a regulation that must be followed. The problem is that to access the now governmental requirement you may have to pay private companies for a legal copy of the standard.

This leads us into the difference between 'voluntary standards' vs. 'mandatory standards'. If you need to go too far down this rabbit hole, just start a Google search. Mandatory standards are common in some fields, especially related to food safety.

Back to referencing another standard in your standard.

Here are a few suggestions on how to do it and what the pitfalls may be and how to avoid them:

1. In brief, my best advice is: never quote another standard in your standard. Just reference the standard, part of the standard or just one requirement that you want to include in your requirements.

2. The 'normative references' section of your standard should list the documents that you reference, either in whole or in part, in your standard as normative. (For more on what we mean by normative see my post on ‘normative vs. informative’.)

3. You can always write your own requirement, even if it covers the same area as in another standard - just don't copy their text.

4. It is usually best to reference standards that your target industry or sector already uses rather than create a new requirement. If your standard references these other standards (or standard-like documents such as codes of practice or lists of principles), the user is likely to be familiar with the requirement and may already be implementing it.

5. Referencing existing standards that are in use in your industry has two other benefits:

   1. If you write your own requirements for practices that are already covered by other standards, you may find that this becomes a disincentive for folks to use your standard. They may see your new requirement as creating confusion or overlap with their existing policies or practices. They may be required to implement two sets of requirements for the same thing (e.g. two different sets of requirements on record keeping).

   2. If you write your own requirements, then it is on you to make sure the requirements are kept current with evolving regulations and norms. If you reference a standard from a credible organization, it is their responsibility to update their own standard as needed. This may be a small matter for some issues but in areas you may not want to spend your resources developing the capacity to be on top of a complex technical issue when someone else has a helpful solution (e.g. a living wage calculator).

Multi-Stakeholder Initiatives

Multi-Stakeholder Initiatives (MSI) are a foundational aspect of credible standard systems.

There is a lot to unpack in this sentence since ‘MSI’ and ‘credible’ are both loaded terms.

First of all, there is no agreed definition of MSI. So…I’ve assembled a set of characteristics that are found in various definitions [1].

MSIs are generally understood to include one or more of these elements:

• Stakeholders self-identify as having an interest.

• Decisions are made by some form of consensus or general agreement of the participants.

• Participants agree to support the agreement(s) reached by the group.

• They are civil society actors.

  • As a result, government representatives that participate (if there are any) normally would have ‘voice’ but no ‘vote’ in the agreements reached.

• They can work on issues of any scale, local, regional, national, or global.

• They can work on any topic that they choose.

• They can form solely for the purpose of reaching an agreement(s) and some may establish organisation(s) that will implement a program based on the agreement(s).

It should also be crystal clear that MSIs are non-governmental[2] and do not include initiatives that are started by, report to or whose scope of work, agreements or decisions are subject to the approval of governments or government agencies.

It should be noted that a well-organized and effective MSI can make governments extremely nervous and sometimes, very angry. Governments tend to consider themselves the only legitimate forum for reaching collaborative agreements. Also, when not actively opposed, the agreements reached by MSIs can be seen by governments as constraints on the range of available policy options. As such, they sometimes resist the work of MSIs. In general, the more successful an MSI is, the more likely governments will resist them, except, of course, when the subject is one that has long stymied governments.

Secondly, there is the issue of credibility. Credibility is a measure of trust and believability (according to Wikipedia). In brief, credibility has both objective elements and subjective elements. For me that means that we can build transparency and accountability into the system to provide objective evidence of credibility but that does not mean that everyone will trust the MSI. Subjective elements can support this by getting endorsements from respected organizations and individuals and finding ways to present the objective evidence in ways that appeal to organizations and individuals. In short, its marketing, but based on evidence.

MSI’s are foundational to standards systems because they provide for the full spectrum of stakeholders to come together and agree a standard and how that standard is to be used for a product or service.

MSIs are not ‘representative processes’; that is where a representative of a sector or group of stakeholders is appointed to develop a standard. In an MSI the actual stakeholders have an opportunity to be part of the process. The only threshold that they must reach is their willingness to be a part of the MSI and to support (or at least not actively oppose) the decisions taken.

MSIs can be a challenge for many organisations and individuals. This is because most people and organisations do not have training or experience in MSIs work and how best to be part of the process (for more on this see the post: How to Build an Agreement).

Other factors to consider include how to establish an MSI, how to design its governance and how elements of an MSI can inform the ongoing work of an established standard system. Maybe these will be topics of future posts.

———————

[1] Multi-Stakeholder Initiative Integrity (MSI Integrity), Centre for Research on Multinational Corporations (SOMO), The World Bank, and Wikipedia

[2] Some people disagree with me on this point, but, quite frankly, they are wrong.

Audit, Assessment, Inspection, Verification and Validation

I know that terminology can drive people crazy. We standards and certification types tend to use lots of it, and it can seem like a secret club if you don’t know what it all means. Good thing that I am here to pull back the curtain…

I have written in the past about audits, but I have not really said what an audit is. To do it justice, we should look at several terms that are used and often get confused. To do this I will take a run at the terms: audit, assessment, inspection, verification and validation. I know that there may be other terms, but usually they are usually a different term that refers substantially to one or a combination of these processes.

I should be clear that I am talking about business processes or management systems and only tangentially about products. There are other terms that are used for the work of testing laboratories - these are the folks that conduct physical tests of products to determine if a physical thing meets specific performance standards. For example, if a helmet is strong enough or if a lamp won’t catch fire. Some of the terms we will discuss in this post are used in similar ways for product testing as for management systems.

All of the four terms are important, and many organizations use some or all of them. That is to say that all these processes play a slightly different role. Just like an internal audit and a management review - maybe I should do a post about these also, but that will have to wait for now.

AUDIT

Audits are normally full evaluations that are conducted by an independent person (or team) and produce findings and a decision (sometimes call a determination). Strictly speaking, an audit evaluates performance against specific measures and thresholds. Findings are the results against each of the requirements, it could be a finding of conformity or non-conformity supported by a record of the evidence found. The decision or determination answers the question: Does the system or process that was audited conform to the requirements or does it not?

ASSESSMENT

Assessments are evaluations that determine the capacity of an organization (or a part of one) to perform a specific function. These are similar to the strict definition of an audit, but it the findings are based to a much larger degree on the experience and training of the assessor. Assessments also produce a finding but rather than being based on a numerical set of requirements it is largely based on the judgment of the assessor.

NOTE: Management system audits include many elements that fit the description of an audit and others that fit the description of an assessment.

INSPECTION

Inspections are evaluations that are conducted by either an independent inspector or by an internal inspector. Inspectors will normally identify weakness or failings to meet a set of requirements but will not produce a decision about overall conformity with a standard.

VERIFICATION

Verifications are evaluations that look a single process or service to determine if it meets its design specification. These can be done by an independent verifier or an internal verifier. Often verifications are undertaken with new processes or products to make sure that what was intended was achieved. Verifications are often done at the design stage to test if all factors have been fully considered. For example, a verification may be undertaken to determine if a batch of a product or an input can be fully traced through all the steps in production.

VALIDATION

Validations are evaluations that look at a single product or service to determine that it meets the need of the user. These can be done by an independent validator or an internal validator. Validations are often conducted once a process or product is up and running to make sure that the result is what was intended. To some degree these are related to quality control, but can extend to cover how the product or service is used in the real world.

As I mentioned above, these terms are used differently in some industries, so the important part is not the term itself, but the concept of what is being done.

For example, audit and assessment are interchangeable for some people as are verification and validation for others.

In the case of third-party certification, the key definition is that of audit. It includes a decision about whether or not there is conformity with a standard backed up by the evidence gathered.

All of these evaluations may be based on a standard, but the need for a formal decision of conformity is less important than in an audit and may not carry as much formal weight since there is no certificate being issued.

Finally, I will note that in some cases, some standard systems will hire an inspector to conduct a full inspection of a client and the decision of conformity is made by the standard system based on the findings of the inspector. In these cases, the certification decision is taken by the standard system, not by the inspector.

If we were to map these terms in a Venn diagram, they would all overlap to some degree or other. The point is that they all have a slightly different use and are all used to some degree or other in a whole range of businesses.

There, now it is clear as mud…right?

What happens to your standard when the auditor goes to work?

So, you are now finished with your standard and all the rules that an audit firm (let's call it a CAB - a Conformity Assessment Body) has to follow.

Before you press send and go live with your new system, maybe it is time to sit back and think through how all of your work will be used in the field. Your system will work best if it fits the needs of the users, the client, CAB, Accreditation Body (AB), stakeholders and anyone else that needs to use your system.

Today, let's consider what happens to your system of requirements when they are used by a CAB. In brief, your system is dis-assembled and re-assembled into a new set of procedures, forms, checklists and requirements that fit the way the CAB is organized... 

• All the requirements that apply to the CAB, including process and timelines are separated from those that apply to the client. The CAB builds the requirements that it must follow in its own internal systems and procedures.

• All the requirements that apply to the client are put into checklists and other audit tools that are used to audit clients.

• The CAB's auditors are then trained to use these checklists and audit tools when auditing a client.

It is important to realize that auditors are not walking into an audit carrying copies of your standard, audit rules or other scheme documents, they are carrying materials that the CAB has developed based on your system requirements that are merged with the CAB’s own procedures. Their goal is to ensure that the audit can be carried out in the most efficient and effective manner possible, at least from their perspective.

CABs conducts audits following a standardized template. Most CABs will use ISO 19011 as the basic template and will add a few tweaks to make it their own. ISO 19011 lays out how an auditor conducts an audit. It includes such wonders as the outline for an opening and a closing meeting and other such innovations. The steps in ISO 19011 mean that the auditor is not just using your standard to evaluate the client, they have a number of steps that they should follow. These steps are necessary to ensure that the auditor does a thorough job and that client knows what is going on and what they have to do.

Before a CAB is allowed to conduct audits, its system must be fully reviewed to ensure that all of your requirements made it into the CAB’s re-assembled system. This is done by the AB and it is a key role that the AB assumes. Unless you have built your own expertise to do this job, you will need a competent AB to fully review the CAB’s system.

If this sounds complicated and a little confusing, it is because it is.

Your standard, scheme requirements, rules for auditors and such are most effective when your system is designed to be taken apart and put back together. It needs to be designed so that an AB can fully check the CABs incorporation of your system into theirs. It needs to take into account how auditors work in the field.

It also needs to take into account time…

The time that an auditor has on site with the client - so as you review your system keep in mind some key questions:

• How many hours does an auditor have per day to ask questions?

• How many questions can an auditor ask (and get answered) in an hour?

• Does the auditor have to travel from one site to another and how much time does that take?

• Auditors need to eat lunch, so allow time for that.

• Remember that the auditor has to include an opening and closing meeting with the client to explain what will happen in the audit and what the audit has found.

The time that it takes for each step in the process:

• For your system to be fully developed;

• For the CAB to incorporate your system into its own;

• For the AB to assess the CAB and make sure that your system is fully incorporated; and

• For the CAB to plan, execute and take a decision on any single client.

What happens during an audit is often as important (if not more important) than the specifics of what is being audited. I do not mean to suggest that what is in the standard is not important, but how the audit is conducted should be considered when your standard, requirements and other rules are developed.

Saving the World, one Checklist at a Time

One day a few years back I heard about some producer who wanted to do a reality TV show about environmentalists. They wanted real life environmenatlists; you know, the people who spend all their time chained to bulldozers to resist pipelines, swinging from construction cranes to hang banners or rescuing whales that are caught in fishing gear. This 'reality' TV idea came up while I was visiting a local environmental organization. Once the concept was explained the room was silent, then someone in the back of the room said: 'They want to film us sitting through endless meetings and while we read multi-volume reports?' Everyone started to laugh.

Needless to say, the show was never produced. 

Just like many other jobs, working in the world of standards, accreditation and certification can be dull. We create standards, guidelines and principles; then we turn them into checklists. With a blank checklist in hand we go through stacks of policies, procedures, practices and all kinds of other files; then we interview people and ask them all the same questions just to fill in the checklist and see if everything has been done and done right.

The work in standard systems is not going to get you a starring role in the hottest new reality show, but it is work that keeps us moving forward. Not just so that things continue to work, but to make them work better. We know that each application of the standard can help save an ecosystem, keep water clean, make sure workers are well treated, keep children in school, strengthen a community and so make the world a better place. 

Transforming markets takes dogged perseverance. It takes work over years to create innovative standards, conduct high quality accreditation, certification and auditing. It takes a hell of a lot of checklists.

So every morning when I start my day, I tell myself that I am saving the world, one checklist at a time.

You are too.

Accrediation - What is it & Why bother?

accreditation: third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks (3.1, ISO 17011:2004)
 

Accreditation is sometime described as 'auditing the auditors'...while that is not really correct it does hint at what an accreditation body (AB) does.

Accreditation is better described as a 'competency assessment' rather than an audit. You may feel that I am splitting hairs, stay with me for a few more paragraphs and I'll lay out why that is important.

First of all, let's focus on what competency means. Some folks think that a qualification to do a job is based on your education; in my case, I spent 2 years earing a masters degree in regional planning and a few more years qualifying for membership in a professional planning organization. As a result, I can call myself a 'Registered Professional Planner'.  I am a qualified planner!  In the twenty-eight years since I completed my degree in planning I have never worked as a land use planner or as a regional planner.  As a result, while I am qualified as a professional planner, I submit that I am no longer competent. Yes, I have the letters after my name and I use many of the skills that I learned in planning, but the degree and the designation does not equate to competency.

Competency is determined by demonstrating that a task or set of tasks can be done to a specified level of performance. In essence, it means the ability to do a job.

To assess the competency of a conformity assessment body (CAB) the AB looks at the systems, process and procedures used as well as how they are executed. Accreditation is granted to a CAB that demonstrates its competency in conducting conformity assessment work. In this context, competency is not a low bar to meet, it is a high bar. 

The areas that an AB would examine include a wide range of elements such as:

• maintaining professional relationships with clients;
• ensuring that all the staff of a CAB are competent to do their jobs;
• the frequency and quality of the training and evaluation of auditors; 
• planning and executing audits;
• ensuring that non-conformities are addressed;
• taking certification decisions; 
• conducting regular management reviews and internal audits of CAB operations;
• and a host of other elements.

Conducting an accreditation assessment is a specialized business, it is not just determining if the results of audits against your standard are producing the results you want. 

Who does what? - An Assurance Ecosystem

This blog focuses on the role of the scheme owner. But, let’s be clear that while the scheme owner is a central player, there are others that have crucial roles. Previous posts have included some elements related to standards development with some references to auditing. In this post I wanted to look at the big picture of how assurance is managed, in what I think of as an ‘assurance ecosystem’.

Before going any further, I want to be clear that while all well run standard systems include all the elements discussed below, who does what and how oversight is conducted can vary greatly. This can be due to the objectives of the scheme owner, the nature of the industry or in response to the way an industry sector or key resource is managed internationally or in specific countries.

A fully functioning assurance ecosystem includes several players, each with their own responsibilities. The diagram below is a highly simplified map of the players and the primary relationships of each.

Assurance Ecosystem

 For this post, as with the all the posts on this blog, the central player in our assurance ecosystem is the scheme owner.

In most cases the scheme owner is responsible for:

• the standard itself, including its development, maintenance and interpretation;
• the rules for the assurance system including the documented policies, processes and procedures that govern its application; and,
• the day to day management of scheme operations including oversight, monitoring and evaluation, risk management, and business operations such as marketing, finance, personnel, etc.

Some schemes delegate some of these responsibilities to other organizations. However, the scheme owner is always ultimately responsible for the scheme. For example: 

• In the case of organizations such as MSC and RSPO, they act as scheme owner and they have assumed all these three core responsibilities.
• In the case of FSC, it established a global set of ‘Principles and Criteria’ that standards must address and works with affiliated national and regional organizations who assume the role of a standards development organization (SDO). FSC, the scheme owner, endorses those standards that conform to its system requirements.
• In the case of ASC, the standard was developed by a group of semi-autonomous bodies, the aquaculture dialogues that were set up by WWF, and the completed standards were given to ASC who now undertakes all the responsibilities of the scheme owner.

The other three players on the assurance ecosystem map (AB, CAB and client) have their own roles to play.

The accreditation body (AB) has the job of providing "third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks (ISO 17011:2004 3.1)". In plain English, its job is to make sure that the CAB has the competence and capacity to carry out audits for a scheme. For some schemes, the accreditation body works in a single country, such as DAkkX in Germany or ANSI in the US, or the accreditation body works internationally, such as IOAS, SAAS or ASI.

The conformity assessment body (CAB) - sometimes referred to as the 'certification body' or the auditor - has the job of carrying out audits against the scheme owner's standard following the policies, practices and procedure specified by the scheme owner. CABs can range from large multinational organisations such as Bureau Veritas, SGS or TÜV to small organisations with only a few staff working in single country or region.

The client, that is the applicant for certification or, when certified, the certificate holder, has the job of making sure that the product, process or service conforms to the standard.

The assurance ecosystem is almost never a single organization that does everything. As a result, the task of assuring that each certificate means that the same minimum performance level has been achieved for each certificate is a challenge that requires skills of facilitation, negotiation, management, collaboration and communication.

In brief, the challenge of being a scheme owner includes the skill and capacity to partner with specialist organization in a range of roles and responsibilities in a way that respects the the scheme owner and their specialist roles so that the whole ecosystem delivers consistently high quality.

Planning for the future

There is a popular quote from Wayne Gretzky. He is reported to have said something like: "I skate to where the puck is going, not where it has been.” 

As cheesy as the quote may be, it draws attention to a fundamental problem that decision makers face - how to position yourself to take advantage of something that is going to happen rather than try to go after the rewards of something that is happening now or has already happened.

My cynical mind sees populist leaders of our age expending tremendous amounts of energy to recapture the glory of a bygone era, whether that is Trump seeking to recreate the hegemony of the US in the 1950s, Putin seeking to recapture the glory of the Byzantine Empire, or the Brexiteers seeking to reclaim the primacy of the British Empire, they all seem to me to be ‘skating to where the puck was…’

Business leaders can get caught in the same trap, that is focussing their resources on a recent trend only to be faced with all their competition going after the same, now diminishing opportunity.

This may not seem like it has much to do with certification; but it does, stay with me.

Certification systems face the pressing demand to solve the problems that they face now. This pressure can be overwhelming and can mean that leaders can be caught in the trap of focussing on where the puck is now, without devoting time and energy to where it will be. Limited resources can also truncate the capacity of leaders to think about the future.

The visionaries that were behind the leading social and environmental standards were thinking about the future and how to bend it toward creating a more just and greener world. They looked at long-term trends and tried to figure out how to take advantage of them.

While leaders in certification systems have to devote large amounts of energy into solving today’s problems, they also need to spend time thinking about were the world is going and how they can get out in front of the changes that are now developing.

All this comes down to the fact that certification systems are constantly changing and taking advantage of the opportunity to change can give us the opportunity leverage to broad changes that are occurring and are likely to occur.

These can include:

• Growth in markets and trade in developing countries
• The ever-increasing rate of technological change
• The globalization of markets, especially capital markets
• Market pressure from activist consumers
• The growth of the size and importance of global megacities
• The growing inability of national governments to control markets

I would not counsel adopting my list, but encouage looking to the research being done on future trends. Study the reports, evaluate the trends and ask the question: How could this impact my standard system?

Even if we just consider the growth in global trade and especially the growth in south-south trade, certification systems will need to adapt to a new shape in global markets. Most certification systems are designed to manage north-north and south to north trade. As a result, their efforts to develop markets for certified products in Europe, North America and Japan will not encourage growth in the rapidly growing market share in the developing world. Will your system be positioned to grow in Indonesia, Kenya and Brazil? Will you be able to deliver markets to your certificate holders for those that produce products in Africa and sell into Southeast Asia?