Every project should have a quality plan. In reality, very
few do. It is something that has puzzled me for some time. A few years back
I had the opportunity to talk to a group of Project Managers about QA. Surprisingly
the two main reasons they didn't produce a plan were:
- It was too complicated to do a plan
- They were overwhelmed by the jargon of quality in relation to compliance
with standards, metrics and a range of acronyms that left them confused
This white paper will provide a common sense approach, and give you
the basic tools you need to put together a quality plan.
So what is quality? There are numerous definitions of quality:
"Quality is fitness for use" - J.M. Juran
"[Quality is] meeting or exceeding customer expectations at a cost that
represents a value to them." - H. James Harrington
"Quality should be defined as surpassing customer needs and expectations
throughout the life of the product." - Howard Gitlow and Shelley Gitlow
A simple layman's definition is to make sure whatever is delivered is within
the quality expectations of the organisation. The expectations of the organisation
are important to understand. If it is NASA building rocket control systems,
the expectations are likely to be higher than if it is a small retailer building
a marketing database...hopefully.
Another important component of the definition is the focus on what is being
delivered. Quality of what is not delivered is not necessarily important unless
it will impact the ultimate deliverable. I know of one organisation that checks
the quality of all documentation including email and memos. The question needs
to be asked
"If I spend an extra 10 minutes on each email to ensure it has the standard
layout, font etc. is it going to significantly improve the quality of the
The answer is that such an exercise cannot be cost justified.
From a business perspective, project quality is usually judged on the following
- Was the project completed on time?
- Was the project completed within budget?
- Did the system meet my needs when it was delivered?
- Is it stable?
From a technical perspective, project quality is usually judged as:
- Does the system comply with corporate standards for such things as user
interface, documentation, naming standards etc.?
- Is the technology stable?
- Is the system well engineered so that it is robust and maintainable?
As you can see, the perspective of quality varies depending on who we are
talking to. Generally speaking however, the "fit for purpose" aspect of quality
is the one we judge. Does the deliverable do the job it was designed to do?
Project Quality v Deliverable Quality
The situation above illustrates the difference between judging the deliverables
and judging the project. You need to decide how much focus you put on the
quality of the project, and how much on the quality of the deliverables.
- The project quality refers to things like applying proper project management
practices to cost, time, resources, communication etc. It covers managing
changes within the project.
- The deliverable quality refers to the 'fit for purpose' aspect mentioned
earlier. It covers things like how well it meets the user's needs, and the
total cost of ownership.
A quality project may deliver low quality deliverables and vice versa. It
is more likely however that a high quality project will deliver high quality
deliverables. You can see that if you were checking project quality you would
look at completely different things than if you were looking at the quality
of the deliverables.
The following definitions will help us understand quality:
||The artifacts used within an organisation to
assist a Project Manager improve quality in the project e.g.
Templates, Standards, Checklists. These materials are used
in "Quality Events"
||How the "Quality Materials" are applied to a project. They
are the activities undertaken using "Quality Materials" to validate the
quality of the project.
||A plan as to how and when "Quality Events" and "Quality
Materials" are applied to a project.
||The implementation of the "Quality Events" in the "Quality
QA is an umbrella term. It refers to the processes used within an organization,
to verify that deliverables are of acceptable quality and that they
meet the completeness and correctness criteria established. QA does
not refer to specific deliverables.
- The preparation of a "Quality Plan" for a project is part of QA
- The development of standards is part of QA
- The holding of a "Quality Event" is part of QA
Statistics captured during the various activities undertaken as part
of "Quality Assurance". Metrics are captured to:
- Identify areas where quality improvements can be made
- Measure the effectiveness of quality improvement activities
|Continuous Quality Improvement
||Use of captured metrics, and lessons learnt to continually
improve quality. They are the main reason for capturing statistics around
quality. They are the main reason for capturing statistics around quality.
|Well Engineered versus Correct
The purpose of quality assurance is to ensure outputs of an organisation
are both well engineered and correct.
- Well engineered means the construction is sound and reliable. It
does not necessarily mean it is correct.
- Correct means the end results are an accurate reflection of the
requirements. It does not necessarily mean it is well engineered.
Many systems are well engineered but fail to meet the business need.
On the other hand, there are also systems that meet the business need,
but are unstable, unscaleable and expensive to run. Similarly a document
can be well constructed but the content is deficient. Alternatively,
the information can be there, but it is difficult to interpret.
The following are examples of "Quality Materials" that might be used in
a quality plan:
||"Standards" are instruction documents that detail how a
particular aspect of the project must be undertaken. There can be no deviation
from "Standards" unless a formal variation process is undertaken, and
||Unlike "Standards", "Guidelines" are not compulsory. They
are intended to guide a project rather than dictate how it must be undertaken.
Variations do not require formal approval.
||"Checklists" are lists that can be used as a prompt when
undertaking a particular activity. They tend to be accumulated wisdom
from many projects.
||"Templates" are blank documents to be used in particular
stages of a project. They will usually contain some examples and instructions.
||"Procedures" outline the steps that should be undertaken
in a particular area of a project such as managing risks, or managing
||A description of how something works. It is different to
a "Procedure" in that a "Procedure" is a list of steps - the what and
when. A "Process" contains explanations of why and how.
||"User Guides" provide the theory, principles and detailed
instructions as to how to apply the procedures to the project. They contain
such information as definitions, reasons for undertaking the steps in
the procedure, and roles and responsibilities. They also have example
||These are examples from prior projects that are good indicators
of the type of information, and level of detail that is required in the
||A methodology is a collection of processes, procedures,
templates and tools to guide a team through the project in a manner suitable
for the organisation.
Below is a list of "Quality Events" that typically are used to review the
quality of deliverables. They tend to have a different mix of reviewing the
structure and reviewing the content. In other words they check to see if the
document is "Well Engineered" and/or "Correct" (see definitions):
Review of a deliverable by a person who is considered an expert in
the area. For example, a review of a data model by a senior DBA. The
person may not currently hold a position (e.g. currently be a DBA) but
has expert knowledge in the area.
This type of review is good when the focus is on accuracy of content
(Correct) rather than of structure (Well engineered).
Review of deliverables by one's peers.
Peer reviews are better suited where the emphasis is on structure rather
than content. A peer review will focus on ensuring the deliverable is
Neither an "Expert Review" nor a "Peer Review" is exclusively focused
on content or structure. They each however, have a different emphasis.
|Multi person Review
A review carried out independently by several people is likely to pick
up more points however it does bring the difficulty of trying to reconcile
different viewpoints. It is best undertaken when the purpose is to gain
agreement between different stakeholders.
Time should be allowed to reach agreement of conflicting opinions.
This may entail a meeting or workshop to resolve differences.
A walk-through is a useful technique to validate both the content and
structure of a deliverable. Material should be circulated in advance.
If particular participants have not done their homework, they should
be excluded from the walk-through. Too much time can be wasted bringing
one person up to speed in a walk-through.
||A formal inspection is a review of a deliverable by an inspector
who would typically be external to the Project Team. The inspector captures
statistics on suspected defects. It is a useful technique for use with
||A "Standard Audit" is carried out be a person who is only
focused on ensuring the deliverable meets a particular standard(s).
In this case a defined "Business Process" is reviewed to ensure all
necessary actions are being undertaken, information recorded, and procedures
followed. A "Process Review" is useful to validate the existing processes
in an organisation.
For example, modification to an existing IT system may be based on
the assumption an existing business process is being followed. If the
business process is either not being followed or is different, the modification
to the IT system may have unexpected results.
For a project quality check, a "Process Review" may be carried out
to ensure proper change control procedures are in place. Typically someone
like a Project Office or Internal Audit would complete a "Process Review".
A quality plan needs to cover a number of elements:
- What needs to go through a quality check?
- What is the most appropriate way to check the quality?
- When should it be carried out?
- Who should be involved?
- What "Quality Materials" should be used?
What needs to be checked?
Typically what needs to be checked are the deliverables. Any significant
deliverable from a project should have some form of quality check carried
out. A requirements document can be considered significant. A memo or weekly
report may not be significant.
For the project itself, it may be appropriate to have the project management
practices reviewed for quality once the project is initially established.
This may be useful to give the Sponsor and Steering Committee a level of confidence
in the team.
What is the most appropriate way to check?
To answer this question requires thinking backwards. If the end result is
that a particular deliverable should meet a standard, then part of the quality
checking should focus on compliance with the standard. This would indicate
a "Standard Audit" could be the best approach.
You also need to differentiate between "correct" and "well engineered".
A "well engineered" bridge may never fall down. If it is doesn't cross the
river at the right place, it is not "correct". Similarly a test plan may be
clear and easy to follow but not test everything it should. Alternatively
it may cover all the testing but cannot be clearly followed. Quality checking
may be for either "correct" or "well engineered", or it may be for both.
When should it be carried out?
Most "Quality Events" are held just prior to the completion of the delivery.
If however there are long development lead times for a deliverable, it might
be sensible to hold earlier "Quality Events".
For example, if development of code for a particular module will take 10
weeks, it may be worth holding a code inspection after 4 weeks to identify
any problems early and reduce rework.
Who should be involved?
Obviously, the producers of the deliverable should be involved. The others
involved will be dependant on the type of quality event. It is also useful
to have some representation from the receivers of the information in order
to ensure you are not using jargon that makes it clear to the producers, but
unclear to the receivers.
What Quality Materials should be used?
The materials used should be a prompt for the reviewers to ensure there
are no gaps. The "Quality Materials" will usually be self evident. It may
be useful to reduce things like standards to checklists in order to make them
more manageable. If the reviewers know the specifications for xyz in standard
abc, they only need to be reminded to check xyz. They don't need the full
standard as the primary piece of "Quality Material". It can just be a reference.
Example Quality Plan
A typical quality plan for an applications project may look something like
|Preliminary Business Case
Template for Business Case
Approved Business Case for Project ABC
|Ensure the information is accurate and well constructed prior to submission
to Steering Committee
|Final Business Case
||Formal Inspection by Sponsor
||Template for Business Case
||Ensure the Business Case is in a fit state to be submitted to the Finance
||Walk-through of early draft
||Template for Project Definition
||Review early draft for completeness
||Peer Review of final draft
||Review final draft for completeness and construction
||Expert Review of physical model
||Standard for Database Design
Compliance with standard
A good story I heard at a conference once referred to the difference between
US and Japanese car makers during the 70's. The speaker said in a US factory,
if a car came off the production line with only three wheels, everyone scrambled
around to find a new wheel, put it on the car and get it out the door. Probably
a few days later another car would arrive with three wheels and the process
would happen again.
On the other hand if a car came off the production line in Japan with three
wheels, they stopped the production line, and found out how it happened. Once
they found the cause it would be fixed, and no car would ever come off the
line with three wheels again.
The world is bigger than one project. What goes wrong in one project, is
likely to go wrong in other projects unless the cause is identified and fixed.
If a template is missing a heading, don't just fix the project document. Fix
the template. If a standard has some aspect that cannot be complied with in
your environment, either change your environment, or get agreement that all
projects are exempt from this part of the standard. If there are no generally
accepted availability criteria for business applications, don't just add some
to your requirements. Get them published as corporate criteria. This is what
continuous improvement is all about.
If we are improving quality, we need to measure progress. This means a baseline
has to be established. Quality metrics are a whole topic in themselves and
are outside of the scope of this document.
Producing a quality plan is not complex. It involves identifying all the
deliverables at the start of the project and deciding how to best validate
their quality. There is an overhead in undertaking quality checks but this
is offset by not having to fix things further down the line. Inevitably, the
later you find a problem, the longer it takes to fix.
It is also going to make your customers more comfortable if they see that quality is being addressed during the project. It can even be a good PR exercise to bring them to a quality review. Not only do they see that quality is being addressed, but it also exposes them to the complexity that usually exists in a project.
Finally, having uncovered the quality issues, be sure you have a mechanism in place to fix the problems. There must be some follow up process to allocate fixes to particular people and ensure they actually make the changes. This implies that time must be built into the schedule for rework following quality events.
To date, 700 people have rated this article. The average rating is 4.07 - Add your rating. Just select a rating and click the button. No other information
Only one rating per person is allowed.
Project Quality Planning
Return to the top