Quality Assurance

Most people spend twelve or more years in an educational system in which grades of 90% or higher, are considered excellent. However, in industry, 90% is not a good quality record. For example, if one out of every ten tires fails, you have a 90% quality rating, but that is totally unacceptable to tire customers.

Motorola developed a concept called “Six Sigma Quality” that focuses on defect rates, as opposed to percent performed correctly. “Sigma” is a statistical term meaning one standard deviation. “Six Sigma” means six standard deviations. At the Six Sigma statistical level, only 3.4 items per million are outside of the acceptable level. Thus, the Six Sigma quality level means that out of every one million items counted 999,996.6 will be correct, and no more than 3.4 will be defective.

Experience has shown that in most systems, a Four Sigma quality level is the norm. At the Four Sigma level there are 6,120 defects per million parts, or about 6 defects per 1,000 opportunities, to do a task correctly.

The key focus of companies implementing a Six Sigma program is to develop a good business strategy that balances the cost, quality, features and availability considerations for products. A valuable lesson learned is that decisions made must be tied to the bottom line for the company. Companies should take care to use correct measurements for each situation, and to consider measuring output of a process over time (not just a snapshot).

When considering a project to improve using Six Sigma, the following characteristics are desirable. If one or more of these characteristics is missing, there will likely be barriers to success.

- The project should clearly connect to business priorities.

- The problem being solved should be very important, such as a 50% process improvement.

- The importance of the project should be clear to the organization.

- The project should have a limited scope that can be completed in less than six months.

- The project should have clear, quantitative measures that define success.

- Management should support and approve the project to ensure resources are available, barriers are removed, and that the project continues over time.

Review
The focus of Review is on a work product (e.g. Requirements document, Code etc.). After thework product is developed, the Project Leader calls for a Review. The work product is distributed to the personnel who involves in the review. The main audience for the review should be the Project Manager, Project Leader and the Producer of the work product.

Major reviews include the following:

1. In Process Reviews
2. Decision Point or Phase End Reviews
3. Post Implementation Reviews

As per statistics Reviews uncover over 65% of the defects and testing uncovers around 30%. So, it’s very important to maintain reviews as part of the V&V strategies.

In-Process Review: In-Process Review looks at the product during a specific time period of a life cycle, such as activity. They are usually limited to a segment of a project, with the goal of identifying defects as work progresses, rather than at the close of a phase or even later, when they are more costly to correct.

Decision-Point or Phase-End Review: This review looks at the product for the main purpose of determining whether to continue with planned activities. They are held at the end of each phase, in a semiformal or formal way. Defects found are tracked through resolution, usually by way of the existing defect tracking system. The common phase-end reviews are Software Requirements Review, Critical Design Review and Test Readiness Review.

- The Software Requirements Review is aimed at validating and approving the documented software requirements for the purpose of establishing a baseline and identifying analysis packages. The Development Plan, Software Test Plan, Configuration Management Plan are some of the documents reviews during this phase.

- The Critical Design Review baselines the detailed design specification. Test cases are reviewed and approved.

- The Test Readiness Review is performed when the appropriate application components are near completing. This review will determine the readiness of the application for system and acceptance testing.

Post Implementation Review: These reviews are held after implementation is complete to audit the process based on actual results. Post-Implementation reviews are also known as Postmortems and are held to assess the success of the overall process after release and identify any opportunities for process improvement. They can be held up to three to six months after implementation, and are conducted in a format.

Three general classes of reviews:

1. Informal or Peer Review
2. Semiformal or Walk-Through
3. Formal or Inspections

Peer Review is generally a one-to-one meeting between the author of a work product and a peer, initiated as a request for import regarding a particular artifact or problem. There is no agenda, and results are not formally reported. These reviews occur on an as needed basis throughout each phase of a project.

Inspections
A knowledgeable individual called a moderator, who is not a member of the team or the author ofthe product under review, facilitates inspections. A recorder who records the defects found and actions assigned assists the moderator. The meeting is planned in advance and material is distributed to all the participants and the participants are expected to attend the meeting well prepared. The issues raised during the meeting are documented and circulated among the members present and the management.
Walkthroughs
The author of the material being reviewed facilitates walk-Through. The participants are led through the material in one of two formats; the presentation is made without interruptions and comments are made at the end, or comments are made throughout. In either case, the issues raised are captured and published in a report distributed to the participants. Possible solutions for uncovered defects are not discussed during the review.

Software Quality Attributes are: Correctness, Reliability, Adequacy, Learnability, Robustness, Maintainability, Readability, Extensibility, Testability, Efficiency, Portability.

Correctness: The correctness of a software system refers to:

- Agreement of program code with specifications
- Independence of the actual application of the software system.

The correctness of a program becomes especially critical when it is embedded in a complex software system.

Reliability: Reliability of a software system derives from

- Correctness
- Availability

The behavior over time for the fulfillment of a given specification depends on the reliability of the software system.

Reliability of a software system is defined as the probability that this system fulfills a function (determined by the specifications) for a specified number of input trials under specified input conditions in a specified time interval (assuming that hardware and input are free of errors).

A software system can be seen as reliable if this test produces a low error rate (i.e., the probability that an error will occur in a specified time interval.)

The error rate depends on the frequency of inputs and on the probability that an individual input will lead to an error.

Adequacy: Factors for the requirement of Adequacy:

- The input required of the user should be limited to only what is necessary. The software system should expect information only if it is necessary for the functions that the user wishes to carry out. The software system should enable flexible data input on the part of the user and should carry out plausibility checks on the input. In dialog-driven software systems, we vest particular importance in the uniformity, clarity and simplicity of the dialogs.

- The performance offered by the software system should be adapted to the wishes of the user with the consideration given to extensibility; i.e., the functions should be limited to these in the specification.

- The results produced by the software system: The results that a software system delivers should be output in a clear and wellstructured form and be easy to interpret. The software system should afford the user flexibility with respect to the scope, the degree of detail, and the form of presentation of the results. Error messages must be provided in a form that is comprehensible for the user.

Learnability: Learnability of a software system depends on:

- The design of user interfaces
- The clarity and the simplicity of the user instructions (tutorial or user manual).

The user interface should present information as close to reality as possible and permit efficient utilization of the software’s failures.

The user manual should be structured clearly and simply and be free of all dead weight. It should explain to the user what the software system should do, how the individual functions are activated, what relationships exist between functions, and which exceptions might arise and how they can be corrected. In addition, the user manual should serve as a reference that supports the user in quickly and comfortably finding the correct answers to questions.

Robustness: Robustness reduces the impact of operational mistakes, erroneous input data, and hardware errors.

A software system is robust if the consequences of an error in its operation, in the input, or in the hardware, in relation to a given application, are inversely proportional to the probability of the occurrence of this error in the given application.

- Frequent errors (e.g. erroneous commands, typing errors) must be handled with particular care.

- Less frequent errors (e.g. power failure) can be handled more laxly, but still must not lead to irreversible consequences.

Maintainability: Maintainability = suitability for debugging (localization and correction of errors) and for modification and extension of functionality.

The maintainability of a software system depends on its:

- Readability
- Extensibility
- Testability

Readability: Readability of a software system depends on its:

- Form of representation
- Programming style
- Consistency
- Readability of the implementation programming languages
- Structuredness of the system
- Quality of the documentation
- Tools available for inspection

Extensibility: Extensibility allows required modifications at the appropriate locations to be made without undesirable side effects. Extensibility of a software system depends on its:

- Structuredness (modularity) of the software system
- Possibilities that the implementation language provides for this purpose
- Readability (to find the appropriate location) of the code
- Availability of comprehensible program documentation

Testability: suitability for allowing the programmer to follow program execution (runtime behavior under given conditions) and for debugging. The testability of a software system depends on its:

- Modularity
- Structuredness

Modular, well-structured programs prove more suitable for systematic, stepwise testing than monolithic, unstructured programs.

Testing tools and the possibility of formulating consistency conditions (assertions) in the source code reduce the testing effort and provide important prerequisites for the extensive, systematic testing of all system components.

Efficiency: ability of a software system to fulfill its purpose with the best possible utilization of all necessary resources (time, storage, transmission channels, and peripherals).

Portability: the ease with which a software system can be adapted to run on computers other than the one for which it was designed.

The portability of a software system depends on:

- Degree of hardware independence
- Implementation language
- Extent of exploitation of specialized system functions
- Hardware properties
- Structuredness: System-dependent elements are collected in easily interchangeable program components.

A software system can be said to be portable if the effort required for porting it proves significantly less than the effort necessary for a new implementation.

Many think that defect-free products and services are not practical or economical, and thus believe some level of defects is normal and acceptable. (This is called acceptable quality level, or AQL.) Quality experts agree that AQL is not a suitable definition of quality. As long as management is willing to “accept” defective products, the entire quality program will be in jeopardy.

Quality is frequently associated with cost, meaning that high quality is synonymous with high cost. (This is confusion between quality of design and quality of conformance.) Organizations may be reluctant to spend on quality assurance, as they do not see an immediate payback.

Quality by definition calls for requirements/specifications in enough detail so that the products produced can be quantitatively measured against those specifications. Few organizations are willing to expend the effort to produce requirements / specifications at the level of detail required for quantitative measurement.

Many technical personnel believe that standards inhibit their creativity, and thus do not strive for compliance to standards. However, for quality to happen there must be welldefined standards and procedures that are followed.

The contributors to poor quality in many organizations can be categorized as either lack of involvement by management, or lack of knowledge about quality. Following are some of the specific contributors for these two categories:

Lack of involvement by management:
- Management’s unwillingness to accept full responsibility for all defects
- Failure to determine the cost associated with defects (i.e., poor quality)
- Failure to initiate a program to “manage defects”
- Lack of emphasis on processes and measurement
- Failure to enforce standards
- Failure to reward people for following processes

Lack of knowledge about quality:
- Lack of a quality vocabulary, which makes it difficult to communicate quality problems and objectives
- Lack of knowledge of the principles of quality (i.e., what is necessary to make it happen)
- No categorization scheme for defects (i.e., naming of defects by type)
- No information on the occurrence of defects by type, by frequency, and by location
- Unknown defect expectation rates for new products
- Defect-prone processes unknown or unidentified
- Defect-prone products unknown or unidentified
- An economical means for identifying defects unknown
- Proven quality solutions are unknown and unused

If achieving quality (i.e., defect-free products and services) were easy, it would have been accomplished years ago. Quality is very difficult to accomplish – it requires the close cooperation of management and staff. Achieving quality requires a commitment and the establishment of an environment in which quality can flourish.

Joseph Juran has explained his model of quality improvement on the basis of the basis of three universal processes which have been popularly named a Juran Trilogy.

The processes are:

1. Quality Planning: As per Juran Triology quality planning is a concurrent exercise which involves all the affected parties related to the product and services, so that they can provide inputs and give early warnings during the planning processes.

The steps of the quality planning exercise are:

- Definition of the project.
- Identification of the customers – those who will be impacted by the actions that are taken to complete the project.
- Discovery of customer needs.
- Development of the product and processes to meet the customer’s needs.
- Establishment of the quality objectives.
- Development of the plans for meeting these objectives.

2. Quality Control: According to Juran Triology Quality control involves the developing and maintaining of operational methods in order to assure that the processes work as they are designed to work and that the target levels of performance being are being achieved. Quality control does not concern itself with improving a process, but rather with the execution of plans. It is primarily to control that occasional spike in error in the process. Quality control is a short term process to check that spike.

Quality control entails the following steps:

- Clear definitions of quality.
- Knowledge of the expected performance or targets.
- Evaluation of the actual operating performance.
- Comparison of the actual performance to goals.
- Action of the difference.

3. Quality Improvement: As per Juran Triology, quality improvement is a disciplined approach that improves the level of performance of the process. This is achieved by a breakthrough improvement in performance; when a new innovation or a completely fresh idea is brought into improve the current performance levels. This ensures that the new levels of performance are achieved, and then quality control mechanisms are in place to sustain that effectively.

Also See:

• Deming’s 14 Quality Principles
• Philip Crosby’s concept of Quality Improvement

Management’s role in Continuous Improvement:

- Establish an organization-wide quality council
- Establish specific quality improvement goals with timetables and target dates
- Provide support
- Schedule periodic reviews

Structure for Quality Improvements:

- Establish a quality Council
- Develop a statement of responsibilities
- Establishing the necessary infrastructure

Essential Improvement Activities:

- Communicate
- Correction Obvious Problems
- Find causes
- Documentation
- Monitor changes

The Scientific Approach:

- Collect meaningful data
- Identify root causes of problems
- Develop appropriate solutions
- Plan and make changes

Identification of Improvement needs:

- Apply multi voting
- Identify customer needs
- Study the use of time
- Localize problems

Development of Improvement Plans:

- Understand the process
- Eliminate errors
- Remove slack
- Reduce variation
- Plan for continuous improvement

Common Improvement Strategies:

- Describe the Process
- Standardize the Process
- Eliminate Errors in the Process
- Streamline the Process
- Eliminate Variation
- Get the process in statistical control
- Improve the Design

Quality Function Deployment: QFD is a practice for designing your processes in response to customer needs. QFD translates what the customer wants into what the organization produces. It enables an organization to prioritize customer needs, an improve processes to maximum effectiveness. QFD is a practice that leads to process improvements that enable an organization to exceed the expectations of the customer.

Benefits of QFD:
Customer-focused: A total-quality organization is a customer-focused organization.

Time-efficient: QFD can reduce development time because it focuses on identified customer requirements.

Teamwork -orientated: All decisions in the process are based on consensus and involve in-depth discussion and brainstorming.

Documentation-orientated: One of the products of the QFD processes a comprehensive document that pulls together all pertinent data about all processes and how they stack up against customer requirements.

Basic QFD Tools:
- Affinity Diagram
- Interrelationship Digraph
- Tree Diagram
- Matrix Diagram

Creating an Affinity Diagram:

- A team of employees familiar with the issue is formed.
- The issue to be discussed is stated without detail.
- Responses of participants are stated verbally and recorded on 3 by 5 cards.
- The cards are spread on a large table, and participants are asked to group cards containing related ideas. Cards that don’t fit any particular group can be grouped as miscellaneous.
- Participants try to find a heading that describes each group.
- The information on the cards is replicated on paper with boxes around each group of ideas.

Interrelationship Digraph:

- Write the problem statement on a 3 by 5 card.

- Place the problem statement card in the upper left corner of a table. Take out all the cards used to develop the Affinity diagram and lay them on the table. Place the card most closely associated with the problem closest to the problem card. Continue on with the next closest until cards are used up.

- When the cards are all laid out, recreate them on paper. Distribute copies of the paper to the group for discussion. The group then makes a final arrangement.

- Distribute the final version to participants and ask them to draw arrows showing what contributes to what. This is the step in which relationships between and among causes are established.

Tree Diagram:

- Clearly identify the problem to be solved. It can be taken from the affinity diagram or from the interrelationship digraph. Write it on a card and place it on the left side of a table.

- Conduct a brainstorming session in which participants record on 3 by 5 cards all possible tasks, methods, and activities relating to the problem. Repeat the question “In order for this to happen, what must happen first?” Continue until all ideas are exhausted.

- Lay all cards on table to the right of the problem card. Put them in order based on what must happen first.

- Duplicate the cards on paper and distribute copies to all participates. All participates are to revise and correct the document.

Matrix Diagram: It is a helpful tool for identifying and graphically displaying connections among responsibilities, tasks, functions, and so forth.

Implementing QFD:
- Form the Project Team
- Establish Monitoring Procedures
- Select a Project
- Conduct a “Kick Off” Meeting
- Train the Team
- Develop the Matrices

Select a Project: It is a good idea to begin with an improvement project. Team members who are not familiar with QFD are at least familiar with the product.

Conduct a “Kick-Off” Meeting:

- Make sure all participants understand the mission of the project team.
- Make sure all participates understand their role on the the team.
- Establish parameters – (length, time, and frequency of meeting).

Train the Team: Team members should learn how to use QFD tools and how the QFD process works.

Develop the Matrices.