Quality Assurance

Review
The focus of Review is on a work product (e.g. Requirements document, Code etc.). After thework product is developed, the Project Leader calls for a Review. The work product is distributed to the personnel who involves in the review. The main audience for the review should be the Project Manager, Project Leader and the Producer of the work product.

Major reviews include the following:

1. In Process Reviews
2. Decision Point or Phase End Reviews
3. Post Implementation Reviews

As per statistics Reviews uncover over 65% of the defects and testing uncovers around 30%. So, it’s very important to maintain reviews as part of the V&V strategies.

In-Process Review: In-Process Review looks at the product during a specific time period of a life cycle, such as activity. They are usually limited to a segment of a project, with the goal of identifying defects as work progresses, rather than at the close of a phase or even later, when they are more costly to correct.

Decision-Point or Phase-End Review: This review looks at the product for the main purpose of determining whether to continue with planned activities. They are held at the end of each phase, in a semiformal or formal way. Defects found are tracked through resolution, usually by way of the existing defect tracking system. The common phase-end reviews are Software Requirements Review, Critical Design Review and Test Readiness Review.

- The Software Requirements Review is aimed at validating and approving the documented software requirements for the purpose of establishing a baseline and identifying analysis packages. The Development Plan, Software Test Plan, Configuration Management Plan are some of the documents reviews during this phase.

- The Critical Design Review baselines the detailed design specification. Test cases are reviewed and approved.

- The Test Readiness Review is performed when the appropriate application components are near completing. This review will determine the readiness of the application for system and acceptance testing.

Post Implementation Review: These reviews are held after implementation is complete to audit the process based on actual results. Post-Implementation reviews are also known as Postmortems and are held to assess the success of the overall process after release and identify any opportunities for process improvement. They can be held up to three to six months after implementation, and are conducted in a format.

Three general classes of reviews:

1. Informal or Peer Review
2. Semiformal or Walk-Through
3. Formal or Inspections

Peer Review is generally a one-to-one meeting between the author of a work product and a peer, initiated as a request for import regarding a particular artifact or problem. There is no agenda, and results are not formally reported. These reviews occur on an as needed basis throughout each phase of a project.

Inspections
A knowledgeable individual called a moderator, who is not a member of the team or the author ofthe product under review, facilitates inspections. A recorder who records the defects found and actions assigned assists the moderator. The meeting is planned in advance and material is distributed to all the participants and the participants are expected to attend the meeting well prepared. The issues raised during the meeting are documented and circulated among the members present and the management.
Walkthroughs
The author of the material being reviewed facilitates walk-Through. The participants are led through the material in one of two formats; the presentation is made without interruptions and comments are made at the end, or comments are made throughout. In either case, the issues raised are captured and published in a report distributed to the participants. Possible solutions for uncovered defects are not discussed during the review.

Software Quality Attributes are: Correctness, Reliability, Adequacy, Learnability, Robustness, Maintainability, Readability, Extensibility, Testability, Efficiency, Portability.

Correctness: The correctness of a software system refers to:

- Agreement of program code with specifications
- Independence of the actual application of the software system.

The correctness of a program becomes especially critical when it is embedded in a complex software system.

Reliability: Reliability of a software system derives from

- Correctness
- Availability

The behavior over time for the fulfillment of a given specification depends on the reliability of the software system.

Reliability of a software system is defined as the probability that this system fulfills a function (determined by the specifications) for a specified number of input trials under specified input conditions in a specified time interval (assuming that hardware and input are free of errors).

A software system can be seen as reliable if this test produces a low error rate (i.e., the probability that an error will occur in a specified time interval.)

The error rate depends on the frequency of inputs and on the probability that an individual input will lead to an error.

Adequacy: Factors for the requirement of Adequacy:

- The input required of the user should be limited to only what is necessary. The software system should expect information only if it is necessary for the functions that the user wishes to carry out. The software system should enable flexible data input on the part of the user and should carry out plausibility checks on the input. In dialog-driven software systems, we vest particular importance in the uniformity, clarity and simplicity of the dialogs.

- The performance offered by the software system should be adapted to the wishes of the user with the consideration given to extensibility; i.e., the functions should be limited to these in the specification.

- The results produced by the software system: The results that a software system delivers should be output in a clear and wellstructured form and be easy to interpret. The software system should afford the user flexibility with respect to the scope, the degree of detail, and the form of presentation of the results. Error messages must be provided in a form that is comprehensible for the user.

Learnability: Learnability of a software system depends on:

- The design of user interfaces
- The clarity and the simplicity of the user instructions (tutorial or user manual).

The user interface should present information as close to reality as possible and permit efficient utilization of the software’s failures.

The user manual should be structured clearly and simply and be free of all dead weight. It should explain to the user what the software system should do, how the individual functions are activated, what relationships exist between functions, and which exceptions might arise and how they can be corrected. In addition, the user manual should serve as a reference that supports the user in quickly and comfortably finding the correct answers to questions.

Robustness: Robustness reduces the impact of operational mistakes, erroneous input data, and hardware errors.

A software system is robust if the consequences of an error in its operation, in the input, or in the hardware, in relation to a given application, are inversely proportional to the probability of the occurrence of this error in the given application.

- Frequent errors (e.g. erroneous commands, typing errors) must be handled with particular care.

- Less frequent errors (e.g. power failure) can be handled more laxly, but still must not lead to irreversible consequences.

Maintainability: Maintainability = suitability for debugging (localization and correction of errors) and for modification and extension of functionality.

The maintainability of a software system depends on its:

- Readability
- Extensibility
- Testability

Readability: Readability of a software system depends on its:

- Form of representation
- Programming style
- Consistency
- Readability of the implementation programming languages
- Structuredness of the system
- Quality of the documentation
- Tools available for inspection

Extensibility: Extensibility allows required modifications at the appropriate locations to be made without undesirable side effects. Extensibility of a software system depends on its:

- Structuredness (modularity) of the software system
- Possibilities that the implementation language provides for this purpose
- Readability (to find the appropriate location) of the code
- Availability of comprehensible program documentation

Testability: suitability for allowing the programmer to follow program execution (runtime behavior under given conditions) and for debugging. The testability of a software system depends on its:

- Modularity
- Structuredness

Modular, well-structured programs prove more suitable for systematic, stepwise testing than monolithic, unstructured programs.

Testing tools and the possibility of formulating consistency conditions (assertions) in the source code reduce the testing effort and provide important prerequisites for the extensive, systematic testing of all system components.

Efficiency: ability of a software system to fulfill its purpose with the best possible utilization of all necessary resources (time, storage, transmission channels, and peripherals).

Portability: the ease with which a software system can be adapted to run on computers other than the one for which it was designed.

The portability of a software system depends on:

- Degree of hardware independence
- Implementation language
- Extent of exploitation of specialized system functions
- Hardware properties
- Structuredness: System-dependent elements are collected in easily interchangeable program components.

A software system can be said to be portable if the effort required for porting it proves significantly less than the effort necessary for a new implementation.

Q 1. In order to bring about improvement in the testing process and help your team members come up with guidelines on

a) How to stop testing?
b) When to stop Testing?
c) How to test with constraint on resources?

Q 2. You are the project leader for the team and the team members are fresh from colleges. During the reviews and testing they are finding a lot defects but they are not reporting all of them for the fear that would reflect in there performance appraisals. How do you handle this type of situations and improve the quality?

Q3. You as a QC person, what type of test end report will you prepare for the PL of the concerned project? What will you report and why?

Q4. You are the new IS Manager of an investment company. Your team needs to develop an online trading system within 2 months with no increase in team size. An online (buy / sell) transaction has to be completed within 2 seconds and the online search should not take more than 5 seconds. How should you approach this to get the software developed with the best quality and within schedule?

Q5. A new team member has joined the project and he has to prepare Unit Test plan for the module. He says that he would prepare UT after completing the code development. As the Project Leader would you agree with this? Explain.

Q6. You are passing by the cubicle of Broadcast project manager, Mr. Paul and you overhead him talking to him on the phone.

Mr. Paul was talking to his client “OK, Initially we planned about 25% efforts to be spent in System Testing but As you know we slipped the Design by 5% so I promise you I would adjust this efforts in System Test by reducing the System Test efforts- instead of 25% – we will now plan for 20%.By doing so I am still hopeful to meet delivery deadlines.

You guessed that Mr. Paul is negotiating the System Test efforts. What would be your action points to Convince Mr. Paul.

Q7. Paul had just finished meeting System Testing people. He seems to be upset. He held a meeting to find out the status of Testing and basically wanted to know how many Faults were Fixed in the Previous build 3.0. The Test Team Leader was away, so he could get hold of 4 testers.

- Two Testers said. We have tested the build 3.0 and there are no showstopper Faults.
- The Other Tester said – we Tested 3.0 Thoroughly and we could find some problems.

Apparently, Paul got confused and wanted to know what really is happening. He asked the Test Team leader who joined late in the meeting. He asked him the same question. The Test Team leader said: we have tested the Build 3.0 and there were many show stopper issues.

Paul could not make anything and just came out of the meeting. Paul feels that there should be some system to get the correct status of Testing.

What kind of approach you will take to eliminate these kind of problems?

Also See:

• CSQA Subjective questions sample paper – 1

Many think that defect-free products and services are not practical or economical, and thus believe some level of defects is normal and acceptable. (This is called acceptable quality level, or AQL.) Quality experts agree that AQL is not a suitable definition of quality. As long as management is willing to “accept” defective products, the entire quality program will be in jeopardy.

Quality is frequently associated with cost, meaning that high quality is synonymous with high cost. (This is confusion between quality of design and quality of conformance.) Organizations may be reluctant to spend on quality assurance, as they do not see an immediate payback.

Quality by definition calls for requirements/specifications in enough detail so that the products produced can be quantitatively measured against those specifications. Few organizations are willing to expend the effort to produce requirements / specifications at the level of detail required for quantitative measurement.

Many technical personnel believe that standards inhibit their creativity, and thus do not strive for compliance to standards. However, for quality to happen there must be welldefined standards and procedures that are followed.

The contributors to poor quality in many organizations can be categorized as either lack of involvement by management, or lack of knowledge about quality. Following are some of the specific contributors for these two categories:

Lack of involvement by management:
- Management’s unwillingness to accept full responsibility for all defects
- Failure to determine the cost associated with defects (i.e., poor quality)
- Failure to initiate a program to “manage defects”
- Lack of emphasis on processes and measurement
- Failure to enforce standards
- Failure to reward people for following processes

Lack of knowledge about quality:
- Lack of a quality vocabulary, which makes it difficult to communicate quality problems and objectives
- Lack of knowledge of the principles of quality (i.e., what is necessary to make it happen)
- No categorization scheme for defects (i.e., naming of defects by type)
- No information on the occurrence of defects by type, by frequency, and by location
- Unknown defect expectation rates for new products
- Defect-prone processes unknown or unidentified
- Defect-prone products unknown or unidentified
- An economical means for identifying defects unknown
- Proven quality solutions are unknown and unused

If achieving quality (i.e., defect-free products and services) were easy, it would have been accomplished years ago. Quality is very difficult to accomplish – it requires the close cooperation of management and staff. Achieving quality requires a commitment and the establishment of an environment in which quality can flourish.

Joseph Juran has explained his model of quality improvement on the basis of the basis of three universal processes which have been popularly named a Juran Trilogy.

The processes are:

1. Quality Planning: As per Juran Triology quality planning is a concurrent exercise which involves all the affected parties related to the product and services, so that they can provide inputs and give early warnings during the planning processes.

The steps of the quality planning exercise are:

- Definition of the project.
- Identification of the customers – those who will be impacted by the actions that are taken to complete the project.
- Discovery of customer needs.
- Development of the product and processes to meet the customer’s needs.
- Establishment of the quality objectives.
- Development of the plans for meeting these objectives.

2. Quality Control: According to Juran Triology Quality control involves the developing and maintaining of operational methods in order to assure that the processes work as they are designed to work and that the target levels of performance being are being achieved. Quality control does not concern itself with improving a process, but rather with the execution of plans. It is primarily to control that occasional spike in error in the process. Quality control is a short term process to check that spike.

Quality control entails the following steps:

- Clear definitions of quality.
- Knowledge of the expected performance or targets.
- Evaluation of the actual operating performance.
- Comparison of the actual performance to goals.
- Action of the difference.

3. Quality Improvement: As per Juran Triology, quality improvement is a disciplined approach that improves the level of performance of the process. This is achieved by a breakthrough improvement in performance; when a new innovation or a completely fresh idea is brought into improve the current performance levels. This ensures that the new levels of performance are achieved, and then quality control mechanisms are in place to sustain that effectively.

Also See:

• Deming’s 14 Quality Principles
• Philip Crosby’s concept of Quality Improvement

Management’s role in Continuous Improvement:

- Establish an organization-wide quality council
- Establish specific quality improvement goals with timetables and target dates
- Provide support
- Schedule periodic reviews

Structure for Quality Improvements:

- Establish a quality Council
- Develop a statement of responsibilities
- Establishing the necessary infrastructure

Essential Improvement Activities:

- Communicate
- Correction Obvious Problems
- Find causes
- Documentation
- Monitor changes

The Scientific Approach:

- Collect meaningful data
- Identify root causes of problems
- Develop appropriate solutions
- Plan and make changes

Identification of Improvement needs:

- Apply multi voting
- Identify customer needs
- Study the use of time
- Localize problems

Development of Improvement Plans:

- Understand the process
- Eliminate errors
- Remove slack
- Reduce variation
- Plan for continuous improvement

Common Improvement Strategies:

- Describe the Process
- Standardize the Process
- Eliminate Errors in the Process
- Streamline the Process
- Eliminate Variation
- Get the process in statistical control
- Improve the Design