Skip to main content

Quality Assurance and Quality Control

  • In ISTQB, Quality Assurance (QA) and Quality Control (QC) are crucial concepts in software development aimed at ensuring the final product meets the required quality standards. These two concepts, while related, have different focuses. Quality Assurance is a proactive process that focuses on improving development and testing processes to prevent defects, while Quality Control is a reactive process that involves identifying and fixing defects in the final product. Here’s an overview of ISTQB terminologies and concepts related to Quality Assurance and Quality Control:

Quality Assurance (QA)​

  • Quality Assurance refers to the systematic activities and practices applied in a project to ensure that processes, products, and services meet specified requirements and quality standards. QA focuses on process improvement and is often preventative in nature.

Key Concepts​

  • Process-Oriented: QA is about improving and optimizing processes, ensuring that they are well-defined, structured, and capable of producing high-quality products.

  • Proactive Approach: QA is focused on preventing defects before they occur by improving development and testing processes.

  • Standards and Guidelines: QA involves establishing standards and guidelines, such as ISO 9000, CMMI (Capability Maturity Model Integration), or IEEE standards, which are used to maintain and improve software quality.

  • Verification: QA includes activities such as audits, reviews, and process evaluations to ensure that the software is being developed according to the prescribed processes and that quality standards are followed. Ensures that the product is being developed in the right way (i.e., are we building the product correctly?). It focuses on activities like process checks, design reviews, and inspections.

Quality Control (QC)​

  • Quality Control refers to activities focused on identifying defects in the actual products or services delivered to customers. QC is product-oriented and involves executing processes to detect, manage, and correct defects.

Key Concepts​

  • Product-Oriented: QC is concerned with the quality of the actual product being developed or delivered, rather than the processes used to create it.

  • Reactive Approach: QC involves finding defects and ensuring that they are addressed and resolved before the product is delivered to customers.

  • Testing: QC involves the execution of test cases to verify that the product behaves as expected, identifying any defects or issues that need to be fixed.

  • Validation: QC includes testing and other activities to ensure that the product meets the user requirements and functions correctly. Ensures that the product meets the user’s needs and requirements (i.e., are we building the right product?). It focuses on actual product testing, ensuring that the product performs as expected in the real world.

QA and QC Processes​

Both QA and QC involve distinct processes and activities. In ISTQB terms, there is an emphasis on structured processes for both QA and QC, which ensures high standards of software quality.

QA Processes​

  • Process Definition and Improvement: QA involves defining and improving processes to ensure quality, such as code reviews, design reviews, and process audits. Metrics and Reporting: QA teams often establish quality metrics to monitor and improve process quality. ie: Defect Density and Test Coverage.

  • Process Audits: Regular audits are conducted to verify that the development and testing processes are followed and are effective.

QC Processes​

  • Test Case Design and Execution: QC involves creating test cases based on user requirements, executing those tests, and identifying defects.

  • Defect Reporting and Tracking: QC includes the process of logging, tracking, and managing defects to resolution.

  • Regression Testing: This ensures that new code changes do not introduce new defects and that previously fixed defects do not reappear.

Verification and Validation​

  • Verification and Validation (V and V) are critical concepts that span both QA and QC. They ensure that the software is both built correctly and meets the user's requirements.

Verification (QA Activity)​

  • To ensure the product is developed in the right way and adheres to standards and processes.

Validation (QC Activity)​

  • To ensure the product meets the specified requirements and performs as expected in real-world conditions.

Continuous Improvement in QA and QC​

  • Both QA and QC promote the concept of continuous improvement to enhance the quality of processes and products. ie: PDCA: Plan, Do, Check, Act

Key Concepts​

  • Plan: Identify objectives and processes to improve.

  • Do: Implement changes or new processes.

  • Check: Monitor and measure the effectiveness of the changes.

  • Act: Take corrective actions and improve processes further based on feedback.

Quality Models for Continuous Improvement​

  • CMMI (Capability Maturity Model Integration): A process improvement model that provides organizations with the essential elements for effective process management and continuous improvement.

  • TQM (Total Quality Management): An organization-wide approach focused on continuous improvement in quality, customer satisfaction, and the overall quality of all processes.

Test Process Improvement (TPI)​

  • ISTQB emphasizes the importance of improving the test process through structured methodologies. One such framework is Test Process Improvement (TPI), which focuses on improving the effectiveness and efficiency of the testing process.

Key Concepts​

  • TPI Model: A framework to assess the maturity of an organization's testing process and provides guidelines for improving it.

  • Test Maturity Levels:

    • Initial: Testing is ad-hoc and chaotic.

    • Managed: Testing is organized and repeatable.

    • Defined: Testing processes are documented and standardized.

    • Measured: Quantitative metrics are used to manage testing.

    • Optimized: Continuous improvement is ingrained into the testing process.

Root Cause Analysis (RCA)​

  • Root Cause Analysis (RCA) is a quality assurance technique used to identify the underlying causes of defects or issues in software development processes. The goal of RCA is to identify process weaknesses that lead to defects and address them to prevent future occurrences.

Key Concepts​

  • Fishbone Diagram (Ishikawa): A visual tool used to identify the root causes of a defect, typically categorizing causes into areas such as people, processes, technology, and environment.

  • Five Whys: A technique where you repeatedly ask "why" to drill down into the root cause of a defect or issue.

Key Metrics in QA and QC​

  • Metrics play a critical role in assessing the effectiveness of both QA and QC activities. Some commonly used metrics include:

QA Metrics​

  • Process Adherence: Measures how closely development and testing teams follow defined processes.

  • Defect Prevention Ratio: The percentage of defects prevented due to QA activities such as code reviews, process improvements, etc.

QC Metrics​

  • Defect Density: The number of defects found per unit of software. ie: Per thousand lines of code.

  • Test Coverage: The percentage of code, requirements, or functionalities that have been tested.

  • Defect Removal Efficiency: The percentage of defects identified and removed during the development process.

Test Policy and Test Strategy​

  • In ISTQB, Test Policy and Test Strategy documents are often used in QA to define the overall approach to quality. These documents help ensure that quality is maintained across different projects and releases.

Test Policy​

  • A high-level document that defines the organization's overall approach to ensuring quality in software products. It may include the company's quality goals, the scope of quality activities, and the responsibilities of different teams.

Test Strategy​

  • A document that defines how testing will be conducted on a specific project. It describes the testing objectives, the approach for different types of testing, the tools to be used, and the roles and responsibilities of the testing team.

Test Automation​

  • In ISTQB, Test Automation refers to the use of specialized software tools to automate the execution of tests, the comparison of actual outcomes to expected outcomes, and the reporting of results. Test automation aims to increase the efficiency, reliability, and repeatability of the testing process, particularly for repetitive tasks such as regression testing.

  • Test Automation involves the use of automated scripts and tools to perform tests without human intervention, typically focusing on reducing manual effort, increasing the speed of testing, and improving test accuracy.

Key Concepts​

  • Automation Testing: The process of automating the manual test execution steps.

  • Test Scripts: Predefined programs or sets of instructions that automate test cases. These scripts can be created using programming languages or specialized automation tools.

  • Test Automation Framework: A structured set of guidelines or rules for creating and managing automated test scripts. It typically includes coding standards, reusable functions, object repositories, and reporting mechanisms.

  • Tools: Test automation tools are software applications that help automate tests. Examples include Selenium, JUnit, TestNG, UFT (Unified Functional Testing), and Cucumber.

Types of Test Automation​

  • Test Automation can be applied at different levels of testing and to different types of tests:

Unit Test Automation​

  • Automates tests at the unit level, testing individual units or components of the software. These tests are usually written by developers and ensure that small pieces of code work as expected.

  • Example Tools: JUnit, NUnit, xUnit, TestNG.

API and Service-Level Automation:​

  • Focuses on automating tests for APIs or services that do not have a user interface. These tests validate the integration and behavior of system components.

  • Example Tools: Postman, RestAssured, SoapUI.

GUI Test Automation:​

  • Automates Graphical User Interface (GUI) tests to validate that the application behaves correctly from an end-user perspective.

  • Example Tools: Selenium, UFT, Appium.

Regression Test Automation:​

  • Automates the testing of previously tested functionality to ensure that new changes have not introduced new defects. ie: Bug Fixes and New Features.

Performance Test Automation:​

  • Automates tests that evaluate how well an application performs under specific workloads, typically measuring response times, throughput, and resource usage.

  • Example Tools: JMeter, LoadRunner.

Test Automation Frameworks​

  • A Test Automation Framework is a set of guidelines, best practices, and tools that help in structuring and managing automated tests. The goal of a framework is to improve the efficiency and effectiveness of test automation.

Key Types of Frameworks​

  • Linear Scripting Framework: A basic automation framework where test scripts are written sequentially with minimal reuse. Each script is written individually.

  • Modular Framework: The application is broken into smaller, independent scripts (modules). These modules are reusable and can be combined to form more complex test cases.

  • Data-Driven Framework: Involves storing test data separately from test scripts. ie: Spreadsheets or Databases.

    • This allows the same script to run with different sets of data inputs, improving reusability and scalability.

  • Keyword-Driven Framework: In this framework, test cases are driven by keywords that define a specific action or set of actions to be performed. It separates the test logic from the test scripts, making it easier to manage and maintain.

  • Hybrid Framework: Combines the best aspects of multiple frameworks to offer a more flexible and efficient automation structure. ie: Modular, Data-Driven, and Keyword-Driven.

Test Automation Life Cycle (TALC)​

  • The Test Automation Life Cycle outlines the stages involved in planning, developing, executing, and maintaining automated tests. TALC is an essential concept in ISTQB to ensure that test automation is well-organized and aligned with project objectives.

Stages of the Test Automation Life Cycle​

  • Decision to Automate: Determine if automation is feasible and beneficial for the project. Evaluate the scope, types of tests, and areas to automate.

  • Test Tool Selection: Identify and select the most appropriate tools for test automation based on the project’s requirements, technology stack, and team expertise.

  • Automation Framework Development: Develop or adopt a test automation framework suited for the project. The framework should ensure reusability, scalability, and maintainability of automated test scripts.

  • Test Script Development: Write the automated test scripts, adhering to the framework guidelines and best practices.

  • Test Execution: Execute the automated scripts on the desired environment. This can be scheduled or triggered as part of continuous integration / continuous deployment (CI / CD).

  • Maintenance: Regularly update and maintain test scripts as the software evolves to ensure they remain functional and relevant.

Test Automation Pyramid​

  • The Test Automation Pyramid is a concept that emphasizes the distribution of automated tests at different levels of the software system. It ensures that most automated tests are focused on the lower levels (unit tests) and fewer on the higher levels (GUI tests), as lower-level tests are generally faster and more stable.

Layers of the Automation Pyramid​

  • The pyramid encourages more tests at the unit and API levels, ensuring fast feedback with minimal overhead, while keeping UI tests minimal but effective.

-** Unit Tests (Bottom Layer)**: Fast, reliable, and easy-to-maintain tests focused on individual units of code.

-** Service / API Tests (Middle Layer)**: Tests the interactions between services or components without involving the user interface.

-** UI Tests (Top Layer)**: Focus on testing the application from the user’s perspective. These tests are generally slower and more fragile than unit or API tests.

Continuous Integration (CI) and Continuous Testing​

  • In ISTQB, Continuous Integration (CI) and Continuous Testing are closely associated with test automation. Continuous Integration ensures that code changes are automatically built, tested, and merged into the codebase, while Continuous Testing focuses on automated testing at every stage of the CI pipeline.

Key Concepts​

  • CI Tools: Tools like Jenkins, Travis CI, and CircleCI are used to automate the building, testing, and deployment of code in a CI environment.

  • Continuous Testing: Involves running automated tests frequently and consistently, usually as part of the CI / CD pipeline. Automated regression tests, unit tests, and integration tests are executed after every code change or deployment.

Test Automation in Agile​

  • In Agile methodologies, test automation plays a key role in ensuring that testing keeps pace with rapid development cycles.

Key Concepts​

  • Test-Driven Development (TDD): A software development process where tests are written before the code. Developers first write unit tests, which fail initially, and then develop code to pass the tests.

  • Behavior-Driven Development (BDD): Focuses on collaboration between developers, testers, and business stakeholders. Test cases are written in plain language using keywords, and these can be automated using tools like Cucumber. ie: Gherkin Syntax.

  • Automated Regression Testing: In Agile, automated regression tests ensure that the frequent changes do not introduce new defects into previously tested functionality.

Test Automation Tools​

  • There are many tools available for automating different types of tests. The selection of tools depends on the type of tests being automated, the technology stack, and the team’s expertise.

Common Test Automation Tools

  • Selenium: A widely-used open-source tool for automating web applications.

  • Appium: An open-source automation tool for mobile applications.

  • TestComplete: A commercial tool that supports both desktop and web application automation.

  • Cypress: A modern web testing tool that focuses on the front-end and UI testing.

  • Postman: A popular tool for automating API testing.

Challenges in Test Automation​

  • Test automation can bring significant benefits but also comes with its own set of challenges:

Key Challenges

  • High Initial Investment: Setting up the automation framework and writing automated tests requires significant initial effort and investment.

  • Test Maintenance: Automated test scripts need to be continuously maintained, especially as the application evolves, to ensure they remain relevant and accurate.

  • False Positives / Negatives: Automated tests can sometimes produce false positives (tests fail without a real issue) or false negatives (tests pass even though there is a defect).

  • Tool Selection: Selecting the right tools that are compatible with the application’s technology stack can be challenging.

Benefits of Test Automation​

  • Despite the challenges, test automation provides several benefits that enhance the overall quality and efficiency of the testing process.

Key Benefits

  • Speed: Automated tests are faster than manual tests, especially for regression testing or repeated test executions.

  • Repeatability: Automated tests can be executed repeatedly with the same accuracy and consistency, reducing human error.

  • Coverage: Automation enables the testing of a larger number of scenarios and data sets than would be feasible with manual testing.

  • Feedback: Automated testing provides fast feedback to developers, helping them identify issues early in the development process.

Module Review​

Click to start the definition to term matching quiz
Drag the defintion to the correct term.
Test type item not available at this time.
Click to start the multiple choice quiz
Choose from the listed options below.
Test type item not available at this time.