Software Engineering

Lecture 12
Software Testing and Validation
● Purpose of Software Testing
○ Ensures a program functions as intended.
○ Identifies defects before deployment.
○ Executes the program with artificial data to check for errors.
○ Evaluates both functional and non-functional attributes.
● Goals of Software Testing
○ Validation Testing: Confirms that the software meets requirements.
■ Custom software: At least one test per requirement.
■ Generic software: Tests for all release features.
■ Includes combination testing to detect unwanted interactions.
○ Defect Testing: Identifies incorrect, undesirable, or non-conforming behavior.
■ Finds bugs causing crashes, incorrect computations, and data corruption.
Validation vs. Defect Testing
● Validation Testing: Uses expected inputs to ensure correct output.
● Defect Testing: Uses obscure test cases to expose flaws.
● Overlap exists: Validation testing may find defects, and defect testing may confirm
requirements.
● Software Verification & Validation (V&V)
○ Verification: "Are we building the product right?"
○ Ensures the software meets its stated requirements.
○ Validation: "Are we building the right product?"
○ Ensures software meets customer expectations.
● Importance of V&V
○ Builds confidence that software is fit for purpose.
○ Required confidence level varies based on:
■ System purpose, User expectations, Market conditions
Factors Affecting Software Reliability
● Software Purpose & Reliability
○ Critical software demands high reliability (e.g., safety-critical systems).
○ Demonstrator systems have lower reliability requirements.
● User Expectations
○ Users tolerate failures initially but expect improved reliability over time.
○ More rigorous testing is needed for established software versions.
● Marketing Environment
○ Competition, pricing, and release schedules affect testing decisions.
○ Companies may release unpolished software to be first to market.
Verification & Validation (V&V) Techniques
● Software Inspections & Reviews:
○ Static techniques analyzing system requirements, design models, and source code.
○ Identify defects without executing the software.
● Software Testing:
○ Validates system performance using test cases.
○ Checks for unexpected interactions, timing issues, and performance problems.
● Advantages of Software Inspection
○ Detects multiple errors in a single session.
○ No need for fully developed systems to perform inspections.
○ Identifies inefficiencies, poor programming styles, and compliance issues.
Limitations of Inspections
● Cannot detect interaction-based defects or performance issues.
● Requires dedicated inspection teams, which may not be feasible for small teams.
● Testing Process Overview
○ Test cases specify inputs, expected outputs, and testing focus.
○ Test data is devised manually or generated automatically.
○ Automated test execution compares actual and expected results.
● Three Stages of Software Testing
● Development Testing:
○ Conducted during development to find bugs and defects.
○ Involves system designers and programmers.
● Release Testing:
○ A separate team tests a complete system before release.
○ Ensures system meets stakeholder requirements.
● User Testing:
○ Users or internal groups test the system in real environments.
○ Includes Acceptance Testing, where customers decide on system acceptance.
Manual vs. Automated Testing
● Manual Testing:
● Testers run the program, compare results, and report discrepancies.
● Automated Testing:
● Encodes tests in a program for faster execution.
● Crucial for regression testing to check changes.
● Limitations:
● Automated tests can’t assess UI aesthetics or unexpected side effects.
● Development Testing Stages
○ Unit Testing: Tests individual program units or classes.
○ Component Testing: Tests integrated units focusing on interfaces.
○ System Testing: Tests integrated components to evaluate system interaction
Development Testing as Defect Testing
● Identifies software bugs before release.
● Often interleaved with debugging to fix identified issues.
● Some processes use programmer/tester pairs for thorough validation.
● Critical systems may involve a dedicated testing group for test development and record-keeping.
What is Unit Testing?
● Testing individual components like methods or object classes.
● Calls routines with different input parameters.
● Ensures functionality coverage of all object operations and attributes.
● Testing Object Classes
○ Ensure coverage of:
○ All operations of an object.
○ Setting and checking attribute values.
○ Simulating all events that cause state changes.
○ Example: Weather station object testing (attributes, methods, and states).
● Challenges with Inheritance
○ Inherited operations may rely on assumptions that aren’t valid in subclasses.
○ Must test inherited operations in all subclasses.
State Testing Approach
● Use a state model to identify sequences of state transitions.
● Example test sequences for a weather station:
● Shutdown → Running → Shutdown
● Configuring → Running → Testing → Transmitting → Running
● Running → Collecting → Summarizing → Transmitting → Running
● Automated Unit Testing
○ Uses frameworks like JUnit to write and execute tests.
○ Test execution provides instant feedback through GUI reports.
○ Automated test consists of:
■ Setup: Initialize test inputs and expected outputs.
■ Call: Execute the method under test.
■ Assertion: Compare actual vs. expected results.
Using Mock Objects
● Simulate dependencies that may not be
implemented or are slow (e.g., databases).
● Allow testing rare conditions and
exceptional cases.
● Example: Mock object simulates specific
times for scheduled tasks.