Software testing

On this page 8

Software testing is the disciplined practice of executing a program or its components under controlled conditions to detect defects, measure behavior against requirements and gather evidence about whether the product is fit for release. Within Software quality, testing is the main empirical technique: it does not create quality, but it reveals where quality is missing and how much risk remains.

Testing can show the presence of defects, never their absence. For that reason, professional testing is organized around risk, choosing which behavior to examine, at which level and with which techniques, so that the most damaging failures are found before users meet them.

How software testing works

Every test follows the same basic structure: known inputs, a defined execution and an expected result. The expected result comes from a test oracle, which may be a specification, a previous version, a mathematical property or a human judgment. When the actual result differs from the oracle, the tester records a failure and investigates whether it reveals a defect.

Testers distinguish three related terms precisely. An error is a human mistake, a defect (or fault) is the resulting flaw in code or configuration, and a failure is the observable deviation at runtime. A single defect may cause many failures, or none at all until a rare condition triggers it.

Testing work is commonly organized as a cycle of planning, analysis, design, implementation, execution and completion, a structure reflected in standards such as ISO/IEC/IEEE 29119. In agile teams these activities shrink into each iteration, but none of them disappears.

Levels of software testing

Testing levels describe how much of the system a test exercises at once. Lower levels run fast and pinpoint defects precisely, while higher levels reveal problems that only appear when parts interact.

LevelScopeTypical ownerExecution speed
Unit testingSingle function, class or moduleDevelopersMilliseconds per test
Integration testingInterfaces between components or servicesDevelopers and testersSeconds per test
System testingComplete application in a realistic environmentTest teamMinutes per scenario
Acceptance testingBusiness requirements and user workflowsCustomers or product ownersMinutes to hours
Main testing levels and their characteristics

A widely used heuristic is the test pyramid: many unit tests, fewer integration tests and a small number of end-to-end tests. The pyramid reflects cost, because end-to-end tests are slower, more fragile and harder to diagnose. Teams that invert it, relying mostly on user-interface tests, typically suffer from long feedback cycles and unreliable results.

Testing types and techniques

Testing types describe what quality attribute is examined, independently of level. Functional testing checks what the software does, while non-functional testing checks how well it does it, covering performance, security, usability and reliability.

Test design techniques determine how testers choose a small, effective set of cases from an almost infinite input space. The table below summarizes the most established techniques.

TechniqueBasisWhat it finds well
Equivalence partitioningInput classes treated the same wayWrong handling of whole input groups
Boundary value analysisEdges of valid rangesOff-by-one and limit errors
Decision table testingCombinations of conditionsMissing or conflicting business rules
State transition testingStates and allowed transitionsInvalid sequences and stuck states
Statement and branch coverageCode structureUntested paths in the code
Exploratory testingTester skill and learning in real timeUnexpected behavior and usability issues
Common test design techniques

Techniques fall into black-box methods, which rely on specifications, and white-box methods, which rely on code structure. Experienced teams combine both, adding experience-based testing where defects historically cluster.

Software testing is often confused with neighboring quality practices, yet each answers a different question. Testing executes the software, whereas several related practices examine code or processes without running it.

PracticeRuns the codeMain goal
Software testingYesDetect failures and measure behavior
Quality assuranceNot necessarilyImprove processes that prevent defects
Static code analysisNoFind defects and risky patterns in source code
Code reviewNoCatch defects and design issues through peer reading
DebuggingYesLocate and remove the cause of a known failure
Telemetry monitoringYes, in productionObserve real behavior after release
Software testing and related quality practices

The distinction between testing and debugging matters in practice. Testing reveals that something fails; debugging and Root cause analysis explain why. Mixing the two in one role is common, but treating them as one activity often leads to tests being weakened until they pass.

Test automation and continuous integration

Test automation uses code to execute tests and compare results without manual effort. Its main value is repeatability: an automated suite can run on every commit in a continuous integration pipeline, catching a regression within minutes of the change that caused it.

Automation is not free. Automated tests are software themselves and need maintenance, and poorly designed suites accumulate flaky tests that pass and fail without code changes, which teaches developers to ignore red builds. The table below shows where automation pays off and where manual work remains stronger.

SituationRecommended approachReason
Stable logic checked on every changeAutomate at unit or API levelHigh repetition, low maintenance
Critical user journeysAutomate a few end-to-end testsGuards release-blocking paths
New or rapidly changing featuresExplore manually firstAutomation would be rewritten often
Visual design and usabilityManual review with targeted checksHuman judgment is the oracle
Load and performance limitsAutomated tools with fixed scenariosHumans cannot generate the load
When to automate a test

Code coverage metrics show which lines or branches the tests execute, but high coverage does not prove good tests. A suite can reach 90 percent line coverage with weak assertions that would not detect most defects. Mutation testing, which deliberately injects small faults and checks whether tests catch them, gives a stronger signal of suite effectiveness.

Limitations and common mistakes

Exhaustive testing is impossible for all but trivial programs, because the number of input combinations and execution paths grows explosively. Every test strategy therefore leaves residual risk, and the key professional skill is deciding which risk is acceptable.

  • Testing only the happy path misses error handling, where many severe defects hide.
  • Testing late turns cheap defects into expensive ones, since fixes cost more after integration and release.
  • Weak test oracles let wrong results pass because nobody defined the expected outcome precisely.
  • Unrealistic test data and environments hide defects that appear only with production volumes or configurations.
  • Treating pass rates as quality ignores what the tests do not cover and encourages gaming the metric.

Testing also loses value when its findings are poorly communicated. A clear bug report with reproduction steps, environment details and expected versus actual results often matters as much as the test that found the defect.

The short version

Software testing means trying out a program on purpose to catch its mistakes before real users do. It is similar to a building inspector checking wiring, plumbing and fire exits before people move in.

Testers feed the program known inputs and compare what happens with what should happen. When the two differ, they have found a problem to fix. Testing cannot prove a program is perfect, but it greatly lowers the chance of unpleasant surprises.

Where it makes a difference

A mid-sized online retailer is changing how its checkout calculates discounts. The development team writes about 120 unit tests covering discount rules, including boundary values such as orders at exactly the free-shipping threshold, and a decision table for combined coupons and loyalty points.

In continuous integration, one boundary test fails: an order of exactly 50.00 receives free shipping twice when a coupon is also applied. The defect is fixed the same afternoon. Before release, testers run five automated end-to-end checkout journeys and spend two hours on exploratory testing with mobile browsers, which reveals a layout problem in the coupon field. The release ships during a seasonal sale without a single pricing incident, avoiding refunds and support calls that a double-discount error would have caused at peak traffic.