Implementing ATDD (Acceptance Test-Driven Development): Differences from BDD and TDD, and Adoption Patterns

ATDD (Acceptance Test-Driven Development) is a methodology in which acceptance tests derived from business requirements are defined before implementation, and development proceeds to satisfy those tests. It can be described as an approach born to prevent situations like "even though tests were written, discrepancies in requirement interpretation are discovered after release." This article is aimed at QA engineers, test automation specialists, and agile development team leads, and explains the implementation steps for gradually introducing ATDD in combination with BDD and TDD. By tracing through the entire flow—how to articulate acceptance tests in language during the requirements definition phase, and how to automate them and incorporate them into a CI/CD pipeline—the outline of a testing strategy that prevents requirement omissions while maintaining development efficiency should become clear.

ATDD is a development methodology that converts business requirements into the form of tests, agreed upon among stakeholders before implementation. Even in the ISTQB definition, acceptance testing is positioned as a means of confirming that requirements are satisfied. When requirements are handed off to implementation in natural language as-is, discrepancies in interpretation between developers and the business side are prone to occur, and there are no small number of cases where such discrepancies surface for the first time only at the integration testing or acceptance testing stage. ATDD is a methodology aimed at surfacing these interpretive discrepancies before coding begins. In the next H3, we will look at how this concept is concretized as a development flow, and how it differs from conventional methodologies.

When developing a new feature, the process branches to start from requirements discussion; when modifying an existing feature, it branches to start from a review of the current acceptance tests. As touched on earlier, what supports ATDD is the very ordering of "fixing requirements first, then writing code," and whether this ordering can be maintained determines the success or failure of the operation. The flow is broadly divided into three stages, but it is worth spending the most time on the initial requirements definition stage.

At the requirements definition stage, the product owner, developers, and QA engineers discuss together and translate business requirements into concrete acceptance conditions. Using a structure like Gherkin notation's Given/When/Then here can prevent conditions from proceeding while remaining ambiguous. For example, a requirement such as "an account is locked after three consecutive failed login attempts" becomes far less prone to interpretive discrepancies between developers and QA simply by making explicit the initial state (an unlocked account) in Given, the operation (three failed login attempts) in When, and the outcome (transition to a locked state) in Then. Conversely, if this stage is skipped and the process proceeds with something like "let's roughly make it behave this way," omissions in conditions tend to surface during test design in a later process, often requiring the discussion to be redone from scratch.

At the test design stage, the agreed-upon acceptance conditions are converted into executable acceptance tests. Tools introduced even in official documentation, such as Cucumber, are characterized by the ability to treat specifications written in plain text directly as executable tests. Tests are created before the implementation code, starting from a failing state.

The implementation stage is comparatively simple: developers write code with the sole goal of passing the acceptance tests. Since passing the tests can be considered evidence that the requirements are satisfied, discrepancies between requirements and implementation can be detected early. The reason this approach can compensate for gaps in business perspective that are hard to catch with unit tests alone is precisely this ordering of working backward from requirements to tests.

In conventional testing approaches, it was common for developers to write test code after finishing implementation, or for QA teams to create test cases by interpreting specification documents. In this ordering, testing remains confined to the position of "a means of verifying implementation," and does not function to surface ambiguities in the requirements themselves beforehand. In implementation review settings, one often encounters situations where, because tests are written after implementation, test conditions end up being adjusted after the fact to suit the convenience of the code. It can happen that a test originally written with "three boundary values are sufficient" ends up, before anyone notices, reduced to two values to match the branching structure of the implementation. This is not so much a matter of someone cutting corners, but rather a problem with the ordering itself—a force that retroactively justifies the implementation ends up acting on the test side.

The greatest difference is that ATDD reverses this ordering, fixing the acceptance conditions before implementation. Even in the ISTQB glossary, acceptance testing is positioned as "formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria," and ATDD can be described as a design philosophy that places this acceptance testing at the starting point of development.

Another difference is who reads the tests. Conventional test code is written on the assumption that developers or QA engineers will read it, but ATDD acceptance tests aim for plain-text descriptions that product owners and business department staff can also read. This difference reduces the risk of requirement misinterpretations being discovered only after implementation is complete.

Isn't there a moment on the ground where you find yourself unsure whether to choose ATDD, BDD, or TDD? If you prioritize building agreement on business requirements, ATDD is the axis to choose; if you prioritize sharing scenarios in a common language, BDD; and if you prioritize the internal quality of the implementation, TDD—but in actual projects, it is common to see all three used together.

The axis for judgment is "who is involved, and what is being verified." ATDD focuses on the scope of what is being tested and who is involved, and is suited to situations where customers, QA, and developers agree on acceptance criteria in advance. BDD centers on the description format and means of sharing, using Given/When/Then in Gherkin notation, and is chosen when there is a desire to share specifications with non-engineers as well. TDD's decision points are the timing and unit of implementation, and it is adopted when there is a desire to verify internal logic at the function or class level before implementation.

Organizing the relationship among these three: ATDD's purpose is the agreement on acceptance conditions itself, while BDD is often used as a means of describing the content of that agreement in a format close to natural language, such as Gherkin notation—the two are not in opposition but rather combine with each other. Since TDD is a methodology that runs the red-green cycle at the smallest unit of implementation, it functions as a lower layer that further verifies, in finer detail, the internal implementation needed to satisfy the acceptance conditions set by ATDD or BDD.

What tends to be most overlooked in practice is whether time can actually be secured for business-side stakeholders to participate in creating test cases. When such participation proves difficult, ATDD's requirements-agreement phase becomes a mere formality, and the operation tends to drift toward being effectively BDD or TDD alone. If a team that cannot secure time for building agreement introduces ATDD, it can happen that only the acceptance criteria documentation remains, while the consistency check against the implementation ceases to function.

It becomes easier to organize the approaches by considering which layer the test target belongs to and when it is executed.

ATDD is a method for finalizing acceptance criteria at the requirements definition stage, and its scope covers the behavior of the entire system. Its distinguishing feature is that acceptance tests agreed upon by the customer, QA, and developers serve as a gate until implementation is complete. Because the "passing conditions" are fixed before implementation, the implementation timing is immediately after requirements definition, before coding begins.

BDD overlaps significantly with ATDD, but its scope is narrower, focused on the feature or scenario level. The typical approach is to use tools such as Cucumber to write Given/When/Then scenarios in Gherkin notation, adding and executing scenarios within each development iteration. The implementation timing recurs repeatedly with each feature development cycle.

TDD has the smallest scope, verifying the internal quality of implementation units such as classes and methods. The implementation timing coincides with coding and occurs most frequently. Since tests are written before implementation, followed by repeated refactoring cycles at the level of a single function, TDD is positioned as the inner accumulation needed to satisfy the outer acceptance conditions defined by ATDD and BDD.

These three scopes form a nested structure, and understanding ATDD as the outer frame, BDD as the middle layer, and TDD as the inner detail makes it easier to judge the order of introduction. When unsure which layer to start with, the basic approach is to first solidify the passing conditions with the outer frame of ATDD, then fill in the inside with BDD and TDD.

If team members are already familiar with Gherkin notation and automation scripts, it is easier to start with BDD. For teams where the requirements definition and QA processes are not yet well established, starting with ATDD's acceptance criteria formulation keeps the introduction difficulty lower.

There is a tendency to prioritize tool adoption, but in practice, it is more effective to establish the team's consensus-building process first. Even if tools such as Cucumber are introduced first, if the customer, QA, and developers have not aligned on how to write acceptance criteria and at what granularity, tests can easily become a mere formality. The ATDD Workshop materials also show a structure that allocates 180 minutes (Most Preferred) for hands-on exercise sessions, suggesting the importance of securing time for consensus-building before tools.

There are mainly three perspectives for judging the difficulty of introduction: test design skill—whether there are people capable of breaking down acceptance criteria into concrete scenarios; automation skill—whether there are developers who can handle frameworks such as Cucumber, SpecFlow, and FitNesse; and a culture of consensus-building—whether there is a regular forum where the customer, QA, and developers reconcile requirements.

If two or more of these are not yet in place, rather than aiming for a company-wide rollout right away, a more realistic approach is to trial ATDD on a single feature, build up successful patterns internally, and then expand from there.

The ATDD introduction process is divided into three steps: requirements definition, automation, and continuous test execution. However, in practice, the requirements definition phase, which comes first, occupies the majority of the time and effort. If the business side and the development side are misaligned at this stage, no matter how precise the subsequent automation is, it becomes meaningless. Therefore, the greatest emphasis should be placed on writing out acceptance conditions in the Given-When-Then format and reconciling them until all stakeholders can agree using the same language.

The two steps of automation and continuous test execution are, rather, processes that can proceed almost mechanically once the requirements definition is solid. The flow of translating this into an existing test framework, putting it on a CI pipeline, and running it continuously tends not to differ much from team to team, and it is sufficient to think of it as the work of converting the agreement built up during the requirements definition phase into code. Rather than uniformly proceeding through all steps with a "complete one before moving to the next" approach, allocating ample time to requirements definition and then raising precision iteratively from the automation stage onward makes the phased introduction smoother.

At what point, and by whom, should acceptance conditions be put into words?

The starting point of ATDD is not developers beginning to write test cases on their own, but rather a three-way conversation among the product owner, QA engineer, and developers. The Agile Alliance's workshop materials show a format for conducting an ATDD introduction session that includes this conversation, within a 180-minute (recommended) or 90-minute time frame, suggesting the importance of securing sufficient time for requirements definition.

As for how to proceed, first the stakeholders verbalize the "conditions to be considered complete" for each user story. Next, those conditions are broken down into the Given/When/Then format, translating ambiguous expressions into concrete input/output values. If corners are cut here, the test cases produced in later stages tend to become ones that "work but it's unclear what they guarantee." Furthermore, identifying at least one exception pattern (error cases, boundary values) can reduce rework at the implementation stage.

At this stage, there is no need to turn the test cases into strict code. This is the concept presented by Gojko Adzic in "Specification by Example," where the goal is to first reach agreement using concrete examples in natural language, and then refine them to a granularity suitable for implementation.

In early-phase projects where requirements change frequently, over-detailing test cases increases the cost of change. In such cases, a more realistic approach is to first finalize only the main scenarios, and add detailed boundary-value tests in parallel with the implementation phase.

Is the automation script written so that anyone reading it interprets it the same way? This is the criterion ultimately at stake in Step 2.

The acceptance conditions verbalized in Step 1 are merely a blueprint. The process of turning that blueprint into an actual functioning building is the creation of automation scripts in this Step 2.

The conditions organized in Given/When/Then, when translated into the Gherkin notation adopted by Cucumber, allow a specification document and test code to be unified into something even non-engineers can read. Cucumber's official documentation describes it as a tool that "reads executable specifications written in plain text," and the basic structure involves associating Given/When/Then steps with descriptions starting with the Feature keyword.

Implementation begins by organizing scenarios on a per-Feature basis and starting with the highest-priority stories. Step definitions (executable code) corresponding to each Gherkin step are implemented using Cucumber or SpecFlow, and conditions involving UI operations are linked with automated operation tools such as Selenium. By deliberately letting unimplemented steps fail, it remains immediately clear where implementation gaps exist.

One point to be careful of is not trying to build out the entire set of test cases all at once. Automating the scenarios for one story at a time, then repeating execution and correction, tends to make it easier to keep rework in later stages under control. Even when choosing a tool like FitNesse that manages tests in tabular form, establishing the correspondence with step definitions first is key to stabilizing the continuous test execution in the next step.

The core of Step 3 is a simple cycle: if the automated script fails, the implementation has not yet met the acceptance criteria; if it succeeds, the team moves on to the next acceptance criterion. Development implementation begins with the acceptance tests created in Step 2 initially failing, and code is written until the tests pass—structurally similar to the unit test cycle in TDD. The difference from TDD, however, is that the target here is a unit of business requirements.

During implementation, a dual structure functions in which unit tests verify the correctness of internal logic while acceptance tests confirm behavior from the user's perspective. It is not uncommon for acceptance tests to fail even when unit tests pass, and this discrepancy itself serves as a clue indicating a misinterpretation of requirements or a gap in implementation.

Continuous test execution presupposes a setup in which acceptance tests are run in a CI environment with every code change. Relying on manual execution lowers the execution frequency, which tends to delay the discovery of defects. By incorporating continuous test execution into the CI pipeline, teams can keep the acceptance criteria for each feature continuously verified.

A common pitfall for teams at this step is judging development to be complete the moment acceptance tests pass. In reality, acceptance tests are an indicator of whether business requirements have been met, and it is important to recognize that verification of aspects such as performance and security must be conducted separately.

The optimal approach to introducing ATDD differs depending on team size. For small teams, a lightweight operation with a limited set of tools is realistic; for medium-sized teams, a gradual expansion of the automation scope becomes necessary; and for large enterprises, integration into the CI/CD pipeline determines the effectiveness of the introduction. Below, we present concrete configuration patterns by team size.

When a small team is starting out with ATDD, where should they begin?

At first, it's tempting to think of automating acceptance tests using the same framework as unit tests, but in practice it works better to introduce a single Gherkin-compatible tool such as Cucumber and start by writing requirements into Feature files. By limiting the number of tools, the learning cost can be kept low, allowing time for the entire team to become accustomed to writing Given/When/Then.

A typical configuration example for a small team is the combination of Cucumber and JUnit. JUnit handles unit testing and implementation verification, while Cucumber handles the execution of acceptance test scenarios. For a team of roughly 3 to 5 developers, this two-tool setup alone is sufficient to bridge requirements and tests.

Wiki-style tools such as FitNesse are also an option, but Gherkin notation is easier to review jointly with the business side, making it well-suited to ATDD's core purpose of sharing requirements. Rather than adding more tools, simply adding one acceptance test execution step to the existing CI environment is enough to establish minimal automation. The decision to expand scope gradually is discussed at the next scale.

Decision axis: How to respond to increases in team size and the number of features.

Whereas a small-scale configuration aims at proficiency with Gherkin notation, in medium-sized projects, multiple teams add features in parallel, making the management of test duplication and the establishment of a review structure a challenge. As Feature files increase per feature, without naming conventions or tagging rules, scenario duplication and obsolescence tend to occur easily, and execution time also grows longer.

For gradual automation, it is effective to start by tagging Cucumber scenarios and running only high-priority acceptance tests daily. Rather than executing all scenarios every time, this approach automates areas with high regression risk first and then expands the scope.

When SpecFlow is used alongside other tools, it integrates easily with unit tests in .NET projects, making it easier to establish a setup where developers can check acceptance test results on a per-Pull-Request basis. When incorporating browser operation tools such as Selenium into UI-layer testing, execution time tends to increase, so designing UI-based tests and API-layer tests separately can help reduce maintenance load.

Regarding division of roles, having QA engineers lead scenario creation while developers handle step implementation tends to reduce back-and-forth in reviews.

When multiple teams develop in parallel, parallelizing test execution becomes a challenge, and when release frequency is high, the execution order and gate design within the pipeline become a challenge. In large enterprises, it is necessary to build a system in which acceptance tests are incorporated as part of the CI/CD pipeline and automatically executed with every code change.

Specifically, a representative configuration centrally manages Feature files written in Cucumber or SpecFlow within a repository, and triggers a subset of acceptance tests alongside unit tests at the time a pull request is created. Since running all scenarios every time causes execution time to balloon, it is practical to continue priority control through tagging and establish a two-stage gate: running only smoke-level acceptance tests on release candidate branches, and running all scenarios in the staging environment before production deployment.

When combining with Selenium or API testing tools, containerizing the execution environment so that the same conditions can be reproduced with every test run makes it easier to isolate failures caused by environmental differences. Additionally, visualizing test results on a dashboard and maintaining the ability to trace which requirement a failed scenario is tied to helps prevent the correspondence between requirements and tests from breaking down. At a scale involving multiple teams, having the QA lead periodically review governance over naming conventions and tag usage is essential.

Q1. What are some representative tools for automating ATDD acceptance tests?

Representative tools include Cucumber, SpecFlow, and FitNesse. Cucumber is characterized by its ability to directly read executable specifications written in Gherkin syntax and execute Given/When/Then style scenarios as test code. SpecFlow provides a similar mechanism for the .NET environment, while FitNesse takes a different approach by describing acceptance criteria in table format.

Q2. What prerequisites should be confirmed when choosing Cucumber?

Cucumber organizes scenarios into units called Feature files, structured with the Given/When/Then keywords. When introducing it, you need to check the language version of the execution environment—for example, workshop materials from the Agile Alliance present a configuration that assumes Java 1.5 or higher. In actual projects, it is advisable to check the supported version in advance against the official documentation, depending on the language and build environment used.

Q3. Do tool selection criteria change depending on team size?

Yes, they do. For small teams using .NET, SpecFlow tends to be selected, while Cucumber tends to be favored in Java or multi-language environments. FitNesse may also suit teams that prioritize organizing conditions in table format, and ease of integration with existing CI/CD configurations is also a decision factor. The priority control through tagging mentioned in the previous section is supported as a standard feature in both Cucumber and SpecFlow.

Q4. Are there any books or materials worth referencing alongside these tools?

Gojko Adzic's book "Specification by Example" (Manning, published 2011) systematically explains the concept of describing acceptance test specifications through concrete examples, and is valuable for understanding the design philosophy behind ATDD. It serves not only as a resource for tool operation but also as material reinforcing the perspective of how to build consensus on acceptance criteria with the business side.

Q5. Is it common in practice to use multiple tools together?

It depends on the case. A common pattern is combining Cucumber and Selenium for UI-layer acceptance tests while handling unit tests with JUnit. Designs that separate tools by layer, such as using a different tool for API-layer verification, are also seen. However, since adding more tools increases maintenance costs, prioritizing a configuration that can be centrally managed within the existing CI/CD pipeline becomes a key decision factor in practice.

This section summarizes points of debate that practitioners often find confusing when considering the introduction of ATDD—such as timing of adoption, applicability to existing projects, and procedures for combined use with BDD. The following three points are explained in concrete detail regarding the decision branch points.

Decision criterion: The earlier you begin—during the planning stage before requirements are finalized—the greater the effect, while adoption after implementation has already progressed requires narrowing the scope.

Since ATDD is a method in which acceptance criteria are defined before implementation begins, it is ideal to introduce it early in the requirements definition phase, at least by the time of the first iteration planning. The process of agreeing on acceptance criteria together with the product owner and business-side stakeholders is itself the core of ATDD, and inserting this process later diminishes its effectiveness.

For a new project, it is reasonable to begin creating scenarios in Gherkin syntax starting from the first sprint. On the other hand, if requirements definition has already been completed and implementation has begun, a realistic approach is to limit adoption to new features or unimplemented backlog items that are yet to be started.

The decision branch point is "whether there are features whose acceptance criteria have not yet been finalized." If such features remain, you can start from there; however, if all requirements have already been implemented and acceptance criteria exist only implicitly, you will need to follow the retrofitting procedure explained next. When the entire team is unfamiliar with the ATDD process, it is considered practical to trial the approach first on features with a small impact scope, establish a way of writing acceptance tests and a review structure, and then gradually expand the scope.

When modifying an existing feature, adoption becomes limited to a narrowed impact scope, whereas applying it to new features yet to be added tends to take root more smoothly—this is the key difference. Retrofitting is similar to running piping through a building whose framework is already complete; it requires the judgment to select locations where walls do not need to be torn down.

When introducing ATDD into an existing project, it is realistic to first start automating acceptance tests from features scheduled for new development. If you go back and recreate acceptance criteria for already-implemented features, discrepancies between the implementation code and the specifications may surface, leading to cases where rework costs balloon. By adding acceptance tests step by step—starting with high-priority bug fixes or frequently changed features—you can expand the scope of application while confirming the return on investment.

On the other hand, if E2E tests already exist, migration costs can sometimes be reduced by re-scripting those tests as scenarios in Gherkin syntax. Projects where the correspondence between test code and business requirements is weaker tend to require greater initial effort for retrofitting, so it is practical in the field to first conduct small-scale trials on features with limited impact scope, and then expand the scope of application while observing the team's level of understanding and the maturity of the automation infrastructure.

ATDD and BDD are not competing methods; rather, they share a relationship in how acceptance criteria are written. When considering adopting both simultaneously, confusion can be minimized by first establishing the pattern for writing scenarios in Gherkin syntax, and then creating a flow in which those scenarios are automated directly as acceptance tests.

As for how to proceed, it is realistic to devote the first few weeks to a period where business stakeholders and developers sit together and practice writing in the "Given/When/Then" format. If the format is not standardized at this stage, the correspondence with test code will later break down, increasing maintenance costs. Next, you move on to the stage of connecting the completed scenarios to a tool such as Cucumber and running them as executable acceptance tests.

A point to be cautious about is the case where the vocabulary definitions of BDD and the consensus-building process of ATDD are carried out in parallel by separate teams. In this case, it becomes easy to end up with multiple acceptance criteria expressed differently for the same feature, requiring integration work later on. By first narrowing down to a single glossary and scenario template, and establishing a system in which the business side, QA, and development all refer to the same document from the outset, both methods can be layered and operated without strain.