What Are Non-Functional Requirements in AI Coding? Why LLMs Struggle More Than With Functional Requirements, and the Standards Humans Must Set

Non-functional requirements are requirements that determine not "what the system does" but "how safely, how fast, how cheaply, and how easily it can continue to be fixed." Coding agents such as Claude Code are increasingly being entrusted with implementing functional requirements—building screens and APIs according to specification. On the other hand, mizutani (@m_mizutani), a security and software engineer, writes in a post on X that "ninety percent" of what he points out to Claude concerns "non-functional requirements such as security, performance, cost, and maintainability."

This article is aimed at tech leads and development managers who have started entrusting implementation to AI. Based on publicly available research, it organizes the reasons why LLMs can handle functional requirements relatively well, and the reasons why non-functional requirements still require a human hand. It then shows what humans should decide and in what form it should be handed over so that the LLM operates according to the standards.

Functional requirements determine "what is returned in response to an input," while non-functional requirements determine "under what conditions, and at what level, that processing operates."

Keeping the standard classifications in mind provides a common vocabulary for teams discussing what to entrust to AI and what humans should keep an eye on.

For example, "a logged-in user can view a list of their own invoices" is a functional requirement. Deciding, for the same screen, that "the list must return within 1 second," "invoices of other companies must not be visible even if the URL is rewritten," and "monthly operating cost must stay within a certain amount" are non-functional requirements. For the former, you can tell whether it's satisfied just by using the screen; for the latter, you can only tell by measuring under matching conditions.

ISO/IEC 25010, the international standard defining software quality characteristics, divides product quality into 9 characteristics in its 2023 edition (see the arc42 Quality Model explanation). Of these, the 8 characteristics other than functional suitability roughly correspond to the perspective of non-functional requirements. In Japan, IPA's (Information-technology Promotion Agency) Non-Functional Requirements Grade has been used as a checklist of items to prevent misunderstandings between clients and developers.

The two classifications take different approaches, but both can be used as checklists to comprehensively identify "properties other than functionality."

The Non-Functional Requirements Grade is meant for agreeing on requirements for the system infrastructure, and does not go as far as determining how application code should be written. When creating standards to hand over to AI, it is realistic to use these checklists to confirm nothing has been missed, and to translate into numerical values only the items that are meaningful for your own system.

Non-functional requirements have three properties that functional requirements lack.

The first is being cross-cutting. A gap in authorization is not confined to a single screen; it spans every API that handles a tenant ID. The same is true for performance—a single slow query drags down the response of other screens as well. No matter which file you open, there is no single location where "writing it here" completes the job.

The second is being a trade-off. Raising availability increases cost, and making things faster with caching creates a window during which stale data is shown. Where the balance is struck depends on the loss incurred when the system goes down, the budget available, and how long users can tolerate waiting. The right answer is not found in the code.

The third is appearing only later. With a few dozen test records, you cannot see processing slowness or the upper limit on record counts. Inputs of the kind an attacker would send almost never show up during normal functional checks. Many non-functional defects have no concrete shape at the design or review stage, and only reveal themselves as violations once in production.

These three properties point in exactly the opposite direction from "what LLMs are good at," which we look at in the next section.

This is because functional requirements can often be verified against a correct answer through input and output, and can often be solved using well-known implementation patterns. These two traits happen to overlap with the strengths of LLMs.

Requirements that can be written as "given this input, this output" can be turned into tests. With tests in place, the LLM can write code, run it, and read the output when it fails to fix it—running this loop on its own. Anthropic's Claude Code best practices also recommend giving Claude checks that return pass/fail results. The explanation given is that without a means of verification, Claude has only "it looks done" to go on, and as a result, a human ends up substituting for the test.

Functional requirements have a nature that makes them easy to fit into this loop. For screen operations, API responses, and calculation results, you can write down the expected value beforehand. Even if the specification text is somewhat ambiguous, showing 2-3 examples of input and output usually lets the LLM grasp the intent. On the human side too, you can tell whether it's correct by running it, so feedback becomes concrete: "given this input, I want it to return this."

Conversely, many of the situations where we feel the LLM's implementation ability has improved are situations where this verification loop is running. Even with the same model, performance drops for requirements where tests can't be written or examples can't be shown. And many non-functional requirements fall into the category that's hard to turn into tests as they are.

Another reason is that many functional requirements are made up of combinations of problems for which there already exist a huge number of implementation examples. Form input validation, pagination of lists, and CSV export all have standard ways of writing them for each language and framework. LLMs are good at selecting the pattern that fits the situation from the enormous amount of code they've learned from.

This tendency also shows up in security research. Veracode, an application security company, has been running a study where more than 100 LLMs are given the same tasks to solve, and the generated code is inspected through static analysis. Under conditions where no security instructions were given, the rate at which SQL injection was prevented averaged 83%, and the rate for choosing appropriate encryption methods was 87% (Veracode's "2026 GenAI Code Security Report"). Countermeasures with a fixed pattern—such as using placeholders in SQL, or avoiding outdated encryption methods—are, in many cases, something the LLM can choose even without being instructed to.

However, the same report also states that while the models' ability to write syntactically correct code has improved, the overall security pass rate has remained flat. The average pass rate is 56%, barely moved from the 55% in the first report. The parts solvable by pattern have improved, while the parts that aren't have not. Those "parts that aren't" are the subject of the next section.

For non-functional requirements, the correct answer lies outside the code, defects appear across the flow of data, and they aren't visible with small amounts of data. Because they don't fit well into the loop where the LLM verifies its own work, the task of setting the standard remains in human hands.

No amount of reading the code will tell you "within how many seconds should this API respond." That's because it's determined by how long users can wait, how much traffic occurs at peak times, how much is lost when the service goes down, and how much can be spent on operations each month. If you ask an LLM to "make it faster," it will add caching, create indexes, and parallelize processing. But it cannot judge how far is enough and at what point further effort becomes a waste of money. That's because the material needed for that judgment lies outside the repository.

In a follow-up post, Mr. Mizutani writes that Claude is still not good at identifying where to land on non-functional requirements, or coming up with ideas for how to achieve that landing point. If a human decides that much, provides guidance, and sets the standard, Claude will follow it to some extent. But his assessment is that getting to that point is still difficult. What's called the "landing point" here is precisely the point where a trade-off is settled.

Security has the same structure. Whether to require multi-factor authentication for an internal admin screen, how many years to retain operation logs, and which country's servers to place personal information on are determined by industry regulations, contracts with business partners, and who will be accountable if an incident occurs. These aren't matters of how code is written, but matters of how much risk the organization is willing to accept—and both the authority and the responsibility to decide rest with people.

There's more to the Veracode study mentioned in the previous section. While the pass rate for SQL injection and encryption exceeded 80%, it drops to 15% for cross-site scripting (XSS) and 12% for log injection. The report explains that these two are affected by the flow of data, the context of the application, and understanding how user input moves through the system. In a later update, the average pass rate rose to 62%, but log injection remained nearly flat (as reported by CybrSec Media).

To prevent XSS, you need to trace where a string received in one function is stored, on which screen, and in what form it's displayed. Log injection, too, can't be found without tracing how many functions the input passes through before being written to a log. Both are defects that can't be prevented just by correctly writing the file in front of you.

Tenant isolation and authorization have the same shape. Writing a single API correctly and ensuring that every API determines the tenant from the session are different problems. An LLM will write coherent code within the scope of the files it's given, but it won't check whether a single rule is being upheld across the entire repository unless asked to. The case covered in Lessons from Vibe Coding Failures, where an application was released without row-level access control set on the database, occurred in an app that was functionally working.

Performance and cost defects rarely show up in the data volumes seen during development. A study called EffiBench compared the execution time of code written by 42 LLMs against the most efficient human-written solutions to LeetCode problems. Even the leading commercial models of the time took, on average, 3.12 times longer to run, and on the problem with the largest gap, 13.89 times longer. All the solutions compared had passed the tests. Code that is correct but slow slips through functional tests.

Something similar happened in our own development work. Supabase's API (PostgREST) limits the number of rows returned per call to 1,000 by default, silently truncating anything beyond that without raising an error. A process that fetched a list and aggregated it worked correctly while the row count was small, but once production data exceeded 1,000 rows, it began silently dropping some rows. We only noticed because a user reported that the aggregated totals didn't add up. Whether the code is written by a human or an AI, if no one has explicitly decided what happens as the row count grows, the tests stay green.

The same pattern holds at the team and organization level. Google's DORA research reports that while AI adoption has shifted to a positive relationship with development throughput, it continues to have a negative relationship with the stability of production deployments. The explanation given is that increasing the volume of changes without automated testing, mature version control, and fast feedback mechanisms leads to instability (Google Cloud announcement).

There are three things humans need to decide: the numerical thresholds and prohibitions that define where to draw the line, the method for judging them, and who enforces that judgment. Once these are decided, most of the implementation can be left to the LLM.

Instructions like "be mindful of security" or "keep performance in mind" convey almost no information to an LLM, because they don't specify what level counts as passing. What needs to be handed over are the numbers that result from human judgment, and the prohibitions that allow no exceptions.

For example, write it like this:

• The list API must return within 500 milliseconds at p95. Limit the number of items fetched per call to 100, and return results via pagination.
• The tenant must always be determined from the session. Never use a tenant ID received in the request body or query parameters.
• Retries for external APIs are limited to 3 per request. Set an upper limit on the number of tokens used per call to the LLM API.
• If a file exceeds 500 lines, propose splitting it before adding new functionality.

The numbers don't need to be correct from the start. What matters is that the fact a human has taken on a trade-off and made a decision is preserved as text. Adding a one-line reason for the decision gives the LLM a clue for judgment calls in unforeseen situations, and also helps when revisiting the numbers six months later. To avoid missing items, it's useful to use a checklist like ISO/IEC 25010 or the Non-Functional Requirements Grade as a reference table.

Even with criteria decided, if they remain as plain text, the LLM ends up self-reporting that it "probably complies" and nothing more. By converting them into a form where pass/fail is returned mechanically, the LLM becomes able to fix non-functional requirements through the same back-and-forth it uses for functional requirements.

This idea has a name. The authors of the book Building Evolutionary Architectures call the mechanism for objectively evaluating architectural characteristics an "architectural fitness function" (Thoughtworks Technology Radar). It's a broad concept that encompasses not just unit tests but also metrics and monitoring.

Here are examples for each type of non-functional requirement. For performance: a test that calls representative APIs with data volumes close to production scale, and fails if p95 exceeds the threshold. For authorization: a test that calls every API as a user from a different tenant, and fails if even a single record is returned. For maintainability: a script that checks file line counts and dependency direction. For cost: a test that counts how many times external APIs are called in a single process. For security: incorporating static analysis (SAST) and dependency vulnerability scanning into CI.

Once built, any of these can be run and interpreted by the LLM itself. If there's an item you keep flagging in every review, that's the first candidate to convert into a judgment mechanism.

Even after building a judgment mechanism, if whether to run it is left to the LLM's discretion, there will be times it doesn't get run. Claude Code's best practices divide instructions into two kinds: instructions like prompts, CLAUDE.md, and skills, where the model decides whether to read and follow them, and mechanisms like hooks, deny rules, and CI, which execute every time regardless of the model's judgment. The former is "usually followed" but not guaranteed, and the recommendation is to move rules that allow no exceptions into the latter.

Criteria that allow no exceptions should be placed in enforcement mechanisms; guidelines where judgment varies by situation should be placed in instructions the model reads.

In our own development environment, we stop reading env files, starting dev servers, and mistakenly editing the wrong target repository not merely by writing it in CLAUDE.md but by enforcing it with hooks. These are all rules where a single occurrence has a large impact, and "mostly followed" isn't good enough. On the other hand, adding too many enforced rules accumulates false positives and maintenance burden. The official documentation also advises keeping enforced rules to a minimum and leaving matters of preference to the LLM's judgment. How to configure hooks is covered in the Claude Code Team Adoption Guide.

If you divide non-functional requirements into 5 perspectives, what's easy to leave to an LLM is "implementation that meets a defined standard," while what belongs to humans is "deciding the standard" and "judging exceptions that fall outside the standard."

If the right two columns of the table are filled in, the work in the left column can largely be left to the LLM. Conversely, if the middle column is left blank, the LLM will proceed with the implementation by plugging in plausible values on its own. If you ask for a list screen without deciding the assumed data volume, it's not surprising if code emerges that fetches all records and then filters them on the screen side. That's because it works fine with no problems at all on small data.

The questions humans check during review can also be built from this table. First, which of the defined standards does this change relate to? Next, did this change pass the mechanism that judges that standard? And finally, is the LLM making judgments on its own about things not written in the standard? Only this last question is hard to replace with a machine, and it remains the part that humans must continue to hold.

Failures in handing non-functional requirements to an LLM tend to cluster around two things: handing them over while the standard remains vague, and mistaking passing tests for passing non-functional requirements.

The most common failure is writing in CLAUDE.md things like "pay sufficient attention to security" or "be mindful of performance" and assuming that conveys the non-functional requirements. Even reading this one sentence, the LLM has no way of knowing what to do or to what extent counts as passing. As a result, easily visible measures (escaping input, exception handling) increase, but less visible parts—such as consistency of authorization or preparedness for data volume—remain unaddressed.

The Veracode numbers cited earlier were also measured under conditions where no instructions about security were given at all. If nothing is said, you need to assume that known vulnerabilities will be introduced in roughly 40% of tasks. Adding an abstract sentence on top of that doesn't necessarily prevent the kind of flaws that can only be found by tracing the flow of data.

The way to avoid this is to replace cautionary notes with standards. Write sentences like "the tenant is determined from the session" or "values displayed on screen must always go through the template's escaping"—sentences where a human can read them and judge whether they were followed. On top of that, prepare one test or check that judges that sentence. If strengthening a single line in CLAUDE.md with "important" or "absolutely" still doesn't get it followed, the official documentation itself says that's a signal to move it into an enforcement mechanism.

When an LLM reports "all tests passed," it feels as though the entire change is correct. However, many of those tests are tests of functional requirements. Code that is correct but slow, and code that is correct but returns data from another tenant, both pass the functional tests.

This assumption also works to strengthen human confidence. In a Stanford University study, participants who used an AI assistant to solve tasks wrote less secure code than participants who didn't, and were also more likely to answer that they believed their own code was secure (Perry et al., "Do Users Write More Insecure Code with AI Assistants?"). Code written by AI comes out looking like it "seems to be done properly" to the reader.

There are two ways to avoid this. One is to have the completion report include "which standard's judgment it passed." If you have the commands that were run and their output pasted in, you can immediately notice that non-functional tests weren't run. The other is to put non-functional tests into CI so that merging is blocked unless they pass. That way, humans don't have to judge every single time whether a report can be trusted. For the full picture, including authority, review, and stop-condition design, see AI Agent Guardrail Operations Checklist.

Here are answers to 3 questions that commonly come up when leaving implementation to AI.

No, you don't need to look at everything with the same level of depth. Leave the parts that can be prevented by types (SQL construction, how cryptographic libraries are used) to SAST and dependency checks, and have humans focus on the parts that determine tenant and permission judgments, and the parts that output user input to screens or logs.

It's useful as a starting draft. If you provide the type of system, the scale of users, and the kind of data being handled, it will list out the points to check and tentative figures, following items similar to a non-functional requirements grade. Sometimes it brings up points you wouldn't have thought of yourselves, such as the volume of data to migrate or the number of backup generations.

However, whether you can decide on those figures as-is is a separate matter. For example, whether you set the target recovery time to "within 1 hour" or "by the next business day" greatly changes the required configuration and operational costs. Which is correct can only be decided by comparing what you would lose when the system stops against the cost you can afford to pay. Use the AI's proposal to check for omissions, and leave the finalizing of the figures to the business and operations managers who can explain that comparison.

On the implementation side, the range of work that can be delegated should keep expanding. Even in Veracode's recent updates, the average pass rate has risen, with improvements progressing particularly in the kinds of defects that can be prevented by types. On the other hand, where to draw the line on how fast, how cheap, and how secure is determined not by model performance but by business conditions, so this remains on the human side even as models change. If you keep the criteria in a form that can be judged, they will keep working the same way even on the day you switch models.

The reason LLMs can implement functional requirements relatively well is that the correct answer can be verified through input and output, and the problems can often be solved with well-known patterns. The reason human involvement remains for non-functional requirements is that the correct answer lies outside the code, in business and operational conditions, and defects span across data flows, only showing themselves under large data volumes or production conditions.

Mr. mizutani also wrote in a later post that, as a skill required of software engineers today, "how well one can design and implement non-functional requirements" will likely be given greater importance.

What humans are responsible for is not watching over every detail of the implementation. It is deciding where to draw the line using figures and prohibitions, putting that into a form that can be judged, and enforcing the parts that allow no exceptions through hooks or CI. Once this much is in place, the range of non-functional requirement implementation that can be entrusted to LLMs will expand through the same back-and-forth process used for functional requirements. Start by picking one item that keeps coming up repeatedly in reviews, and turn it into a criteria statement and a judgment test. The approach of preventing AI agent failures through systems and mechanisms is also explained in detail in What Is Harness Engineering?.

Our company also supports development processes using coding agents, as well as building criteria for non-functional requirements. Please reach out through AI/DX Support for inquiries.