The green test suite was not the release

When I first started building software with AI assistance, a green test suite felt like the closest thing to an objective finish line.

The code compiled.

The tests passed.

The build succeeded.

The pull request looked clean.

That was reassuring, especially when a model had written most of the implementation. I could not personally inspect every line with the same confidence as an experienced developer, but I could at least point to a set of checks and say the change had been validated.

Then I started opening the preview deployments.

Sometimes the implementation worked exactly as described and still did not feel finished.

A panel could expand without bringing the new content into view. A button could be technically present but visually disconnected from the item it affected. A layout could satisfy every assertion while feeling stretched, unbalanced, or awkward on an actual phone.

Nothing had necessarily failed.

The test suite had answered the questions it was asked.

The problem was that I had not known every important question before seeing the result.

That distinction became increasingly important as my projects got more complicated.

There is a difference between code that works, an implementation that satisfies the issue, and a feature that is ready to ship.

Working means the underlying behavior happens.

The issue being satisfied means the implementation matches the written requirements.

Ready to ship means I have seen the result, used it in context, and believe it belongs in the product.

Those states overlap, but they are not interchangeable.

This was especially obvious with visual and interaction work.

A browser test can prove that tapping an inventory item opens a details panel. It can prove that the correct text appears and that the expected action is available.

It may not tell me that the panel opened below the visible portion of the screen and the user has no obvious reason to know it appeared.

The implementation can be correct according to both the code and the issue while still producing a confusing experience.

Sometimes the issue itself is the thing that turns out to be incomplete.

I can describe what I think I want, review the plan, and approve the implementation. Then I see it running and realize the spacing is wrong, the information is grouped strangely, or the whole idea would work better another way.

That is not always scope drift or an agent misunderstanding the task.

Sometimes seeing the finished version gives me information I could not have had while writing the issue.

The preview is part of the design process.

Once I understood that, I stopped treating human review as a ceremonial step after the “real” validation had already happened.

The automated checks and the preview were answering different kinds of questions.

Tests could tell me whether important behavior remained correct.

Type checking could catch broken assumptions between parts of the code.

Linting and builds could catch structural problems.

An independent code review could look for mistakes the implementation agent and its tests had both missed.

The live preview could tell me whether the feature made sense as part of the actual product.

And I still had to decide whether all of that evidence added up to something I was willing to merge.

That led to the workflow I use now.

One issue becomes one branch and one pull request.

The pull request begins as a draft.

The implementation gets tested and independently reviewed.

The preview deployment gives me the real interface to use on my phone.

Visual problems, unclear interactions, and changes of mind become one focused correction pass instead of being waved away because CI is already green.

Only after that do I mark the work ready and merge it myself.

No single part of this process is the final authority.

A review model can misunderstand the product.

A vision agent can spot a visual oddity without understanding why the design exists.

A passing test can preserve behavior that was poorly designed in the first place.

And I can react too strongly to a first impression or request a change that creates a different problem somewhere else.

The value comes from the checks disagreeing with one another.

A test says the behavior is intact.

The reviewer says the implementation is structurally sound.

The preview says what the experience actually feels like.

I decide whether the result matches the reason I wanted the feature at all.

That last part matters because the “why” is often difficult to encode completely in an issue.

An agent may correctly implement the requested control while missing that the real goal was to make an action noticeable.

It may preserve all the existing behavior while making the page feel more crowded.

It may solve the literal problem and weaken the larger experience.

Those are not always coding failures.

They are product judgments.

I do not think this makes automated testing less valuable.

It makes testing more honest.

A green suite is strong evidence that the implementation passed the checks I knew how to define. It is not proof that I defined every important check, that the interface communicates clearly, or that the original plan was correct.

The tests protect the parts of the product that can be expressed as dependable rules.

They should not be asked to pretend that every quality judgment is a rule.

The same is true of AI review.

I use a capable model to plan work before implementation and review it afterward. That catches real problems and gives the cheaper implementation model a much stronger path to follow.

But the reviewer is not the person who will use the feature from my phone, understand the history of every design decision, or live with the result after it reaches main.

It provides another layer of evidence.

It does not provide permission to stop thinking.

The biggest change in my workflow was realizing that completion is not an event produced by CI.

It is a decision supported by several different kinds of evidence.

The code needs to work.

The tests need to pass.

The implementation needs to survive independent review.

The actual product needs to be inspected.

And sometimes I need to admit that what I asked for was not quite what I wanted after all.

A green checkmark is still satisfying.

I just no longer mistake it for the release.