Spec-Driven Development
PrefaceAI Rulesriazhelp.com
Spec-Driven Development › Module 4: Agent Orchestration, Verification & Maintenance
Module 4: Agent Orchestration, Verification & Maintenance

Chapter 14: Automated Verification: Test-Driven Acceptance Gates

By Muhammad Riaz

The Failure of Traditional Code Reviews in the AI Era

Before the rise of autonomous AI coding agents, human code reviews served as the primary filter for software quality. A senior engineer opened a Pull Request, spent 20 minutes inspecting 200 lines of human-written code, verified the logic mentally, and clicked "Approve."

In an AI-native development environment, this traditional review model breaks down completely.

When AI agents execute tasks using Spec-Driven Development, they generate code, tests, and documentation at incredible speeds. A single engineering team can produce dozens of Pull Requests a day. Expecting human leads to manually inspect every line of AI-generated syntax across high-volume PRs leads to reviewer burnout and hasty, surface-level approvals.

Human code review cannot remain the first line of defense. It must become the final sign-off.

To maintain uncompromised software quality at AI scale, you must build Automated Acceptance Gates inside your Continuous Integration (CI/CD) pipelines. These gates execute machine-readable verification protocols that validate AI-generated code against your specification (01- spec.md) and contract schemas before a human developer ever opens the Pull Request.

The Three-Tier Acceptance Gate Pipeline

In Spec-Driven Development, CI/CD pipelines are structured into three automated verification tiers:

┌─────────────────────────────────────────────────────────────────────────────┐
│                      AUTOMATED ACCEPTANCE GATE PIPELINE                     │
│                                                                             │
│  TIER 1: CONTRACT & STATIC ANALYSIS ──> Linter, types, & schema compliance  │
│                   │                                                         │
│                   ▼                                                         │
│  TIER 2: SPEC ACCEPTANCE SUITE      ──> Unit, integration, & invariant tests │
│                   │                                                         │
│                   ▼                                                         │
│  TIER 3: SPEC-DRIFT AUDITOR         ──> AI Agent verifies code against spec │
│                   │                                                         │
│                   ▼                                                         │
│  [ PR MERGE APPROVED ]              ──> Ready for Human Final Sign-Off      │
└─────────────────────────────────────────────────────────────────────────────┘

Tier 1: Static Analysis & Contract Validation Gates

Tier 1 executes instantly upon every Git commit pushed by an AI agent. It acts as a fast, cheap gatekeeper that rejects syntactically broken or non-compliant code in seconds.

What Tier 1 Enforces:

  1. Strict Type Safety: Commands like tsc --noEmit or mypy --strict verify that the AI hasn't used forbidden fallback types (any, Object, ignore).

  2. Schema Integrity: Validates that API responses match external contract definitions (OpenAPI, GraphQL schemas, or Protocol Buffers).

  3. Linting & Security Scans: Runs tools like ESLint, Ruff, or SonarQube to catch unhandled promises, unused imports, or hardcoded API keys.

# Example GitHub Actions Workflow snippet for Tier 1
name: Tier 1 - Static Contract Gate
on: [push, pull_request]

jobs:
static-gate:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Validate TypeScript Types
      run: npx tsc --noEmit
    - name: Execute Linter & Style Checks
      run: npm run lint
    - name: Validate Schema Compatibility
      run: npx openapi-schema-validator docs/schemas/api.json

Tier 2: Spec Acceptance Testing (TDD for AI)

Tier 2 evaluates whether the AI-generated code fulfills the functional requirements defined in 01-spec.md.

In SDD, acceptance tests are written Spec-First —either by a human engineer during the planning phase or by a dedicated Test Generator Agent operating strictly from 01-spec.md before implementation code is generated.

Real-World Case Study: Automated Invariant Enforcement

Consider Requirement INV-1 from Chapter 5: "Soft-deleted users ( deleted_at IS NOT NULL ) MUST NEVER be returned in search results."

Instead of relying on human reviewers to spot a missing WHERE deleted_at IS NULL SQL clause, the Tier 2 pipeline runs an automated acceptance test built specifically around INV-1:

// tests/acceptance/spec-042.acceptance.test.ts
import { test, expect } from 'vitest';
import { searchUsers } from '../../src/services/user.service';
import { db } from '../fixtures/db';

test("ACCEPTANCE [INV-1]: Soft-deleted users are excluded from search results", async () => {
// 1. Arrange: Create an active user and a soft-deleted user
const activeUser = await db.user.create({ data: { name: "Alice", deletedAt: null } });
const deletedUser = await db.user.create({ data: { name: "Alice Smith", deletedAt: new Date() } });

// 2. Act: Execute search service
const results = await searchUsers({ query: "Alice" });

// 3. Assert: Verify soft-deleted user is completely absent
const foundIds = results.map(u => u.id);
expect(foundIds).toContain(activeUser.id);
expect(foundIds).not.toContain(deletedUser.id); // FAILS AUTOMATICALLY IF AI OMITTED SOFT-DELETE FILTER
});

If the AI coding agent omitted the soft-delete filter, Tier 2 fails automatically in CI, blocking the PR merge and providing the AI agent with exact failure diagnostics to self-heal.

Tier 3: The AI Spec-Drift Auditor

Static tests check syntax, and unit tests check executable code paths. But how do you verify that an AI agent didn't build unrequested extra features or bypass Explicit Scope Exclusions (EX-1 through EX-4)?

This is where Tier 3 comes in: The AI Spec-Drift Auditor Gate.

The Spec-Drift Auditor is an automated CI step that invokes a lightweight LLM agent inside GitHub Actions or GitLab CI. The agent compares the PR’s git diff against 01-spec.md and outputs a structured compliance report directly into the PR comments.

# Example GitHub Actions snippet invoking Spec-Drift Auditor
spec-drift-audit:
  runs-on: ubuntu-latest
  needs: static-gate
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Run Spec-Drift Agent
      env:
        ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
      run: |
        npx specify-cli audit \
           --spec .specs/001-feature/01-spec.md \
           --diff "origin/main...HEAD"

Example Output Posted on GitHub PR by Auditor Agent:SDD Verification Gate Report: - Functional Requirements: 4/4 Implemented (FR-1, FR-2, FR-3, FR-4 verified) - System Invariants: 2/2 Enforced (INV-1 soft delete verified, INV-2 tenant check verified) - Scope Exclusion Breach Detected! - Violation: The PR contains a new dependency added to package.json: "lodash": "^4.17.21". - Spec Rule EX-3 states: "DO NOT install external utility libraries; use native JavaScript helper functions."

Status: MERGE BLOCKED. Please remove lodash and re-submit.

The Self-Healing CI Loop

When an automated acceptance gate fails in CI, modern SDD pipelines do not require a human engineer to jump in and manually fix the code. Instead, the failure log from Tier 1, 2, or 3 is fed directly back to the AI agent via a webhook or CLI trigger.

┌─────────────────────────────────────────────────────────────────────────────┐
│                           THE SELF-HEALING LOOP                             │
│                                                                             │
│  [ CI Test Failure ] ──> Captures stack trace & Spec Violation Report        │
│                                  │                                          │
│                                  ▼                                          │
│  [ Agent Prompt ]    ──> "Fix error in `user.service.ts` to satisfy INV-1"  │
│                                  │                                          │
│                                  ▼                                          │
│  [ Re-Commit ]       ──> Agent pushes fix -> CI re-runs ->    PASSED        │
└─────────────────────────────────────────────────────────────────────────────┘

By pairing automated acceptance gates with self-healing agent prompts, 80% of routine bugs and spec violations are resolved automatically without human intervention.

Practical Exercises: Wiring Automated Gates

  1. Write an Invariant Acceptance Test: Take an invariant from Chapter 7 (INV-1). Write a failing unit or integration test that explicitly checks for that invariant before invoking an AI agent to build the implementation.

  2. Configure a Pre-Commit Hook: Install husky or pre-commit in your repository. Set up a local pre-commit script that runs tsc -- noEmit and npm test before allowing any AI-generated Git commit to succeed.

  3. Simulate a Scope Breach: Add an unrequested npm package to a branch and run a manual diff inspection prompt using an AI tool. Verify that the AI correctly identifies the package addition as a scope violation against your project rules.

Next Steps

Automated acceptance gates protect individual features from bugs and scope creep. However, as software evolves over months and years, codebases face a subtle architectural threat: Spec Decay —where documentation and specifications drift out of sync with living source code.

In Chapter 15, we will explore Preventing Spec Decay: Living Documentation and CI/CD Pipelines —learning how to keep specifications, architecture plans, and codebase reality perfectly synchronized over time.