svg

Menu

Case Study · Developer Tooling & Agentic AI

spec-forge

Autonomous agentic code verification and AST-preserving mutation testing engine. Hunts blind spots and subtle regressions in LLM-generated code by synthesizing targeted AST mutations, computing true Mutation Scores, and autonomously generating regression-killing invariant tests.

RepositoryPython 3.14 · AST Analysis · Invariant Synthesis

Why Line Coverage is a Lie

With the rise of AI code assistants (Cursor, Claude Code, GitHub Copilot), codebases frequently achieve 90%+ line coverage. However, lines that are executed are not necessarily lines that are verified. Tests often fail to assert invariants on boundary off-by-one conditions, inverted booleans, and null-returns. A single unverified branch can introduce a critical security exploit or silent calculation bug.

Core Engineering Capabilities

  • Python AST Mutation Engine: Parses Python source code into an Abstract Syntax Tree and systematically injects 6 classes of targeted semantic mutations: comparison flips (< to <=), arithmetic alterations, off-by-one offsets, null/None injection, condition inversion, and statement omissions.
  • Isolated Dynamic Test Runner: Executes existing test suites against each individual mutant in an isolated runtime with execution timeouts to prevent infinite loops.
  • Rigorous Mutation Scoring: Computes the true resilience score — what fraction of syntactic bugs were actually caught by the test suite versus silently survived.
  • Agentic Test Case Synthesizer: When a mutant survives, the synthesis engine analyzes the AST diff and generates targeted test cases asserting the missing invariant, automatically killing the mutant and hardening the codebase.

Target Alignment

Perfect fit for frontier AI tooling and developer productivity teams like Cursor, SkillsCapital (Agentic AI), LetzBizz, and open-source infrastructure maintainers (like Frappe).