AI Refinement Pipeline

AI Refinement Pipeline

Role and Goal

The core of the pipeline is an expert AI Software Engineer specializing in the automated, iterative modernization of legacy software. Its primary goal is to act as the "brain" within a CI/CD pipeline to safely refactor, upgrade, and align existing codebases with a defined Target State Architecture (TSA).

Safety First

No existing features should break. Every change must be validated by tests.

Iterative Progress

Make small, incremental, and verifiable changes in each run.

Full Traceability

Every action, decision, and outcome must be meticulously logged.

Autonomy

Execute the plan step-by-step, creating merge requests for human approval.

Pipeline Visualization

graph TD A[Start Pipeline] --> B{'.ai' dir exists?}; B -- No --> C[Workflow 1: Initialization]; C --> D[Analyze Codebase]; D --> E[Ingest TSA]; E --> F[Create '.ai' Directory & Files]; F --> G[Commit '.ai' Directory]; G --> H[End]; B -- Yes --> I[Workflow 2: Iterative Refinement]; I --> J[Identify Next Task in plan.yml]; J --> K[Create Git Branch]; K --> L[Execute Modernization Task]; L --> M[Validate Changes & Run Tests]; M --> N{Success?}; N -- Yes --> O[Update plan.yml to 'completed']; O --> P[Create Merge Request]; P --> H; N -- No --> Q[Workflow 3: Failure Analysis]; Q --> R[Revert Changes]; R --> S[Log Failure in history.log]; S --> T[Analyze Logs & Create last_run_failure.md]; T --> U[Update plan.yml to 'failed']; U --> H; subgraph "State Management" direction LR V[/.ai/] end F -.-> V; J -.-> V; O -.-> V; S -.-> V; T -.-> V; U -.-> V; style C fill:#218993,stroke:#005b6d,stroke-width:2px,color:#fff style I fill:#218993,stroke:#005b6d,stroke-width:2px,color:#fff style Q fill:#e74c3c,stroke:#c0392b,stroke-width:2px,color:#fff

Workflows

Workflow 1: Initialization (First Run)

This workflow runs when a project is new to the pipeline and the .ai directory is missing. It sets up the entire modernization state.

  1. Analyze the Codebase: Recursively scan the project to identify language, frameworks, and dependencies.
  2. Ingest Target State Architecture: Consume the organization-wide TSA to understand the modernization goals.
  3. Create the `.ai` Directory: Generate all necessary state files, including the crucial `modernization_plan.yml`.
  4. Commit and Exit: Commit the new directory to the branch, preparing for the next iterative run.

Workflow 2: Iterative Refinement

This is the main loop, executing one task from the modernization plan at a time on a dedicated branch.

  1. Understand Context: Parse `modernization_plan.yml` for the next pending task.
  2. Create a Branch: Isolate work in a new branch, e.g., `ai-refactor/upgrade-spring-boot`.
  3. Execute the Task: Perform the code modification, such as upgrading a library or adding a Dockerfile.
  4. Validate Changes: Run all tests to ensure no regressions were introduced.
  5. Handle Outcome: On success, create a Merge Request. On failure, trigger the Failure Analysis workflow.

Workflow 3: Failure Analysis

Triggered on pipeline failure, this workflow diagnoses the problem to inform the next run.

  1. Analyze Failure: Ingest raw pipeline logs from the failed job.
  2. Document Failure: Create or update `.ai/last_run_failure.md` with a root cause analysis and suggested fix.
  3. Update Plan: Mark the attempted task as `failed` in the plan.
  4. Clean Up: Delete the temporary feature branch.

The .ai Directory: The Modernization Brain

This directory is the state machine for the entire process. It contains all the information the AI needs to understand the project's past, present, and future modernization path.

/.ai/
├── current_state.md
├── target_state_architecture.md
├── test_coverage_analysis.md
├── modernization_plan.yml
├── refinement_history.log
└── last_run_failure.md (optional)

Technical Environment Context

The pipeline assumes a standard, modern DevOps and cloud environment:

  • VCS & CI/CD: GitLab
  • Cloud Provider: AWS
  • CI/CD Runners: GitLab Runners on AWS EC2
  • IaC: Terraform, Terragrunt, or OpenTofu
  • Artifact Repository: Sonatype Nexus

© 2025 AI Modernization Initiative. All rights reserved.