Continuous integration

Continuous integration is the practice of merging code changes frequently and verifying each merge automatically. The shape of that verification layer has grown considerably more complex as teams scale, as AI agents write more of the code, and as the infrastructure itself becomes an attack surface.

At PostHog’s scale, 575K weekly CI jobs and 33 million test executions, the volume of failures exceeds what humans can triage manually. Mendral’s AI agent addresses this by ingesting log data, tracing flaky tests to root causes, and opening fix PRs automatically What CI Actually Looks Like at a 100-Person Team. Flakiness at this scale is not an edge case; it is a primary cost center.

Test stability is a design problem as much as a tooling one. Playwright suites break during UI refactors when tests couple to CSS classes, DOM structure, or element position rather than semantic roles and accessible names Designing Playwright Tests That Survive UI Refactors. Where you run those tests also matters: staging catches regressions cheaply but cannot reproduce production-only conditions, so some flows warrant splitting between environments Playwright Testing in Staging vs Production.

The merge queue layer introduces its own failure modes. A GitHub bug built temporary branches off the wrong base commit and silently deleted thousands of lines from main; Trunk’s architectural choice to avoid pushing temp branches to the main ref sidestepped the incident entirely What Happens If a Merge Queue Builds on the Wrong Commit. Correctness at the gate matters more than throughput.

Below the workflow layer, the orchestration infrastructure itself is an engineering problem. Depot CI’s scheduler uses AWS Lambda durable functions with a two-layer Run/Workflow hierarchy and callback-driven coordination, keeping the system stateful and checkpointed without a long-lived process Building CI with Lambda durable functions.

Security enters CI from multiple directions. The TeamPCP supply chain attack poisoned four SAP-ecosystem npm packages with a credential-stealing payload that targeted cloud secrets and used VS Code configs as persistence vectors SAP-Related npm Packages Compromised in Credential-Stealing Supply Chain Attack. MarkdownLM addresses the policy enforcement side differently, centralizing engineering standards into a knowledge base and blocking non-compliant code at the Git layer before it merges MarkdownLM.

AI-generated code adds a review problem that standard diffs do not surface. Vet reads an agent’s conversation history alongside the diff to catch mistakes like silently skipped tests or swapped-in fake data Vet: Catch your coding agent’s mistakes. The platform hosting all of this also comes under scrutiny: GitHub’s reliability decline has prompted calls to migrate to alternatives like Codeberg or self-hosted forges GitHub is Sinking, and one developer wishlist explicitly asks for pre-commit remote CI and signed, offline-usable Actions as first-class forge features If I Could Make My Own GitHub.