Discuss your project
Web architecture

When a slowdown alert deserves a code fix

A catalog page slows down after a deployment. Use OpenAI’s engineering case study to see how to verify the cause, prepare a fix, and measure recovery after rollout.

Quand une alerte de lenteur mérite un correctif

A catalog page slows down after a release. An alert arrives twenty minutes after the first change. You have response-time charts and a deployment timestamp, but you still need evidence that the release caused the slowdown.

What OpenAI described on November 19

OpenAI’s engineering case study presents Performance Factory, an internal system that turns an alert into a task, assigns work to Codex, and keeps findings in a task service. Agents can gather evidence, challenge a diagnosis, and open a proposed fix. An engineer decides what to merge and deploy. The system checks recovery and resumes the investigation if the slowdown persists.

In a manual review of 100 proposed causes, OpenAI judged 70% accurate. The rest needed work or lacked evidence of causation. The figure describes the team’s review and explains its decision to check diagnoses separately.

Replay two catalog alerts

  1. Use a test service and synthetic traces with a baseline timestamp, a duration, and a request category. Introduce a repeated call for the first alert. For the second, change the traffic mix alone.
  2. Give the agent and your usual method the same data. Ask for the first measurable change, a comparable request group on each side of it, and an alternative explanation. The 95th percentile is the duration below which about 95% of responses fall.
  3. Review the traces, deploy the approved fix in staging, and replay the original traffic mix. Check errors and latency on the same time window. The second alert should lead to a finding about traffic composition, with any code change requiring separate evidence.

The agent can prepare evidence and proposals with read access. The team controls merge and deployment. Start with a task log that records evidence for the diagnosis, the review decision, and the follow-up measurement.

Share this article