News/Guide
Guide · Aug 18, 2026

What your team does when the AI tooling goes down

A 20% error rate is harder to work through than a total outage, because most things still work. The first hour is always spent debugging your own code. Here is how to spend it better.

361361 NetworkEditorial team4 min read

On 17 August GitHub ran an incident from 13:40 to 21:15 UTC. Error rates around 20% on web and API, roughly 50% on archive downloads, and Git operations, Actions, Issues, Pull Requests, Webhooks and Copilot authentication all affected.

Most teams handled the first hour of that badly, and not through incompetence. Partial failure is genuinely hard: four out of five things work, so the natural conclusion is that the problem is yours.

Why 20% is worse than 100%

A total outage is self-announcing. Nothing works, everyone knows within a minute, and the response is to wait.

A partial outage looks exactly like a bug in your own system. A push succeeds, the next fails, a retry works. CI is red on a run that passed an hour ago with the same code. An agent completes one task and fails authentication on the next.

The cost is not the downtime — it is the hour your team spends bisecting commits, re-running pipelines and questioning a change that was fine. That hour is entirely recoverable with one habit.

The first five minutes

The habit is to check upstream before you check yourself, and it is cheap enough that being wrong costs nothing.

  • Check the vendor status page first, before opening the diff. Thirty seconds
  • Ask whether more than one person is seeing something odd. Intermittent failures across unrelated work is the strongest signal there is
  • Check whether the failure follows the code or follows the retry. A bug is deterministic; an incident is not
  • Say it out loud in the team channel. "Anyone else getting auth errors?" resolves this faster than any dashboard

Telling an incident from a bug

Three tests, in order of how quickly they give an answer.

Does it reproduce? Run the same operation three times. If it succeeds twice and fails once, you are almost certainly looking at infrastructure, because your code does not usually change its mind.

Does it correlate with content? A failure that hits one repository, one branch or one file is likely yours. A failure that hits unrelated work at the same time is not.

Does the error mention identity? Authentication failures were the long tail of the 17 August incident, and they surface in confusing places — a CLI that suddenly cannot push, an agent that cannot reach a service, an integration that stops mid-run.

What to do with the time

This is where most of the recoverable value is. A degraded platform blocks some work completely and leaves other work untouched, and the difference is worth knowing in advance.

  • Local work continues: writing code, running tests, refactoring, reading. None of it needs the platform
  • Anything involving pushing, review, CI or agents queues up. Do not fight it with retries — retries during a partial outage add load and confusion
  • This is a genuinely good moment for the work that never gets prioritised: local test coverage, documentation, the refactor nobody scheduled
  • Do not start a large local change that will need a painful merge when everyone else's queued work lands at once

Fallbacks worth having in advance

Set these up on a working day, because the middle of an incident is the wrong time to discover you do not have them.

  • Know whether a local model is available to you. Copilot for JetBrains added Ollama as a BYOK provider on 11 August, and a local model is the one AI assistance that does not depend on anyone's uptime
  • Know which of your CI failures are infrastructure and which are real — a pipeline that cannot distinguish them costs you debugging time on every incident
  • Know what depends on your code host for identity. If it is your SSO, the blast radius is much wider than your repositories
  • Have a communication channel that does not live on the affected platform. An incident discussed only in GitHub issues is an incident nobody can discuss

Afterwards, ask three questions

Not a formal post-mortem for someone else's outage — just three questions while it is fresh.

What could we not do? That is your dependency list, discovered rather than theorised, and it is more accurate than any architecture diagram.

How long did it take us to work out it was not us? If the answer is more than fifteen minutes, the fix is a habit, not a tool.

What would we do if it were a full day rather than seven hours? Most teams have never asked, and the answer usually reveals one dependency worth a fallback.

More news