Copilot in a real team: two months of data

GitHub Copilot was still in technical preview when we trialed it across four projects. It excelled at boilerplate with a known shape: test mappings, table-driven test data, HTTP client code, JSON serialization patterns. Situations where the pattern was clear and the variation was predictable.

Those wins were real and measurable. Developers moved faster on boilerplate because there is less to think about. The tool nailed the pattern. The variation was predictable. Those are situations where human brains would rather be doing something else meaningful.

The failure case was messier and more complex. When requirements were ambiguous or business logic was novel, Copilot produced plausible-looking code. It read well. It compiled. You had to read it line by line carefully to notice what it got wrong.

The underlying problem: reading code costs nearly as much attention and time as writing it does, maybe more. If you are going to review every line of Copilot's suggestion anyway, the time savings you hoped for evaporates quickly.

We adjusted code review before we adjusted the tool usage. Code from a model needs skeptical reading, not just a scan for syntax. That is slower and more careful than normal review, which meant Copilot could not be as fast as advertised.

The permanent lesson was about the human process, not the tool itself. Fast suggestions require slow, skeptical readers who will actually understand what the code does. We have not found a way around that fundamental asymmetry.