First of all: I am not against AI. I use AI to work and I use AI to vibe code as a hobby. There is nothing wrong with using AI to write code, it can write better syntax than humans and it can write faster. My whole gripe with AI is the process of using it, which many people get wrong. I will elaborate and try to construct my train of thought step by step in this post, but in short:

A human has to review 100% of the AI generated code and understand what it does and why. That is precisely why if you use AI to write code 8x faster you can't deliver code 8x faster, because you will need to read it all and investigate what, how and why, and that is what takes time.

If you do not do it, you will lose track of your code base. If you are a programmer you know how much harder it is to work on a codebase you did not write and how smooth it is working on a codebase you wrote from scratch. If you don't review properly you will be in a perpetual state of "working in a new codebase" because the code base was not written, or better said, not reviewed, by you.

But do I know about AI?

Admitedly not so much as in I have never developed an AI model or created AI automation tools. I use it to code and I used it when I worked at Uber and I use it too now that I work at Pinterest. In both companies I did get training directly from frontier AI labs and we have access to almost all AI tools available. So I think that I can say that professionally I have contact and I use AI at a very high level.

While admitedly I am not an AI developer I think I have earned the bragging rights to say that I am a developer who uses AI in projects that have global scale, managed by thousands of devs and reach millions of people, with billions of budget, in 2 publicly USA listed companies.

Had to brag, I am sorry, couldn't hold it LOL. But now for the real post:

AI can write a lot very fast and the code can even work

AI can code absurdly faster, to the point that nowadays, with frontier models, you can ask for a full working app and it may get it right on the first try.

The problem is that it will probably get it right but it will most likely be incomplete or have usability bugs or security vulnerabilities. Well, none of that is a problem, you can spend 100 hours running an AI to find and fix those problems, correct? No.

Context window and attention

The problem is multilayered. Context window is the first of the problems.

I have read a nice Medium post that elaborates on the context window problem:
https://medium.com/@tahirbalarabe2/understanding-llm-context-windows-tokens-attention-and-challenges-c98e140f174d

But to try explaining it with my own words:

The AI doesn't know your full codebase. The best AIs have 1 million tokens of context window. That is a lot, but the more context the worst attention it has.

With worse attention comes worse code that fits poorly in the codebase or has issues.

How do you prevent codebase issues form being pushed and merged into your code? You need to review the code the AI wrote.

Well, if we automated the code writing, the code debugging and the security checks we surely can automate the reviewing process, right? No.

Reviewing and coding is the same for AI

When an AI reviews it does roughly the same processes it did when coding.

As a tangent, you know what was one of the most signifcating changes for AIs? The thinking/reasoning capability. It does not mean that the AI is thinking like a human. I know this is an oversimplification, but AIs are still predicting and generating statistically likely sequences of tokens. The reasoning part allows them to explore, test alternatives, backtrack, and see what lands.

Without thinking/reasoning, AIs need to get it right on the first try. With thinking/reasoning, the AI tries many alternatives until it finds what lands.

When reviewing the AI follows similar procedures. It tries to find alternatives and tries to connect the dots and it can even think more broadly in terms of the codebase.

It is still flawed though. The AI still can only hold a limited amount of the codebase in context, and the more tokens you add, the worse attention it gets. It is actually quite common to get lost in AI review hell where the reviewer AI makes a suggestion that sounds reasonable, you apply it, then the AI reviews again and realizes actually there is another way to do it, or something else is wrong, or a previous problem returns.

Then you get into a loop where at each review there is a different problem. Sometimes they are real problems. Sometimes they are not. Sometimes the AI is improving the code. Sometimes it is just moving the mess around. All that is due to context and attention.

But wait, humans also have limited attention and possibly hold less than 1 million tokens in memory, right? No.

Humans think different

That is when human intervention becomes HIGHLY VALUABLE. If you are a human programmer and you program a software from scratch you know things work and even though you don't remember exactly the syntax or where something is you remember the implementation details or at least you remember WHY.

An AI does the exact opposite of a human! An AI may remember the syntax perfectly but not know why something was done and that is the most important part.

If an AI creates software from scratch it will soon forget why it did something or how stuff is supposed to work, the more code the AI writes the more disconnected it gets from its own code, the idea codebase for an AI is a short, simple and self contained one, without many interactions and outside components.

inb4 yes there are skills, agents.md and other methods to prevent that but all those CONSUMES CONTEXT and just moves things around. It it better than not using, those are highly valuable AI development techniques, but do not fix the problem, just patch it.

If a human creates software from scratch though that is precisely where a human will show is strenght. A human may not remember the exact syntax of state management or account authentication, but it remembers that there is state management and account authentication and won't create a new competing state manager and a duplicate auth method.

So all we need to do is read the code? I will stop with the rhetorical questions.

Yes, kind of, all you need to do is read and understand the code

It is as simple as that, we all should use AI to write code, it can write with perfect syntax, but it will make mistakes, but not only that, you need to know what changed in your code at every commit.

Up to Opus 4.7, the best model IMO, it still writes javascript code that does map and fiter instead of using reduce. It is known inneficient code syntax but it still makes such mistakes (probably because it is small, hard to notice so devs ignore it and it is now everywhere on the training data). This is fixable, just read and tell the AI to change the filter with a map to a reduce, it improves performance. You can prompt your way into imporovements but you can't prompt your way into understanding the code base.

So AI is not bad, it is just that AI keeps the human bottleneck! You write code 8x faster? That is non-ironically legitimately great! But now you have 8x more code to review!

I know that been I have been there, I wrote the code for all my tasks of the next two weeks on a Monday and spent a whole week reviewing it! Wrapping my head around why and how. If I do not do that 2 things can happen, either someone else will have to review thoroughly for me (because a third party review is mandatory) or I will not understand why something was done and that will bite me back in the future. It is not even worth mentioning that the code may be buggy at this point, that is the least of your worries because bugs can be fixed, but misunderstanding or lacking knowledge of the codebase compounds the problems.

That is the real danger of AI coding. Not that it writes bad code. Sometimes it writes great code. The real danger is that it allows you to merge code faster than you can understand it and if you keep doing that your repo becomes a foreign territory to you yourself.

The machine can type for you. It can suggest, refactor, debug, test and sometimes even architect. But it cannot own the consequences, that part is still human.

But if someone reading this thinks they are better than that, go ahead. Keep coding and delivering at 8x.

I can only wait and watch, maybe with popcorn and ad bug tracker opened.

PS: I have used AI to generate all the images in this post. I had a blast prompting away to explore illustration ideas. I finish this post with so many generated but unused images but the best are here IMO and I hope someone appreciates, although I recognize it took 0.1% of the effort and yielded 0.1% of the quality a real illustration would be able to achieve. It was a fun write and even more fun AI-driven illustration process!