Creative Flow vs. Critical Review

Something I struggle with when writing code in the current state of AI is the quick oscillation between creative flow and critical review that many AI tools force upon their humans. Here’s what happens…

You’re building something, and it’s going well, and you find yourself in that magical zone of creative flow. Ideas wash freely through your mind and into your keyboard-clacking fingers. There’s great clarity and an intimate sense of forward motion and progress.

Sometimes, AI plays along like a good improv comedy partner that says “Yes! And…“. But mostly, my AI tries to complete my sentences in ways that require very close scrutiny. I think: Is that right? Does that struct actually define that method? I’d like the first part of that, but the second half is nonsense, and so on… The larger the edits, the harder it gets. A single-line completion can be reviewed relatively quickly; multiple lines take more thought. Edits in many files or an agent that’s been working for many minutes generating all sorts of code? Good luck.

Now I’ve lost my flow state and I’m in reviewer mode. In reviewer mode, I’m a critic. I’m thinking about potential bugs, security vulnerabilities, resiliency of the system, design and maintainability, and what the user experience will be when things go wrong. In critical reviewer mode, I care deeply about precision and correctness and about saving myself work down the road. Unfortunately, this mental state is hard to square with continuing to tap into your creativity. Overcoming the initial inertia of getting started creatively is hard, so once you do that, you want to keep moving—it’s better to plow through the typos to get your ideas out and then circle back later to revise.

The problem is, it is very hard (for me) to oscillate between these two states of mind. Maybe this is something the next generation of engineers will be much better at, and it’s a skill to be practiced? For now, it’s like having a comedy partner that says “Yes! and…“, and sometimes hilarity ensues, but then, frequently, you find yourself in jail at the end of the night. So you can’t always say “yes, and…“ back: you have to correct, reject, edit, scrutinize. It’s much more like reviewing code from a junior developer or a new teammate. Until they ramp up, you know they don’t have enough experience, skill, or context to write high-quality code, so your goal as a reviewer is to a) level them up and b) make sure the changes that land actually accomplish the goals and are as high quality as possible. Even if the AI-written code was incredibly high quality, you still need to map that back into your brain and sync your own mental context—and reading code is much harder than writing it.

In a more traditional work style, this moment of critical review still happens. It’s just that there’s a big gap in time between getting the ideas out and bringing in the experts to give feedback and help refine. And the different roles are often assigned to different people.

I think “vibe coding” might be just embracing that improv “yes, and…“ with the long-term consequences be damned! And maybe that does make for a fun toy application or an amazing natural language-to-working-app demo. The problem is that subtle errors and misunderstandings add up very fast in engineering systems. We know that catching these early makes them much easier, cheaper, and faster to fix. Is it possible to vibe code with an AI on real, revenue-generating, production systems and not end up buried under a mountain of tech debt and existential business risk? Maybe the future of coding is more like software archaeology, where we’re all sifting through the midden trying to make sense of it all.

Secondarily, any effort I put towards reviewing the AI is essentially lost in the void. The AI system isn’t getting better when I correct it. And worse, my new colleague, who would absorb and learn, isn’t even in the conversation (because I’m talking to my AI). So I’ve both wasted my breath on the AI and missed an opportunity to level up a coworker (not to mention the benefits of sharing knowledge and context and the old-fashioned fulfillment of human interaction).

I’m not sure what to make of all this yet. I’ve been mulling over a few recommendations and ideas though:

• AI systems should narrow the scope of their changes and engage only when the probability of acceptance (by me) is above some threshold (90%+). As they improve, AI systems can take on more, but until then, they need to reduce verbosity and noise. I’d like a dial that I can control that’s basically: don’t suggest a change unless the AI’s confidence is above some threshold.
• AI systems need much deeper context and an internal loop to validate output before presenting it to me. Don’t suggest something with a typo (Copilot does this all the time) or something that won’t compile or something with fundamental performance problems. I understand that some of the reasoning and thinking modes of the latest models can do this, but the interaction lag is way too slow.
• Humans should pair together and then add an AI. Senior developers should be especially sensitive to this because I observe that more experienced and senior engineers gain significantly more productivity by using AI compared to junior or early-career developers. The senior engineer can rein in the tool and has the skills to review deeply. The junior engineer is just happy that the model spit out a bunch of code that does a thing and then gets lost down wrong turns and convoluted solutions with code they do not understand.
• There has to be a way to personalize and/or further train the AI systems that I use so that my corrections aren’t going into the void. I don’t know if this is fine-tuning my own models, custom model memory systems, intricate personal system prompts, or what.
• Generative coding AI tools need to provide proper attribution for code provenance (preferably at the span level). It’s important to know who or what wrote a section of code (co-authorship is fine). This is important for normal git blame development/debugging but also for future training of models or fine-tuning on your private code.
• Everybody needs to level up in how they read and review code. Learn to read code and read it in a way where you actually understand what’s going to happen when that code runs. It’s also important to have some insight into whether that code is going to stand the test of time. Ask yourself: What will it be like to read, debug, and fix this code 1 year from now?

NOTE: I had GitHub Copilot lightly edit the final draft of this post for spelling, punctuation, and grammar. It did a pretty good job making concise and minor edits as asked.