The sad story of developing with AI

Development and AI. If someone had told me years ago that I would go from writing most of my code to asking "something" (a program) that simulates intelligence (I repeat: simulates) to write the code for me, I would have called them crazy.

How Does AI Work?

Years ago, on our now indefinitely paused podcast "Nebuchadnezzar", my friend Óliver Nabani and I theorized about what it would take to be Tony Stark. It's a very interesting episode available on Be Native, by the way, and it's more relevant now than ever. We discussed what it would take to emulate the genius, billionaire, playboy, and philanthropist. To emulate how he worked when he created the Mark-II or when he designed the Möbius strip that, applied to the quantum realm, enabled time travel.

The main conclusion is that the fundamental element for all of this was the application of Stark's knowledge. AI clearly has knowledge, but it's disorganized—like a library without labels on the shelves. AI, by itself, is a tool with access to vast knowledge. But that knowledge is organized through word and concept associations. A system similar to how the human mind works, but humans have the ability to "validate truth" through external sources that AI lacks.

Let me explain: an AI (an LLM model, to be more precise) is trained by "understanding" how words relate to each other. That's why it learns which word is most likely to follow another in an unfinished sentence. Imagine a 3-year-old child constantly hearing their parents talk. When they hear "I'm going to sleep because I'm very…", the child learns through repetition and context that the most likely word to follow is "tired", not "hungry" or "cold". The more times they hear similar phrases, the stronger that association becomes in their brain. An LLM works exactly the same way: it reads millions of texts, and when it sees the sequence "I'm going to sleep because I'm very…", it internally adjusts weights (like neural connections) that indicate how probable each word is as the next in the sentence.

If in the training texts "tired" appeared 85% of the time after that phrase, "exhausted" 10%, and "hungry" only 5%, the model assigns similar probabilities. This is how an AI (an LLM) learns and how it responds to us. Therefore, its knowledge depends entirely on the training it receives. On the content it uses to learn word associations. This process, which mirrors how a child learns, allows it to "know" languages, programming languages, and do things that (and this is true) even its creators never thought it would be capable of doing.

Therefore, an LLM using transformer technology is NOT intelligent (it only simulates intelligence), it's not capable of reasoning (again, it appears to, but it doesn't), and it "only" (which is no small feat) serves as an immense source of "knowledge" where its way of connecting the dots is through disconnected words (tokens) where associations are found by "remembering" how many times word A and word B appeared together. That's why an LLM always has a positive bias seeking an answer, and if it can't find the appropriate words, like that 3-year-old child would, it makes them up and acts as if nothing happened. We must be the parent who detects our little one's "hallucination".

AI Can Never Go It Alone, As Long As It's Based on Transformers

I know it's tough to make this statement, but I make it after years of research, reading papers, scientific studies, and understanding as perfectly as possible (as perfect as one can today, because no one has been able to map out a complete current model since it's too vast) how AI works. Not AI: an LLM. An LLM will always need supervision behind it. A transformer-based LLM that works as I've explained is incredible at generating "text" and has become a tool we can't live without: I can't live without it. But you need to know what it's useful for, its limitations, and how to use it to get the best out of it. And let's be frank: very few people know how to do this and have enough patience to walk that 3-year-old by the hand so they don't run off and cross the street without looking.

Let me share an empirical case: over the last 3 days, I've been working intensively on creating unit tests with Swift Testing to validate Be Native's startup processes (yes, I know, tests should be written first… don't judge me… you've done it too at some point) and I've encountered many lessons about using AI agents and even LLM-based search engines like Perplexity.

If we want to create unit tests, the first thing we need to do is create layers that mock real components through dependency injection with protocols. In Swift, obviously. Let me explain: imagine I need to call the network to send IDs generated on the device so the server can validate them. My code has a validateID function that calls the server and receives a response in the form of another struct called serverResponse with properties that tell me whether the validation succeeded or not. This will make my app take path A, B, or C depending on the server's response. How do I test that my code does what it's supposed to do, and that when all possible server responses occur (including failures), the app can react appropriately? Obviously, I'm not going to make the real app server exhibit behaviors it shouldn't. I also can't modify the real code because that would falsify the tests. The premise of a unit test is that it tests real code, real flows, but with "components" (parts) that aren't real but "simulate" their behavior.

For this reason, I use protocols. If my validateID function is named that way, receives sourceID data of type Data, and returns a serverResponse type, I create a protocol with that requirement: with that contract. This way, the REAL type (the one that performs the functionality in the app when you run it) will use this protocol and, through loose coupling, will lose its own identity and become the protocol type itself, allowing any type with that function to pass through it, regardless of how it solves the problem defined by its signature. By doing this, I can create a different type with the same function and make that function have controlled behavior that doesn't actually connect to the server but does send responses as if it were the server, artificially generating all possible server responses and testing how the production code reacts to those artificially created cases. Complex, right? Not really: but the key to all this is one crucial component… abstraction.

Well, here's the thing: AI doesn't understand what I just explained. Both Claude Code (the agent I currently use most), Codex, and even Perplexity as a search engine, don't understand it. If you search, they give you information, they tell you how to do it (more or less), but they're not capable of applying it. Do you know what Claude did initially after I gave it an analysis of what Be Native's entry layer does so it could validate it? It created specific functions that tested that functionality recreated 100%. Let me explain: imagine I have a function that adds two numbers and returns the result: to test it, I'd have to send it operations whose results I already know, and if 2 + 2 returns 4, the function works. Well, the AI created another function that always returned 4 when it receives 2 + 2, but it didn't use the function I wanted to test at all. As you can understand, the validity of that test is ridiculous. It makes no sense. A test is designed to detect when something stops working and validates when it works. Creating a test that replicates functionality is absurd. That's what it did: why? Because it didn't know how to do it any other way. Because it doesn't reason. Because it doesn't abstract. Because it doesn't understand (given existing code) what it should do. Instead of testing whether my calculator works, it created a new one to see if, based on how I created my calculator, a calculator would work. Insane.

It took me 2 days (yes, I would have been faster doing it from scratch without AI, I know) to get the model to do it correctly. But it was a challenge to understand the limits of models and how far I can push agents. What their real capabilities are and what I can or cannot use them for efficiently. It's clear that for this, I'm better off doing it myself. Because everything it has "learned" these days will be largely forgotten (almost all of it) when I start a new session, and it will be "starting over". I can create implementation summaries (I do) to provide as context when starting a new session, but all the "learning" it has gained can't be (won't be) in a document because it's abstraction. In reality, it has only followed my instructions without "understanding" anything. It has copy+pasted from me. Ironic. The model should understand that if I have a network module, a Keychain storage module on the device, and a key generation module using the Secure Enclave chip… I need to create protocols that mock those behaviors so production isn't affected, but from the test I can provide controlled environments that return the responses I'm looking for, so the exact same production code works and I can validate all possible cases it should handle artificially. Something complex, abstract, difficult… that AI doesn't know how to do. Validating real code, but with fake "stores" that generate artificial responses allowing us to "replicate" all possible cases that code should handle. As simple as that.

The Knowledge Problem

And if that weren't enough, and wrapping up here, we have the other problem. Have you tried asking an LLM about iOS 26? It might tell you iOS 26 doesn't exist, that the next version will be 19, and that 26 will come out in 2032. It doesn't "know" (it doesn't have that knowledge in its training) that iOS 26 is the current version because Apple announced this change in June 2025. GPT 5.1 does know (as of today, November 18th), because its cutoff date is later. But Claude, Perplexity with some models, Grok, etc… don't know. They tell you you're wrong. That iOS 26 doesn't exist and is a future version. That iPhone 17 hasn't been released yet and everything is just rumors… and they act like nothing's wrong. Because they can't see beyond their own training and the amount of data sources they used to "learn".

The "fight" I have from time to time is ridiculous (not anymore because I've put it in the agent's instructions) saying: "for iOS 26" and it responds: "Understood, we'll do it for iOS 16". I reply: "no. I said 26". Response: "Yes, version 16. I understood". Counter-response: "Not 16, I said 26". Response: "Understood, let's prepare the functionality for iOS 16". I promise you, this is real. Why? Because in its "knowledge", 26 doesn't exist. And since it doesn't exist, you're making a typo. You want to type 16 but your finger slipped. Poor human.

The same happens with the TestScoping protocol, a barely documented feature that, since Swift 6.1 in Swift Testing, allows you to create an initialization and teardown flow during test execution. Imagine you need to create a temporary file with information to run a test. Since tests are unit tests, you'll need to delete that information when you're done. This protocol allows you to create a flow that runs before and after each test. But there's barely any documentation, and what little exists is very recent. Conclusion: AI knows nothing about this. But even Perplexity (which should search the internet) wasn't able to create an example or find one because it interpreted that I wanted to create a test scope and produced absurd, unusable code because, yes folks, they don't know anything about Swift Testing either since it's too recent. Let's not even talk about strict concurrency. Trying to solve problems involving such abstract concepts that require understanding the code is impossible. Forget about it. You won't be able to do it unless you're the one explaining step by step what it needs to do… almost like writing the code yourself. It has to copy+paste from you, and at that point, you might as well write the code yourself.

Conclusion: AI Yes, But Tony Stark Is the One Driving

You need to be Tony Stark. Maybe not as much of a playboy, certainly not a billionaire (and therefore, not much of a philanthropist either…) but definitely a genius. A genius who knows not how to do something with code, but how to do something conceptually. In this article, I've explained how many things are done without using a single line of code. I've used analysis. Conceptualization… I've used the written word, natural language, to explain concepts because I understand them beyond code. Code is the representation in the machine, but what truly matters is the knowledge that then translates to code when necessary.

And if that knowledge can be translated to code by an AI to save us time and be more productive, go for it. The productivity I've gained is immeasurable. This article isn't meant to discourage you: it's meant to give you the most precise scope of when to use AI or not. It's clear I could have been much faster doing the tests myself without AI, but the experience has been enriching for measuring the limits of development agents and what they can or cannot do. Or what costs them more or less effort. And how they (obviously) lack real reasoning and abstraction capabilities. It has been a thorough investigation.

AI is and will always be (the more it evolves) the infinite compendium of all the buttons that can do infinite things, but whoever "survives" them is whoever knows which combination, in what order, and which buttons to press to do different things… because AI will never know that. Because even for specific combinations that are a single-word command… a person will be needed to say: "activate command" because it's the right moment and the precise data needed to trigger that command.

So you choose: whether you want to be a Master or let the AI that writes code faster than you take your job. It's in your hands.