Challenging the AI
2026 March 4 - João Porto
Challeging the AI day after day
Challenging
Once AI drew my attention, I have been in a mixed state of skepticism and excitement. My initial impressions were skeptic and not long time after pivoted to excitement with AI agents for coding.
Currently, I am in a constant state of “challeging” it. That is, I have been using AI in a daily basis as much as I can for coding and to solve problems that were not possible to tackle with conventional programming. For both cases, I have been facing problems that I do not see any clue that will be solved via AI in a mid (or even long) term perspective.
Regarding to use AI for coding, although some recent impressive milestones as Anthropic’s Claude building a complete C compiler “from scratch”, my experience with AI agents and vibe coding reached a ceiling point that contradicts all these fency showcases. That is because, in most of the cases, when I am vibe coding I reach a plateau state where to prompt “do this”, “fix that” etc is not enough and then I have to read and edit the code to unlock progress. Even without this plateau state, I do not trust non-reviewed AI code anyway, then I always read the code to ensure consistency and security, to avoid architectural flaws etc.
In the end of the day, I am basically writing prompts and reviewing what AI produces (just as before with conventional coding: writing ~20% of time and reading ~80%). Of course, this ceiling point is far better than what I had before with conventional coding, the productivity indeed increased. However there is no such thing as to prompt “do X” and got X instantly if X is a fairly complex app that contains some novelty — that is, a C compiler is much more complex software than any of my side-projects, but it is not quite new.
On the other hand, I am trying to use AI to solve problems that deterministic programs would not solve solely at all, but that would be feasible when we integrate AI into program’s pipeline. One of these problems is Discourse analysis (DA) and, even using AI, I have not reached the quality I aimed yet, because is very hard to ensure consistent results for non-deterministic stages of program’s pipeline just consuming responses from regular LLMs in the market. Of course, “use AI” is quite broader than “consume LLMs”, however that is exactly the point: for the general public, AI = LLM, but there is no such thing as “plug X stage to a LLM to solve the problem” and got the expected result instantly for hard problems as DA.
In summary, I think AI progress would be represented by a logarithmic function and we are reaching the point of diminishing returns, though I can see progress beyond what we have fostered by AI.
In fact, since the first computer machines were built in the mid of 20th century (in some way, even before, with Babbage and Ada), we have been evolving progressively the Human–computer interaction (HCI) to extract from machines what we need. Such extraction was feasible in the past via punched cards, later via the first programming languages and compilers and currently by modern coding tooling boosted with AI.
Nonetheless, the different essences of natural and formal languages will always produce a noise when we are trying to translate from the former to the later. Currently AI is not able to solve this noise for us. It does can help us, as punched card once did and programming languages still do alongside AI, but it can not solve the noise entirely for us.
I dream with the day where HCI will be we thinking and the machine processing the signals to produce what we want throught a fair interface. But I think we are far distant of that, even with AI.
Thanks to my fellow co-workers José, Ale and Lucas that sparked this topic.
References
“Building a c Compiler with a Team of Parallel Claudes.” Anthropic.com, 2026, www.anthropic.com/engineering/building-c-compiler.
Quora contributors. “Where Do Natural Languages (like English) Fit on the Chomsky Hierarchy?”
Quora, 2019, www.quora.com/Where-do-natural-languages-like-English-fit-on-the-Chomsky-hierarchy. Accessed 4 Mar. 2026.
Wikipedia contributors. “Chomsky Hierarchy.” Wikipedia, 9 Dec. 2021, en.wikipedia.org/wiki/Chomsky_hierarchy. Accessed 4 Mar. 2026.
Wikipedia contributors. “Formal Language.” Wikipedia, 8 Oct. 2021, en.wikipedia.org/wiki/Formal_language. Accessed 4 Mar. 2026.
Wikipedia contributors. “Human–computer interaction.” Wikipedia, 4 Mar. 2026, en.wikipedia.org/wiki/Human–computer_interaction. Accessed 4 Mar. 2026.
Wikipedia contributors. “Discourse analysis.” Wikipedia, 4 Mar. 2026, en.wikipedia.org/wiki/Discourse_analysis. Accessed 4 Mar. 2026.