This article is inspired by and adapted from a FB post by Аntоn Аntіch as I think he provides a great overview on what RAG is and why it is important, and how that remains absolutely relevant after the release of GPT-5.
It seems we need another AI literacy session. Let me break down, in the simplest terms possible, what’s actually happening inside modern neural networks and address the recent hype around GPT-5.
First, the hard truth: there’s no breakthrough here. No new architectures, no revolutionary advances. So when Altman and his fans talk about AGI (Artificial General Intelligence), divide their claims by a very large number. At its core, GPT-5 still uses the same transformer architecture described below, with incremental improvements like internal “critics” that evaluate each other and internal “reasoning” processes before generating responses.
While these improvements offer some nice enhancements for the applications we’ll discuss, they don’t and can’t provide the quantum leap that’s being promised.
The real breakthrough would be implementing System 2 thinking – genuine reasoning using logic and mathematics. This doesn’t exist, and it’s impossible to implement on the current LLM architecture.
This means asking ChatGPT about a birth year of your favourite singer or solving quadratic equations (in memory, without calling external tools) is still pointless – it’s like trying to crack nuts with a microscope.
However, for data conversion, agents that systematically call different tools with human oversight – that’s a categorical YES. Even early tests show GPT-5 performs noticeably better in these areas.
What Modern AI Chatbots Actually Are
What is modern ChatGPT (Claude, Grok, etc.)? It’s an LLM (Large Language Model, which is itself an artificial neural network) plus various hand-programmed wrapper systems. To understand how it works – and ideally inspire some of you to build your own agents – and to grasp its limitations and why this “programmed wrapper” is necessary, you first need to understand what an LLM actually is.
The Core Reality: Token Prediction
An LLM predicts the probability of the next token (part of a large word or a small word) based on the context of all previous tokens fed as input. That’s it.
There’s no “memory.” No ability to “calculate” anything. No ability to “go to the internet.” Nothing exists beyond this basic function – no state whatsoever. You give it input, it produces output, according to probabilities “learned” during training. And no, it doesn’t continue learning beyond its initial training. It sits there, completely static and stateless – you stuff a bunch of symbols into it, you get another bunch of symbols back.
That’s all. No magic. No hidden gnomes. Just learned probabilities.
What This Means in Practice
1. The Knowledge Problem
Without additional context, facts that appeared many times in training materials are learned “well” and will likely produce correct results. Facts that appeared rarely in training materials are learned “poorly” and will likely result in “hallucinations” – convincing but incorrect answers.
The classic example (which recent versions have manually fixed due to frequent questioning): Ask “Who is Tom Cruise’s mother?” and you’ll get the correct answer. But ask “Who is the son of [Tom Cruise’s mother’s name]?” and it can’t answer, because the “fact” that “this woman’s son is Tom Cruise” rarely appears in training materials. It has no logical reasoning – it can’t invert the statement.
This is why the phrase “I asked ChatGPT” as an argument is laughable. If you’re not asking in deep research mode (see below), you’ll likely get a convincing hallucination about lesser-known facts.
2. The Solution: Quality Input Equals Quality Output
Remember: output depends on input. If you feed an LLM good, correct information, the probability of hallucinations drops to nearly zero, and you’ll get a good explanation of the topic. This leads us to RAG (Retrieval-Augmented Generation) – our first encounter with “programmed wrapper systems.”
Understanding RAG: The Game Changer
Here’s how RAG works: In response to a human query, we first search the internet, corporate databases, etc., substitute relevant results as part of the input context, combining them with the user’s request. Now the LLM generates output based on relevant information that was separately provided to it. This information either doesn’t exist internally or was learned “poorly.” But since it’s in the immediate context, it takes priority.
Without RAG, no proper AI agent is possible.
ChatGPT uses RAG maybe once in a hundred queries, or if you explicitly request it – then it writes “searching the internet.” But even then, it essentially gives you an answer based on the first link it finds, and you can trust that answer exactly as much as you’d trust that random link.
Deep Research is much better – it investigates not one but multiple links from different sources until it finds an acceptable result or exhausts its search depth. This process is far more reliable but consumes many more tokens.
Bottom line: STOP WRITING “I asked ChatGPT” as an argument. You’re only demonstrating your ignorance.
Addressing Common Misconceptions
“But What About My Chat History! It Understands Context!”
With each new query, some (truncated somewhere to avoid killing the token window) history of your previous “conversation” is inserted into the input context. This is done through external programmed wrapper systems. Nowhere in the actual LLM are your chats remembered or can they be remembered.
“ChatGPT Started Remembering Our Conversations!”
This works by periodically summarizing conversation essence and recording it in compressed form in a database or even a simple file (as OpenAI still does). Then this information is substituted into context with every query. No magic, no “learning” by the network itself – still external wrapper systems.
“But What About Perplexity!”
Perplexity can be built in two minutes on platforms like Integral. It’s simply: 1) regular internet search, 2) substituting findings into LLM context, which then “explains” them.
“But How Do They Access the Internet, Draw Pictures, Calculate, etc.?”
Again, programmed wrapper systems. In the bot’s instructions (which are also just part of the context), it’s specified that if you need to call some external function, return the answer in a specific format (usually JSON). This response is analyzed by external software, which calls the corresponding function. There’s nothing magical inside the actual LLM beyond what’s described above.
The Path Forward: AI Agents
From everything described above emerges the necessity for agents – which are far from being just LLMs. They consist of:
- Multiple LLMs
- Different RAG systems
- Various strategies for updating external knowledge
- Additional software enabling calls to necessary tools
This combination creates something genuinely useful in AI, while publicly available bots are basically training wheels for the lazy.
Conclusion
Understanding these fundamentals is crucial for anyone working with or evaluating AI systems. The magic isn’t in the LLM itself – it’s in the sophisticated orchestration of multiple systems working together. The real breakthroughs will come not from bigger models, but from better architectures that can truly reason, not just pattern match.
Until then, use AI tools wisely, understand their limitations, and always verify important information through multiple sources.