The Evolution of AI: From Symbolic Reasoning to GPTs and Agentic Systems

Introduction

It seems like Artificial Intelligence (AI) has suddenly appeared in everything, everywhere, all at once. What feels like “five minutes ago”, there was “pre-AI life”, and now we have AI assistants that speak like real people, apps that create images, music, and video from nothing, and AI agents that do work for us. So, it’s natural to ask:

Hang on… How did we even get here? And where has all this AI come from?

The reality is, AI has been around for many years. However, many of its key advances have happened quietly in the background through academia and research, only becoming more overt as computer power and the abundance of data we have access to advanced. Technologies like ChatGPT have merged AI into everyday conversation, whilst image and video generators like DALL-E and Sora consistently capture the public’s imagination.

This article looks back at how AI evolved from early symbolic reasoning to generative and agentic systems, walking a path across the key milestones that led to where we are today.

Foundations: Alan Turing and the War Years

Before AI became a real field (or even computing), Alan Turing set the stage. During the Second World War, he worked at Bletchley Park to decrypt the Enigma ciphers using an electromechanical machine he helped design called the “Bombe”. It didn’t think like a human, but it showed how logic and computation could solve problems that seemed impossibly complex for humans to process.

After the war, Turing proposed the idea of a test to qualify machine intelligence called “The Imitation Game” (Now known as the “Turing Test”). He suggested that if a machine could hold a conversation well enough to be mistaken for a person, we might “reasonably” call it intelligent. These ideas gave AI both a philosophical goal and a practical taste of systematic problem-solving.

Figure 1 - In the Turing Test, a human interrogator engages in a text-based conversation with both a machine and a human respondent and attempts to determine which one is which.

In the mid-1960s the MIT chatbot ELIZA became one of the first chatbots built with the specific goal of passing the Turing test in mind. Using one of its scripts called “Doctor” to imitate a psychotherapist, the system used simple pattern matching and substitution rules to reflect a user’s statement back as a question (e.g. “you say you are anxious. Why do you feel anxious?”).

Figure 2 - ELIZA: a simple chatbot that mimicked a Rogerian therapist by reflecting user statements back as questions.

Although ELIZA failed the Turing Test and had no real understanding of human language, many users reported feeling “understood” by it. These early experiments demonstrated the power of conversation and that even simple rules could create the illusion of “intelligence”.

The Early Days of Symbolic Reasoning

Initial AI systems represented knowledge using words and logical rules in a process called “symbolic reasoning”. This design was convenient and effective for situations where the rules were clear-cut, and the input could easily be transformed into symbolic representations:

Figure 3 - Symbolic AI used named concepts (symbols) and explicit rules to derive facts and make decisions.

This process could be used to make conclusions, and from the 1950s to the 1990s, it defined what we now know as Good Old-Fashioned AI (GOFAI). Early programs such as the “Logic Theorist” and “Samuel’s Checkers player” demonstrated that by following basic rules, machines could prove theories and even play basic board games.

Figure 4 - Samuel’s Checkers used symbolic search: the search engine expanded a game tree, scored leaf positions with human-crafted heuristic rules, and performed a minimax backup to select the next move.

By the 1980s, this grew into “expert systems” as expert knowledge was encoded within multiple “if–then” rules. One example of an early expert system was MYCIN, an AI designed to assist doctors by suggesting what bacteria could be behind a serious infection and what treatment is advised.

These symbolic systems were clear and could often be easily interpreted by a human to understand why a decision was made. However, symbolic AI had limits. Building and maintaining large knowledge bases took a lot of effort, and it turned out the real world was also messy and full of exceptions. As a result, enthusiasm and funding for this type of research declined in a period known as the AI winter. The field needed a way for machines to learn patterns from data, not be limited to following rules handcrafted by humans.

The Power of Search

Before learning methods took centre stage, one high-profile success came from another direction. In 1997, IBM created a system called “Deep Blue” which was able to defeat the world chess champion Garry Kasparov, in a game of chess. Deep Blue worked by combining fast hardware with a powerful search feature, that, for each move, explored a vast number of possible positions and picked the moves that led to the best outcomes further down the line.

Figure 5 - Deep Blue used the same symbolic state representation as earlier programs but used a much deeper minimax search with alpha–beta pruning on dedicated hardware.

This showed that with enough compute power and well-designed search features, machines could match and even surpass human skill in well-defined domains - such as the game of chess. However, the machine’s knowledge was still human-crafted (e.g. chess opening books) and there was still no ability for learning during play.

The Artificial Neural Network

Alongside symbolic AI and search, researchers had been exploring mathematical models inspired by the brain called neural networks. The earliest iteration of this was the Perceptron in the late 1950s. Here, inputs were multiplied by weights, a bias was added, and a hard threshold was then used to produce a binary output. These were simple yes/no gates and could only learn linearly separable patterns (AND, NAND, OR, NOR), but they famously could not model non-linear patterns like XOR, which limited their capabilities.

Figure 6 - Single-layer Perceptron solving AND using a hard step. Errors resulted in a change to the output layer weights and biases by a small amount, scaled by the learning rate and the input values.

Soon after, researchers proposed networks with hidden layers and smooth activation functions (e.g. sigmoid, tanh, ReLU) that produced real-numbered outputs. These early neural networks were called Multi-Layer Perceptrons (MLPs), and with non-linear activations, they could model non-linear relationships - such as XOR.

Figure 7 - A single neuron trained with backpropagation to adjust all layer weights and biases in proportion to how much they contributed to the error (left); An MLP solving XOR using neurons with smooth activation (right).

The emergence of backpropagation in the mid-1980s made training these networks practical. However, progress stalled as computers were slow, datasets were small, and networks were shallow, and by the 1990s, other methods, such as decision trees, often still performed better on practical tasks. Even so, this work established the basic architecture for artificial learning that later advances would continue to build upon.

The Rise of Machine Learning and Deep Learning

From the late 1990s onwards, research moved more towards statistical machine learning. The idea was that, instead of writing rules by hand, researchers could train neural network models on labelled examples so they could learn the correct mappings from input to expected output. In practice, the model would make predictions (the forward pass), a value would be defined to measure how wrong the guess was, and an optimiser was used to update the model’s numbers to make future predictions more accurate.

This early form of machine learning still relied on hand-engineered features. For example, an engineer would count edges in an image or measure the frequency of words in text, then pass those numbers to a classifier like a decision tree or a support vector machine(SVM). These methods did well, but were still limited by human involvement, and designing good features for each new problem took time and expertise.

Figure 8 - Machine learning relied on manual feature extraction

As datasets and compute grew, neural networks re-emerged at scale. This increase in power and depth gave rise to the concept of deep learning, expanding on existing machine learning techniques. The key difference here was that instead of humans telling the models what to measure, the model could discover those measurements for itself using multiple deep layers of neurons. In images, lower layers detected edges, mid layers captured textures and parts, and higher layers represented the whole object, removing much of the manual feature work.

Figure 9 - Deep learning merged feature extraction into the neural network itself

A key moment arrived in 2012 when a neural network called AlexNet won the ImageNet image recognition challenge by a significant margin. It was trained on millions of images, used GPUs to run quickly, and learnt features directly from pixel data. This showed that deep learning could scale well and deliver significant gains over hand-engineered pipelines.

Different network types tackled different data sets. Convolutional Neural Networks (CNNs) worked well for images because they looked for repeating visual patterns. Recurrent Neural Networks (RNNs) could rapidly process speech and text sequences by keeping track of what had come before, and a design called Long Short-Term Memory (LSTMs) helped them remember information over longer spans.

A famous demonstration of this new era came in 2016 with DeepMind’s AlphaGo. Go had far more possible positions than chess, which made the brute-force search of earlier systems such as Deep Blue impractical. DeepMind overcame these limitations by combining deep neural networks with reinforcement learning and a guided search over promising lines of play.

Figure 10 - AlphaGo combined learned policy and value networks with Monte Carlo Tree Search (MCTS). The policy guided where to look, the value estimated who was winning, and MCTS repeatedly simulated and backed up averages.

This concept of reinforcement learning effectively allowed the AI to play many games against versions of itself, learn what moves tended to lead to wins, and used that experience to direct a focused search during real-time play. When it beat the world champion Lee Sedol by four games to one, it marked a historic moment. People could now see that learning systems could master complex tasks that had once seemed out of reach.

The Birth of Generative AI

While deep learning excelled at recognition, we soon shifted towards creation, and in 2014 Generative Adversarial Networks (GANs) emerged. These systems consisted of two neural networks trained together. One to generate synthetic images and one to identify real images from fake ones. Across iterations, the generator improved until its images looked extremely convincing. For example, when trained on the photographs of human faces, it could produce new faces that resembled real people but didn’t actually belong to anyone.

Figure 11 - The Generator (G) turns noise into images; the Discriminator (D) learns real vs fake. They train together and D learns to spot fakes whilst G learns to fool D, until the fake images look real.

Around 2020 another family of models rose to prominence called “diffusion models”. The basic idea was that instead of attempting to generate images in one step, we could take an image, add some noise to it, then train a neural network to reverse that process by removing the noise. Once trained, the model could take an image of pure noise and denoise it into an image based on a text description of what the image should be.

Figure 12 - Unlike GANs, which generated images in one step and used a discriminator, diffusion models created images by iteratively denoising static, aided by text, reference images, and structure maps.

By 2022, generative tools had reached the public, and people were typing prompts and receiving detailed images from systems such as DALL-E 2 and Midjourney. Creators used them for concept art and design, musicians experimented with models that composed short passages, and developers tested generating 3D assets.

In early 2024, OpenAI then revealed Sora, which could generate short photoreal videos from simple text prompts. Sora combined the core ideas built by diffusion models and Transformers to produce clips with consistent motion and scenes. This was shortly followed up by additional advances like Google's Veo-3, which could not only generate photoreal video, but the appropriate audio as well.

These advances marked AI’s creative phase, where AI systems no longer just labelled or classified content, they created it.

The Transformer Breakthrough

The next leap came in 2017 with a research paper titled “Attention Is All You Need” that introduced the “Transformer” architecture. This was a brand-new way to process sequences of words, sounds or other tokenised data formats. Previous models such as RNNs would read a sequence word by word, making it hard to capture relationships between distant words in sentences. The Transformer took a different approach. It looked at all the tokens in a sequence at once and used a mechanism called “attention” to decide which tokens mattered most to each other, then transformed the sequence into a rich, context-aware representation.

Figure 13 - Attention between every word in a sentence

Imagine skimming a paragraph and letting your eyes jump to the words that seemed most relevant, rather than strictly reading every word left to right. Because the transformer processed tokens in parallel, it could be trained extremely effectively on vast amounts of data and could learn context across a long range of tokens.

Figure 14 -Transformers let every token look at every other token (attention) while retaining token positional context. The results: Parallel training, long-range context, and fluent sequence generation.

Soon after the paper was released, researchers began to stack many Transformer layers on one another to build extremely large language models (LLMs as we now call them). OpenAI’s Generative Pre-training Transformer (GPT) family and Google’s Bidirectional Encoder Representations from Transformers (BERT) were some of the first well-known examples of this at the time - both models being trained on an extremely vast corpus of data.

And with that, the foundation was set for the modern wave of generative, conversational, and agentic AI we now see every day.

The Creation of Conversational AI

Human language benefited in a similar way. In 2020, OpenAI created GPT-3, a large language model with 175 billion parameters. At its core, it was a Transformer, simply trained to predict the next word in a sentence. However, as it was trained at such an enormous scale, it learned to produce patterns that emulated human reasoning and style. Suddenly, there was a tool that could produce essays, answer arbitrary questions, and even generate code.

This exploded into the public domain on the 30^th of November 2022, with the launch of ChatGPT. The application wrapped an improved GPT-3 model in a simple chat interface, and within days, millions of people were using it. The system followed instructions, kept track of conversation history, admitted to mistakes, and provided helpful human-like replies.

Figure 15 - ChatGPT’s user interface

Other companies soon followed with their own AI chat assistants and behind the scenes, developers continued to improve these models using reinforcement learning from human feedback - a technique where human preferences could be used to guide the model towards more helpful and safe responses.

Conversational AI rapidly changed people’s expectations and everyday use of technology. Users could now simply ask for what they wanted in plain language and get it in seconds, rapidly making AI present and ingrained in many areas of everyday life.

Computer Using Agents and Agentic AI

One of the more recent shifts moved beyond simply answering questions and generating media and enabled the AI to take action. Researchers explored agentic AI by creating systems that could plan, invoke tools, and carry out multi-step tasks with minimal supervision. Early community experiments in 2023 chained model calls together to break a high-level goal into AI-capable subtasks. Across 2024, this matured as vendors added function-calling, tools, and protocols such as the Model Context Protocol (MCP) to standardise how models accessed external services.

Figure 16 - MCP acts as a broker between an AI model and external tools. The model issues calls to the MCP server, which in turn executes them via the relevant service APIs, returning the result.

In early 2025, OpenAI then introduced Operator. This pushed the idea even further with a Computer-Using Agent (CUA) trained to perceive screen elements and operate a browser autonomously. Users could now ask this CUA to complete multi-step browser-based tasks, and it would type, click, and navigate away - pausing for confirmation when needed.

Figure 17 - A CUA carrying out tasks inside a browser on a virtual machine, streaming screenshots and results back to the user, who can take control at any time.

This hinted at a future where AI assistants were not limited to generating content, but could handle entire workflows. It also shone a light on the need to design safeguards and security controls around these systems, as the drastic and rapid move from passive assistant to active autonomous AI agent changed the risk profile of the landscape, reinforcing that AI technology needed to be designed reliably - and with care.

Conclusion

The story of AI has not been a single breakthrough, even if the past few years have made it feel that way. It has been a layered progression in which each era prepared the ground for the next: symbolic systems taught clarity and structure; search proved that brute computation and branch pruning could master complex games; machine learning replaced hand-written rules with the ability to learn patterns; deep learning embedded feature extraction into the model itself; Transformers provided the ability to scale and keep context; generative models created new content; conversational systems made AI assistance accessible; and, most recently, agentic systems began to act on our behalf.

Several ingredients were needed along the way to unlock each of those leaps. Algorithms changed (attention, diffusion, optimisation). Compute power grew (GPUs/TPUs and distributed training). Data expanded as the web and video content scaled. And the interface created a chat window that brought large language models from research into everyday life.

Looking ahead, the direction is clear even if the details aren’t. Models will expand into all domains, take more initiative to plan and act, run on smaller devices, and leverage outside tools and real-time knowledge for up-to-date information. The real challenge now is less about how to scale AI, and more about how we can make it dependable, trustworthy, and safe! Designing AI systems that people can understand, trust, and use safely is extremely important in a world where misinformation and dangerous information can already be disseminated so easily and at such a huge scale.

AI has moved from rules and learning to creating and acting. The next chapters will for sure continue that journey, driven by human curiosity, restrained, hopefully, through care, and shaped by all of us who every day choose and influence how these systems will be used.