AGI is Now in Sight and is Inevitable
15-seconds summary
Over the past week, I've been playing with reasoning models and Deep Research tools and have become convinced that AGI is no longer a distant possibility—it's in sight and it's inevitable. Using a framework grounded in first-principle information theory and a hiking analogy, I will explain why AGI should be seen as an unavoidable outcome of current advancements. Central to this thesis is the transformative role of Chain of Thought reasoning, which allows AI to perform ad-hoc gradient descent on each user-generated query.
The Hiker's Journey: Intelligence as a Gradient Descent in Information Space
At its core, intelligence can be thought of as a process of gradient descent within a vast and intricate information space. For any given question, this space contains an "absolute minimum"—the optimal answer. Using this representation, AGI can then be defined as the ability of an AI system to consistently outperform humans in navigating the information space, arriving at better answers with greater speed and efficiency across a wide range of queries.
To make this concept more intuitive, imagine the information space as a sprawling, mountainous terrain. The AI is a hiker tasked with finding the deepest valley—the point of absolute minimum height. Initially, the hiker has no map, no clear understanding of the landscape, and no tools to guide their journey.
GPTs: Teaching the Hiker to Walk and Find Direction
The journey toward AGI took a monumental step forward with the advent of GPTs (Generative Pre-trained Transformers). These models enabled machines to generate coherent sequences of words, and as their scale increased—with larger architectures and more parameters—they unlocked higher-order cognitive skills like deductive reasoning.
In our hiker analogy, it is as if GPTs introduced the ability to undertake directional movement toward a point, traversing the terrain purposefully, rather than randomly changing direction.
However, GPTs are not without limitations. While they excel at general-purpose tasks, they often lack the depth and precision required to tackle highly specific, user-defined queries. Humans, by contrast, are adept at zooming in on narrow slices of the information space, leveraging domain expertise, contextual understanding and reasoning to locate optimal answers to specific questions with precision.
In recent years, the core challenge for AGI has essentially been bridging this gap: empowering machines to dynamically adapt and refine their exploration of the information space to meet the specific demands of each query.
Chain of Thought: Transforming the Hiker into a Trail Runner
Recent advancements in chain of thought reasoning have introduced a transformative dimension to artificial intelligence, enabling AI to focus deeply on specific topics and perform deep optimization on user-generated queries.
This progress has been driven by three key technical innovations:
Automated Web Browsing and RAG: Machines now have the ability to pull in far more knowledge than humans can, almost instantaneously, by browsing the web and retrieving information into context. For instance, tools like OpenAI and Perplexity's Deep Research can read and synthesize insights from 20-30 sources in just minutes. This provides a level of depth and breadth on any specific topic that would take humans days or even weeks to achieve, dramatically accelerating the process of gathering and contextualizing information.
Reasoning-Action Sequencing: Chain of thought reasoning allows AI to perform gradient descent-like optimization for user-generated queries. By iteratively observing previous steps, planning the next move, and executing it, AI systems can traverse the information space in a structured and purposeful way. This iterative process enables the AI to refine its understanding and move closer to optimal solutions with each step, mimicking human-like reasoning but with far greater speed and efficiency.
Simulated Annealing Techniques: Inspired by the metallurgical process of annealing, AI systems now incorporate strategies to take larger exploratory steps initially, backtrack to escape local minima, and gradually reduce step sizes as they approach an optimal solution. This approach allows for a thorough exploration of the information space while avoiding pitfalls, such as getting stuck in suboptimal answers. As the AI nears the absolute minimum, it takes smaller, more precise steps, ensuring that it converges on the best possible solution.
Why AGI is inevitable
The ability of AI to perform inference-time optimization within the information space represents a transformational breakthrough that will inevitably lead to AGI. This capability is inherently scalable: with sufficient time and computational resources, machines can explore countless pathways, achieving levels of precision and efficiency far beyond human reach. Unlike humans—who are constrained by time, cognitive limits, and the imperfect transfer of knowledge across generations—machines face no such barriers.
Moreover, continued advancements at the core algorithmic level, such as enhancements to GPT models, will further improve AI's ability to perform abstractions, recognize intricate patterns, and derive deeper meaning. These foundational capabilities will only grow stronger as scaling progresses, driving AI closer to its full potential and paving the way for AGI to become a reality.
"We are now confident we know how to build AGI as we have traditionally understood it" — Sam Altman, January 2025
Returning to our analogy, imagine the hiker evolving into an ultra-efficient, infatigable trail runner. With each iteration, the runner assesses their progress, recalibrates their path, and accelerates toward the valley. Unlike the hiker, who might tire or get stuck in a suboptimal spot, the trail runner can explore the terrain tirelessly, covering vast distances and escaping smaller valleys with ease.
Closing thoughts
Artificial intelligence is fundamentally about searching the vast information space for optimal solutions. With advancements in technologies like chain of thought reasoning, we are equipping machines with the tools to achieve AGI.
And the best part? This journey is not exclusive. The market for intelligence is expansive, the barriers to entry are falling, and the ecosystem is shaping up to be inherently democratic. AGI will not only exceed human capabilities in certain domains but also create opportunities for innovation and collaboration across the board.
The future of intelligence is bright, open, and within reach. At Crosscourt we are excited to be at the center of it.