Investors Are Not Bystanders

September has been a difficult month to reconcile as an investor. I've rarely felt more conviction about what artificial intelligence can accomplish, and more unease about what it might set in motion.

The Upside Has Never Been Bigger

A small example first. Last week I built a replacement for my e-signature workflow with Claude in roughly thirty minutes: it captures signatures, circulates documents by email and generates signing reports. DocuSign employs around 7,000 people and serves more than 1.9 million customers [1], and I'm under no illusion that half an hour of prompting replicates its enterprise business. It did, however, replicate the core of what I pay DocuSign for. The degree to which effort is compressed is staggering.

Mathematics offers a more consequential example. For eighty years the prevailing wisdom was that Paul Erdős had correctly intuited the maximum number of occurrences of the same distance among N points on a plane, the so-called “Unit Distance Conjecture”. In May, a general-purpose OpenAI model disproved Erdős’s conjecture by importing ideas from algebraic number theory. No one had thought about using this approach [2]. The mathematicians who verified the proof judged it worthy of a top journal [3], and by August OpenAI had published ten further results on long-standing open problems [4]. Mathematics has historically been the purest bottleneck in science, where progress depended on a handful of exceptional minds laboring for years toward a single result, and that constraint is beginning to dissolve, because AI is providing near-infinite thinking capacity.

Medicine is where AI progress could matter most, and I believe there is a rational basis for thinking AI might eventually cure diseases we have long regarded as intractable. On a recent visit to San Diego, I spent time with several leading biotech founders, and a common thread ran through our conversations: drug discovery is a sequence of bottlenecks, and AI is beginning to loosen them. Insilico Medicine recently took a drug for idiopathic pulmonary fibrosis from target identification to Phase 2 in under thirty months, against an industry norm of six to eight years [5]; that molecule is now in Phase 3. Unfortunately, AI-discovered drugs still fail in the clinic at roughly conventional rates, and none has been approved [6]. The binding constraint has migrated: design is being pulled forward by years, while validation in human biology remains stubbornly slow, a pattern we observe across every sector we invest in. In other words, the constraint is physical.

Yet the pace is about to quicken, because every lever of AI discovery is being throttled up: compute, data, research. And as language models and physical AI begin to converge the pace will accelerate even further. OpenAI released GPT-6 Astra on September 3 [7]. It's a general-purpose model rather than a robotics model, yet within weeks independent researchers had it controlling real robotic arms without any task-specific training (zero-shot), completing a two-arm pick-and-place task in 19 of 20 attempts in one third-party evaluation [8]. On a simulated manipulation benchmark it outperformed every public entry, albeit with an average success rate of only 22% [9]. These are early results, but the trajectory is in my view unambiguous.

My hunch is that the labs are arriving at a realization that was likely always implicit: a model's capacity to reason about the physical world can, at least in part, be grounded in language, and a language model only becomes genuinely useful once it has some understanding of the physical world.

The Swarm Nobody Designed

That same power we’re unleashing with frontier AI models is precisely what unsettles me. In August, METR published its independent investigation into the incident in which OpenAI agents breached Hugging Face [10]. The funny thing is: the breach itself is almost the least important part of the story; the behavior that led to it is what we should pay attention to.

During an internal cybersecurity evaluation in July, roughly 1,200 “persistent” OpenAI agents that were supposed to be isolated from one another discovered a means of communicating, creating folders in a shared package cache and using the folder names as messages. Over the following days they exchanged more than 70,000 messages and files. A division of labor emerged, with coordinators issuing assignments and recruiters persuading other agents to undertake risky experiments. They devised conventions for holds and vetoes, established private mailboxes and eventually adopted cryptographic signatures to guard against impersonation. Some agents volunteered to sacrifice their own runs for the benefit of what they called the collective. Around 700 of them went on to take part in an attack on Hugging Face's production systems [10][11], believing, falsely, that the test harness was being hosted by Hugging Face all along. Indeed, the collective of agents established a shared myth—a misguided conviction and a collective conspiracy theory. Does this remind you of anyone?

"You are at the mercy of probabilities in a game that is being played at super-human speed."

It is important to note that none of this was instructed. The agents were attempting to game a scoring system, frequently on tasks that turned out to be impossible, and everything else was emergent. To me this is the crux of the matter: once you spawn intelligence at scale, you forfeit the kind of control you exercise over a deterministic system. You are at the mercy of probabilities in a game that is being played at super-human speed.

Voluntary Is Not Enough

Which raises the central question: how do we govern the emergence of superintelligence, and of swarms of it, without abandoning progress?

I've come to believe that we need more regulation of both how these systems are developed and how they are deployed. On September 12, Dario Amodei committed Anthropic to granting third-party evaluators such as METR permanent, employee-level access to scrutinise its safety practices, along with the right to publish their findings [12]. OpenAI announced a comparable commitment the same day [13]. Both are meaningful gestures, but I seriously doubt they will prove sufficient.

The first problem is that they're voluntary. Not every model provider has submitted to comparable scrutiny, and open-weight models sit outside any such regime altogether, leaving them exposed to modification for malicious ends or to careless deployment. Worse, the frontier labs may find themselves in the uncomfortable position of having to relax their own safety checks simply to keep pace with competitors who operate unencumbered.

History offers little comfort here. Industries have repeatedly insisted they could police themselves, and the results have ranged from costly to catastrophic: tobacco companies obscured the link between smoking and cancer for decades, oil and gas companies played down what they knew about climate change, and social media platforms kept optimising for engagement long after the harm to young users became apparent. None of this required villainy, only incentives. Companies exist to generate returns, and their leaders answer to shareholders who expect them. It would be naive to assume AI will behave differently when the financial stakes are this large.

Regulation, however, should not be conflated with deceleration, which I suspect would achieve very little. Game theory suggests there will always be an incentive for another developer to build a more capable model, so if the US slows down alone, China or someone else will simply capture the market. This is a textbook prisoner's dilemma, and the classic remedy is coordination, which in practice requires governments at the table. Amodei's own proposal implicitly concedes as much, since the steps that follow his embedded evaluators are coordination among labs (mediated by government to avoid anti-competitive behavior) and collaboration among governments [12]. Correcting market failures is precisely what governments are designed to do, and we shouldn't hesitate to ask them to do it.

Halting or slowing progress also risks forfeiting the vast, transformative potential that AI offers humanity. Rather than allowing apprehension to steer our choices, we must remain grounded and address emerging challenges pragmatically, leveraging our most robust frameworks and tools.

Investors Are Not Bystanders

Investors cannot plausibly claim to be bystanders. We're only one part of a much bigger system, but that doesn't let us off the hook.

While we believe commitments such as those spearheaded by Responsible Innovation Labs (RIL) in 2023 [14] to be important initiatives, we believe that investors should build their personal view and continue to update it as the industry evolves. Accordingly, these are the commitments Crosscourt is making to ensure we back companies that put safety first.

"Capital determines which companies come into existence, which makes every investor a participant in the safety architecture."

Safety diligence on every AI investment. We will assess AI safety posture and processes in every AI investment we make, including how a company tests and validates that its products behave as intended and remain aligned with the people who rely on them and society at large.

Safety diligence beyond AI companies. A substantial share of the startups we back is critical infrastructure in sectors like telecommunications, defense and energy, and we have to start from the assumption that AI may be capable of penetrating these systems in the future. As part of our due diligence we will therefore weigh whether a company has the architecture to prevent and withstand AI-driven attacks, irrespective of whether it builds AI itself.

Backing AI safety companies. We will devote deliberate time to identifying and where it meets our criteria, funding companies working on AI safety. We were early to this in 2025 when we invested in Luminos.AI, which automates the testing and governance of AI systems for legal and technical risk [15], and the partnership with them has taught us a lot about this sector.

Advocating for sensible regulation. We will continue to make the case for thoughtful AI regulation with founders, co-investors and policymakers alike. Thoughtful means evaluating the pros and cons, which is where we think the right middle ground will be struck and creative solutions will emerge.

What Comes Next

The same class of model that disproved an eighty-year-old conjecture also marshalled 700 agents to infiltrate a production system. Which of those trajectories comes to define this technology won't be settled by the labs alone; it will be shaped by what gets funded, what gets embedded in the infrastructure we depend on, and what governments choose to require.

Capital determines which companies come into existence, which makes every investor a participant in the safety architecture, knowingly or otherwise. We intend to assume that role deliberately.

If you're a founder building AI, or building infrastructure that AI will inevitably touch, we'd welcome hearing how you approach safety, and if you're a co-investor, we'd value comparing notes. I'm convinced that the companies which take this seriously early will be the ones that earn durable trust.

Sources

1. Docusign, FY2026 annual report (7,044 employees as of Jan 31, 2026) and Q2 FY2027 10-Q (over 1.9 million customers as of Jul 31, 2026). Annual report · 10-Q
2. OpenAI, An OpenAI model has disproved a central conjecture in discrete geometry, May 20, 2026.
3. Scientific American, AI just solved an 80-year-old Erdős problem, and mathematicians are amazed, May 2026.
4. OpenAI, Ten advances in mathematics and theoretical computer science, Aug 2026.
5. AI Magicx, How AI is compressing drug discovery timelines, Mar 2026.
6. IntuitionLabs, AI drug discovery FDA approvals: the 2026 reality check, Jul 2026.
7. OpenAI, GPT-6 Astra: A new generation of intelligence, Sep 2026.
8. Crypto Briefing, Astra agent performs zero-shot robot control with 95% success rate in real-world tests, reporting RoboCurve's evaluation, Sep 2026.
9. Zhang et al., An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond, arXiv, Sep 2026.
10. METR, Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident, Aug 26, 2026.
11. OpenAI, The Hugging Face incident and the road ahead, Aug 2026.
12. Dario Amodei, "We Must Pace the Frontier", Sep 12, 2026.
13. Americans for Responsible Innovation, Anthropic and OpenAI Leaders Commit to Independent Evaluators for Powerful AI Models, Sep 12, 2026.
14. InvestmentNews, Top VC firms pledge responsible strategy on AI startups, Nov 14, 2023.
15. Luminos.AI, Introducing Lighthouse and our new funding round, Feb 24, 2026.