Deep Reads

Quoting SpaceX S-1

We have the ability to use compute resources to support our proprietary AI applications (such as Grok 5, which is currently being trained at COLOSSUS II), while also providing access to select compute capacity to third-party customers. For example, in May 2026, we entered in

How fast is 10 tokens per second really?

How fast is 10 tokens per second really? Neat little HTML app by Mike Veerman (source code here) which simulates LLM token output speeds from 5/second to 800/second. Useful if you see a model advertised as "30 tokens/second" and want to get a feel for what that actually loo

Google I/O, Gemini Spark, Antigravity

It's hard to find much to write about Google I/O this year because I have a policy of not writing about anything that I can't try out myself, and a lot of the big announcements are "coming soon". I actually prefer to write about things that are in general availability, becau

llm-gemini 0.32

Release: llm-gemini 0.32 New model gemini-3.5-flash for Gemini 3.5 Flash. See also my notes on Gemini 3.5 Flash, and the pelican I drew using this upgrade to the plugin. Tags: llm, gemini

Why We Think

Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute (Graves et al. 2016, Ling, et al. 2017, Cobbe et al. 2021) and Chain-of-thought (CoT) (Wei et al. 2022, Nye et al. 2021), have led to significant improvements in

Reward Hacking in Reinforcement Learning

Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. Reward hacking exists because RL environments are often imperfect, and it is fu

Extrinsic Hallucinations in LLMs

Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model makes mistakes. Here, I would like to narrow down the problem

Diffusion Models for Video Generation

Diffusion models have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation. The task itself is a superset of the image case, since an image is a video of 1 frame, and it is much

Common pitfalls when building generative AI applications

As we’re still in the early days of building applications with foundation models, it’s normal to make mistakes. This is a quick note with examples of some of the most common pitfalls that I’ve seen, both from public case studies and from my personal experience. Because these pitf

Agents

Intelligent agents are considered by many to be the ultimate goal of AI. The classic book by Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, 1995), defines the field of AI research as “the study and design of rational agents.” The unpre

Building A Generative AI Platform

After studying how companies deploy generative AI applications, I noticed many similarities in their platforms. This post outlines the common components of a generative AI platform, what they do, and how they are implemented. I try my best to keep the architecture general, but ce

Measuring personal growth

My founder friends constantly think about growth. They think about how to measure their business growth and how to get to the next order of magnitude scale. If they’re making $1M ARR today, they think about how to get to $10M ARR. If they have 1,000 users today, they think about

Moving To Substack

I’m freezing this blog and starting to post on my Substack instead. The authoring experience is much more convenient for me there. Please follow me there, and check out The Illustrated DeepSeek R-1 if you haven’t yet. And check out our How Transformer LLMs Work course!

Generative AI and AI Product Moats

Here are eight observations I’ve shared recently on the Cohere blog and videos that go over them.: Article: What’s the big deal with Generative AI? Is it the future or the present? Article: AI is Eating The World

Remaking Old Computer Graphics With AI Image Generation

Can AI Image generation tools make re-imagined, higher-resolution versions of old video game graphics? Over the last few days, I used AI image generation to reproduce one of my childhood nightmares. I wrestled with Stable Diffusion, Dall-E and Midjourney to see how these commerci

The Illustrated Stable Diffusion

Translations: Chinese, Vietnamese. (V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version) AI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visua

[AINews] OpenAI GPT-next disproves 80 year old Erdős planar unit distance problem for under $1000

a quiet day but a nice result in AI x mathematics

Railway: The Agent-Native Cloud — Jake Cooper

3M Users, 100K Signups/Week, Own-Metal Data Centers, $200K+ Coding Agent Spend, and the Death of PRs

[AINews] Google I/O 2026: Gemini 3.5 Flash, Omni (NanoBanana for Video), Spark (background agents), and Antigravity 2.0

Google has been busy!

[AINews] How to land a job at a frontier lab (on Pretraining)

a quiet day before google i/o lets us amplify a notable blogpost

Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.

An eventful month with one flagship release after another

How open model ecosystems compound

Further reflections on China's high-participation, open-first AI ecosystem.

Notes from inside China's AI labs

Lessons from my trip to talk to most of the leading AI labs in China.

The distillation panic

‘Distillation attacks’ is a horrible term for what is happening right now.

I Don’t Want a Learning Dashboard for My Child

Often debates about education are framed as non-tech versus AI approaches, but too often, AI ed tech just magnifies the same failures of traditional school.

Breaking the Spell of Vibe Coding

Vibe coding is the creation of large quantities of highly complex AI-generated code, often with the intention that the code will not be read by humans. It has cast quite a spell on the tech industry. Executives push lay-offs claiming AI can handle the work. Managers pressure empl

How To Use AI for the Ancient Art of Close Reading

Close reading is a technique for careful analysis of a piece of writing, paying close attention to the exact language, structure, and content of the text. As Eric Ries described it,“close reading is one of our civilization’s oldest and most powerful technologies for trying to com

Stop Saying Boredom is Good for Kids

Chronic boredom is harmful to adults, causing stress, disengagement, and poor well-being. Academic researchers have shown that boredom in the workplace can be just as damaging as burnout. But search for information about childhood boredom and you’ll find the opposite message: art

After Orthogonality: Virtue-Ethical Agency and AI Alignment

Preface This essay argues that rational people don’t have goals, and that rational AIs shouldn’t have goals. Human actions are rational not because we direct them at some final ‘goals,’ but because we align actions to practices[1]: networks of actions, action-dispositions, action

AGI Is Not Multimodal

"In projecting language back as the model for thought, we lose sight of the tacit embodied understanding that undergirds our intelligence." –Terry Winograd The recent successes of generative AI models have convinced some that AGI is imminent. While these models appear to capture

Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research

What is the Role of Mathematics in Modern Machine Learning? The past decade has witnessed a shift in how progress is made in machine learning. Research involving carefully designed and mathematically principled architectures result in only marginal improvements while compute-inte

What's Missing From LLM Chatbots: A Sense of Purpose

LLM-based chatbots’ capabilities have been advancing every month. These improvements are mostly measured by benchmarks like MMLU, HumanEval, and MATH (e.g. sonnet 3.5, gpt-4o). However, as these measures get more and more saturated, is user experience increasing in proportion to

AIE Singapore: The Agentic Nation

i gave a little talk as closing keynote for the first AI Engineer Singapore. burned some bridges but said what i felt.

What you can do in a decade

I turned 40 today. For my 35th I did principles, but for my 40th, I wanted to offer perhaps more useful reflections.

How to Thought Lead (2026)

I first started compiling "How To Thought Lead" in my notes 5 years ago, at first as an ironic parody and then slowly becoming sincere, and never published it, 1) because I don't know if I ever really nailed it / have a complete picture, 2) I was somewhat worried if I published i

Constance Crozier: Forecasting s-curves is hard

There was a famous Covid era chart that I always struggle to find, showing how hard it is to estimate an S curve while living through it. in the early days it seems that everything is exploding as an exponential and you always get hypey essays about how YOU, YOU DUMB DUMB, DONT U

The Sequence Opinion #864: Every AI Agent Needs a Computer

The raise of agentic sandboxes.

The Sequence AI of the Week #863: The Model is the Interface: Inside Thinking Machines' Interactive Models

Thinking Machines’ interactive models turn real-time conversation, vision, audio, and tool use into one continuous learned system.

The Sequence Knowledge #862: Learning About Text Diffusion Models

One of the most credible alternatives to transformers.

The Sequence Radar #861: Last Week in AI: IPOs, Interactive Models, and Recursive Dreams

Cerebras monster IPO, three new innovative frontier AI labs

Sign of the future: GPT-5.5

One impressive step on the curve

Claude Dispatch and the Power of Interfaces

We often lack the tools for the job, even if the AI is capable enough

The Shape of the Thing

Where we are right now, and what likely happens next

A Guide to Which AI to Use in the Agentic Era

It's not just chatbots anymore

Do AI Risks Require Extraordinary Government Intervention?

Let’s not skip the hard work of AI governance

Open-world evaluations for measuring frontier AI capabilities

Introducing CRUX, a new project for evaluating AI on long, messy tasks

New Paper: Towards a science of AI agent reliability

Quantifying the capability-reliability gap

AI Won’t Automatically Make Legal Services Cheaper

Applying the AI as Normal Technology framework to legal services