Category: AI

In 2013 Google started work on TPUs and deployed them internally in 2015. Sundar first publicly announced their existence in 2016 at I/O, letting the world know that they’d developed custom ASICs for TensorFlow. They made TPUs accessible to outside devs via Google Cloud in 2017 and also released the second generation that same year. And since we’re plotting a timeline here, the Attention is All You Need paper that launched the LLM revolution was published in June of that same year.

OpenAI got a lot of attention with GPT4, a product based on the AIAYN paper, putting LLMs on the map globally, and Google has taken heat for not being the first mover. OpenAI last raised $6.6B at a $157B valuation late last year, which incidentally is the largest VC rounder ever, and they did this on the strength of GPT4 and a straight line trajectory that GPT5 will be ASI and/or AGI, or close enough that the hair splitters won’t matter.

But as OpenAI is lining up Oliver Twist style asking NVidia if “please sir, may I have some more” GPU for my data center, Google has vertically integrated the entire stack from chips with their TPUs, to interlink, to the library (TensorFlow) to the applications that they’re so good at serving to a global audience at massive scale with super low latency, using water cooled data centers that they pioneered back in 2018 and which NVidia is getting started with.

Google has been playing a long game since 2013 and earlier, and doesn’t have to create short term attention to raise a mere $6 billion because they have $24 billion in cash on their balance sheet, and that cash pile is growing.

What Google has done by vertically integrating the hardware is strategically similar to SpaceX’s Starlink, with vertically integrated launch capability. It’s impossible for any other space based ISP to compete with Starlink because they will always be able to deploy their infrastructure cheaper. Want to launch a satellite based ISP? SpaceX launched the majority of the global space payload last year, so guess who you’re going to be paying? Your competition.

NVidia’s margin on the H100 is 1000%. That means they’re selling it for 10X what it costs to produce. Google are producing their own TPUs at scale and have been for 10 years. Google’s TPUs produce slightly better performance than NVidia’s H100 and is probably on par when it comes to dollar per compute. Which means Google is paying 10X less for GPU compute than their competitors.

And this doesn’t take into account the engineering advantages derived from having the entire stack from application to chips to interconnect all in-house, and how they can tailor the hardware to their exact application and operational needs. When comparing NVidia to AMD, the former is often described as having a much closer relationship with developers and releasing fixes to Cuda on very short timelines for their large customers. Google is the same company.

As a final note, I don’t think it’s unreasonable to consider the kind of pure research that drives AI innovation as part of the supply chain. And so one might argue that Google has vertically integrated that too.

So amidst the noise and haste of startups and their launches, remember what progress their may be in silence.

The $60 million deal that Google cut with Reddit will emerge as incredibly cheap as foundational model providers realize amidst the data crunch that Reddit is one of the few sources of constantly renewed expert knowledge, with motivated experts in a wide range of fields contributing new knowledge on a daily basis for nothing more than social recognition. The deal is non-exclusive as was demonstrated by a subsequent deal with OpenAI, meaning Reddit will begin to print money.

Google’s vertical integration of hardware via their TPUs, their software applications, and their scientists inventing the algorithms that underpin the AI revolution is going to begin to pay off. Google will launch a number of compelling AI applications and APIs in 2025 that will take them from an academic institution creating algorithms for others, to a powerhouse in the commercial AI sector. Their cost advantage will enable them to deliver those applications at a far lower price to their customers, and in many cases, completely free. Shops like OpenAI lining up for NVidia GPUs will be the equivalent of a satellite ISP trying to compete with Starlink who have vertically integrated launch capability.

DeepSeek will continue to demonstrate unbelievable cost reductions after delivering V3 for less than $6 million as the group of former hedge fund guys continues to sit in a room and simply outthink OpenAI, which has been hemorrhaging talent and making funding demands approaching absurdity.

OpenAI will be be labeled the Netscape of the AI revolution and be absorbed into Microsoft at the end of the year. But like Netscape, many of their ideas will endure and will shape future standards.

As companies like Google and High-Flyer/DeepSeek prove how cheap is to train and operationalize models, there will be a funding reset and companies like Anthropic who raised a $4 billion series F round from Amazon in November will need to radically reduce costs and we may see down rounds.

We will see new companies emerge that provide tools to implement o1 style chain of thought in a provider and model agnostic way. Why pay o1 token prices for every step in CoT when some of the steps can be done by cheaper (or free) models from other providers?

China will continue to rival the USA in AI research and in shipped models. The new administration will rethink the current limits on GPU exports which will prove ineffective at accomplishing their goals of slowing the competition.

And finally my personal hope is that the conversation around the dangers of AI will shift from a fantastic Skynet scenario to the practical reality that out of the $100 trillion global GDP, $50 trillion is wages, and that is both the size of the AI opportunity and the scale of the global disruption that AI will create as it goes after human labor and human wages.

We need to acknowledge this reality and hold to account disingenuous companies and founders who are distracting from this through AGI and ASI scare mongering. This “look at the birdie while we steal your jobs” game needs to end. The only solution I’ve managed to think of is putting open source tools and open source models in the hands of the workers of the world to give them the opportunity to participate in what could, long term, become a utopian society.