Anjney Midha on AI Infrastructure, Outputmaxxing, and Frontier Labs

Transcript

We're in periodic labs with FungeMita, CEO, founder of AMPLICAL. Welcome. Thanks for having me. I Google, if you'd like to say, so there's two types of utilization, usually, right, that you're measuring in these clusters, one is node allocation, and then the other is MFU. So node utilization is usually, like, work percentage of cards in the data center are just like used, and that, if it's not at, like, 95%, it's no excuse. It doesn't excuse, right? I think 95% of Google, which is where Michael Funder's sub came from, he built a board of Explore GQM scheduler at Google, and there, I think, 95% was considered an outage, so 96% node utilization should be standard, and most single-down clusters are not running at that. So that's one. And then MFU utilization should be, I would say, the best in class today, somewhere between 16-70%. I think this is a leadership question, right? Is there, and fundamentally, it's an alignment question, which is are the people who are funding the cluster, and then deploying the cluster actually aligned? And sometimes, theoretically, there are, but in practice, the number of people in the chain, the supply chain between, like, the capital, and all the way to whoever's managing the cluster, and then whoever's measuring what the output is, are just so many degrees of separation away, that, like, the, you know, heard that, sort of, you know, radium metaphor, which is at the beginning of an arc, if you have two arcs that are two lines that are just off by a few degrees, that it spreads out, right, at scale. And I think what's happening is a lot of cluster implementations and infrastructure, a lot of Frontier labs and other teams, that's what's happening is they initialize the plan, which is kind of like not snort star with a team that wants to do good, but then they're required to scale so fast, instead of iteratively, that the wasted just compounds really fast at scale. And so I think we know the answer, which is just to iterative bring-ups, you know, if you spend time with people who've been in the semiconductor industry or the DSN industry for a long time, this is not new, and I don't think it should be an excuse, like, sure, something, what is new, okay, we have a lot of new capabilities, but that doesn't mean just abandoned common sense, common sense should always be in fashion, you know, AI scaling doesn't change the, in fact, if anything, AI scaling should be putting a premium on the value of common sense in infrastructure because the margin of error now is so much lower and the cost of wastage of so much higher. And the cost of wastage, by the way, is not just economic, I mean, obviously I'm an investor or I'm an investor by background with the last few years, now we're running an AI infrastructure business called, you know, AMP. And I think that it's okay to say this time is different on the capabilities front, like we are genuinely getting capabilities of the kind we haven't had before. That doesn't give you an excuse to say this time is different for everything, especially infrastructure. So, look, I love the hacker mindset and the hustler mindset, and that's great for the start-up mindset. But you remember this moment where Zalquan from saying, move fast break things to like move faster stable infrastructure, faster stable infrastructure, I think now we need to move fast with like responsible infrastructure. Yeah. They're going to say, like, where is the impact, you know, there was a really in our class yesterday, Scott Nolan, who was a founder of General Matter came by at Stanford to speak about energy bottlenecks. And he had a phenomenal idea, he said, if you look at the marginal unit economics of compute power, instead, it's called like $4 an hour. If you're having to bring up a new data center in a new community, why not just say we're going to charge $4.50 an hour and that marginal impact or that marginal increase, we just literally take that and give it to the local community as cash. I can tell you as a customer of that compute, I would love that. I'd be happy to pay an additional $0.50 cents per hour at scale. Because if that means the public benefit is so clear to the communities that the data centers are coming up and I'm going to feel like that compute is much more reliable. You know, up to 20% of all data centers this year in the in the U.S., my understanding is our at risk of community backlash of not getting the community support they need to get brought up. Wow. That's a huge idea. Now, we, I think we should dig into about that numbers. I think it's a little bit of overstated. These things can get overreported, but they don't just care about jobs. They care about all the other stuff around it, right? They care about power grid, care about environment. Power grid, permitting, and so on. And imagine, I think if you said there's a new AI deal, if we're bringing up a data center in your community, we're actually going to reduce the cost of your electricity bill. Okay, now we're talking, right? The community's going, okay, now this is a deal. I feel like a partner in this, right? Now that's not happening. There will be audits. There will be investigations. And when the, when the regulators come, I don't know when it's going to be. The folks who are moving fast and breaking things in the name of the AI progress better be prepared. That's certainly not how we're procuring compute. Or we're trying as much as we can to work with partners who have long-term track records. Many of whom, by the way, are not like AI providers. I think this whole idea of neocloud being somehow this new category is a lot of marketing speak. There are really good, reliable, trusted data center providers in America who've been around 20 plus years. I love those folks. They know how to, sure, have they, are they sponsoring happy hours at Neurips? No. Are they allegedly bitterless and build? No. Are they hanging out in, you know, in like, situationally aware parties? No. But there are adults. Yeah. I trust them. They can run land. They can run high on our shell. They have credit histories. We sit down. We have a conversations. Many of them live in Silicon Valley. They've, you know, they've had to deal with the boom and bus cycles of the internet. And I love those folks. You know, they are stable infrastructure partners and thinkers. And I think there's a lot of short-term thinking going on in the compute layer. And it's going to catch up to us. It's not going to be good. You talk about aligning incentives. And, you know, I would think that aligning incentives means you have the full stack in one company, which is XAI and open AI. Right? So, you as a standalone infrastructure layer, why are you somehow more aligned to your portfolio companies than people who just own the whole thing? In systems design, right? There's, there's two regimes of architecture. Right? You have integration. And then you have fooling and utilization. Right? So, the, or rather, the way to increase utilization often is you can do systems integration where you collapse a lot of process into one node. Or you can pull out a process from a node and share that amongst various, that resource amongst several different nodes. And so, we see the, the amp grid, which is, uh, what, the system we're building here, which is basically a compute grid. You know, we're doing trying to do for compute what the electric grid, yeah, what the power grid did for electricity, this is a pooling and utilization layer across clouds. And so, we're actually the opposite of a full stack integration. It's like a super hardware. It's much more horizontal. And it's, you know, it's multi cloud, it's multi silicon, um, the goal is to try to make me go, you know, flops flow like mega watts. And that is very hard to do today for many reasons, like there's stranded pools that compute all over the place and there's no fundurability. And so, right now, we do it at the level of scheduling. And we often do it at the economic layer. But as we start an ounce, what we're working on, it's extraordinary, like how many folks are coming out of woodwork and saying, hey, I'm actually working on a way to make compute fundurability at this part of the stack and that part of the say, and as a grid, we'd like all of these folks to participate on the grid. This, you know, people often ask me under your new cloud and they could know actually, new clouds are suppliers. Oh, sometimes the last car you eventually capital for my going to actually, they are, they are demand, like sort of off-takers of the grid. We see ourselves as what's called an independent system operator. So if you study the history of the electric grid, once it became legible to a lot of factories and industrial sort of participants that hey, actually turns out pooling is a good idea. We should pool our generators instead of all having a app, a generator running at half capacity in our backyard. There was a need for an independent entity who could coordinate all these parties, transmission line, you know, power generation facilities, transmission lines, factories, and that neutral coordination mechanism is very critical. In order, if you study, like the history of the grid, the most enduring ones, were those that never owned their own assets. They were ones that had often started with long-term anchors who are uncorrelated sources of demand, a steel factory, a shoe mill, or whatever, in a particular town who weren't competitive, where the steel factory wanted to spike up at night, the shoe mill wanted to spike up during the day, so then you pool and you share. So each of these guaranteed some base load, but then you kind of schedule your spikes to drive a peak utilization across the town. The gold standard sort of speak historically has been these utility companies like PGM Interconnect and the Northeast of America, where they over many, many years became this, what was called an ISO, an independent system operator of the grid. So that's how we see ourselves. Economically, that's what we are from a technical perspective. We started at the scheduling layer, because Sab and Mihai who run engineering here built that scheduling. They did that at Google. And you have infrastructures from discord as well. I don't know if discord is like the primary identity, but whatever, I'm just... No, discord was using a well-known name. Well, I was trying to develop a platform there. The internal infrastructure I was not responsible for. That was actually a guy by Naomi Marksman, who is extraordinary. And yes, discord did pool. So discord is actually a counter example. I guess I had the chance to learn a lot about fully full stack and for there, because it's the other architecture, which is um, discord built its own WebRTC voice-in video in front. So like discord did not, for communication, discord did not use third-party in front. It was all built in house. And then the way you maximize your utilization was you pool demand from the world's 200 million plus monthly active gamers. And so that's how those stacks were constructed. Again, in systems design, the two concepts that keep coming up over and over again are abstraction and composition. Bundling and unbelievable. Bundling and unbelievable. That's abstraction composition and like verticalization and and horizontalization. So in that sense, AMP is an independent system operator of the grid. We pool demand. We pooled supply from a number of partners. We trust at about 1.3 gigawatts scale over 4 years. And then we pool demand from some of the world's best, you know, research labs and so on. We're sitting at one periodic labs who need extraordinary long-term demand. And the idea is that, you know, each of them is guaranteed base load on the grid, but they can spike up and non-flexibly on for compute with much shorter timelines as needed. That was roughly the design of the program I came up with at a 16-z called oxygen. The same design of the GQM, morgue, export GQM, and plantation at Google that me and Seventh-Bilt, which is that how do you allow teams inside of Google on the internal infrastructure to be guaranteed capacity for their base workloads? But when they need to spike up on research, how could they ensure that that was sufficiently there? Of course, the big innovation that was discovered, not discovered, but kind of implemented in the space, this in first space, maybe three, four years ago at Google was the idea of interoperable demand, right? Where you just queue up a bunch of jobs and through this sort of credit system, that can be a bidding mechanism. It's a dynamic proposition, basically, and jobs can get interoperable based on somebody else who's saying, you know what, I have 10 tokens, 10 credits I want to spend on this job, and other like team lead, research leads like Genie 3 or whatever is only worth five credits, and nanobinana 2 is worth 10 credits, and so the nanobinana gel gets priority. That's a made up example. It's very real, brand marketplace was real, and we've covered this on the pod with David Luan, who is there, and the criticism is that, well, actually, sometimes you need central commands to go all in on the thing, and actually, sometimes capitalism via credits doesn't work. It's not a criticism of AMP, I'm just saying, like, this is a thing that has been tried internally within Google, and it led to Google missing GPT. Like, we structured ourselves, essentially, very similar to Google, we are structured as a wholeings company. So, you know, alphabet holdings is alphabet holdings, and then they've got these subsidiaries called Google and other bets, other bets, and so on, we've got, you know, AMP holdings, and we've got our infrastructure business, and then we've got a capital business called Foundry that incubates new front area labs and invests in them as venture capital, like periodic, you know, we put a few hundred million dollars into anthropic from our fund earlier this year. So, wherever we feel like teams are making progress, especially researchers, and so on, who've pushed the frontier inside of existing labs, like Deepvine, I find, you know, there comes a point where they feel misaligned with the dictatorship of alphabet holdings, and at that point, sometimes the dictatorship doesn't want them anymore, and they're like, thank you, you've done your job here, you've kind of helped us through the zero-to-one phase, and for whatever reason we're going to de-prioritize, you're amazing, like omnimodal or whatever it is, and instead we're going to prioritize coding, and I think that's a tragedy, but I get it, like they're, you know, Sergey and team are running their own business there, but that doesn't mean we should, the rest of us should sit around waiting for that progress to get unlocked for the rest of the world in humanity. I mean, if you think about how much extraordinary research has happened inside of Deepvine over the last 10 years, I mean, Demis and Sergey and those guys did such a great job, but at the end of the day, so much of that has never seen the light of day. Yeah, they're like papers only, but they never actually shifted to production or, I mean, what's worse is the paper is actually not even being published anymore, because there's a six month embargo inside of Deepvine, right? Like we've heard about this, where a paper comes out, and then I think there's a six month embargo window where if anybody on the business team says this could be interesting, it's embargo for life. Exactly. So, the stuff that gets published is the stuff that's not good enough. That's basically it. Yeah. At this point, it's a common complaint at New York's, by the way, that's like, well, why, why look at the papers that are the trash of GDM? Again, I think it's a tragedy. I mean, I get it, they're running their business, but the rest of this pay, I think there's negative externalities of research being hoarded, and so there's a market failure, and somebody needs to unlock that research, and we can't do it on our own. We only have one point to get what's a compute. That's nothing. That's about $40 billion of cloud spend. We're going to need another one. That's a new number I haven't come across that, they can't get what number. That's huge. Yeah, and to be clear, we haven't secured all of it. That's how much demand we have started to secure. I think publicly, we haven't actually confirmed how much we have for this year. We really want to get to. I think the steady state would be that we have a base load pool of 1.3 gigawatts at all times of base load capacity. For spike capacity, right now, my estimate is we need roughly 6 gigawatts over the next four years for all our teams to feel like they were able to keep moving the frontier, whatever they're working on, whether super-conducted discovery over here, there's a new investment we're working on right now, which is in the end-of-life prediction space in health care. It's extraordinary how much you can you can give. This was actually my graduate school work. I went to Glasgow for Bioinformatics at Stanford Med. Yes. And I know we call it MCS Bio. I was just really weird. I was never satisfied with my major options. I said 1. I was an econ major, then I was a CS major, then I was a MCS major called Mathematical Computations. They decided they were going to end that major. I took all that coursework and I applied it to grad school. My graduate degree in Bioinformatics, which was the master's program, and then I thought I was going to do a PhD. I never ended up doing a drop-down and went to work a cleaner. But I was lucky enough to apprentice with this professor at Stanford Med. His name is Nygom Shah, and he was working on end-of-life prediction. Stanford is one of the only research facilities in America that has a longitude no patient data set that's larger at scale. I think it's at least 12 million patient lives. The only larger data set is the VA, the Veterans Affairs of America. And to do research, like do any deep learning and so on on that data set, I was called the Stride data set at that time. You had to be a Stanford Med school affiliate, which is why I went and enrolled in the Bioinformatics department. And look deep learning was early. Nygom Shah had the vision to see that you could do end-of-life prediction to help value the data. In America, over 30% of all Medicare Medicaid spend, at least at that time, was spent on end-of-life care. And what's, you know, we grew up in Asia. So I want to speak for you but I have a very different relationship with death than I find folks who grew up in America do. In America, spiritually and culturally, especially in Western societies where Christianity the Christian tradition sort of frames death as this terminal point. There's often a judgment day and so on. The way we view death is with a finality. In Indian culture, in Hindu culture, you know, death is what is Buddhist as well. You're Buddhist. Yeah, so it's one step in a journey of many lives, right? And so I grew up in the city called Chennai in the South of India. And when people die, you dance on the street. You know, there's like a procession where your body is carried to be committed and your family, like, like celebrates. And there's drums and so on. It's a huge thing. And it's because the idea is that you're going to be reincarnated. You know, you've been liberated from responsibilities of this life and now you're onto your necks. It's a new it's like going off to a new college or whatever, right? And so it was so alien to me when I got here as an undergrad that the medical system works backwards from that assumption that we have to view death as this terminal thing and delay it, postpone it. It's a bad thing. And so at the time, clinical decision support in the United States was this very primitive field. Even to this day, physicians in the United States often will tell you when you have a terminal disease. This is your we've diagnosed you, which is great. Our really diagnosed here is extraordinary. You have some of it in six months to six years to live. What do you do with that information? The aerobars are so high that then you, you, you, in times of uncertainty, we default to culture. And when the culture is, let's, this is a bad thing. I've got to prolong my life. Then you start doing things like and just to sort of from a systems perspective, what's going on there is physicians often feel like they need to provide such high aerobars because there's always some uncertainty in end of life diagnosis. And if you provide the wrong diagnosis or recommendation to your patient, you can be sued for medical malpractice. And then your license can be taken away. It can be catastrophic for your career. In contrast, if, in countries where that's not the case, what you often observe is that patients, like physicians are quite prescriptive with their recommendation, they say, hey, this is your condition. The literature says that you probably have this much time on earth left. My expert opinion is that you are an outlier or whatever. And they try to be more prescriptive. And that empowers a patient. Because a patient can say, I trust my doctor. They said, on average, I have six months to live. But if I do these things, I may have a shot because of my particular predispositions or my genetic history or whatever. And that empowers you to go about your life in an actually more scientific way than leaning on religion, culture, spirituality, and so on. In contrast, here, because of that medical malpractice sort of thing, looming over your head, a physician never gives you a clear recommendation. So instead, you say, okay, Doc, well, that's dried off. And then you start a whole regime of drugs and therapies and then you often spend weeks and weeks in the hospital and that deteriorates your quality of life. And when that deteriorates your quality of life, like you instead, instead of spending your last few days doing the things you love with your family, you're spending it on a hospital bed. And that ends up being 30% of Medicare and Medicaid. So it's worse for the patients. The doctor is feel terrible. The American taxpayer is being a huge amount of money. And so this is why Niggum Shah, who was the professor, Stanford said, on, if there's, I kind of sat down with him, I was this young guy, you know, I was 21 and I was like, I want to work on a big problem and he's like, the big problem is end of life care. And so we tried to do deep learning to say, we start trying to learn deep learning on these dried patient data sets to say, could you have any eye system make a recommendation that is orders of magnitude more precise about how much time you have left once you've been diagnosed with a terminal condition, then a human. And then if we can get that precision to be high enough, then you can empower the patient. And it turns out the tech works. Once you get the data set, like RL works, honestly, even regression models work, you don't need to get that fancy. At the time, we're just trying to doing like very simple neural lens. Today, what we can do with RL is extraordinary, the problem remains, then and now is regulatory, because you actually can't shift the burden of the wrong clinical diagnoses from the physician to the AI system. And so at that time, I got quite dissolution 10 years ago, for the 12 years ago, because I felt I just didn't have the resources to influence regulation. Today, I'm very lucky. I'm in a different place. I'm a lot of older, and so I've been thinking a lot of time on my next incubation, which is, how can we unlock the patient empowerment by training AI models to end of life prediction, much with much more precision and you're still focused on this. I haven't been able to get this out of my mind a single day for the last 14 years. This is the hill I would like to die on. This too, I would say. You know what, I actually prefer not to die? Yeah, yeah, but I think two bipartisan issues, I think two issues that should be bipartisan in America are how do we empower patients to make the right clinical decisions at the end of their life such that we're reducing the tax peer burden with science. It's just good old science and AI can help here, and the second is net positive data centers, because I think that's the biggest critical bottleneck on training in good Navi-I models to help people at the end of their life. So they sort of two sides of the same scaling bottlenecker, but those two, you know, we formed an amp as a public benefit corporation, my wife and I who you've met, you know, you've met with her passion is education, her family is a long line of educators and so on, and of physicists, and so this class is my attempt to stopping the black sheep of the family and be an educator, but if I'm not educating, the thing I would be doing is working, you know, on these two problems, whether on the political spectrum or as a researcher back at, at, in some lab. And my hope is if anyone's listening to this podcast, if, you know, if they're passionate about either of those two topics, I'd love to hear from them, which we can share the contact and the show notes, but we're looking for people to join both of those missions on the political side as well as on the medical side, on the research side. You know, you said, this is a discipline that you want to form. You call it, it's called, various, very say called frontier system. It's very released, very recently called one person frontier lab. What is the ideal name or shape of this, like the, what is the mission of the class of the discipline that you're, I guess, exploring, right? Like that, the classes called for intersystems. But like, for me, maybe one phrase is like, you're just anti-waste, right, which is wasting, wasting GPUs, wasting in human and in Medicare. Is there a broader theme that maybe you can encapsulate more, succinctly? Yeah, yeah, from an engineering perspective, it's very simple. It's output maxing. You know, it's the department of output maxing. That's also what we have. Exactly. I'm a huge believer in optimal outcomes. You know, I think both in America and other countries, we are losing our appreciation for nuance. And this is the thing of, yeah, it's the same case. Oh, the bitter lesson holds, okay, fine. But that doesn't mean you're just like throw 500 GB 300, 500,000 GB 300s that you're like, you know, sub-optimal model, scaling any waste a bunch of compute. It also doesn't mean that, you know, the most optimal is have like 50 different architectures where there isn't enough standardization. Like, one of the reasons and tropic has had extraordinary sort of velocity. It's because they pick the transform architecture and said, this is simple, let's double down on it, right? And now, luckily, there's enough investment going to the space that we can afford other architectures. But at the time, investment was just too fragmented into other architectures. So that arguably unlocked scaling. So I think there's a philosophy. I think we all owe it to ourselves to do output maxing with a new capability called AI on a global level. I think if I was starting a new department at Stanford, depending on how fuzzy or technically I want to be, I'd probably call it the department of alignment. You know, like, it's a role-related term. But it is, but alignment really is a hard problem. And I think when you unlock it, full stack alignment is super hard in any organization, in any system, like in a venture capital firm. If you can have full stack alignment between your limited partners and the founders who are creating the value and ultimately the public that owns the IPO stock, that is a gift that keeps giving. And when you study the history of these systems, when they start off, they usually start out small scale where the feedback loop is actually so tight. That there's alignment. And then the more you try to scale the more division of labor happens, the more specialization happens, and each step you add abstractions. And wherever there's an API interface, there's like loss. That's communication loss. And so I think a really cool thing would be for us to figure out, is there a way for us to have our cake needed to as an engineering discipline? Is there a way to actually scale up and scale out without losing any alignment without lossy transmission? I mean, standards. So standards is one way. The other way is you just have net new capabilities. So like what we're trying to do here is discover new superconductors. A room temperature superconductor would be a lossless transmission mechanism for energy. We would have flying cars. We have, right, within a few years of having a room temperature superconductor. So I think those that you either have to standardize on protocols or API specs that allow lossless communication, or you can come with a whole new capability that unlock so much abundance. The standardization doesn't matter, because you just unlock net new capacity. So this is what I spend my days thinking about. I mean, no, I think every in-for-person at who once scale and wants to output max does eventually end up thinking about this. We don't have time to go into it, but we have done an episode with SF Compute. It is trying to standardize the future's contract for Compute. Right. I don't know how that's going by the way, but at some point this will be part of it. So I think Evan is awesome, and SF Compute is the kind of effort that I hope we can accelerate, because what often happens is these exchanges are very hard to get, it's hard to bootstrap them, right, because they often require there's many inefficiencies between parties. This trust boundary inefficiencies in infrastructure, because you don't trust one part of the stack doesn't trust. Now the power is stacked to give them visibility, the capital markets inefficiencies is operational inefficiencies, so if you can inject a single shock to the system of a ton of compute demand or supply, then you can accelerate these new flywheels, and so my hope is one day, or soon, if SF Compute needs extra capacity, it has access capacity, they just hook it up with the grid, and they get flooded with demand from us. On the other side, if they have a ton of demand, but they don't have supply, again, hook up to the grid, and it's a two-way protocol, where they can just hook up to our capacity. And I don't think we're too far from that, you know, today, our working implementation of it is mostly through a group of labs, universities, and a few sort of trusted parties who all feel like they're in alignment, to borrow an over sort of used word, but I hope is to just have it be an open protocol that anyone can hook up to. hook up for demand, or hook up for supply, primarily demand, it sounds like, like you don't want to offer demand. Both, yeah. Unfortunately, what's happened last six weeks is, you know, we thought we'd have a bunch of access capacity by the end of this year. It's all gone. It's exploding. Yeah. It's all gone. And so, my text messages are full of friends, I mean, we know many of these people, these are founders who raised billions of dollars in San Francisco going on, and he chance you have, like, 15 nodes in the next few weeks. What is the scope for non-invidea, right? You have Lisa Sue coming and Reiner Pope is one. And so, there is a lot of demand for more performance, alternative architectures and all that, at the same time, this hurts your standardization. I don't think so. So actually, Reiner's a great example. Reiner's a CEO and founder of Matt X. I actually had him buy for office hours in the class earlier today. And there was an insight he brought up that I hadn't considered before, which is, when they decided to pick the standard for their data center, they picked the NVIDIA reference architecture. So the Matt X trips just plug in to any site that has an NVIDIA bring up plant. It's just software then, it's not the hardware. Well, from an input and out, IO perspective, it's the same footprint as NVIDIA rack. And where they have done innovative a bunch from what I can tell is on systems code design, which is where a lot of the gains are to be had. And so he picked, he was like, there's just so much work to do when you're building a new chip company. You can just can't fight on every front. Because so my question to him was, well, you're working on this new chip. Their tape out is next year, you know, who are you going to partner with to host the chips? And he said, whoever will host them, that's just not my focus. And I said, but how did you, like, you decided, you know, back to our earlier systems quite design question, he decided that like, he didn't want to be a fully integrated chip provider, the bottleneck they're focused on is the logic die. And he feels they can crank out a ton of performance gains to code design there. But then that means you delegate, you know, to our question earlier, like it, he's like the data center providers a different part of the stack. And so then he's dependent on that part of the ecosystem to host his chips to get the performance gains to the customer. So now you have another abstraction and you might have lost. So I asked him, how do you prevent loss? And back to your point, he said, I just picked the NVIDIA standard. Because I didn't want to, like, I wanted a piggyback off of an existing protocol. And that, what's great about NVIDIA is that reference architecture is known, it's open, they've published it. So Jensen's actually enabled someone like Rainer to build a chip company like Maddox. And I don't see them as competitive. The compute demand is so high, like I don't, I think NVIDIA's not able to meet the demands of productions. So we just need more chips. And I think it's very smart what Maddox has done, which is, say, we're just going to, we're not going to innovate on the data center design, because actually, thank you, Jensen, you've done all the hard work, where we can innovate somewhere else. And I think that's very healthy, I think that's how we unlock new bottlenecks. And my view is these chip teams like Maddox who have arrived at the insight that code design is the way that primary bottleneck for them is trust boundary. You do code design well, you need visibility into the next model generation as soon as possible, because it takes two years to tape out. So if by the time I bring my chip to market, your model architecture is changed, I'm host. Now when he was inside Google, he was sitting next to the Gemini to use on palm or whatever. His co-founder was one of the palm guys, I think. Yes, yes, exactly. So when you're inside the trust boundary of Google, then your systems code design loop is super tight. When you leave as a founder, one of the biggest risks you take is now your outside the trust boundary. And so what I love doing is helping chip teams who can help us unlock more capacity for the independent ecosystem access to trust. Because if I've been involved with a lab from day one, and I was lucky enough to work with anthropic, and then I'm on the board of maestro, and I have black force labs get started. I think at this point I'm on six or seven different teams. Only six. I feel like my mental number was going to be 13, but yet it's. No, I go deep with one at a time. You were founding CEO of Arena? No, that was an administrator. That was an administrator. It was an administrator five month gig where we learned how stasis were graduating from their PhDs, and they didn't need a product team. So I helped recruit the head of engineering product and design. So Anastasis has always been the CEO of that company. I played a pinch hitting job. I'm an intern. I was CEO intern for five months. I interviewed him, and he's very, very well spoken, and I think he's a former debate champion, but also very quantitative in the medical which is such a unicorn. You know what's amazing about him? If you look at his output, he's an output Maxor. By the time he was graduating from a PhD, which he only had graduated last year, he had published more work with a citation count than like people twice as age. But at the same time, he'd already started a project called Alamoreena that was being used by millions of people as a side project. And time and time again, what I've realized is venture capitalists suck at seeing human beings as like dynamic agents where they want to put you in a box. They want to put you in a box. They want to put you in a box. The first time I got introduced to Anastasis, somebody had told me, oh, he's amazing, but he's a researcher. Yeah. I was like, what, what do you mean he's a researcher? That's not a CEO, not a founder. Not a CEO, exactly. I was like, are you crazy? Have you met Dario? Dario is a scientist. He's gone from zero to like what will soon be a trillion dollar company in four years. Being a CEO, nominally speaking, is not that hard. Being a good CEO is hard. Being a great CEO actually requires a level of performance that scientists who have already published the top of their field have accomplished. It is super hard to be a competitive scientist to publish in academia over the last 20 30 years to make it a top of your discipline at a place like Berkeley. You were a star athlete. Like you are an athlete on the mind. When you perform at a highest level and to get there, whether you're, you know, Anastasis or we're in at Berkeley, or you are Robin, who would like for us to create a stable diffusion, or if you're like, young at meta, who created Lama, before we serve in straw, like the amount of human leadership you have to demonstrate to get the resources, like get the trust of the organization, publish it, put it up. I mean, I would just fund researchers all day, right? Who have contributed already to the field? If they put soda out there, their star athletes already. If they haven't done soda, look, they can still be good CEOs. But then I find the failure mode is that they just don't want to be CEOs. They probably want to publish. And that's okay too. You know, one of the things we do with the Amp Grid is we donate excess compute. We have two nonprofits like university labs. We carved out like a couple thousand H-100s. But I do think there's extraordinary research being done on university campuses. You know, my father-in-law is a physicist, he's a professor. Extraordinary work in physics. And we need that. But if you want to be a CEO, what you need to be willing to do is be super confrontational, you know, outside of science. Like within the scientific community, some of the best researchers are very confrontational about their convictions, right? This architecture is right. To be a great CEO, you basically have to be willing to be confrontational up and down the stack. If you're on team, if you're on team, higher and higher customers. Well, I would say, yeah, pretty much to have. Yeah, right. I feel a little bit of that in my own work, but like, yeah, I can't imagine the stakes that Darryu has to go through. It's like, I don't think the stakes are that different from how you're feeling it, right? Stakes are personal scaling vectors, right? The stakes that seem so low to you, like having this podcast, where you can talk to somebody and just have a, I mean, you're an extraordinary communicator, right? Like, already in this conversation, you've pulled more out of me than most people, you know, and I've been on 12th podcast in the last two years. I think we've just seen each other enough that there's some base trust in this. And I know that you know that I've done my homework and like, I know that trust is a big deal for you. So, right. Yes. I think trust is about consistency and you and I've seen each other in the community for years, right? Like, I remember the first time we met was at New York's and New Orleans, I don't know if you remember that. Oh my god. Rayco had, yeah, Rayco's amazing and he's set up this luncheon. I was like, who's this discord guy, I'm like, okay? No, you were like, no, you made some investment. You were much less polite. You were like, who's this VC? You're like, no, I was like, oh my god, I was so sorry. It was visible on your face. I'm so sorry. But like, you weren't, the introduction was bad. I was like, I don't know who you were. See, this is the thing with context, right? But then I think I heard your accent. Yeah. And I was like, are you? Are you Singaporean? And you're like, yeah, and I said, I went to J.C. in Singapore. And then the ice broke, right? But this is the, you know, there are in the scientific community. Sometimes the stakes are very high for people who haven't had the emotional, we know what it's called EQ, coaching and mentorship, right? Which is like, to have scientific impact. You often need to be an extraordinary emotional, like emotionally intune person with the folks you're trying to influence. And so what comes so naturally to you is actually super high stakes thing to other people. And so I wouldn't assume that Dario's more stressed out than you, you know, these things are, like you'll be surprised how similar and small sometimes the problems are to you that some of the world's biggest leaders are facing. And that's what I've learned from this class, you know, the guest speakers are Sam, Sacha, Jens, Aykuchella. Aykuchella. Yeah, they say Aykuchella, right? So we got to get all the headliners and I'm very lucky that some of these people have either mentored me over the years or I've done business with them. And when you take the performative stuff out and any assumptions you may have about these people that you read in the press or on Twitter, we're all just humans, we're not trying to get along. And what's so special about this moment is AI's forcing, like scaling, the bitter lesson is forcing a lot of people to revise their assumptions for how the world works and go back to first principles or go and educate themselves. So the people, you know, I was, I won't name who this person is, but I was at an event last week in Texas and ran to somebody who said, you know, I came across the class. What do you think about real-time action prediction models? And I was, you know, I don't know how happy it made me feel when they asked me that question, I know they've done the work. If John says, I'm, you know, they didn't ask me, what do you think of world models? They said, what do you think of real-time action prediction models? World models don't get me wrong or cool in everything, but you and I both know that that is a layer of abstraction that is sometimes not used fully precise enough. Yeah, right. It's like four different kinds of world models exactly. We've done the part with general intuition, by the way, which is very focused on. Oh, cool. Yes. I love them. Great. And this is what I love about people who've done that level of work. They realize they're not in competition with people who arrest the world things they're in competition with. Yeah. Right. They're not in the category, they're in the specific thing they're trying to do. They're focused on their mission and they have a systems understanding of the bottle, like they're trying to, like, you know, solve. And when somebody else says, I'm working on real-time action prediction models too, Tim goes, oh, I love that person. I want, I can learn from them. But the minute they're like, oh, that person, the world model person is like, ah, like, which type of world model person, mostly they just try to figure out if they're wasted that time because we don't have enough time. So, you know, Tim, for example, is super loves this other company, I work with, we've talked about, called Black Forest Labs, you know, and he's mentioned multiple times that he's, so he thinks what Flux is doing is really cool, you know, Andy Blattman came by and spoke in the class and what I find over and over again is for people who do the work, who can be usefully precise enough about, like, what is actually going on in the world of frontier research. The sense of camaraderie is still well in a life, but it gets lost sometimes when you have to, like, abstract the technical complexities in, like, business terms and then the VCs are, like, how are you different from that world model come, is it? Yeah. Where do I even start to explain this stuff and then the misaligned. Yeah. I think, like, people listening get a sense of, like, what is, like, to operate at a real level, like yourself, rather than, like, at the journalist level where you have to sort of put everyone in, like, a rough category and create an narrative of competition, and who's winning today, who's behind it. Yeah. Yeah. This idea of winning is so weird to me. You do want to win, you want to competitiveness. No, I think you want to lead. No, I think you want to lead. Yes. So, you want to push the frontier, you want to push the state of the art, you want to do something that has been done before you want to capture value, but you don't want to capture so much value that, like, people think you're on a line with your mission or trying to do what's best for the world, you want to capture enough value that you can keep innovating, right? And I think that people want to lead. They don't really, this idea of winning and losing, like, you know, again, I love Jensen. He's a, he's a leader. The mindset that he talked about on Darkish's podcast, right? He's like, I didn't wake up with a loser mindset. I think that was awesome. Right? Because he's, he's an engineer, Darkish has done the work, so there's at least, even though to me, it was very obvious they talked about the same thing in this past each other. Like this had to, like, you know, basically, Jensen has this, like, five layer kick abstraction of how the industry works. And Darkish had, I think from that podcast, had more of, like, a pre-training, matrining post-training systems loop concept. It's just a factor of who he talks to, right, again, it's just a system, it's the abstraction, the mental models, the, it's, the whole, do so much of the problem in the world is reasoning by analogy. And then the assumptions that are held invisibly. Yeah, I've, I've said, like, this is actually the best time in human history for Frisbee's world thinkers. Yes. Because everything you think will happen is actually now coming true, correct. And the venture capital community is, like, notorious for this, where people look, in terms of uncertainty, they like, cling to axioms that end up being true from the previous era. And they, they kind of, like, proclaim them with confidence as if they're truths, but they're not. And it's very important to see the distinction between a heuristic and an axiom. Yeah. And axiom can be proven, like, from an internal consistency, a heuristic is a, where you kind of a shortcut. And my god, the number of people I have had to put up with over the last few years, who proclaim, like, use heuristics as axioms to judge people, to judge which companies are going to succeed. I mean, the number of people were like, oh, yeah, yeah, yeah, anthropic, they're just training models right now, but this one continue to be, yeah, which, which over the fullness of time, if you squint at it, maybe, but the way you arrive, they're so important that you can, you just, you can dismiss people. Here's what happened, right? What happened is anthropic basically achieved takeoff in October of last year. That training run. Whatever. Three, seven. I forget the numbers now. But whatever, that checkpoint was, we saw the commission. Yeah, right? You probably, to those of us in the community, especially once post training was done, and it was released in December. Yeah. Yes. But I don't know if you have a perspective, maybe you don't, I just, the number one question is, how did anthropic crack coding, right? Yeah. Because Claude 1 Claude 2, okay, like it was part of it, but it wasn't a big deal. And leaving hypothesis, it's a lucky dice roll that was then compounded, right? Like it was like, mildly better, but then they saw it, and they were like, okay, let's really invest. I had this very annoying teacher. Yeah. I went to this boarding school called Rishivali in India, which is like, this bird preserve, like 358 acres of bird reserve in rural India, and there was no technology for seven years. There was this teacher. I won't name them, but they would have this, I hated it every time he said this to me. He was like luck favors the prepared mind, which is like a common thing, but the way he delivered it, like always great, because he's always, like, I was always one of those kids who got, like, a good grade without trying very hard, because like high school in middle schools, not that hard, if you're generally like paying attention so on. And there was this one time where, but then I would get an 80% grade, and he would keep pushing me to say, like, the reason you didn't get the 95 plus percent is because you're not that lucky, and I would say, what do you mean? Because I would think that I deserve that grade, and I would sometimes argue with him, and he'd say, you didn't have a prepared mind. If you want to get lucky again, there was basically one time where I got like 95 or 96 on this subject, and now that I felt entitled, I was like, okay, I'm going to keep doing this, and I didn't, and then he was like, luck favors the prepared mind. You got lucky last time, but you got to stay prepared, and I didn't understand what he meant. Now as I'm old, I'm like, okay, these are adults who actually knew a thing or two. Andthropic has been the most prepared company for four years. And so then when the right, like, context data comes in, the right developers start sending in, you know, the right context diffs, sure you could say you got lucky, but, yes, me, they're pretty, they're prepared with paranoia for like four years, and you have to remember, it was so hard for them to get going early on that they had to do so much more with so much less, that you just have to be prepared to be so efficient. Yes. The numbers on their burn compared to an orphan AI, I've written about it, but they are so much more efficient. It's not even close, but it's so clear, right? Like, how do output max for the world? They have been prepared, and you could call that luck, but luck favors the prepared mind. This one of the things that I was going over some of your old lectures, and you were like, you know, data people think it's a mode, and like, actually it's culture, and actually it's team. Yeah. There's different levels of modes, and this is the ultimate one that determines everything else, which you can think about. You're saying culture is the ultimate one, yes. Yeah. But the thing about culture is it's very fragile. So modes, I don't think there's very few modes I found that I actually modes. It's a nice concept, but in reality, you have to replenish your culture. Ben Horowitz was the speaker in CS 153 on Tuesday, and I asked him this question about the culture bottleneck in teams, because you know, there's several of the items in his book, like hard thinking about how to do it. Hard thinking about hard things, but more concretely, there's so many AI labs today that have all the cash they need. They have all the compute they need, and they're still not able to ship anything soda. And then you start seeing people leave and so on. And my diagnosis is it's the culture. And so I asked him then, you know, there was, he's been one of the most aggressive investors in AI labs. He goes back to this thing, which resonates in my mind a lot. When I used to work at A16Z, I would book a conference room, and right outside the conference room, which is close to the toilet, because it was the fastest way for me to go use the bathroom between Zoom meetings. Oh my god, I'll put an X-ing my toilet, optimization. Okay. It was on the outside, but maybe this is DMI, but anyway, outside that conference on the wall was this quote that was printed that said, culture is not a set of beliefs. It's a set of actions. And it's by Bouchito, Japanese philosopher. And if you stop taking the actions that demonstrate the mission alignment to what you've said to your team and to your world matters to you, then your culture starts to fray. So it's not actually a mode. So it's a very, very brittle, fragile thing that requires daily tending to like a garden. But if you figure out the system to keep that garden tended, which I think ultimately comes down to knowing yourself, because you're most naturally, if you're authentic and so on, you will naturally make trade-offs that seem effortless to you, but that reinforce your culture. And then that becomes this very hard thing for other people to catch up to. When an anthropic from day one, you know, there was this mission, like missionary like zeal and belief that hey, these capabilities will scale. These systems are stochastic, not deterministic. There will be error bars and until we crack interpretability, there's risk. And at some point, people will stop using cloud just for coding. They'll use it in some mission critical context where there's, it'll throw off a bug. And then people are going to come blame them. And they want to be on the right side of history, where they said, yes, this is a powerful technology. We think it's going to change the world. And we want to be very measured and scientific about the fact that, hey, guys, these are statistical models, that's how statistics works. Like ultimately when you're training neural nets, it is just a statistical system. And I think that that belief that safety is important and that it might seem toy-like. And the early days and sometimes you could say, oh, they totally over exaggerated the risk, you know, like two years ago, when they said, let's not launch cloud one or whatever. Well, okay, maybe in hindsight, but hindsight is 2020. And at the time, they didn't know how that model would be used. And to them, it felt existential if somebody came and said, you weren't responsible. This wrote a bug. The liability associated with that is massive. So how do you prevent against that? Well, day in day out, you say, safety, safety, safety safety safety and when you start deviating from that, you have the team holds you accountable, you have the world holds you accountable. And I think that becomes a mode over time. At some point, that mode will get challenged and so on. And then it will come fragile. I hope it indoors because that's the beauty of having founders run the show because they can make really hard trade-offs to do mission alignment. The hardest part is in the earliest days when you don't have a group of people who are going to difficulty, stress, crisis together, then your culture doesn't get defined sharply enough. And that's what I'm worried about right now is there's so much money going to these labs. There's no hardship. There's no 21 news. There's no 21 news. And that, in hindsight, was a feature, not a bug, went floppy. The number of people who said, no, the number of people who said, sorry, we're all doing investors and opening eye. One is competitive with difference. It forces you to really understand what is the hell you want to die on at the expense of everything else. What's the P0? And there, P0 from day one was coding. The reason mechanism system there was if we crack coding, then we will crack HGI. In our mission is HGI, we want to get there safely if we focus on coding. It's such a generally powerful capability that it can accelerate all kinds of work on a computer. We can get to HGI. You know, as a result, they have had to say no to so much other stuff. Here, superconductivity is the mission. Coding is not the mission. So we use cloud. We don't care about that. The mission defines everything. And I think teams who can raise too much money, too fast, too early, who don't have to define what the P0 is. Because that's the only thing when he has scarce resources. You've got to invest in those cultures and the most fragile in Britain. They almost don't want to make it to take off. So let's apply this to periodic sense for here. What is the constraint or the hardship that they were forcing themselves to go through? Do it here. Here, are you crazy? No. Well, yeah, okay. So on a technical level, it's physics. It's the reality. I mean, by the way, is there another one that's like the complete building? Yeah. Yeah. When Liam was a co-creator of JATGPT. And Doge was skip level from Dennis a deep mind, had created genomes. So one of the most important tools to come out of deep mind. At that time, I was a visiting scientist at the Stanford Physics Department. And we had started benchmarking onto your models on physics and science capabilities. They were not very good. They were good at doing things like summarization of papers. But if you said, yeah, could you analyze the scientific data coming out of a condensed matter physics lab? That was in the condensed matter physics group at Stanford. There's terrible. So it was not popular 12 months ago. Periodically, and I wouldn't go into details, but there were people who said, as recently as a few months ago, who said they wanted to join the company. And they, for whatever reason, took a job elsewhere, they kind of reneged on their commitments. They took a job elsewhere that offered more money. And then we had a technical breakthrough, created a soda system, and like, okay, I was excited. We'll be doing a separate part on periodic. And then they wanted to come back and I said, no, no way. You come here. You had your shot. You had your shot. Because it's actually about culture. Of course. The first principles. You know, and look, I believe in second chances and so on, but time will need to heal. Some of those wounds were, they will leave deep, deep, for them will leave deep scars. But because I started my company at 2425, I went to the whole cycle of betrayal and drama. So you realize, you know, Silicon Valley is both a very missionary place. It's also a very mercenary place. Sometimes people lose their minds, when big money gets involved, which is, and the grand scheme of things quite small money, like, you know, I guess you're taking a kind of changing to me, maybe less to you, but you know, a lot of people have not been talking about it. And yeah, we didn't come up from like that privilege of a background. I was a street doctor. Yeah, I was yeah. I look, I grew up in Rishivale, we didn't have, like, this was enforced brutalism. Juducperson when you start the school was like, you will sleep on a hard, slab of stone. My mattress was this thin, you know, and you go up in Singapore when I got to Singapore, I used to sleep. I was part of the scholarship program, but which was amazing. I'm very grateful to Singapore and government. But I was at St. Andrews, J.C. And our dorm, which was by Boon King, you know, MRT, was not a prestigious neighborhood. It was a transition dorm, because you're building this beautiful, like residential campus on-site at S.H.J.C. in Portong, Passeur. But we were the last, I think the second last batch to be in the transition site, which was some old, like, I think it was like an immigrant laborer. Yes. I keep the people who work on the factories and stuff, right? So I lived in my 11th and 12th grade. I slept in a bedroom, the size of this, like, literally from there to here, right? They were like bunk beds. And so one bunk bed here, one bunk bed there, one on top, one on top, one more here. And then here was where we kept our toiletries and clothes and stuff. And when one guy would climb onto his bed there, this one would shake. Oh, my God. And one of my roommates, who was from, and it was amazing. I loved every minute of it. You know, my roommates were a guy who was a top ranked, Dota player from BRC, from China, didn't speak a language, English, loved him, amazing guy. I mean, all the same for scholars are fantastic. And honestly, we should treat you guys better, because of what you go on to do. It's good to know. Yeah. What I'm saying is I don't need much to be happy in life. You know, when you've lived through that, money is a way that I think sometimes we measure ourselves, but you know, when it stops becoming, you know, at war, good-harts law, when it stops becoming, just a by-product and more of a measure, it stops having meaning. You use it to do more meaningful things, resources to pursue your mission. I've kept you longer than I suppose to, but we should continue this. And it's hard to, you know, I really enjoyed this. Yeah, I mean, you're so inspirational. And yeah, there's more I want to dig into about how you've set everything up. Every single one of your investments, how app is going, but we're running out of time for that. But thank you so much for joining us. It's great to see you, man. Let's get chicken rice sometimes. Yes. I'm actually, tomorrow. I'll send you a, I'll send you details. Okay. I'm posting a birthday party. And I don't get any advice. It has to be a single for your birthday party, yes. The only thing in my right now. Okay. All right, thank you. All right. Thanks, man.