DeepMind's Gemini, On-Policy RL, IMO Gold & AI Research in Singapore with Yi Tay

Source material

Transcript

Did I think that I find the most useful about like these models in general is like when I have these big spec sheets of a lot of results and I just need to be a plot survey I didn't models can quite go to the screenshot and then go plot this I hate making this method like stuff about it so annoying There were so many moments this year where AI is a sunny cross that like that imagine Like the AI code is one of them that we just discussed I think like none of them I also got to the point where I usually like to make this images That's like it's really for fun.

You just throw your friend or something like that.

No, no, man I actually really got so good Welcome back.

Yeah, are you?

Yeah, I'm good.

I'm good.

Great to be back.

It's really one one.

I have years Yeah, it feels like a long time.

So last time we talked you were at rica.

Yeah, and then you joined GDRM again working for a clock again.

Yeah, and more recently you've started GDRM Yeah, is it GDRM so about gemizing up all good?

I don't know if you've named the team I I think we have a Gem and I didn't say no, Paul.

Yeah, I think it's called reasoning and age I yeah Reason is AI is it important to have EGI in the name?

I was like a white sting that we flew agi in yeah, I think that like One the reason why we work on this model is that we want to get to agi and I just was a wife thing They would edit the agi to the job posting here But it's not like for one name of the dinion unit.

Yeah, but it's basically the Gem and I didn't single pop here.

I mean, I think people are like Trying to triangulate Amazon as an EGAT and you guys have a EGAT and then Let's see Meta and I as a super intelligence team what are people signaling when they choose these names for their teams?

Do you have all we have a plan?

Or is it just vibes?

You try to fish on hot sticks on the?

No, you have officially EGI in your job title.

No, it's not a din name Is this is not that dinion?

Yeah, is this you know, we just want to signal the north style We'll be doing these models.

Good to each guy.

Yeah, no, and was it really fishing hot sticks?

Okay, so you rejoined GDM yeah, and I think last time when you talked about I listen met a little whole thing It was an amazing episode last time you were talking about how it's like externally war in brain and came out and now you're back in GDM yeah Or no one's just general reflections just plugging back into the Google infrastructure oh, yeah, so I guess coming back It's very interesting because it felt and we turned to Google like everything including out there We all use the name it's all the same it's like play Pokemon you leave it aside and I go back and you continue to keep yeah, you say game and continue game It's like that obviously the last one point five years while it's a way many things have changed Brain is now part of GDM and stuff so I think that obviously a lot of things have changed But I think overall the coming back has been pretty simplest of us obviously I love Google infrastructure and I think the views are great and stuff That yeah, and I'm very glad to be back to yeah to go in front.

Yeah, and was the intention always that you were going to work on D think Not really I think I miss research a lot like doing like research Not like super fundamental research, but like close to model research, right?

But I really miss being at a frontier and trying to go beyond that, right?

So I really missed that a lot and I think when I came back a big thing wasn't thing and I don't think there was any plans Actually it was just like I'm just going to work on research and see what happens.

Yeah, but I'm sure I guess there was some inclination The reasoning is the thanks frontier and that's like obviously the Most rewarding research path especially this year Yeah, the reasoning this day's reasoning and RL is like boy, be quite It's um RL reasoning comes I spend a lot of my Past life I call it the past art working on like architecture and positioning But I think now I more I have a condition more into RL I'm not like old school RL but the games are around the old school RL and and and to be honest I had almost no RL background coming back, but I think like Like RL is the main means of modeling this days and So I think it was pretty easy to jump back in and I think a lot of fundamental skills in research is for general purpose and universal and and it's it's quite easy to innovate even in a tool set.

They are not super used to And yeah, so I think RL is basically the main modeling tool set that we play around with.

Yeah, this days.

Super officially I see some you know in your URL too and at T5 work so some overlap of like you know the focus on objectives and the focus on the stuff that you're trying to incentivize so I would have maybe guessed there was more overlap than you are saying right now It's just interesting But I understand it's really awesome with the shiva is objective and there has some like overlap right Yeah, I think it's just merely like the on policy and of policy list of designing these things that change home Like also the learning algorithm itself right let's just introduce this kind of terminology to people If they're not that familiar with these sort of RL policy I do think that a A lot of people are like trying to Understand what is working about this generation of RL research Anyway, so Jason had this interesting Post which I think you were co-signing which is basically you always want to be on policy instead of mimicking other People's success with trajectories to your elections and nothing to reward given by the environment At least he correct your own path instead of trying to imitate other people's path.

Yeah, yeah, and First of all, he writes really well and I wish that more people will relate him But I don't know whether it's a reflection on that or your addition on top of that Yeah, so I think like the biggest energy of the on policy and all policy is basically that all policy is basically like when you SFT something is all policy Basically you take some but other model larger model stuff and then it's basically like this off put somebody else's Generator outputs treasure theories and whatever I mean all policy is mainly like the core idea of like modern LMRL where you Like generate and then you grew up in the model based on its own generations and then the model trains on its own generations Yeah, so it's more it's a bit like Self this decision to start it's then you the model generates on output and then you reward it and then trains on it on output So I think on positive knows it's basically this idea of like Modern training on this one outputs and letting the model Like generates on trajectories and then let letting somebody work very fight and then the model changes on outputs I think this is more generalizable in general.

I think there's still a lot of like size out Still to be done about the gap between SFT and and how itself but I think basically on policy And I think bring this analogy back to real like life I mean this on policy knows it's more like Like humans we are more on policy because we go around the world we make mistakes and then we are okay.

This is but Like imitation learning is supposed to be somebody else well first principle Yeah, it helps not work to do a day.

That's copy So I think yeah, this philosophy bringing back to first philosophy to life is quite like powerful Like when I like now I have a kid and everything like one my kid to try stuff and then you tell them like okay This is like where this went wrong where this went right and stuff rather than okay.

You just copy everything somebody else does Yeah, there's a Montessori schooling is mostly that right like very unstructured learning like you discover your own path and we just give you a safe environment to do it Yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, what is the Point in which you should transition from imitation to on policy?

I do bounce back and forth Humans right not models right I will say models it seems like the mostly has been a very concrete like first you imitate And that's betraying and then you are already that the SFT is doing meditation, but I think for humans That's a little bit of this right because if you basically like like spots, right This spots you start off by imitating like hardcore imitating But you cannot imitate forever because you need to like imitation I don't know where this is good energy, but watching a lot of tutorials and stuff is more like imitating We learn try to learn some of the movements and stuff like that, but then like on policy unless it's like going to the game itself and trying to get away What's enough from that right, but so I think that humans do need some form of imitation learning, but like I think everybody starts out by imitating But then again, the human and model kind of is not it's just fun to have analogies, but we shouldn't like take things like super literally and stuff Actually, I am quite a serious Take care of machine learning insights into human learning That's what we learn from models now.

Yeah, because I think like machine learning is the most scientific way we have ever studied learning Just in general That's where we have to invent curriculum from like scratch that that's really and things like learning rate If you learn in race too high, but learning race too low about like where do humans even have a learning rate?

So I do tell people to to keep an idea of their own learning rate and to be wary of it being too low So for example, if you've been wrong once you should ask where else have I been wrong and typically usually let's say Okay, you know what I mean people usually update slower than they should when they have been wrong When is it stubbornness?

stubbornness?

I don't know is that is that the right word for it it could be like they're they're too Beijing when actually It's like they're prior assumptions are wrong and they need to completely throughout their previous assumptions because one Counter example invalidates all prior experience you're an entire world model is wrong So Beijing actually wrong let's say you live for 10 years under some assumptions and you have one example that breaks your narrative Okay, you shouldn't be like okay now I have 2% update no actually you should be like oh like something's really freaking change Everything I've seen for the last 10 years is probably wrong what else am I wrong in and update 20% of the 50% not 2% You know, I mean that's your that's a learning rate thing for me So my my direct example is the whole getting into AI stuff.

I was watching games for 10 years Is it been 10 years uh 2012 15 times lies yeah I was watching games and I was like okay, this is cool.

It's getting more detail Not than impressive.

They're all of a sudden stable diffusion came up and you can run it on your laptop And that was my learning rate okay like fuck like my mental model Genitive image should images did not include this and so I was like okay Like I am very wrong and I need to pivot everything and that's how I started later space So it means that like your your learning rate is high yes, I will not get up.

I schedule my learning phase Because a warm model has been validated Okay, I think the good is a good strategy.

I think also this brings a little bit to like When new paradigms happen like how fast people are to adopt it or like to invalidate their understanding of things I think as scientists we definitely have a lot of times we do have to keep as the few progress We do have to keep like either getting our own warm model.

It could be like a certain ways The way to do like something all along and suddenly something comes along and Yeah, you can be very proud of your priorities until it's like becomes your prison Yeah, I know There's actually very dangerous Yeah, okay, there was a bit of a tangent.

I don't know how we go there You did highlight Denny's LLM reasoning lecture is where he got chased the intellectual history of reasoning in LLMs Yeah, I've changed a thought to to know our LFT and then one part there was going to prompt you a little bit It was also self-assessency, right?

Yeah, I think people Roughly know and I think it's more cruelly implemented with open AI then with you guys where It is straight up they have Inferences and they judge or whatever But I do think that also is relevant to unpolicy Distillation where it's like literally you have a different path and they're all from the same model It's like checking my intuition there.

Basically the stuff that you're saying about White-on-policy is important and Using let's say an external verifier to to improve your reasoning You can also do that with parallel reasoning Oh, yeah, yeah, I mean like when we train our models to just assemble multiple times, so yeah, to somebody stand this some form of fear Self-conductions is that directly.

Let's talk about this area.

Yeah, yeah Self-conductions is a little bit more gene and more nonsense version of if we talk to Denny with that He's not majority voted for sure, but it's more a mega agree.

Yeah, it's more nuanced version of that biting parallel thinking Definitely is really good to self-consistency.

Yeah, yeah, I think for those people How many I was actually put out some interesting papers on majority voting versus other forms of like Multiple output consensus then basically like you that the highest level is actually add actual LM judge that Decides like this is actually a worthwhile trajectory that is more valid based on something internal Consistency or just like inspecting the chain of thought, which is kind of real cool that we can change models through that Yeah, yeah, yeah, yeah, I still want to see it's a big like a big fundamental idea in I mentioned of thought itself Also so a big idea and then self-authenticity was was like a big fundamental idea in in in in More than like power.

I am a literature.

Yeah, I'm using okay, so let's bring it to I guess One of the headlines of this podcast is going to be about diving into the I am over.

Yeah, so this was around about May March March and July.

You guys announced oh, this very nice photo here.

This is the photo I was looking at this is in London and the leaf where you had the shoe Yeah, they shoot me.

Oh, you got to be able to like you got to be at a photo taking to get a credit This bullshit right?

No, no, no, no, no, no, no, no, no, no, no, no, no, like to get the credit for being the I'm more If I see this but we hit it in the water.

So is that the joke?

Is that it?

Yeah, but anyway, okay, could you tell the story of Studying this I'm all thing apparently was done in one week.

So let me like be a bit more tri-file a lot of things right?

So the I'm what I felt has been like very long standing so tongue and basically and quote has been was he Working on this even last year right last year because like I was not back over at the time Like you had the alpha drum tree stuff and then they were like alpha proof and stuff so it's it's a very long Extending effort but I think this year was the we wanted to try to use actually use Jeremiah and do and model basically No, no second system no, no, no effects in text out.

Yeah, I'm model and even that was Not into a thing I covered the silver results from last year and I was like, okay, it's pretty close like as one point off from this It's just try to be how that you get go That decision to abandon it.

Well, I think it was pretty bold.

I don't know I personally was belief always belief in if you are not like in virtual space easy to say this, but it's a bit like If the model can get to IMO go then can we get to a G i basically so it's basically at some point We have to use these models to try this on the technical and to competitions and I think that one of the goals this year was like Okay, we're going to do it and to and like in that some model That's where like my involvement came in so basically I was not like involving the IMO effort Only until the model training part.

So I have to say that the tongue did most of the IMO thing I just trained the model a week for a bunch of other cool old orders that work involves What are something so so basically we we just prepared the model 10 points for the actual IMO itself right so that that's also something There's easy overlook about the IMO thing was that many times you you want to chase benchmarks or stuff that is always Like thing that you can kind of keep running and running a hue climbing and do you get there and then you But like the IMO was a life completion like some members of the team were in Australia for the thing and it was like This happened anything was happening life was unfolding life.

It was a very awful goal You receive the thing you like punch it into your system and then you like yeah So like some of the professors some tongue steam were like and when they went to the IMO itself and stuff like that the conference I don't even know where the IMO is the conference, but it feels like they'll people there Like in Australia And then so it was a life thing and they were people who actually the job was to run inference on on on on this IMO P1 the P6 that came out and it also came out on Like different days was like different sets like one day one day do something like that So the fun part is that I knew nothing about IMO like at all I'm not like a kid that took by an IMO there.

I asked two down for that You're kind of clear and yeah, I would be able to be able but what I only knew was that okay We delivered the checkpoint and that checkpoint was used to do the IMO goal But then like there was somehow a week in London where everybody gathered there So everybody was flying into London and then this photo was taken there And then you get to see how all the different parts like come together and Like also being in the other rooms in the room with the other like co captains and then it felt a little bit like a hike or something So yeah, I think this was like the training process of this IMO model itself was like Maybe a OB also Not the extra like the whole like basically yeah, yeah, I think the question is I'm still not over the decision to throw away alpha proof.

Okay.

Yeah, basically I think it's very major and I understand that you have this goal of AGI obviously like at some point One model should do it to do all of it, right?

But I think if you point it gun at me and sit in 234 What do you need is to do IMO and IOI and on CPC not the other stuff that you guys did was you need the LLM An existing that knows how to operate a computer and knows how to write lean and run a lean verifier and all these But basically you RLFT the lean verifier into the chair of that That Yeah, like it's so basically like it's not obvious that you can do that Okay at all because I think it's not okay.

So that's what you mean is that like some in some weird system One essentially is encoded and you need the others of them all the somehow.

Yes Hmm Yeah, I mean, it's just whether at any other day you just believe in like this like Connection is one one model lots of programming.

So I mean, there's also two use, right?

There's also two use which you know And stuff like that, but I think do some I stand the model with Like I think we should be able to get to a point where Like in the past when the LLM first started the model could not even be a calculator now you can somewhat be a calculator So technically like a tool like a calculator is somewhere encoded in the parameters of the model.

So I think we we eventually get a point where whether there's things that cannot be Express in the parameters of the model is like an open question.

Let me we don't know where it's that limit, but I think we will keep pushing and pushing this limit.

So whether like a Something like a lean system or like some other things to solve other like metaphysics engine or something Where you could it's still we still continue to push that that boundary.

Yeah, but I actually don't know like whether there were a lot of the bits about Symbolic system versus and do we have that's the what I was turning I actually don't really know whether there was like to me I was just like all that's training the model and then then some someone told me to train the model and then I train the model Basically there was like overacting like I people like the I'm more I thought that that decided this and I also think that because the Eskidis Specialized systems are very like one off Systems that are like we could create like chemistry and genome creative math and genome creative the thing right, but And they you want one model for everything so yeah, so I think this kind of fits that direction a little bit more way Have one model for and then this model was also like launched as Gemlike deep thing as a general population Gemlike deep thing so it's basically unchanged, but with maybe some config tone down a bit.

Yeah, so the the Infantime config was like the one stuff to most people is different But and though full I'm more like Infant's config was with like ship to some and the better mediation is that's because of the Infant's cost right, but that was good enough to be a general purpose models.

I think my take is that this intuition was what let to the Trying to go to a one model instead of course this Specialized systems.

There's no end, right could create many specialized systems.

Yes, that the most I can see in the future is there'll be a model then That there's something like really cannot be subsumed by a model You just use a tool or something.

Yeah, right, but my prediction is that like the most things can be Subtune by the model.

I think yeah, let me be fair.

Yeah, which is just like quite good here climbing Wait, he's sure you would say that you have a lot of evidence backing you up.

Is this the model?

I'll put that this is it, right?

This is what yeah, I think this is a model.

Yeah What do you see when you look at this?

You just see obviously it looks like a well written problem It looks like something a real human methodization would do I people did compare yours versus the opening I one Where open eyes a lot more raw or have to clean up the abrogents We don't have to talk about open eye, but I just I think what is interesting to you when you saw this kind of output I want to give a little bit like a special disclaimer is that I know nothing up Right, so I think the wonderful thing about this era of lm is that like you can be like Yeah, I'm sure and you know and you don't have any domain knowledge and you can still yeah Software get a gold medal is a unit also so that you don't know anything about it.

I can't pass this at all I guess it's foreign to me, but maybe a proof is a particular kind of chain of thought Well, I would say that the other interesting thing that some of the some of your collaborators were talking about You just use like oh, this is the first example of reasoning in a non-verifiable domain Which to me isn't proof by definition very viable?

I just want to give you Things to reform or debates that are like the worth digging into So I think that's good.

There's a lot of stuff on a process a lot of domains that are like non-verifiable and I think Not easy to verify so it's like when people mean non-verifiable with like non-trival to verify or like Just not as easy as like the solution of like a math problem because we post a long form and it's also That's why it's non-trival to do very unless you convert it to lean and then you do all the kinds of things right?

So I think there's a lot of work to be done in like they start non-verifiable Doments here.

I barely do this territory.

We are not sure whether what I can say what can I say?

Okay, so yeah sure that's it.

I think another thing that is An open topic of debate was how much domain specific work or post training was done because if you then went on to do the iOi and CPC stuff as well, right?

And the same model.

I was not directly involved in the iCPC, but I was Related to size then that's all I can say yeah.

Yeah, any other interesting Call out maybe I want just on the team you call that Jonathan as someone who's co-capted on on this effort and Yeah, basically how does the effort like that?

So I think they were four Captains for the iM2 from London.

Jordan was from modern view.

I was from Singapore.

So I think For us basically trained this model together and I think one also trying to see what Tang was saying But I think one one interesting thing was that like we all in different time zones and we off And there's something also very interesting about like passing on the job.

There's no like really like fixed work So how to work together between captain so it's more like oh, I'm going to bought the plane now Be like FK for child hours.

So it's how many decades.

We just really be sitting around sometimes they're Bucks and start the job comes down sometimes.

So basically it's very I hope and it's very very really between the captains how we decide to like work together And yeah, but I think it was a kind of interesting Kind of also because we were all flying like I think the London folks would not have in the fly But I had to fly and Jonathan had to fly and then like when you visit another country You have another like if you visit an office you have many meetings so it was in our meetings and both are pretty Like interesting and also thing that we nobody really knew whether we would get go at a time because I'm more actually Has hasn't happened.

Yeah, it was interesting.

I think and then I think like the whole Process of this getting very fight by the I'm all Committee and you know like you know that was like okay, but we're not going there But I have to learn a lot about how the I'm work for I apparently the ghost call is not even it's not your fixed number It's like a belt.

Yeah, I'll cover it on the right so it was like the time where you just look at a score You're like like I was even like looking at the watching the human participants and then seeing like what does cause World because whether Gemini would get go depends on like how the humans do so you're like looking at all If I said the percentage you were like what's the story?

Yeah Yeah, so some extent you can go have any control over that so Yeah, but you're just curious right because yeah, but so I would say that it is definitely more in time like a sighting Like there's more adrenaline than like just Rob running on a benchmark and get the anomalies was like a process that that took some time.

Yeah But it was more if you had specific questions you could ask or so, but I think this whole thing has been a highlight for me Is I more effort has been right?

Yeah, I would say most people if you asked them Maybe two years ago whether model could get and then I moved or they would have said it like impossible Yeah, then the silver helped right from last year, but like the fact that you control that system completely and then just take Existing Gemini and scale up deep think and then just running for I am a gold I think it's also like very non-consensus compared to last year.

Yeah, definitely to start extent I think researchers will also surprise I would I want to say like surprise, but like it was more like a petal on the back house surprise Or we actually made a lot of progress in we as in collectively that all the engineers and researchers working on Gemini It's a lot of progress being made.

It does look at how much we went in one year.

Yeah, and I also think that it just five years ago Like that not do it like five or you just you just imagine that I'll come like you just look at the state of AI now Like just generally the bit I am all and the ICPC go and like also even like things like that open And if you just look at the AI progress now and fire the gold.

I think people would think that we really reach like Age some form of age you're some form of age.

Yeah, we're just moving is if you just travel like you take this checkpoint Any travel back into five years ago Some of you will make a drama about this, but I think it's really quite impressive like how the fuel has more so quickly.

Yeah, yeah The hard parts you would say were scaling inference in what aspect like heart and does that even as massive Heart as in maybe the most amount of brain power to expand it on the team I saw some comments where there are like actually the hardest part was the inference optimization Or like the very very long horizon inference that deep thing needed compared to normal Gemini Stuff like that.

I didn't work on the inference type scale.

Yeah, yeah, I wouldn't know here that is mostly that Then it was this the code name was apparently I'm more cat which you named after your desk Okay, that's not like it was in the so I think I do the body at point at some point right Yeah, yeah, so the I'm quite a cat was basically like okay, it's not like an official Code name or something is just like the name that the config of the job was like I'm more cat This is the you just need something for the name.

Yeah, I mean, I just like you know I just I like cats in India.

Yeah, there is mostly it and I'm more unless you want to bring out anything else We have other sort of researchy topics, but beyond before it's going to sort of researchy topics I did want to maybe leave the floor to cover what else should people know about the reasoning effort that's going on at GDM?

Let me think of where to start yeah, please what do people need to know So it's very good.

Yeah, but that's a lot of people maybe an easy one to start with would be a lot of people were focusing on maybe Academic benchmarks two years ago last year made the other marina this year for command For killing it's very interesting reasoning Visual reasoning and just general long horizon agent planning yeah benchmark and I don't know you seem to focus on it a lot and I think generally did very well, so obviously I think it's something that is easy to talk about At the publisher we were just in this and there's actually nothing specifically done for Puking one first yeah, of course, there's nothing specifically done for Puking one and I think that I didn't Logo had this bit recently about the recent gem my tree on Pokemon crystal Yeah, so much more special yeah, I think Pokemon is like so I used to play a lot of Pokemon and I'm a big Pokemon fan in general and I think it's a great like you say it's a great long horizon benchmark and stuff like that And I think it's a good to check in once in a while on this batch box that like Almost never get contaminated all that people actually like don't spend time to Like you climb it's like kind of silly to like build like okay, like what are you working on if somebody we were all I'm working on Amy I'm working on his yaw yeah, I'm working on Pokemon mixing or something Okay, that's kind of like funny.

We did interview the crop is welcome on yeah, I think it's name is David and it showed serious flaws in anthropics screen understanding vision capabilities Yeah, couldn't literally couldn't tell I'm trying to like get past this wall, but you keep just keep running into it There's no need to walk there and so there's not any special reason at all I mean that's not what we could be like a harness like a harness thing or like also whether the model has access to like game state information Is it compare visual yeah, because implementation is very game stay heavy.

They dumped effectively like all the The memory of what's going on in the email later.

Yeah, yeah, I see it here.

I didn't look for I don't know where I'm dropping off tangent or something.

I think solving Pokemon is going to be more of like how fast you Sort of it and then like the thing that I have not really seen so far is like whether the model can complete the booky decks What is that?

I don't need more challenging.

No, but completely so hot there You need to plan you need to like you need to search up like information like there's something See if you don't like go online and amazing you need to have a little bit of the research in yeah in this here The model with us never know like that it needs to trade.

Okay, if it's able to go on like personal forums and then find someone to Like a call crash rate review is Pokemon to evolve some Pokemon needs to be treated to be involved Yes, or meet it Yeah, but anyway, I don't know.

I have not seen the model figure to complete the book.

That book of the book is really hard actually for model So I was I think that's actually an interesting Into like like an interesting one.

Yeah, I wonder what the real world analogy would be once If we have a model that is capable doing that what can we make it do that we cannot do today But there's a lot of planning involved right just really research like planning there's a lot of planning in both and it's more I did copy Pokemon is put but the Pokemon game is very linear right the in the bug legs is involved a lot of like backshirking Research yeah, a lot of research and a lot of yeah, so it's a probably a different nature Yes,elf is that as interesting to you as for example a lot of other people in the AI for science world are trying to discover things that you can not look up Right, no one knowledge.

Yeah, I don't want knowledge Because basically what you're saying is we're not even there yet We're at the place where models cannot Consistently apply knowledge that they look up Right, like you give Gemini access to a web search and you say okay, come try to collect all the Pokemon in a Pokédex You don't have high confidence that you'll do it.

I don't know if someone's actually tried probably no right But I think the hard buys actually like Trying to use the slice in the size of the web knowledge and then blow in in the game itself with all their visual state going on and stuff It probably will be stopping one or yeah, it's not it is challenging.

It's not interesting.

It's not interesting that you're basically just The task really is can you look up the guide to do it and then can you apply the guide?

That's it.

We know what's even more intelligent than that creating the guide Like being the first to figure out how to create the guide Which is what didn't oh, yeah, but then when you come to this at least mostly that's that is also such thing A model to try and try like humans are great for that Yeah, okay, so that's actually less interesting to you interesting.

Okay, I should be okay.

Well, you think what is like not super super interesting But it's okay.

It's just that.

I had not seen a model to try to do this so yeah, that's right Yeah, I think like efficient search of a novel idea space Is interesting.

Obviously you can move forth anything But we're not talking about before saying we're talking about trying to create a I think this But novel knowledge is actually an interesting thing that that I think it's gonna be quite a big thing Being able to generate a novel go-go has done stuff there Which I don't you probably not that close to those teams that is then yeah, I think this work There's some things that I've been it's like for example if we freeze the model with at like 2015 The freeze time at 2015 and then you even with the car model.

Let's say you have okay That's assumed there's no looking or information somehow If our model was the best ML Okay, not 2015 like 2012 or something.

It would just tell you that SVM's are the best right This is a way that machine learning works in general, right then the question is can you invent the transformer Right it might not be able to like even do the models they might not even be able to invent the transform Or like if you freeze the time at certain time and even you bring the tech I mean the model is a transformer so I just say that's no As soon as there's no education.

That's what you're like no, it's only possible.

Yeah, so so I think there's still a lot of Questions of all I whether the model can really you know, you know, wait then generate like already know well Yeah, not all it's yeah, yeah one related question on that which I think is related to the Any people which is I think people have this sort of mythicism on what reasoning is and reasoning If you really demystify a lot it's whatever happens inside the channel tags Right and you post you you you you're illiciting that reasoning behavior from some stuff that is already Latent inside of the pre-trained Corpus is that that's one version of this interpretation.

I think these days like reasoning itself It's very vague and it's very open so it's like most mostly different people have different definition of what reasoning is right So I agree that like this chain of thought is like basically when people think of reasoning They associate you change thought and also this is what happens in in the thinking and then that okay reasoning right But I think these days is more like I say in earlier, but this like reasoning RL is almost like this like basically Anything that is Post-training to let us see capabilities this basically it's like like RL and post-trained that's it could be so I think the Actual like technical definition of reasoning is making more or better with thinking and post-training Okay, yeah, so basically like RL in the model to think better Right and thinking is more like thinking traces and thought trajectories and stuff like that Right, there's also this line of work for later.

Like latent thinking and stuff like when the leader thinking and discrete talking thinking is like Going to be the same thing or like something like that is like open question meaning adding extra tokens to your cap That represent all just all these I don't know what's the name like this a kind of a person that do this Loopy things or try talking all right the basically is that of Decoding discrete tokens you actually simulate this by doing this in latent space right so when you do Chain of thought thinking and like reasoning with basically the core extra tokens hide in the thinking back And then you did the code stuff, but like latent thing is between you just don't decode yeah Go into this don't bother by it.

It might start speaking the native language of thinking is number is not Passing you through some filter of English and sometimes you must start thinking Chinese or something else Yeah, I generally I'm not of I'm not really I don't really believe them all all thoughts have to be the same with human thoughts I'm actually like you're generally in ML.

I'm more of the school of thought off let them all the way Once in general.

There was a discussion there's a latent representation hypothesis paper that I think you are maybe Sympathetic to if you haven't already read it to me some obvious the basically image models will have the same idea What a laptop is versus a text model will have the conversion like the same latent Yeah, and obviously you can align them and you can do all those stuff with them And so it totally makes sense that their concept would be just a vector of numbers that represents laptop That's the concept.

Yeah, and okay, maybe you have some numerical differences between one models idea what a laptop is with another, but it mostly would be the same Yeah, very interesting the question I was going to lead into was that because there's now where in this age where Dell M text is in the corpus of like stuff that we train on where it's a little bit of a recursive loop right like The reasoning tokens are out there now and so pre-trained models themselves To pre-trained base models that are also capable of reasoning and they're increasingly so as More and more reasoning text goes into the corpus Isn't that interesting or is that worrying?

Do I actually see much reasoning trace on the internet?

So I've no seen those though.

I would say that I'm hugging a face.

Yeah, people are publishing that Specifically as not as the weather or not people are researchers are actually including that in their training Compuses who knows right like the stunster their choice, but I would say that percentage on common crawl That has COT tokens in there went from zero to zero point zero zero one Percent and it will just go up over time because like people are publishing it Yeah, but I think if the sources are like quite clear.

You can actually like filter away those because it's usually people put it on get hop Do you want to filter maybe don't that there's a choice for the for the yeah quite really quite literally the whole reason why I don't think we covered this in our previous part, but two years ago a lot of people were like oh, it just include more Coding tokens in your pre-trained corpus But the coding tokens are different from like calling token.

It generalizes outside of code for reasoning Oh, there was like you don't believe that.

No, no, no, I think like I don't know if it's still true today, but I see Yeah, that was just our general coverage of reasoning.

I would say that there's a lot of interesting work here and More to do maybe I'll cover one thing which I know that you have personal inputs on which is that you have started using AI coding Oh, yeah, so I actually don't really use my AI coding the path, but I think we really should point where AI coding has started to become really useful like okay, so before I code in the most did I think that I find the most useful About like these models in general is like when I have this Big spec sheets of a lot of results and I just need a lot of it.

I think models can quite go to the screen showing me a plot of this I hate making this method like stuff about it's so annoying.

Okay, but that's basically Like one thing that I can remember about like how I use AI in the past by the AI coding has started to become the point where I run a job I get about I almost don't look at about I pass it into like anti gravity and like I thought it that would fix the bug for me And then I reload the top that like it beyond life like coding is more like like Training right now or something like that now I would say it does be well most of the time and it's actually That's like there are classes of poems that is just generally I know this is actually really good for an effect maybe for me better and I would have to spend 20 minutes to find like figure out the issue and then Think of thing and everything so yeah, it's very interesting because I do say like level one by coding is you actually know What to do you just too lazy.

Yeah, it's just I just do it for me like I've done it a thousand times like just go fix it like I know exactly what to do here You're seeing it's like on the next level.

Well, you actually don't even know it's it's investigating it for you This one is like the answer looks right you're just stupid.

I just thought I was a little bit like I did check it Look at the thing and then at some point I'm like okay, maybe the model looks better than me So I'm just going to let you do it stuff and then I really really launch a job based on the fix that the model gives me And I think the models with this kick keep getting better and better.

Yeah, so yeah, it's something that that yeah I also think that I really recently does this anti-gravity and then I think also because these tools will not like that in Google infrastructure is not that easy to You don't I'm not that familiar.

What it's fair about outside and when I was they started I didn't really I didn't the models for not like so good like one and a half years ago So it's also like a forcing function that like so people like or try a degree of these game changer and stuff And like okay, so I just but they're using and yeah, yeah, you spend some time with the ruin recently What do you want it?

Well, no, I really did say hi and great.

Oh, okay, I guess you were telling me you're my AI researcher that doesn't use much AI and like now you actually like yeah I feel as a user there was so many moments this year where AI suddenly crossed that like the image and things like the AI CoD is one value just discussed.

I think like nanobana.

I also got to the point where I usually like if you make this image It's just like the way for fun.

It just troll your friend or something But like nanoma actually really got so good that you can use it for their best that you can use it for like basically Yeah, so it's getting really good and I think yeah, did this year the stuff like and even things like the past Maybe this I also like helped this house in it things a lot, but now I just trust it automatically I think we just I people are just enjoying the utility but by Yeah, but by these models so not I'm like I was always AI feel like AI is a good thing like I don't see how anybody Yeah, but yeah, but you're actually using it for things that you are high expertise in which is your own ML work Yeah, yeah, and just and just to come you are you do have a special version of Gemini that you use internally that we don't have access to or you're it's like public Gemini I think it's the budget Gemini.

Okay.

Yeah, I was just saying like it would be entirely reasonable to train Gemini internal for only your code base and your work.

Oh, actually I'm not sure though You see these things are just like I really like right there.

I'd rather wait for me Yeah, but obviously if they obviously improve it improves your productivity by I don't know 10% yeah worth it Right, yeah, so I think that's interesting and there's the interesting thing levels of how much do you trust it?

How much of your jobs do you draw to me the way yeah no longer need there's also the question I guess about how people come up and train in a field if they you no longer need Juniors because the Gemini is your junior ML at ML researcher So I think this is all interesting questions.

I want to say one quick thing first, right so I think that when it comes to like Whether a model can be like a junior sweet or like something like that, right?

So I think if you think of it at least way of if a job from one one X sweet one time sweet can be based by a model yourself But let's say you are a manager, right?

The objective that the measure you try is like your time and then if you can have a model that saves you like the same amount of time as The work that you're supposed to but you don't actually replace one percent per say, but you yeah a little bit from everybody Yeah, right then you can I definitely agree that okay like when you count a net time and save There are times where the model can fix box that like what I call me like one day right one day is huge In these things are definitely like what if you I don't know whether anybody has done any like real like metric evaluation or these Things but if you use time as a map real metric and then not as in number of okay Maybe suit I was like kind of map metric right but these things are not like like going to replace one percent of what like SSEs but more like a passive or are that buffs everybody the in-game thumbs right Yeah, I often think of myself as a bard because I tell stories and I plus everybody around me I think that's that's an ideal situation for me in a D&D group.

Oh Okay, yeah, and I don't bet the ND ball.

Okay.

Yeah, I got I get I get I get I sit This is the coolest that the king's a factor more a plus of what he wrote so the spot spot.

Yes.

Yeah, okay AI support.

I think is very encouraging.

I think like where is it still not working for you?

That you've tried and you're like oh man, I expected it to be better Oh, they are kind of a models get try to get lazy and try to Fix something in a like they are still we will have they get lazy and then they try to like Guess like me into thinking that like the bucket fix.

Uh, so there there's two classes of their classes of bronze They are like very easy for the model a great hot for humans.

There are some things they are very easy for humans Very hot for model or a expert or and stuff.

So it's still right how to characterize this this things into this proper Corridor and stuff like that.

So I would say that the capabilities on model systems are good enough to be like Really helpful, but like it's still you it's a bit like it still has some but yeah, I think this world Like this I don't think there's anything that to be done to specifically like Focus fire and these things is more like General capability in performance models that get better over time and then these things will just like go away.

You say that okay, so Yes, I think obviously in the grand scheme of things just trust the process keeps killing Every dimension and things will just fall away things will emerge But you've also said in the past I can't remember the exact to where you're like each additional data set compounds Over time it they're just small additions and I would say that we sort of say things like focus fire on Things that you would think humans it's easy for humans hard for machines Those are easy wins where you can just add a data set that wouldn't focus fire on that and Isn't hill climbing just as sequence of doing that until you VGA Okay, so I get your point.

I think that it's true that sometimes a lot of progress on the whole is just a series of Small incremental changes that yeah that push added.

I think that's accurate.

That's true There's also it also feels that there's also a lot of like small might like seemingly Miner for the lack of better work like they push AI to the step ways today So I definitely agree so nothing like against people who will like focus fire, but it's just that when I mean that like It may be not easy to focus fire on things that are like not very easy to characterize So is that like when he has just something targeted right, you know, okay.

I want to improve this Capability at some data so I think we're defying like the emails defying the problems and stuff That it's like characterizing it and if you can't be characterized and then okay, then fine, but I think like Why I was trying to save the coding is that these things are not even some of the class of problems are like I don't work on coding, but like people maybe work on they know like they have like terminology for different types of failures But so maybe somewhere somebody is focusing fire on these then make the model better.

That's great for everybody Yeah, I mean that's what I think thousand people are like get all these things together AI is definitely like a big collective effort this days.

It's a big mission.

Yes.

It's really crazy.

Okay, so I just wanted to brought it But not to general things people are talking about in the community on research Which again, I know that you are very Locked in so you don't know necessarily have read all the people is anything, but we can just riff on I think obviously ask me what I think as well.

Yeah, he's attention all you need So attention and transformer has been it's like a core idea in the recent times Like pre-training and scale is the thing that made attention and transformers like actually shine right because it without I think the first transformable people was just like a machine translation thing and then basically LGBT and birth were the ones that like actually showed the full Like big potential of this idea So in terms of it is attention like really really all we need like a part with no But I think it's like one of the form architecture point of view also maybe no, but It's not all you need is but you've been in it definitely So like what else are you thinking about on that table level are you talking about Emily stuff?

What do you mean we say it's not all you need?

Not definitely like you definitely need like the scale of pre-training you need okay All the tokens you need like are out.

I think when I say when people say I say it's attention You need is mostly from all kinds of point of view.

Well, will transrovers get us all the way to HIV right?

Like yes, it's the so it's also basically when we get to HIV it's the problem.

It will be will it still be a normal architecture or Like meaningfully different you'll be a transso lighting really like like you We like people it depends on what you call it, but I think unless the paradigm shift completely Which is I mean as the scientists cannot like Like completely like saying all the like like that this would never happen But my feeling is that it's been like what like almost 10 years since transform In 2017 nine years of GIS like since this is the transform I think we have not replaced self attention like it's some form of it like you could rename it You could name it something else you put something something to do local global I'm letting yeah, it's still a transformer and then and I think like that's not going like anyway Unless the whole thing with like backprop like everything like goes like the whole thing this changes completely Like then there's a different story.

There's a different conversation to have but if it's still we didn't the same School and months so I spent a lot of time thinking of about architectures and like whether there's alternate architectures There's stuff that either Okay at the sequence processing level Like there is the ultimate yeah, it's sequence the sequence transformer.

It's probably the self attention There was this whole big era which I was also involved in this era where people try to like And the mind that attention as much as possible like they try to remove it Simplify it if we shouldn't like it's really like like efficient attention era and then they the outcome was always like Or remove all the attention but we have one layer of self attention that it still works like this at end like after Like always a story which even known a character he published some stuff about how he has some ratio of mixing of local and global attention right like you basically still attention But modifying it quite a lot.

I will continue the local and global attention to be like City or Beware attention Just I how much is keeping yeah, the only question is that if the formula changes too much your QKV becomes like ABCD yeah, have she do or something like that or some okay Maybe I'll give you some motivating constraints.

Yeah, I'll do this you guys.

There's still charging 2x for over 180k token context or 2x40 something like that and the max is there are equal max is 2 million tokens Right, what if we need 200 million it is there some point in which where Even this concept of input token context is irrelevant because you are doing continual learning That kind of stuff where you're modeling it as okay The AGI will be achieved through a sequence sequence transformation Therefore annotation is the best sequence sequence Model or architecture therefore attention is all you need but I think other people are like sequence sequence doesn't actually capture Intelligence but it's just not really about sequences because there's more about like the whole The gradient is sent and back procting right it's not architecture itself that there's a problem that we that is more of like the Learning paradigm itself rather than architecture is how I think architecture is this basically like the interface between the learning algorithm and the tokens I think it's more about the learning algorithm itself and this conflict like continuing learning this like There's many ways to think about processing many like insanely lush like context, right like to hire a medium one Beating tokens is something that they're right like whether it's going to be like you have a new learning algorithm that every time you run Inference you you learn on it, right then you can Kind of have some kind of memory like human being is learning as I'm talking to you right So that's also like one way the other ways like whether okay, maybe you somebody will say that okay The attention is like just two expensive for 200 million One billion context will be new architecture or some people will say that or we just improve the Chips like accelerators So I think many ways to interpret it, but I think it's like if it's about that there's a lot fundamental things that if it's about continue learning and stuff that it's a lot of fundamental things about the learning algorithm and stuff that it's well here There will have to change I think like the learning paradigm and architecture is not that of course like hand in hand And I think as the field progress and I just just take on one another, right?

So there's also this thing about The idea that it was supposed has to be compatible with all the work that have been done before doing to shine Right, it's a bit of very enough this hardware lottery like by Sarah is not the hardware Retreat that I wrote about the GPUs and failing, but it's the original hardware lottery But it's more like a bit of lottery of like the things proposed have to play well with the things that will propose before So it's a bit like going down this Local of the minima to some extent So now we are like in this local minima of like transform was everything everything right Maybe it's not easy to like get totally out of this because also a lot of people's Investionalization only might have been have been done so the things that play well needs to play well with the ideas Before and the way I see it now is it's very difficult to like yeah come out of it Okay, I'm not entirely convinced I see what you're saying But let's call it Jen AI fucking hate that term.

It's still a very young field and so yes, there's been it years of work on The transformer, but what's that in the grand scheme of things maybe we're in a local minima and kind of Not sure so sort of it.

I do want to leave that open ended.

I don't have an idea I do think that people are in this call it what you're so scared of us in following the age of research right Like we're like okay, we scale that what we can scale up There's we know what the next maybe one or two orders of magnitude look like in scaling on every dimension that we know about But what is the next dimension to scale?

there's this Like miss Understanding a little bit about like all the last five years has just been Like scaling things skills.

Okay, please tell me right.

Yes.

I didn't you made that yoga bus Now we scale research salaries.

Okay.

Let's not go there, but but I think that ideas Like matter and I think that there have been a lot of good ideas in the last five years.

It's just that maybe it's just not It's always not been like blindly like if you talk on MLP or I just like without a self-attraction Okay, I'm going to troll like hundred trillion dollars on this scale up and thing and the thing It's never going to work.

It's never going to work.

Yeah, yeah, so there's no like there's part of it There's also like I think the beta lesson gets used too much in like To conveniently use around but actually there's a sort of a little bit of Not a bit there's a sort of suit as an way it's like ideas a matter and I think even to do do they right like people don't play ideas and stuff like that.

Yeah, do you think the rate of new like without being specifically about what ideas because obviously you can't share But like didn't a rate of ideas has increased or decreased because there's like kind of a lot of diminishing returns These small ones.

I think the number of ideas always proportional to the number of researchers working on a certain problem So by definition by definition to increase but I think the number of ideas that actually work It's not decreasing compared to the last like it's we're not in the era of diminishing returns yet.

Well So I think the ideas are still right bottom and they're still very good ideas.

They are game genius.

They are Being invented.

Yeah, yeah, and I think I know the answer to this, but is the close let advantage Increasing versus open source or decreasing Like the Chinese labs.

They say keep publishing open source models and some of the American labs as well Publish open source models Would you say that the ideas that I see there Nvidia has Nemo Tron opening a history PTO assess These are all basically checkpoints on what is publicly known about training models as of this year You know, okay, okay, there like it's declassified information because everyone there Like okay, yeah, everyone does this.

I think that the gap is increasing.

I don't think it's it's completely predictable from stuff that you said before The gap is definitely increasing.

Yeah, I think that would mean that justifies Researchers otherwise was the point of having researchers if not finding new tricks that compound over time.

Yeah Yeah, but definitely adding this is a increasing risk.

Yeah, okay I'll do a site tangent.

I don't know if you have any comments on this.

Okay, so then this is very related to Nvidia's recent purchase of Grok Which I don't know if you have views because you're a very tip-use entry, but I'll remember your compute bound And this is relevant to the trustworthy discussion of like in terms of what life-saving Exactly, I think the classic view is that we are compute on because we just need more compute for portraying and or else then in print But actually the Counter argument I would make or August this is I actually have these charts of more laws I wish I could just pull it up easily more laws of the scaling of compute versus scaling of memory versus the scaling of network and bandwidth and Compute has a much higher slope of scaling than the other two memory was the cheap memory like Honestly, I don't think about this this memory bound that much from maybe doesn't Yeah, so I would this may be but I don't have high confidence in and because you're mostly on the research side less on the Inference side yeah, there's maybe the inference guys will be like yeah, yeah, yeah, I don't think about I don't pick up I think also, so yeah, maybe I don't think about the inference Yeah, my previous line of discussion here was like Videos very for foresighted by millenocks because it actually is the real bottleneck in scaling because it has the lowest more law And then the second one now is memory Which is very interesting But honestly, I don't think about that so yeah, well, it is this the the much yeah understand okay Detail efficiency so this is a joke, but implicitness is that there's some kind of a maximum data exposure, right?

And so I so previously I would say that a lot of The training paradigms is like one epoch is all you need that says yeah, you need title of this idea I would say that the real number maybe is between three to four epochs and I do wonder What theoretical limit of data efficiency of a model in terms of training and compression should be?

I don't know that means they don't have efficiency and the missing but you're asking the question in a way I think like how much we beat this horrible is that one and we've in tolerable is Contigion on does it actually improve in meaning?

Oh, it's not about like you actually wanted to do it for its foys on sick But do you think there's that and then there's also just the sheer amount of stuff that we can learn with limited data So you see let's say like we are you're not compute bound you're not memory bound But let's say you are Data bar right last time that we were on a podcast we talked about chin shell services If it's after more training, but now actually I think a lot of people are even talking about like data of the Moduring like given limited data set how well can you learn from it?

I think that's an interesting research direction that nine of people are Talking about maybe there's something that is Common place in the labs, but it seems very clear that we are Very unoptimized with regards to how much we We learn from our data Not just put it there.

I think in general the Like learning most like extra thing more from very data points that we available But I think there's a related effect that we like running off tokens in the world So I Don't book on data for betraying and I think things that I say with general speed of industry General speed is it right so I think that the I don't know even know whether data has Like diverge like the way that these things are done.

They have diverged too much across Dislapse and no open.

There's a lot of cross-pollination for sure Foundation okay, okay, yeah, but I don't think about data that much like the betraying data They're much this yeah, maybe Earlier this the first half of this year yeah, I would have said that kind of pre-training is that And that everyone's like just funneling all their work up towards RL and You had this like rock chart which are very interesting Where we're sitting same amount of compute on side grain as to you think it's a side out No, I don't know.

I don't know idea.

Yeah, I think that people are taking seriously they are like yeah, okay Whatever especially in the agent labs like connection cluster They're taking the open source models from from whoever and then adding Let's call it pre-trained scale RL on top of it if they have that level of info we should they that wish they do Which is very interesting.

I would say yeah, this data efficiency argument yeah, I need to me is also more Trying to Discover new paradigms of learning in order to get where we want to all go which is okay And the existence with his humans right you are to your doctor can it's much more capable than Then an LM in some things having seen way like It orders a magnitude less data That's very interesting Yeah, comparing human learning machine learning is definitely like surely is an interesting existence proof that we could probably do better three examples of dog Yeah For example of identify anymore.

I can probably tell it's a dog as a human but machines Classically you think 20 they're efficiency for humans is definitely way higher than then than models Yeah, the weird question is that where does this they come from is it actually like Within more flock on Larry talking oh Like maybe it's like back to the question about whether the trans one was the optimal architecture maybe it's back prop It's the one maybe it's the off-policy nurse maybe it's the yeah, so what is the like where's the buck right Exactly, oh, but I don't know but so not okay If I identify it and probably it'll be a while to get this across so this is the kind of data efficiency and and talking about I think it's imaging basically at the end of every year I like try to take batches to care what will be the big themes next year.

Yeah, I think there's more them Yeah, people are really trying to focus on because you're feeling this data crunch Even though everyone's like still investing in data.

I forgot to mention that I would say that I've been wrong on Pre-training being dead.

Yes, I've now met featuring Leats from both anthropic and off-ni-i and I've so seen the the talk from the deep mine guy recently And so everyone's investing in pre-training still which is like Nice to see Nobody's saying pre-training was dead.

There you go No, it's a theory that we're trying to disprove all or proof anyway, so I think okay Let me one back to my general idea, right?

So yeah, data efficiency Seans worthwhile you would treat it as like okay, well show me where the buck is and I'll go fix it Yeah, we don't know where the buck is.

Yeah, we just have existence proof that it could be better And then I think the final Logical chain in this from me is that everyone is focusing on Some idea of world model as a version of this form of efficient learning which potentially might not take the Form of a sequence sequence transformer I don't know how that works like definitely a little bit automated that here to me that is more efficient Because every world must be instantly consistent and if the next piece of evidence coming in and Invalidates those walls, they no longer need to pursue those halves ever And you can just narrow in on the wall that you've identified and so to me that is Learning where you're learning to fit world models Auto actually, yeah, oh, so yes Maybe you can treat the learning process as comforting yeah, so you're learning the world instead of learning the The world model.

Yeah, it's a learning the world model, right?

Okay, by sampling multiple world models and then finding out we shouldn't fix the data The best so I guess my query is this what people talk about is this if you I mean?

Obviously feel free to attack it because and just be bawling and but this is what I pick up from talking to multiple people about Okay, what are you talking about with world models?

What are you talking about data efficiency and learning efficiency and like how do you Jull it all together in a cohesive sense of the future where we can actually Before what's the definition of world model at the start of on start yeah, yeah two kinds Okay, go on yeah first kind is the view okay, okay, oh, the what's the other one?

Yeah, and then deep my hands, which is the sort of video world model.

Yeah, you model everything with some kind of Got inspired so whatever and you're like you inhabit those that 3D space yeah second let's call it the Yannluku slash meta school of thought which I don't know if you're definitely with it He has published the jepark architecture and then Separately fair is also published the cold world models where you're basically specifically for cold very interesting You are executing cold and modeling the internal state of the execution environment as you go land by land Okay, the LM actually like learns to predict that those things yeah, and actually it seems a lot more efficient at the skill that they've tested it up Which is recall which definition are you anchoring on the third one The court lines the court world model that's the second one.

Oh, so you know those two are bundled together Yeah, the jepark fans are probably hitting me right now because I'm lumping all methods work and the one school of thought Yeah, but whatever Okay, the first one is vio genie If they use like super in special intelligence those kind of video based world models second one is some execution or some sort of explicit modeling in the As you as you sort of run through the the corpus yeah, and I think that the third one is this Immortless thing which I think people are trying to get to where They are doing what I said about the Resolution of possible worlds in your cuff fitting as you learn as your inference Yeah, but what is the world model itself itself is it like it is some mental model of where everything is and How you think the world works how what I think you think Like everything but in the technically it's like it is something in the latest ways Okay, so you can understand basically it could be just be like go transform model between and yes Yeah, so the like only that is the most coherent thing to the current paradigm Which is you could actually do this in current transformers I think the way that you train it over for we have to be different Okay, I see I see I don't have any conclusion here.

I'm just throwing it out at something where I know you're interested in this kind of stuff and I don't have that many Knowledgeable people will talk to about it.

No, I don't think about one model that after that I mean because one model is not just not really well defined in the first place.

Yeah, but I think so don't say a world models But the problem is did I love the efficiency and maybe I guess Accuracy or like Agi capability that it's not easily unlocked right now on our current path of scaling.

Yeah, so I didn't I even It comes to like like data efficiency.

I think it's more like a Unbelievable finding ways to spend more flops for the token right because you actually Basically if you are data bound you want higher data efficiency because you can learn more from every data Pointy squeeze squeeze out more points right so things like that can extract more can use more flops on every token It's definitely like a form of Data efficiency then that's the learning algorithm right because I think that's this that's a different scaling law for like humans This machine is these like dogs are these cats are these that is this different my ex One like earlier chart right.

Yeah, yeah, that is this famous famous famous shot And point one point two at this like not entirely like the different things because it could be that better architecture is actually just adding more flops But token so if you are you come to a point where you are Very data bound but not completely bound or you just find algorithms that spend a lot of computer every talking on every token So I think So the overarching point is just that okay that that it's learning algorithm thing for data efficiency and then if whether the correct ways Actually just apply more flops, but token that's the squeeze out more from every Or read data every data point also because humans actually don't like like they are exposed less When you say less or more data is the right ambiguous because they are technically like on Make them in the force as seven and then you have things mostly visual level of like different thousand inputs right And whether they actually spend more flops on everything that listens is also a question because maybe they are that's better If you should just because I mean, I did so many things to call like how much flocks and brain use to process like how much Maybe they're just spending more compute than every token.

I know some of you are learning algorithm is different So I but I read that data efficiency is very important given that that I think we're going to like Definitely maternal of data in the world one more thing before we go into the SI You know how like we're talking about RL and it's working on RL stuff.

Why are people paying so much for RL environments?

Where so who is paying for RL environments open the eye and topic at least?

I don't know Be say anything about the mind so a lot of the model labs that are not you are well known for paying at least seven figures for external startups to create RL environments for them to train in okay and I think the question is if you are your models are so good at coding what do you do yourself?

And so I think there's some amount of expertise that's being distilled from human experts into an RL environment that you can then That your agents run well then but I'm curious if there's any other deeper insight in that because I'm not satisfied with my own explanation RL environments that are like death a lot domain expertise are probably very valuable and I don't know if I say about what RL environments be but I actually especially buying But what was the thing that you're not satisfied by like the it was a sort of variable and a lot of people are saying like looked it's Next year's app instead of a Docker container that logs stuff out when you send the inputs in Then before we do yourself internally, why you pay so much for some standard that you don't know to do it for you Actually, I have no clue of a lot like why did you just have money?

Yeah, I have no clue and a classic example would be like if you want to build computer use agents for buying things on a in e-commerce You would once our environments that Perfectly replicate maybe the thought does an e-commerce website.

Yeah, and then he just Parallelville, I don't know all of them.

Does that seem unbelievable?

I don't know.

All right cool DSI and Ellen Rex is a big bet for me this year for my conference was we actually like started focusing on Ellen Rex's The other actually was the motivation behind starting on Rex's track.

Yeah, I think Rex's is the king AI problem in consumer Is these single most valuable thing all your feeds any even search is Rex's But it's search basically like which we will is the god problem Right because Rex's is ranking but they're also filtering also personalization also re-indexing and in like performance it is the god problem and you've get paid a lot for it Engineers are not that excited by it which is very weird Because they don't see a lot of them don't work on Rex's and they probably never will Yeah, but they don't see the monetary value that can come out of a good Rex's The other two pieces of updates for me, which I actually didn't even know that DSI like directly tied into this was one Twitter publicly adopted their feed algorithm as an Ellen Rex's I don't understand this is everywhere now, but what it is actually like a Like anything Ellen like when it is like a generated retrieval type of yes models like it's like another Question correct is it we don't know all we know is that they have said that we they have swapped out their current Rex's for an Ellen based Rex's and that's all they found okay But what has what is published is YouTube where they actually adopted some of the ideas for YouTube's Rex's yeah And YouTube is obviously a big deal is it like public information?

Yes, okay, okay They came in did talk about it with us okay, and then it published a V2 this year as well Okay, I think and I just so basically the last time we you were on the podcast We didn't talk about this I didn't much but you will have actually some background in IR Are you care about IR?

No, I don't care what I are but I think yes, like okay, like DSI or gender retrieval was like I think one of my favorite works in the all of like I have some IR background in like when I was Do a PhD at this time with Rex's world.

I did some Retrieval work with Rex's and stuff that so I have some IR and Rex's background So I think gender retrieval and gender vaccines is rare or very Conflated DSI started as a retrieval thing so we did like natural questions like ranking of like the index Yeah, I talking documents that were everything the it started off as I mean that's actually we did Interview with Yannick like me and don't meet it We went do a bit of people came out like long time ago So at the time we wanted to like really imagine retrieval and search so we wanted at that hour And if we still didn't he find models at that time it was like not not in the LM era Yeah, it was pre-alarm era.

It was like okay Works kind of thing and then there were like some pushing models around So we wanted to really imagine retrieval right, but retrieval Rex is the all the same formulation ranking We should have a problem right and then that's where we started to imagine retrieval as one giant I am the the enclos everything in the memory But we tried so many different like so many ideas actually my clip the wind was the one that came out I just So many ideas that I did that basically and the start of this whole gender retrieval was actually basically Literary just trying to give a document like identify it and that's predicting like raw before Predating like this actually like it actually works because the models can memorize something if you look at a literature from All the way to things like talk to back of this very off like Model the words have no meaning that this ID and in a vocabulary so not this number right and Then we did a model chef enough capacity to put pretty pretty but in somatic ideas was an idea that that basically you have some Like somatic association and then you actually Try to break down the subspace hierarchy All right, so how this would evolve in the races was at a time after DSI came out right, so And she's group and I'm a hash the guy who who they did some exploration of applying DSI to to access and that's how that gender-tifled generative Racist Recommended system paper Came out yeah, I didn't even know he was involved And crazy there was like basically asked like transferring this like basically okay DSI will treat we try to try it on on-rexies and then I think the recommend system people have a slightly different way of doing so many ideas but It's basically just because the domain is slightly different but after that I think we were done with the invention but then we just saw The rest is details that the rest are details over time.

I also left Google and stuff like so over time Distinct evolved a little bit here every day.

I think I also saw like something like Spotify is also using something like you just what if like like they they used this stuff like some of the ideas This stuff DSI like models I think from the christian community point of view like the DSI was the first one that like decodes the Medic tokens but then when we went to I don't know I can't tell you it's like strange in a way that like they were doing things like Oh, this is general retrieval.

It's not generally racist.

It's like they'll do this type of like random things that is like a bit strange But yeah, I know this was like the whole history of this gender retrieval Apparently that there's also a lot like of people working on I don't follow actually I don't follow at all now.

It's not even in my mind.

But yes, there was one I went to even in the Singapore office.

There are suites actually like working on Gender retrieval.

They don't work on it.

I don't know where they're just working on it But I met a person that tried to explain gender retrieval to me.

It was quite funny that you know I kind of like oriented gender retrieval.

But yeah, I think this is this whole IRR thing has been it's just a Interesting face and I think the exercise one of my more creative works that I've done There's not like really our and what it's like another the general principle for play a melt everything If the Googler is working on gender retrieval would that be like yeah over views is that there's something similar I've no idea.

Okay, well if you want this thing I did have a track there Then you just type in a I dot engineer and you'll get it where the gender guy was talking sorry the YouTube guy was talking about how to use Gemini or their access.

I don't know what size of Gemini because they he didn't talk about it But this is public work now and basically Every YouTube video uploaded gets encoded into some kind of code book and they they retrain this every the On some kind of batch job Yeah, just interesting.

So yeah, I don't know if you even know what yeah What Gemini's been used for?

I don't know if it's just in this days.

I do think like in the sense of like for people who are not Still not getting it Applying intelligence and then the general intelligence of an LM to the retrieval to the recommendation task Means you can accommodate such weird Recommendations like such weird queries as well that Normally like no classical system can ever handle and I think like it's also somewhat emergent in a sense that when you were using T5 You just couldn't actually add that much value on top of a normal BM25 retrieval technique which we've seen as accurate.

It is not that's not just about paraphrasing It is about understanding query intent.

I'd be able to device a really strong baseline.

I should be I'm going to do this like a really strong baseline.

Yeah It's like like and like I sorry I don't I don't know the the the converted data versus T5 for you guys versus BM25 but I don't expect it to be very high and I expect it to be a lot higher for For a true LM base axis depending obviously on the query set I didn't really think about it this way before but because I've done modeling in many different domains It's going like such and you know in the such community I'm a community as a social badge box and stuff like that for like that people who claim on some like There's Amy of like a of I don't know why they call this this anymore, but generally the modeling dynamics of IR task is very different from Like it records these tasks it's very different from standard language tasks or like vision tasks and stuff Like when we kill kind of out and where you train models into like the way that modeling things interact with This environment is very different so I think that I honestly I hated working on like raxies and which you first off Okay, I'm just looking about all days when you work on like T5 you work on you change architecture You try to improve your complexity Super good like this is the old and this you are on like even now when you train our ms You just do zero short through short stuff like that you know things The you because as a researcher and you know you just interrupt and rhyme a lot by this you're just like okay RL like this environment but Raxies and IR has a very Strange feeling to be Strange feeling in the sense that it feels like your the like whatever works is like you are in the in a wall with the gravity is different Or like you are in a world where the modeling things that feel intuitive are not intuitive So it feels like a very strange space to so I wrote some people's back in my days We don't like raxies and stuff like that.

What every time I ran some modeling experiments for for xys and stuff like that I didn't enjoy the if you feel like everyone was rude and feels like the vibes are just like like when is it rude This feels transactional no not not not not not not not not not transactional like I don't know how to describe it For example, like if you play like spots like that and it's a perimeter or you hit the ball You have a very nice feeling like hitting the sweet spot And when you do modeling in traditional don't when you get the feedback back you feel like everything sounds right everything feels right everything Right for raxies and IRIs like a problem with it's like you hit the shortcut and you hear a glass shatter like randomly It just feels like we say world that That caused it if I were too far apart like it just feels strange and then something's maybe the metric like I think raxies they use like all the NCCG effects and then the BM25 strong and then you just and then you get like rules that look like that the BF35 But in the day when you stack two hours the M2 trial is there you will like whoa, I see life is just The game on rewarding area to work in is just weird also the IR community and we should work on it So so like always behind the mainstream and then now is just probably going even more worse because of our Stuff so okay, I'm waiting to hot dictator for you, but it's just like it's just like behind New routes and I see a mountain stuff.

Yeah, some conferences like this like did this like applying things that that this Downstream of yeah downstream.

Okay, so he always feels right on his partying go to do to work on on this Look there's a reason that you left and but it was like a site quest like where I look on here.

It's a side quest thing Yeah, yeah, okay, and understand I still think it's an important business problem even though maybe it's an rewarding field I don't understand why because the academic benchmarks for those tasks are just so far There's from yeah, there's so far detached from what in their trees.

I didn't want on any of these like in in like Like the thing but that's from academic point of view.

Oh, they don't only need it on the investor, right?

Yeah, I need to send it.

Oh, okay.

That would have been a different experience.

Yeah, that is mostly our sort of topic Research topics coverage and everything.

I think we're just gonna end on a very simple one on gdm Singapore You organize a Symposium here.

We wrote Jeff Dean quark and all the others basically what's the general message or the impetus for studying gdm Singapore So we talk about the event first so the event was mostly So qualanized I'm gonna start team and then I think before I came back We discussed this was some time Jeff was very supportive of this He was in the region many times in Vietnam in Singapore last like about around the time where I was going to come back And I think that so this event itself was quote and Jeff was visiting and we just inspired the community here I think that it's also a bit more like a soft like setting the tone right from the start off the gym Nightingale in Singapore and I think it's a very rare instance where you get somebody like Jeff and quark The true pioneers of AI in the world to be in one room and then are you with that as well?

And I think like many people told me not to find any of AI but yeah, I was there to drive to it I didn't have in them all in one room and then leaving these thoughts like maybe we came out to me and said they were very Despite by Lendier presence in the region so starting a team and starting something is also very Like there's also no one moment that is no care press the button.

It starts right it's like a process So we hire people and then people join one by one and stuff stops up something like that right So I think this event was more I would say like a do set the vibe I think it's possible for Singapore to be close to the frontier and I think that we're having the true pioneers of AI here We want to give this a bit more like inspiring thing and also get let for quote and Jeff do me to people here And so Jeff was here last year about quote hasn't been fit for some time and he's going to have team here So it's like also nice to bring him around and meet people here.

Yeah, so it was a really like amazing event We met along as well and Who a lot of people don't know has a CS degree is like one of the few pms with a CS degree Yeah, I would say that the context of the meeting was more like partially like also He wanted to learn more about the I am off Start for really and then also about Jeff He wanted because they might have you without knowing that these guys were coming or something like that I was a business that is like the some quote that you have Jeff and quote and me What we went to visit the cherry at the stunner and We discussed a little bit on the thing discuss a bit like I am all and then I think the rest of it was more like Jeff and this is always talking more about like we became less about AI and that more about very macro Mechanical boy the good thing which was I was very our element with so I was just I just you were in the suit How I was just talking about the dick thing and I am always stuff that yeah, but he seems to be generally quite surprised that AI has reached this point So I didn't but he was a both interesting Yeah, I would say for people like you have done something that is unique in Singapore A series so far you like you are establishing a frontier research lab in Singapore Which is a accomplishment.

I think the other thing also that I guess I'm Still trying to wrap my head around is does geography actually matter like you're all working on the team You have your London people with a lot of new people and Mostly you're just like Collaborating with them anyway, you've collaborated with them your whole life I don't even really know what countries mean anymore when it comes to Research or just AI in general because this thing is just inherently International from the start this is a very good like question also is related to letting a boy identity Right because I think you were so moved from S7 Singapore quite a bit right.

I was in my model We were just like one two weeks ago and like I'm here, but like almost all my If it doesn't look at massive a side for my family like everybody I talk to is like somehow in the period or like that's because of work and everything I think the geography matters Okay, firstly the the most like things is like logistical is like probably the time zone So you literally want the 24 hour coverage around the world They are they are advantage not what one things that the difference is possibly for like it's also more like Like people define the location more the location defined okay the people somehow okay the time So we get to the time zone a bit like the important concept.

I think that's post-incons right The position thing only is just thing for people the talent pool I didn't remember you find like raising amazing amazing people But I would also have to say that this type of things is more like talent attract talent I think most of the time like people are very excited like the vibe I get is the people are very excited because it's Like costume and my team maybe working on like very core Things related to to a GIS so I feel like the talent we can get from the region It's really good by Sony It's only because it's asked so we can a lot of these talent otherwise might join some other place I move to to the US yeah about the identity wise I would say that I definitely agree with you that like why it doesn't matter like that I think that the advantages of single boy itself or like just anywhere like it's also that you are like Okay, the words very good so technically you can interact as much as you want you kind of solve that But I think single boy has this advantage where you can go close and you can go far I think the bearer is like So much about I have so friends in London in New York that will just never move to the bearer So I'm not against bearer.

I think bearer is a great place, right?

But it's just yeah, yeah, yeah, yeah, everywhere Right, I think sometimes if you have some like mental space and energy to like to have Some other culture and then like you know the London single boy New York they have their own culture right by the bearer culture is this like AI right Like you just go anywhere you just hear AI everywhere right even the billboards and stuff I think it would be much yeah Although I did see some billboards down here.

Oh, yeah.

Oh, yeah.

Oh, it's like what is this?

It's a culture infecting it's equal.

I don't think that to start a stand if you want to do research in You need a little bit of peace and quiet somewhere, right?

So this island may be good for that But then you can you still still like able to like be connected, right?

Okay, so I think that's mainly kind of wise.

I think people have Strong here Yeah, so far enough away, but he's still connected You have strong talent.

What are you hiring for?

He's still hiring right.

We're hiring like like my team will work out like our Our reasoning for Gemini and Gemini deep thing.

I didn't care more about like telling that to Do you know so we're not like also like growing that big this small Deen first just because computer computer is probably like important and yeah, so I think that's something that That we're hiring for now.

We see I personally just there's a lot of hands, but I think like Generally there's a lot of people like who are very capable But I think what I'm looking for many is I don't you have like a track record of RL research or like some Even not necessary the RL, but all like some exceptional achievement in like coding competitions All like some exceptional achievements somewhere Then there's like the kind of people that like we want yeah, because he don't strictly require I do remember something about your record days where you're like you like to train your own Videos from scratch Right, so they don't I for what if I say that or not, but Too fast then just do to stay yes, then yeah, I think we'll be definitely very happy with people They are like very high Stats and just like we wouldn't be that much.

She's just gone on No, no, that's really like the step point show with it like this high-tech that point people like this raw Like you like that that pattern people that is I think all like strong engineering skills Amel can be learned easily.

Yeah, I think maybe one version of this is can it be done on the student budget Where they wait?

Can you do something and interesting anymore on the student budget?

I would say relevant to the point where conferences of criteria these days I did do an interview with one of the best people winners Where they worked on thousand-layer neural network RL and That was done on student budget.

It was very cleanly executed pieces and paper and in good findings Look, I'm not sure if production models were ever go to the thousand-layers But they stretched it in an interesting direction and found some good recommendations and the guy immediately go high by opening I and I think that's encouraging for the grad students in the market where I like okay Well, do I need to know somebody who works at these labs in order to get in my uncle works there?

I get the internship, whatever No, actually you could just do it on a student budget with those good advisors.

Oh, I just really think one thing Interesting is that for most of the people that I she went to recruit them like personally Right, so it's I just you see the work and then you send them the ends, right?

So I get a lot of success paper.

No, no, no like life or hiring it and really So generally I have almost like I do your point is that you almost like you can just do good work put it online And then somebody will contact you right it's actually super easy, but super hot at the same time because No, I can tell you like I taught a few of these grad students.

They don't know what good work means Right because they don't know There's so many things their professors have their gender they're forcing on them Which let me not be right because it's not like their professors know what to work on either So yeah, they just think I did they just need to hey work on these like five things it show me Interesting result in any of them Okay, so if somebody comes out with something and then does something that you feel that it's very tasty and it aligns with what like researchers in The labs like like one and they come out with that independently you know that the function that brought this in It's good right like if you just go and tell somebody to do this like you you can you just get the signal that is guys Can execute right so I think there's some value and people that Yeah, they demonstrate taste research.

Yeah, yeah, very interesting.

I feel like I could give people Yeah, I do care about this in some ways the research direction is work that I do is Kind of build that like it's low accountability for me because obviously it's just But experiments, but I think for a lot of people it's like their career is Founded by can you demonstrate research taste with this like short before years or that you have and just do it Yeah, yeah, I would say then this is more there's so much competition just because of like the everybody wants to get into AI It's just more of like how to Like mostly it's more like how you're going to prove yourself Yeah, it must be hard these days to be a great student trying to prove yourself it's definitely harder But yeah, yeah, now you drop.

Okay.

That was it.

Do you have any other sort of friends or topics that you had to queueed up before we wrap?

I don't know but it was great.

There was a great time.

It's fun chatting with one fun chat.

Yeah, you know even I even love last time We were supposed to do not say we meet at the symposium with supposed to record But even we just ended our hanging on trading.

It's just nice to get the debris dump of what's going on in your world Because yeah, we're working on really funny stuff.

I'm always great to chat with you and since I see you Yeah, good to chat partying words on the sort of weight loss and work out journey Because there's also a big thing for you.

I think being healthy is important to be to do research Right, and I think I think I've been probably in one of I probably in a pick physical health now Yeah, I look great there.

Thanks.

And I think it's also impact on my work in a good way you did the sort of cup of tea inspired by you hacking I didn't go to a stream, but I was white also quite Data driven when I came I would like have my own about and then I would track this Then I was I'm still supposed to make a blog post about this, but I feel I'm not like really at the end game yet So like when I get there, but yeah, just to just for people who don't know I like I think I lost 20 3 kilos this year actually across one year yeah one half years.

Yeah, so the industry yeah It's literally from the last forecast to now.

Yeah, yeah, it's that's the abolition study now 20 kilos yeah, and I think like my HRV hardware variety has went up by two times And my rising hardware has dropped by 30 beats per minute 30 beats per minute Like it was like 80 90 and now it's 160.

Oh, we are 80 many super high.

Yeah, I was unhealthy.

Yeah Okay, yeah, so I think anything when it's hard like what do you have a thing that can be going?

You know a lot of people they maybe they're they're focused on yeah and including myself right I do prioritize work.

I enjoy work.

I don't enjoy the fitness side, but obviously it It feeds into your intellectual work like the sort of log-off and go for work, eat better All that kind of stuff.

Obviously feeds in but like people seeing positive example like you They will get inspired to do the same thing.

So I think it is good to set yourself up.

It's an example I think there's a house like when I do this thing for my health.

I just think that it's also part of work because It helps me to get better in my job.

So it's important as well.

I think it's important as well.

Yeah, I like the HRV of the better.

I have no idea what mine is But yeah, there's a general question about what is productivity and how do you measure it what really matters and it's still unclear to me But I do think general energy level and hunger almost like you almost have to like experience Physical hunger in order to have intellectual hunger and I don't know if that's like I think the way that hungry you're just think of food.

So I think to me it's like this is destroying things.

But why you why it's hard to do what when you're hungry.

Yeah, okay Thank you so much.

Yeah, thanks.

It's really great.

Yeah, have a great time.

You