Inside Google Labs: Building the Jules Coding Agent with Jed Borovik

Source material

Transcript

[MUSIC PLAYING] OK, Jad Borovic, welcome to "Lanespace."

Yeah, thanks for having me.

So we're sitting here at F.inc's beautiful podcast studios.

And we're actually meeting at GitHub Universe.

How's it been so far?

It's been great.

I mean, yeah, the keynote today was awesome.

It was fun to see Jules up there a little bit.

We have a lot of folks from our team here.

Jules is, yeah, partnering with GitHub for the new agent HQ stuff, which we're excited about.

And also, this is an incredible podcast space.

So yeah, I'm excited to do this here.

I'm glad for them to loan us this space.

You are also an MC for AI Engineer Code.

That's exciting.

In New York, where you went to college, but you don't live there anymore.

Yeah, I know.

I spent a bunch of time in New York.

You know, it's funny being part of the New York Tech scene.

I actually think it's great having big major conferences there.

So there's a lot, obviously, that happens on the West Coast.

But being someone to tech on the East Coast, it's just awesome to have stuff there.

So yeah, you mentioned you fly over to SF a lot.

And what's the scene like in the East Coast?

Obviously, we are pretty new.

This is our first year coming to New York.

What else happens in the New York?

What are the highlights for you in the New York Tech scene?

Yeah, I mean, there's so much.

There's obviously a ton of great companies.

I think the thing that's interesting about New York is it's such a big city with so much going on.

And so your tech is a huge part of it.

But there's also so many major issues there, whether it's-- SAS and finance.

It's always, I think, that helps push the tech and do all kinds of stuff.

But yeah, the East Coast is a great city.

All the schools, there's all across the East Coast a ton of great schools and great students doing all kinds of stuff.

So yeah, I went to school there.

The hackathon scene there was amazing.

Really fell in love with tech and programming there.

Is there a big NYU hackathon in Stanford where CalHacks and stuff?

Yeah, so there's-- But tree hacks?

Yeah, there's one that was put on by-- this was a while ago-- but were put on by NYU in Columbia.

We do a hack and why.

So there's a bunch of events that we did together that would bring people across New York City, students across New York City.

And those are super fun.

It'd be at Columbia one time, then NYU the next, and recycle back and forth.

So yeah, a lot of cool stuff was made there.

Nice.

So you've been at Google for a while, nine years.

You worked on a bunch of things, including with Malta, which is also another guest that I'm interviewing today.

How do you get into Jules?

What's the AI journey?

Yeah, so this is going to sound really cheesy, but I've told a story a couple of times to folks when they're like, oh, how'd you end up doing this?

But it is actually very true.

So I worked on Search for a long time, and specifically news and freshness.

And then when Stable Diffusion came out, that to me was the first Gen AI moment.

I know some people talk about chat GPT as the first thing, but for me, Stable Diffusion was a couple of months before chat GPT came out.

It was a huge thing.

I was falling at a ton online, and there were two groups of creators having reactions to it.

There was one group that was, this is stealing my art.

This is stealing everything that's near and dear to me.

I hate this.

This is ruining my life.

And there was another group of artists and creators who were like, oh, this is a tool to create better art.

And so I was watching-- It's a new brush.

Yeah, exactly.

And right around then, I was having conversations with a couple of people who would say things like, if I had a kid in college, I wouldn't recommend they study computer science.

I was like, what?

And this was long before Jensen Huang and people had been saying this kind of stuff.

I was like, well, why?

And it was like, oh, AI, software engineering is going to change.

It's going to be so-- who knows if they're going to be jobs?

And I was like, I love being a software engineer.

I love programming.

And I was like, wait, this is my Stable Diffusion moment.

This is either it's going to take my art, my craft, or this is a tool to create better art.

And I was like, I definitely know which path I'm taking.

So I got very into building coding.

And I was still working on search.

But I spent a bunch of time making stuff for my own time and playing with things.

And you ultimately tried to find a role that would-- the most exciting role I could find to do this stuff.

And that was to join Google Labs and Jules, where we were right around then, we were starting to build these kind of coding agents at Google.

And yeah, the timing worked out well.

And I joined.

And yeah, it's been awesome.

Can you-- since we're talking about Google Labs, I am actually unclear about where Google Labs starts and the rest of-- and then DeepMind and the rest of Google.

Like, what is the org chart layer?

Yeah, yeah, yeah, yeah.

That's a great question.

So Labs' mission is to build kind of new, like, innovative products that the rest of Google isn't well positioned for.

Yeah, which we've had, like, the rise of an Opa-Gel M.

Yeah, exactly.

Exactly.

So Opa-Gel M is maybe the most wide-- The most-- yeah.

And then it's called Nano Banana.

I don't know if it's-- Yeah, so some of the-- so the thing about-- that's really exciting about Labs is we work incredibly closely with DeepMind.

Yeah.

So all the stuff in terms of the-- we're building a product, but we work so closely for the model.

And one of the nice things about being at Google is you have this opportunity to really build an end-to-end AI product, right?

From like, pixels on the page, through the infrastructure, through the model and the training and all of that loop.

So Labs is here to build new products.

And we're really like a product org, but a true AI product org, where we work incredibly closely with DeepMind, but also other parts of Google as it makes sense.

Yeah.

Just on the history of AI coding, I had heard that actually Google had an internal version of Copilot or something like that that was never released.

Is that true?

What can we say about it?

Yeah, so I think there are-- Google has published papers in this space for a while.

And so yeah, we have-- in Google, we built a lot of our own tools.

And Cider, which folks maybe have heard of, is our internal IDE.

And we've had all kinds of capabilities and tools there for a while.

So yes, we certainly have had pretty good tools for a while, but they were for internal use.

Yeah.

Yeah, I think it was interesting because I think one of the hype moments when Google started getting into the sort of like LM game, like basically when everything rebranded to become Gemini and starting to push out Gemini, where people were like, oh, did you know that Google-- probably Google's entire repo is probably about the same size as GitHub.

And there must be some interesting data in there.

Oh, yeah.

I mean, and that's one of the things in building a lot of these internal systems.

The data is incredible, especially when it's not only is the model and the training in-house, but all the data on the usage and whatever.

So we could build really kind of sophisticated things there.

Yeah.

OK, so let's introduce people to Jules on your website.

Jules Autonomous Coating Agents.

We've seen lots of these.

They're not octopuses.

They're not purple.

So you got that going for you.

But what really is the core thing you're trying to nail in a very crowded Coating Agents landscape?

Yeah, so what we think about and what we set out to do-- back when I joined, it was like, where are coding agents going to go?

And as these models get more and more powerful and sophisticated, what is that experience going to be?

And let's build for that future.

And so when you think of a really powerful agent that can run for a really long time, doing really complicated things, that's when the products start to take shape for us.

So for example, Autonomous means it has its own computer.

So for Jules, it's the end of the-- Exactly.

So tons of agents that run locally or in your workspace with you while you're coding.

But if you want something that's going to run for hours, let's say days, you might want it to have its own environment where it can do its own work.

So that's just one of the pieces that's important for this Autonomous Coating Agent.

But it's really like-- think about this future where they're incredibly powerful.

You can spin up tons of them.

They're autonomous, but also we're thinking about what does it mean for it to be ambient?

Like it's kind of-- when it has its own infrastructure, its own computer, its own ways to interact with it, how does that start to change what it can do?

For example, we have an API.

So people are using it for all kinds of things, triggering it from when something happens.

And we saw an example where someone has-- they're triggering Julesford to do all kinds of updates to their site.

And then they have a GitHub action that is going to automatically merge Jules and we have the Jules CLI we launched a couple weeks ago, which lets you interact with it.

By the time this podcast comes out, we'll be integrated with the Gemini CLI.

That's what I was thinking.

Like you have a number of CLIs.

I'm not sure.

Exactly.

So Gemini CLI, all kinds of places where we're going to kind of mix and be able to harness this power, because developers work in all kinds of spots.

So making it easy to have this autonomous, ambient agent that can really do all kinds of work for you.

Yeah.

What is your journey?

When you started out, did you find any assumptions that were quickly challenged when working with Gen.

AI and coding agents in general?

I guess you're maybe not too unfamiliar with it, because Search uses a lot of machine learned, like, black boxy type things, including BERT, which was a major update a few years ago.

Yeah, so just fill us in.

What is your AI engineering journey?

Yeah, totally.

So I think one of the things that keeps coming up is the model makes such a difference.

I mean, maybe it sounds obvious, but it's like the quality of the model really changes what you're able to do and how you engineer around it.

So for example, when we started, this was with relatively early models of Gemini, the agent scaffolding around it was incredibly complex.

I think one of the things we've seen is the scaffolds get simpler and simpler over time as the models get better.

And in some ways, the scaffolding is almost a crutch for things the model struggles with.

For example, really complicated subagent systems.

We've played with that.

We've experimented with that.

Can you give an example of a kind of subagent that you had to abandon?

Yeah, which is basically like you have-- you give JUULZ a coding task to do, and it's going to have different agents for whether it's making a code edit or handling a subproblem or doing any kind of action with an integration and having full subagents for different parts of it.

Like a reviewer agent or even like people-- sometimes people do these different personas where you're like, one of the things that cracked me up was like, you're the product manager agent, and then you have the code reviewer agent, and the lead agent, and we didn't go that far.

But I think a lot of these things aren't as in favor.

I mean, certainly people do-- I don't want to say the agent harness isn't sophisticated.

It certainly is.

But as the models get better, like less is more, especially as it comes to being able to improve through whether it's machine learning or just regular maintenance.

I think certainly we found that we're finding that less is more.

I think that we were talking about a little bit before we started recording, like RAG and code-based indexing and all that stuff.

It seems like not just for jewels, but kind of across the industry, that agent-based search, it's maintaining embeddings is hard.

But getting the chunking right is hard.

In terms of the black box aspect you mentioned, a lot of that is hard to improve upon.

Yeah, I would even say it's maybe not even hard so much as it will never be good.

Well, tell me why you say that.

Never.

Because a chunk that happens to capture the thing you're looking for will fail to capture something else.

And so if you only retrieve based on your embeddings of a chunk, it uses very arbitrary boundaries that are drawn with some hope of the semantics being captured.

But you could just throw attention at it.

And you can scale probably much better using grep.

So I think that's an example of these harnesses how they're simplifying.

Yeah.

Well, I haven't abandoned it completely, because one of the things that we were doing-- I don't know if you saw the cognition suite grep work-- was basically using semantic search and chunks with embeddings as a tool.

But on the same level as the other tools, like code rep and file access and glop and whatever else other variants you have.

So I think that makes sense.

Don't abandon it.

Just don't reify it into the only way to do things.

And to be clear, this is the area of research we're doing tons of work on.

And I actually expect in the coming months we'll be talking about some stuff we're doing here too.

But it's not the-- I feel like when we started, it was like rag.

It was like embedding-based rag.

It was like the thing everyone did.

And it's interesting to see how it's changed.

People asked me for where are the code embedding models.

And I pointed them to a few Chinese ones.

There are some-- Nomek was working on one.

And then we found that we didn't need them.

Yeah, exactly.

Exactly.

Very bitter lesson.

So I think these are good things.

I think when Jules came out, it was kind of a preview.

I mean the trusted testers group.

So I got to see a little bit.

But now it feels like more of a real product.

What's that transition like?

Is there a process in Google Labs to promote things when you feel like there's this attraction?

Yeah, absolutely.

So I think the Google Labs is not about just experiments.

So like Nomek, we thought it was-- It's not very serious.

It's a incredibly successful product.

It's really money.

It's really, yeah.

And for us, I/O was kind of a little bit of a turning point.

In May, when we announced Jules, it was like great reception following I/O.

And that was a real moment of us to turn this into very much a real thing.

I mean, it's something that we always intended to.

It wasn't ever intended.

I talk about my journey.

It was always a goal to build a real product here.

But for us, that was kind of a very key moment, very key milestone for us.

And so yeah, now it's very much a real thing.

As mentioned in talking before, Jules being talked about in the GitHub keynote, it's certainly here to stay.

We're excited to kind of keep building and expanding.

Awesome.

Let's talk about just the coding engines in general, your coming to MCD, AIE code summit.

It's going to be your first time at AIE and the MCE.

What do you want to know?

Yeah, yeah, yeah.

Well, tell me, why would someone want to-- Oh, guys.

Yeah, this is-- yeah, let's turn it on.

Oh, boy, this is embarrassing.

So I mean, fortunately, we're in our third year, fourth year now.

And we have a bunch of prior art.

We can just point people to it and say, look at our YouTube.

You like that?

You like this?

There's some great talks that I haven't been before, but I've watched the talk.

Yeah, yeah.

This is a lot of good stuff.

Yeah.

And I'm proud that it features content from all labs.

And basically, we are like the-- this is a pattern I've seen across my career in terms of every industry needs its focal gathering points to just trade tips and stuff.

So I've seen that in JavaScript.

I've seen that in Cloud Native.

I've seen that in data engineering.

And I was like, probably AI engineering will need something like this.

And then also, the concurrent thread to this was I went to a bunch of the academic ML conferences, NeurIPS, ICML, iClear.

And a lot of them, like NeurIPS is 40 years old, and that hasn't really changed.

And it's very focused on academics and PhD students.

Whereas, I think, really, the transition in AI, going from research to industry, is that you gradually see a shift, unfortunately, less open source, less papers, and more products, and more startups, and closed models, and what have you.

But people still want to share.

People still want to hire.

They want to promote their work.

So they need a place to do that.

You can always do that at your company conferences, obviously I/O, and GitHub is GitHub, and Microsoft is Build, and Ignite.

But there usually is one place where it's the industry-neutral thing, where everyone is on the same playing field and me the best person to win.

And honestly, some people like that.

You're not going to be treated as the VIP.

You kind of have to earn your spot.

But when you earn your spot, I think people give that requisite level of attention better, because you had to.

Yeah, yeah, cool.

Let's say I've watched the videos online.

I kind of get a sense for this, but what's happening between that?

For someone who hasn't been before, what goes on other than the talks?

Oh, yeah.

A lot of, well, just logistical stuff, of invoicing, and vendor selection, and venue selection, and did you know we have five different pieces of software to coordinate speaker logistics, and booth logistics, and AV.

But what I was going to do, an attendee.

So I'm going to go.

Oh, yeah, sorry.

Yeah, what am I going to get?

So actually, it's really weird, because I'm the content guy for AIE.

I curate the speakers.

I invite them.

But I actually know that the content is the least important part, because all of it's films, and we're going to edit it and post it for free on YouTube, anyway.

But the reason you come is because, one, you can talk to the speakers, but also you can talk to each other.

And so I always say the hallway track is the most important track.

How do you get the most out of the hallway track?

What's your guide?

Begin a more hallway track.

I don't have as collected a thought as I should.

One, I think if you have some prior history of what you're interested in and work on, so basically the best intro to somebody is if they've seen you online before, so they can skip the whole who the hell are you part, and just get into, like, oh, yeah, I saw you wrote that thing.

Let me talk to you in person about this.

They're both here.

That's way better than who are you, what do you do?

And that's a very cold interaction.

Ideally, people come warm, or they can come with some clear idea of, like, here's why I'm here.

Here's what I'm looking to get out of this.

Because I think if you show up with no real intention, or if you're in and out for your thing and nothing else, then you don't have the space and the mental energy for the unstructured serendipitous connections.

And the thing about EIE, at least in our scale, outside right now, especially for the summits, which is the one that you're going to, everyone had to apply to get in.

So usually, our first summit, we had something like a 10 to 1 applicant to invite ratio, invited spots ratio.

This one's going to be-- it went up to 10 to 16 to 20 something.

This one's going to be 23.

There were 23 people who were invited in.

Yeah.

So yeah, it's a lot.

I think, like-- and really, it was trying to filter for people who would be speakers at any other conference.

They are at the top of the field.

They are either founders or, honestly, enterprise buyers of the best companies you can find in New York.

And that's another reason for our New York conference, which is we're bringing the best of San Francisco or tech to the finance sector, really.

There is a little bit of media, but mostly finance.

And yeah, that's great.

So what I'm trying to say, I guess, you're there to meet the other people.

So make time to meet them.

Have a calling card, like, who are you?

Like, a quick, like, what are you?

What do you do?

What can you help with?

What are you looking for help for?

That kind of intro stuff is really good.

Going with friends is really good.

Obviously, like, we actually offer-- for the world tour, we offer bundle discounts.

This one, I don't think we do.

But just reach out if you need something.

But yeah, I mean, like, I think, like, the idea of getting immersed in the code agent community is really important.

And I think maybe the last part of bringing up is that we themed it for the first time, right?

So you used to be-- these are just generalists.

Here's the state of AI, the best because we can get at any point in time.

But now we're really trying to push ourselves to theme everything.

So we have the best people in code, the best people in data sets, the best people in RL.

I want to do a Mac and Terp one.

That'll be fun.

Cool.

That one I'm thinking will be in London, because the people I want to target are in London.

But yeah, I think when you do a summit, it should be focused.

Everyone there should have an agenda of trying to learn what's the state of the art, trying to have off-the-record conversations with their peers, doing the same thing at the other companies.

And who knows what could happen?

That's the weirdest thing.

I organize the thing, and I don't even know half the things that go on, just because my job is to provide the nexus of people to just connect.

Last time we were in New York, there were 13, maybe 15 side events organized by people, just dinners, meetups, whatever, around the summit.

And we encourage it.

We just want people to meet up.

Yeah, I was going to ask, is there a whole off-menu set of events happening?

How do people know?

They organize it.

Honestly, if you're not scared of strangers, you should organize your own.

Like a little dinner.

We leave all the evenings open.

So just organize a dinner or a meetup, focus on your thing.

We have people doing only voice.

So if you want to do voice, great.

If you want to do code review agents, as a small subset of generalist coding agents, do that.

And I think you'll find it.

Or you can do AI in finance, AI in bio, whatever this particular sector might be.

And I think that is honestly the highest signal way to get a bunch of people who really resonate with your thing to meet and have high bandwidth conversations.

Yeah, yeah.

Are you going to do the autonomous coding agent dinner?

Well, no.

My job is to float.

You're the best at it.

Yeah, yeah.

My job is to handshake, ask how everyone's doing, fight fires.

So I tend to just leave myself open until the end.

But yeah, it will be a sprint.

It's always a mad rush.

And because then I have to do my own talk.

And I don't know yet.

I think so far-- so the last time I did this summit, I was talking about how this year had to develop as the year of agents.

And it's really played out a lot.

Obviously, now the trendy thing is to say, it's not just a year, it's a decade of agents.

But this year, I think agents really took off and most people got it right.

The consensus was correct.

You don't have to be too spicy or counter-consensus to say, if you worked on an agent, you're probably a lot better off.

You probably made a lot of progress this year.

And maybe you can tell me how it feels on the Jules point of things.

I didn't see myself at the start as you're joining an agent company.

And I ended up doing that.

But I've gone so agent-filled to the point where people come to me with startup ideas for infra companies.

They're like, what if we made agent framework so that other people couldn't build agents?

And I'm like, well, don't you just build agents yourself, bro?

There are a lot of these frameworks.

Frameworks and infra companies.

And all of these guys are just like-- they're good developers with no conviction whatsoever in what they want to build.

They don't know what customer they want.

They're just like, we want to build developer tools, so that's why we feel comfortable.

But honestly, it's not that hard to actually take a stand and be full stack and verticalize in some particular agent field that you want.

Because guess what?

The business and the economics are aligned that way.

And I'm not saying that you cannot make it as an infra company.

There's some fantastic infra companies that are sponsors and that I admire and I would invest in myself.

It's just that comparatively, those are a lot harder.

And agent companies seem like they're shooting fish in a barrel.

They seem like they're ramping up in AR a lot faster.

And you seem like their margins are better, so why not?

Yeah.

So I mean, I think for us, it's really been-- you're the agent.

As the model-- what was your term?

What is-- let's build drills for where things are going.

And as the model's got better, I think it just becomes clearer and clearer that agents are super powerful.

We have-- you were talking about before high context and management and all that stuff is important.

We hit people.

We had-- this is a funny story.

We store some data for a session, but it only lasts-- we only store for 30 days.

And so after 30 days, your session becomes locked.

And when the first user starts hitting that, they were upset.

We were like, there's no way anyone's going to be using a single session for 30 days.

Like, people do a single track of work for 30 days.

But you're just like, how powerful that could be.

And how do you compress context when you run into it?

Yeah, so we have-- I mean, I can't talk too much about it.

But we do a lot of the standard things.

And it is also-- we're developing a bunch of stuff.

It's an active area of research for us.

I think like-- just to-- I'm not asking you for how exactly Jules does it.

There's just a number of approaches, right?

And you just have to pick one.

Because you can't just use up your 2 million token context window.

Is it 2 million?

It is up to 2 million.

Especially for coding agents.

Because you're reading files.

So you're running commands with huge outputs.

I think coding agents are a really interesting area, both product-wise and the impact they're having, but also for research.

They really push the limits of what other domains are you running an agent for 30 days?

And what other domains are you accumulating so much context in so many turns?

And so it's-- yeah, coding agents are, I think, a special spot of super interesting product, impact, research.

Yeah.

I see the AMP folks drop the auto-compaction for a hand-off mechanic, which was pioneered by the agents SDK, which is basically the sub-agents pattern where you speed up a sub-agent and do a thing.

You don't need all their context that a sub-agent is doing.

And then you can come back to the main thread.

Totally.

Yep.

You have your groups.

Yeah, it's a good pattern.

I would also add this challenge is having to make sure enough stuff, your information is going back and forth.

But that's the pattern.

Summarization is a pattern, kind of externalizing some of that context, where it's like writing it to a note kind of thing is a common pattern.

So yeah, there's tons of things to try and do.

Yeah, yeah.

And one thing I do want to get more consensus about is what is the best?

Because I don't think I've read any papers about which methods compare better.

Yeah.

It's also just like as models change, the answers change a little bit too.

Yeah, yeah.

Claude, you probably know Claude externalizes too much.

Yeah.

Yeah, yeah, beep, beep.

How much does your work actually-- I feel like I switched back to Jules mode.

Yeah, yeah, we're flowing here.

Well, I mean, how much does your work inform the model creation, right?

At the end of the day, you obviously are a very big consumer of Gemini models.

But also, you are not the only consumer, and they have other priorities than you.

Yeah, totally, totally.

I mean, I think we're lucky in kind of how we're positioned.

We have very close relationships with DeepMind.

So we have-- and coding agents are an important area.

Let's be honest, right?

For any kind of company building models, you can see it in all the labs.

Coding agents are important.

Coding capabilities are really important.

Yeah, my OG image of the AI code-- I wrote something obnoxious.

Code is the first spark of AGI, which is probably true.

Totally.

It's important from an AGI perspective.

It's important from a dollar's perspective.

It's important for all of it.

So I think we're in a really lucky position.

We're able to have a lot of kind of good collaboration.

But both ways, all kinds of capabilities that are being developed.

And it's interesting.

It's a whole host of things, right?

Because in terms of AGI and the capabilities of things, it's also like computer use models and browser use models.

And so it's models of output code.

But it's also the whole suite of things that you want an intelligent agent to be able to do.

So it's multimodal.

It's all kinds of stuff that goes into it.

What would you want to find out from your peers at other coding agent companies?

Because you're going to meet all of them, basically.

Yeah, so I think one thing-- and I don't think of this as a zero-something.

I think this is really like there's this tide that's going to lift all of our boats.

And we're inventing a new way to do our art and how to create good art as a software engineer.

And so what does that look like?

And how does that feel?

What is the experience we want to create?

I think as people working in AI, sometimes we don't do a good enough job describing this beautiful future we're creating.

I know the CEOs and heads of these labs have started writing their think pieces on this.

But for software engineers, what is this beautiful future we're creating?

And I think it's inspiring.

It makes it maybe less scary for people who are thinking about these tools.

But also, if we can't articulate it and think about it, it's less likely we'll get there.

So what is this great place we want to create?

The grinding software is so hard.

There's so many companies.

It's such a-- especially big companies.

It becomes so challenging to manage a code base and create.

And what can we do to make being a software engineer an absolutely incredible experience?

And how do you want to interact with your model?

How are you doing things locally versus in the cloud?

And how does that interop?

So I think as an industry, we're trying to-- which has changed.

We're in some ways inventing.

And there's this movement to change how we do our art.

And the better we can create this experience, we all win to some degree.

So yeah, I think that'd be one thing where it's like-- yeah.

The local to cloud sync is the most contentious or important, I guess, topic for a lot of people.

I wonder if we'll ever get some kind of interrupting.

Probably not.

But a man can dream.

[LAUGHTER] Tell me more about what was your dream?

What's your dream flow here?

I don't know.

Start with JUUL CLI and up in Devan.

I don't know.

Oh, you're not between age.

You think you should name these guys?

It's probably meaningless.

No, but I'm not actually serious about it.

But like-- [INAUDIBLE] So I think codecs-- or is it called Cloud Code?

Cloud Code Web can do this teleport, where they just basically dump the entire history.

And you can pick it up in Cloud Code on your desktop.

And probably that's the right move.

Yeah.

Maybe there's some more elegant things, but they were first.

So why not?

And actually, maybe the real thing is maybe it's not the conversation.

Maybe you don't need to teleport if the artifact that you pass back and forth is the linear ticket or the GitHub PR.

So you don't need the full JSON.

You don't need the full chat history.

You just need to pick up where other people left off because that's how humans do it.

Right, right, right.

I don't transfer my brain state to you.

I just tell you what it did.

And then if I forgot to say something, you find out eventually.

Right, right.

You see, the cloud agent dumps some kind of summary onto the ticket or whatever kind of it needs to pass on to the next-- - In Slack or in there and whatever.

Yeah, that's interesting.

We have-- there are some patterns emerging, like IDE, CLI, cloud.

Right, like these are the pieces-- - VS Code extension.

- VS Code, yeah, like whether it's-- - You guys don't have yet.

- VS Code extension.

There's a-- the surface area is like standardizing, it feels, a little bit.

And yeah, how these things that are up, how you can kind of make this great experience with all of those.

I think it's really interesting.

- Yeah.

Yeah, I think like-- and then the other point, I just want to backtrack a little bit to something else you said, which is like what the thick pieces that the-- - Yeah.

- These CEOs and stuff do.

I think there's a lot of question about the impact that coding has on the software engineer industry in general, the humans.

Do we end up-- do we stop hiring juniors altogether?

Do we-- is it actually increasing productivity or do you just feel like you're increasing productivity?

I don't know if you have any take on that stuff.

- Yeah, it's totally so.

I mean, I-- some people spend a lot-- I spend a lot of time talking and thinking about with folks.

And you know, I also spend time talking to people at companies.

And you know, I think sometimes working on these tools, it's interesting to see-- it's not as like diffused.

This technology isn't as diffused across software engineers as I sometimes expect, right?

There's plenty of places that I think are not really using AI a ton.

A lot of companies, a lot of software engineers aren't.

That being said, I'm very kind of excited about what this-- what the future of software engineers are.

Like, could you imagine going back to not having these tools?

No.

That sounds horrible, right?

Like that-- and so that's one aspect of it.

I also think-- you know, I don't really buy this like, you know, that we're not going to hire more software engineers story, I think, like, for a few reasons.

I mean, this is an example that often comes up, but is it like kind of the elasticity of the demand for software?

Yeah, Jevon's Paradox.

Exactly.

And you know, like, a lot of the cases sometimes come up as you look at like farming, right?

And so there was a time in America where like the vast, vast majority of Americans were farmers, right?

And then technology happens, and today it's like less than 1%.

Yeah.

And that's one example.

But the flip side of that is your electricity, which like, as that gets cheaper and cheaper, people just consume more and more and more electricity.

And with food, there's only so much food we're going to eat, right?

There's a kind of an inelastic demand for that, whereas just a very elastic demand.

It seems like software, you know, software keeps getting better.

But like the ability, like we're creating more and more software from like, you know, obviously punch cards to where we are today is like remarkably different in terms of how you're able to create software.

So much more software is being made.

And software just keeps becoming more and more of our GDP, right?

Like it's a-- so I'm bullish on kind of the amount of software we'll be able to create, how it'll be created.

I think there's also something here about, you know, as an engineer, being able to be more productive, like, encourages more investment in people building software, right?

If it's, you know, the job of a software engineer can now, you know, they can do 50% more, 100% more, 10x more, like justifying investment dollars into projects like dramatically changes, right?

And so, yeah, I'm bullish on this idea that it's actually going to be great for softwares, both for our ability to kind of do our craft or art, but also just what it means for the number of companies and the amount that's made and the quality of it and what we're able to do with it.

So, yeah.

That's a nice rose-colored glasses take.

Rose-colored glasses indeed.

Yeah, I have this take on the different kinds of work.

Like we're splitting up the different kinds of software work.

And there's a lot of commoditized work that we used to spend a lot of time on.

And now we can basically entirely delegate to agents.

And then that leads us, ideally, for more strategic, important, novel, high-risk, whatever, work, deep-focused work that is something I posted here on the semi-async value of death, where basically you kind of need to-- on the extreme end, you can delegate to async agents, which is jewels, you know, cloud code, whatever.

But then over here, you kind of need the sort of deep involvement in understanding the code base and not vibe coding and whatever the opposite of it is.

Actually, that's my talk, which is-- I've been thinking about this.

So I tweeted out this phrase, because I think it feels in the air that the term vibe coding was obviously coined by Andre, and he's super influential, in February.

And people have just come to kind of use it as a blank check to just yolo on prompts and stuff and then create the worst code imaginable and leave other people to clean it up.

So I think people are kind of at their limits with this.

It was probably maxed out in terms of popularity.

But we don't have yet what's next.

So my talk is really challenging every attendee, every speaker, to come up with what is the aspirational good version of vibe coding that we can actually trust.

Yeah, what is it?

Well, the punchline right now, what is it?

I mean, the current leading candidate is agent decoding, which is what Darmesh Shah, who's like-- I don't know if you know who Darmesh is.

He's pretty good track history when he's naming things.

It's just too many syllables.

I don't think it just has the-- it doesn't have the joy that vibe coding invokes, which I think people want.

But then people also want care and craft and reliability and all that stuff that-- But if we don't have the term I would describe, but maybe we don't have the catch this phrase for it.

What does it look like, even if we don't have the phrase?

Yeah, that's a great question.

Well, we have some speakers who are going to be pitching spectrum and development that you have to really be thoughtful and effectively write a PRD.

I think that is obviously correct in terms of like, basically, it's just a glorified prompt, but a very, very, very good one.

And models are tuned to follow your prompt, for good and for worse.

And if you prompt sloppily, you're going to get slop.

So a spec sounds good, I think.

I don't know how often it will be followed in practice, because effectively what that transitions us to is a waterfall development approach, where you spend three days writing a 50-page document, and then you kick off the agent.

That doesn't seem right.

So obviously, I have some advice here, because cognition has from the start believed in interactive planning, where you kick off a thing, you get some feedback.

And then you're like, oh, that's not what I meant.

Let me correct myself, because I don't know what I wanted when I started.

So you work with the machine to discover what you wanted.

And the machine works with you to either get you what you wanted or show you the areas of your ways, and then you correct it from there.

Yeah, I mean, one thing we talk about, which very align is what you're thinking, is like, there are kind of like two problems as these things go.

But one is like, how do you specify what you want?

And the other one is, how do you verify that what you got is what you're-- Yeah.

Yeah.

And so yeah, whether it's specifying to respect or this interactive plan or whatever it is.

But then yeah.

And then on the flip side with the vibe cutting thing is you might specify, but you never come back and verify.

You're like, it's more hands off the wheel.

Maybe I'll click around the app a little bit and see how it works.

But I'm not really engaged with the code.

So how do you verify and making sure that it's-- To my knowledge, you guys don't emphasize tests that much, right?

It's not like you volunteered to write my tests.

Yeah.

It depends like we-- if there are tests in your code base-- It's right.

It's right out of the picture here.

Jewels around your test feed.

Exactly.

Exactly.

So-- But it's not like after everything, everything must have a matching test to the prompt that was mentioned.

No.

That would be the extreme of what we mentioned.

I don't know if people always want that.

Maybe it would be helpful to do that to kind of show that it was right.

But let's say I don't write tests in my code base.

Like, I want to merge that pull request that is introducing tests just for this one thing.

Like, I think in some ways the engineer should be able to control what kind of outputs they want.

If it helps, then they want it.

Absolutely.

And then do you think there's other innovations on specifying apart from just chat?

Oh, totally.

Totally.

I mean-- Agents, MD.

Yeah, agents.

I mean, the spectrum of development, I think, is in this category.

I think one of the reasons is multimodal.

Like, if I'm going to show you a bug on our website, do I want to come and type it with words to describe it, or am I going to point the picture?

Yeah.

And so with Jules, you can upload images now.

But kind of more-- we have certain ways we communicate as humans that are easier in certain situations.

And let's just bring that to our engagement with the actions.

So of all people, I expect you guys to be best at this, because Gem and I has video understanding.

I want to submit a video, because some things like do cannot be screenshots.

It's more about the behavior of things appearing and disappear.

Yeah, I mean, I would love that if you guys did it.

Because no one has it yet.

I know.

I would love it too.

Oh, yeah.

I'll tag you a little bit.

On my side, the version of that that we're exploring is computer use.

Computer use was kind of introduced by Anthropic.

And then OpenAI did their towing with Operator, and now Agent Mode in Atlas.

I don't know if you guys have done anything super splashy on computer use.

But anyway, it's coming back.

I can feel it.

Yeah, I think, yeah, definitely.

And it ties into coding agents.

It ties into just using AI systems in general.

But basically, your VM now needs to render a UI or a browser.

And then you need to let the agent click around in it.

Absolutely.

And you need to have precision and speed and cost and affordable cost.

It's a lot.

Yeah, I know.

I mean, what can I-- projects are so fun.

There's just so much to build.

There's so much-- I think also as a software engineer working in this space.

I think one of the reasons you see so many companies in this space is partly like it's just so fun.

There's so many things to build.

There's so many tools that seem like fun sci-fi.

It brings up a demo of what I've worked on.

It's clicking around.

I can see a video of it, or I can even take over and use it.

So yeah.

Awesome.

OK, so just moving towards wrapping up, if people run into you at AIE, they've heard your pitch out jewels.

Yeah.

What else should they also talk to you about?

What can you help with versus what are you looking for?

Anyone should feel free to come up and talk to me at any point.

You're obviously very interested in anyone who's doing stuff with coding agents, or someone who's using coding agents in an interesting way.

So I'm always curious about workflows people have with their data agents, whereas whether it's, hey, I'm using this tool in this way, and I've configured this crazy thing, I always love hearing how people are using it.

I also love hearing people who are having bad times with it, where it's like, actually, maybe they're not coming to this conference.

But I've tried all these tools, and I don't like them, and I don't use them, and here's why.

So I'm totally open for any side of the-- all the way from full AI pill and coding AI lovers to people who hate it.

As far as I'm looking for, I think really just going to connect and meet people, I think we are always hiring.

So anyone who's interested in working on this stuff, I'm always happy to talk.

But yeah, really just meeting people, spending time, geeking out on this stuff.

Yeah, there'll be lots of geeking out.

All right, thanks for your time.

Looking forward.

Yeah, same.

[MUSIC PLAYING]