Web site for AI Transcript

Idx	Time Stamp	Speaker	Excerpt
1	0:00:03	Aarush Selvan- moderator	Hi, everyone. My name's Arush. I'm a product manager here at Google. Today, we're going to talk about Gemini Deep Research, why we built it, how we built it, and what it can do for you. Sonal, how would you describe Deep Research to a fellow engineer?
2	0:00:15	Sonal Gupta	This is a new agentic system we have developed where we have breakthroughs in how do we do goal decomposition and multi-step planning and reasoning, fetching information from the sources, and a self-critique loop where we can refine the answers into a comprehensive answer that we can put in a report.
3	0:00:31	Rushin Shah	Yeah, so I would tell an engineer this is a system that combines three really powerful concepts, best in class LLM reasoning over long context, really novel long compute infrastructure, sophisticated browsing capabilities, and it all comes together in one of the first production grade agents out there.
4	0:00:47	Aarush Selvan- moderator	And Jeff, how would you describe this agent perhaps to someone who's a friend that doesn't know anything about AI?
5	0:00:53	Geoff Barnes	I think to a friend, I'd probably describe this as like a legwork doer, something that can go out there for you and accomplish a lot of time consuming, maybe regular tasks, maybe mundane tasks, but important ones like figuring out where your daughter can go to swim camp next summer.
6	0:01:09	Mukund Sridhar	Yeah, I think it's a great starting point if you want to dive deep into some topic or plan something more elaborate that takes researching over 10, 20 tabs. This does all the legwork for you and puts together a great starting point.
7	0:01:22	Aarush Selvan- moderator	Who do you think will get the most value from a feature like this?
8	0:01:24	Sonal Gupta	Definitely, students. You can use deep research to learn about a new topic, they have a class going on, you don't understand the complex concepts and you want to really go deep. And deep research can help you get all the information so that you are ready to go.
9	0:01:36	Aarush Selvan- moderator	Rushan, what about you?
10	0:01:37	Rushin Shah	Yeah, I think it's going to be very widely useful, but speaking as somebody who has a young kid and another one on the way, I think is going to be massively useful for working parents. There's just so much to figure out with school and vacation and, you know, after-school activities and the right kind of diet, all of these things for kids. And I think having something like this is like a superpower.
11	0:01:59	Nihal Balani	Yeah, I think it would be people who want to get expertise really fast. For example, you know, parents or students or people who are analysts. In particular, I'm more excited about people who want to learn about a new topic, learn about neighborhoods you might be moving to, or insurance or mortgage. I think they will get a lot of value from this.
12	0:02:16	Aarush Selvan- moderator	Mokan, could you explain a bit more about how deep research works under the hood?
13	0:02:20	Mukund Sridhar	Sure. A user comes to deep research and asks for their research query. Oftentimes, it's underspecified. So the first thing the model does is it tries to really decompose the user goal and tries to lay out this kind of a plan of what are the different tasks and attributes that the user might be looking at. And here you might discover as a user things that you're not thought of or to the contrary things that you really don't care about. So you have the power to iterate on this plan with deep research. And once you've edited this plan, we kind of enter the research phase. And here the model is clever enough to understand which of these tasks can be spun out and be done in parallel versus things that have to be done iteratively and have some dependency. And each step, it's trying to ground all the information from various different sources and try to understand, is this sub-task really done? or are there things to still go out and look for? And then by the end of this, there's a ton of information. So it prepares multiple drafts, tries to critique itself, to prepare this final report with every single fact and a lot of sources along the way for the user to continue exploring their journey.
14	0:03:29	Aarush Selvan- moderator	What's one thing that really impressed you or you were surprised by when building deep research?
15	0:03:35	Rushin Shah	Yeah, there's a lot of focus on making LLMs faster. And even though this would take minutes, it would give you something that would save you hours worth of work.
16	0:03:43	Aarush Selvan- moderator	Yeah. I think in terms of taking time, at Google, we're used to fractions of fractions of a millisecond in latency. And for the first time, I was using a stopwatch that could only do seconds and minutes to time this. And that was perfectly adequate. And we could use that to measure the latency. It was definitely a new experience for a lot of us at Google. So Jeff, when working on Deep Research, what was an aha moment that you had when it really came to life for you?
17	0:04:10	Geoff Barnes	Yeah, I knew that we were building something to kickstart research. What I found was that we were building a massive gateway to really intensified discovery.
18	0:04:19	Aarush Selvan- moderator	Mokan, what about you? What are some of the aha moments?
19	0:04:22	Mukund Sridhar	Very early on, we wanted this to be open domain. The first time we were able to show that with very minimal data, it's able to generalize. That was a moment of relief. And my aha moment of the project.
20	0:04:34	Aarush Selvan- moderator	What was one of the technical challenges you had to overcome for deep research?
21	0:04:38	Nihal Balani	creating a system which is reliable, because this takes a lot of time, and it would be very unfortunate if there's a transient failure which just ends the task. And so we built this asynchronous system which can run very reliably. The user doesn't need to have their browser open. They can just come back later and see that their task has been finished.
22	0:04:56	Aarush Selvan- moderator	So deep research typically will take around five minutes, sometimes a little bit more, based on the complexity of the query. But what I'm hearing you saying is this is something that could scale to even agents that take an hour or two or even a day to get things done for you.
23	0:05:07	Nihal Balani	Absolutely. There is no constraint in the system or the design which prevents that. We should see more agents like this do way more complex tasks. Wow.
24	0:05:16	Aarush Selvan- moderator	Could you talk through some of the challenges and some of the ideas that you guys came up with in designing the UX for this experience?
25	0:05:22	Geoff Barnes	Yeah. When we were designing this, the team thought about a handful of things. One of them, I think, at the top is just how deeply curious humans are and how humans do things right now. Yes, we're going to save a ton of time. Maybe we're going to let you close the 100 tabs that you have open and your forever tabs in your browser. But in order to go off and do all of this autonomously and agentically, we have to engender trust, right? So presenting a plan that has some transparency and helps engender some of that trust is a critical part. giving the user the ability to edit that and collaborate with the model a little bit and say, no, actually, I want you to do something a little bit differently than what you've proposed, or add this, subtract that. What we're trying to do is lower the barrier to humans being able to satisfy and cultivate that fundamental curiosity.
26	0:06:09	Sonal Gupta	But the system is actually really complex behind the scenes. So we have figured out this really nice balance between UX and LLM technology to solve the problem.
27	0:06:17	Rushin Shah	Yeah, so there's a lot of focus on making LLMs really fast. When we first proposed that we were going to build something that would take a few minutes, we got some questions. But we had conviction that it would be something that produces such high quality responses that even if you had to wait for a few minutes, it's okay because it saves you hours of time. So Arush, let's turn the tables a little bit and let me ask you, what was the spark for building deep research?
28	0:06:41	Aarush Selvan- moderator	Yeah, so I believe there was just a really visionary product manager who wrote a really detailed spec, and then everyone just went and executed, right, Mukund? Is that how it plays out or not? Exactly how I remember it.
29	0:06:53	Rushin Shah	Visionary and humble.
30	0:06:56	Aarush Selvan- moderator	Yeah, they can cut this out, right? So we were thinking about how we could use LLMs to not just give quick answers, but actually help users with more complicated problems. Google's mission is really to organize the world's information and make it universally accessible and useful. And so what we were thinking about is why don't we help with that specifically around research and helping people with more complex research problems or the kinds of things where you have to do a bunch of research but you don't even know where to get started. So that's really the idea that we wanted to explore. So where does this go from here? What does it mean for the Gemini app?
31	0:07:30	Geoff Barnes	Yeah. I think for the Gemini app, this bodes really well for the future. It's super exciting. Deep research is the first step in my mind. It's the basis for making Gemini a more agentic, capable agent, and I just can't wait to go further on that path.
32	0:07:45	Aarush Selvan- moderator	Fantastic. So I'm actually curious to hear more about some of the use cases. Have you guys been having fun with this tool? Sonal, what are some of the interesting ways in which either you've used it or seen people use it?
33	0:07:54	Sonal Gupta	I think it's a very fun tool for just you have a concept in your mind you've heard people talk about and like I don't have time to go and research on this topic and you just put it in deep research you get a report and you don't have to like basically go and do all the like work and it could be really random stuff I think I don't understand how new young folks talk these days it's a very good tool to go learn how are people talking about like what are those language people are using
34	0:08:16	Aarush Selvan- moderator	Or you could just ask your product manager. I don't know. I like to think of myself as pretty young. Maybe I should use a tool first. Maybe I can use Deep Research first. Jeff, what about you?
35	0:08:26	Geoff Barnes	I recently learned a whole lot about the West Coast versus East Coast hip-hop beef of the 90s.
36	0:08:34	Aarush Selvan- moderator	With that, thank you for joining us as we went deep on Deep Research. We're excited to see how you use it. Thank you, guys.