Sarah Nagy on AI-powered search: Looking forward with AI

Sarah Nagy is a former astrophysicist and a data scientist with a Master in Finance degree from Princeton. She’s now the co-founder and CEO of Seek AI, a platform that uses generative AI to support business users in querying business data sets, freeing up data scientists for more creative work.

We talked about a range of AI topics, with a particular focus on search. The interview happened shortly after rumors of OpenAI launching a search engine began to circulate. (Interview edited for clarity and length.)

Q: Is generative AI going to be a disruptive force in search, much as Uber and Airbnb were in their spaces?

A: I definitely think so. Search engines have been more or less the same for a long time and there is so much more value you can unlock by adding generative AI. It does a lot of the work for you; it can find things in the pages it’s searching and surface that for you. Do I necessarily think that traditional search engines are going to go away? Probably not. Right now, the speed is greater with traditional search engines; people don’t necessarily want to wait for text to generate all the time. But as the latency of the models goes down, that will probably cause people to switch more and more to generative AI search engines.

Q: Using a genAI search engine like Perplexity, rather than getting a summary followed by links, I get a generated text with a handful of citations. That gets us to the position where the AI is, in effect, choosing a limited number of search results for me. That’s surely got to be worrying for brands and publishers?

A: It’s not necessarily that different from a traditional search engine. No matter what, you’re always going to have limited results displayed to you. With Google, the incentive comes from ads; Perplexity doesn’t have ads yet, so the results are being chosen based on what’s going to be most helpful to the user, not who’s paying.

Q: Any thoughts about the rumors swirling around ChatGPT? Would it be a surprise if they got into search?

A: I don’t think it would be a surprise, but at the same time there is Bing and some of the complications that come up when you think about how Bing relates to ChatGPT; that could be a reason that they did not release a search engine. But the model is being improved. A year ago, its knowledge was very dated; I think it was 2023 and its knowledge ended in 2021, if I remember correctly. That’s not the case any more, so ChatGPT could continue being more up-to-date without being a search engine.

Q: Some people are talking about ChatGPT being able to report the news. Are the LLMs getting close to being real time?

A: Closer and closer. That has definitely been improving. Never say never. We have many years ahead of us as a species. Something we learned at Seek is that any sort of data, including news, needs to be collected efficiently and that data collection right now is very hard to do without humans. Journalists need to be boots on the ground, reporting. How are you going to automate that with AI?

Q: That relates to claims that AI will one day be so smart that it doesn’t need us any more. But if it’s that smart, it will realize that it needs hardware; putting together hardware and sourcing the materials for hardware is very much a human undertaking.

A: I totally agree. The hardware is harder to figure out than the software. But the hardware side is advancing quite a lot. There’s funding going into embodied AI, humanoid robots. It’s making progress.

Q: Watching the conversational capabilities of the new ChatGPT-4o reminded me of the Turing Test. Conversational AI could now pass the Turing Test with flying colors, but surely that just points to the limitations of the Turing Test because the AI really is just responding based on data analysis and predictions, not actual intelligence?

A: There are different schools of thought about this. Is the Turing Test a good benchmark? Our machine learning team at Seek will say it isn’t and there are other ways to measure intelligence. One alternative is called the Arc benchmark. It measures visual reasoning. I would argue that LLMs have been passing the Turing Test even before ChatGPT.

There’s the stochastic parrots school of thought that these models are really just next-word predictors, just statistically producing the next word. Then there’s the OpenAI school of thought; all of the machinery that is producing that next word is running through a simulation of reality to get to that next word. One is more pessimistic, one is more optimistic, but I’d say they’re both saying the same thing. For me personally, there’s no doubt that the models are next-word predictors, but there’s so much we don’t know about what goes on underneath the surface. Are humans really so magical that we defy the physical world and there’s no way a large language model could live up to how god-like we are? I don’t know if I believe that either.

Q: We still don’t fully understand how humans do what they do, which makes it hard to say if a machine will one day be able to do it.

A: Exactly. There’s a lot we don’t know about LLMs and there’s a lot we don’t know about the human brain either.

Q: To conclude, what are you doing at Seek AI?

A: We are a natural language interface for anyone within an organization to be able to ask questions about large datasets and get accurate answers. It’s an emerging field that I am calling “quantitative AI” because the AI is crunching numbers, it’s getting numerical or data-driven results. We’re primarily intended for businesses to more efficiently query their own data, but we do also have partnerships with data companies to provide their customers with more efficient ways to query their data.

One example, we partnered with Prodigy to create Prodigy Plus; that’s the Seek AI engine plus the proprietary marketing data of Prodigy. Prodigy’s customers are able to ask questions in natural language, like “How much more did I spend on this year’s Super Bowl than last year’s?”

Q: So you’re cutting the data scientists out of that process?

A: I started Seek because I used to be a data scientist and it was intended to be able to do those low level tasks that no one really wants to do. Often they involve writing code that was too complex to automate with traditional methods, but not complex enough to be at all interesting or worthwhile for a human to be doing.