As AI becomes bigger and better there is a problem which may dramatically limit its usefulness. The problem is that it only know what we tell it and we're unlikely to ever tell it everything.
If you're already familiar with how these large language models work, the idea is that you build a very large computer system and then train it by feeding it all of the data you can. That data can be books, music, art, essentially anything that can be found on the internet is potentially useful. In fact, these models are already so large that there is concern they are running out of data from which they can learn new things.
For years, the people building powerful artificial intelligence systems have used enormous troves of text, images and videos pulled from the internet to train their models.
Now, that data is drying up.
Over the past year, many of the most important web sources used for training A.I. models have restricted the use of their data, according to a study published this week by the Data Provenance Initiative, an M.I.T.-led research group.
The study, which looked at 14,000 web domains that are included in three commonly used A.I. training data sets, discovered an “emerging crisis in consent,” as publishers and online platforms have taken steps to prevent their data from being harvested...
“We’re seeing a rapid decline in consent to use data across the web that will have ramifications not just for A.I. companies, but for researchers, academics and noncommercial entities,” said Shayne Longpre, the study’s lead author, in an interview.
That story was published just over a year ago. Bigger and bigger data centers keep getting built but we're already at the point where limiting factors for their capabilities are in sight.
But there's an even bigger problem with these systems which limits their usefulness. These models may have scooped up a significant portion of all the digital information on the internet but they still only know things that people are willing to tell them. And it turns out there are a lot of holes in this information, things that can't be googled and therefore can't be passed on to ChatGPT. The Free Press just published an article about this. There is a whole world out there of things that people don't talk about.
Fascinated by the existence of such un-Googleable gold, I began work on what would become my new book—Schott’s Significa—which explores the language, signs, and jargon of 53 curious worlds, including Venetian gondoliers, bespoke tailors, paparazzi photographers, and casino workers.
Let me give you an example: One of my first interviewees in the Diamond District of New York, told me about The G — a common term for customer. I mentioned this to another dealer who gave me Kitty with the G—which means, “Talk to the customer to make sure they don’t leave the shop.” Armed with this, a third diamantaire disclosed Sherry the G—which means, “Get this idiot out of here.” And finally, a dealer disclosed Tee the G—which means, “Follow the customer once they’ve left the shop to make sure they’re not collared by a competitor.”
Now, there’s no way any one diamond dealer would have disclosed all these internal terms in a single stroke; they had to be lured gently, over a period of months, from many expert minds. But, when you uncover such gems—gems that are truly un-Googleable—you know you’ve found priceless treasure.
The omnipresence of Google has made us impatient to mystery: How many conversations are derailed by a smartphone consultation? But some information must still be earned. The slang of the Diamond District, like the jargon of any secret world, is valuable because it can’t be scraped or simulated, only passed by word of mouth. In a world obsessed with instant answers, such hard-won knowledge reminds us that meaning is still made in the margins and on the fly.
Now you could argue I guess that none of this information is really that useful. Does it matter how people in the diamond district talk about their potential customers behind their backs?
It's a fair point, but that's just one small example. Think about all the other things that people in the diamond district know that they aren't sharing with everyone else. There is potentially a lot of expert knowledge, decades of it, which you probably can't get from Google or from AI. Even if you could get it, there is a big gap between the ability of hands with decades of experience and those with none.
And the same is true in other fields. I came across this video by music producer and YouTuber Rick Beato earlier this week and he was really making the same point. People have the belief that AI knows everything but actually there's a lot that AI doesn't know because the people who do know are few and far between and those people just aren't talking.
In this clip he spends some time demonstrating how AI's ability to correctly answer a math question about shuffling a deck of cards may be dependent on its ability to ingest the best available answer from a single YouTube video. In other words, unless a human expert has explained it, AI struggles.
From there he asks AI some questions about how to properly mix an album, something he knows a lot about because he's been a producer for decades. AI offers very confident answers but they are also very shallow. As Beato points out, there are a handful of real experts in these topics, people who've made millions by perfecting these crafts over decades. And guess what, they don't talk about it with strangers. The volumes of detailed and specific information that they know are not being shared on the internet, can't be googled and therefore will never end up as part of a large language model.
If you're trying to win bar trivia, AI is a great tool. If you really want to do something specific with exceptional skill, the information you need to get there is going to be a lot harder to access.
Editor’s Note: The Schumer Shutdown is here. Rather than put the American people first, Chuck Schumer and the radical Democrats forced a government shutdown for healthcare for illegals. They own this.
Help us continue to report the truth about the Schumer Shutdown. Use promo code POTUS47 to get 74% off your VIP membership.
Join the conversation as a VIP Member