this post was submitted on 29 Apr 2024

315 points (97.3% liked)

News

22890 readers

4173 users here now

Welcome to the News community!

Rules:

1. Be civil

Attack the argument, not the person. No racism/sexism/bigotry. Good faith argumentation only. This includes accusing another user of being a bot or paid actor. Trolling is uncivil and is grounds for removal and/or a community ban.

2. All posts should contain a source (url) that is as reliable and unbiased as possible and must only contain one link.

Obvious right or left wing sources will be removed at the mods discretion. We have an actively updated blocklist, which you can see here: https://lemmy.world/post/2246130 if you feel like any website is missing, contact the mods. Supporting links can be added in comments or posted seperately but not to the post body.

3. No bots, spam or self-promotion.

Only approved bots, which follow the guidelines for bots set by the instance, are allowed.

4. Post titles should be the same as the article used as source.

Posts which titles don’t match the source won’t be removed, but the autoMod will notify you, and if your title misrepresents the original article, the post will be deleted. If the site changed their headline, the bot might still contact you, just ignore it, we won’t delete your post.

5. Only recent news is allowed.

Posts must be news from the most recent 30 days.

6. All posts must be news articles.

No opinion pieces, Listicles, editorials or celebrity gossip is allowed. All posts will be judged on a case-by-case basis.

7. No duplicate posts.

If a source you used was already posted by someone else, the autoMod will leave a message. Please remove your post if the autoMod is correct. If the post that matches your post is very old, we refer you to rule 5.

8. Misinformation is prohibited.

Misinformation / propaganda is strictly prohibited. Any comment or post containing or linking to misinformation will be removed. If you feel that your post has been removed in error, credible sources must be provided.

9. No link shorteners.

The auto mod will contact you if a link shortener is detected, please delete your post if they are right.

10. Don't copy entire article in your post body

For copyright reasons, you are not allowed to copy an entire article into your post body. This is an instance wide rule, that is strictly enforced in this community.

founded 1 year ago

MODERATORS

rjc@lemmy.world

Thekingoflorda@lemmy.world

gedaliyah@lemmy.world

Blackbeard@lemmy.world

little_cow@lemmy.world

jeffw@lemmy.world

315

ChatGPT provides false information about people, and OpenAI can’t correct it (noyb.eu)

submitted 4 months ago by gedaliyah@lemmy.world to c/news@lemmy.world

41 comments fedilink hide all child comments

It’s clear that companies are currently unable to make chatbots like ChatGPT comply with EU law, when processing data about individuals. If a system cannot produce accurate and transparent results, it cannot be used to generate data about individuals. The technology has to follow the legal requirements, not the other way around.

top 41 comments

sorted by: hot top controversial new old

[–] GenderNeutralBro@lemmy.sdf.org 100 points 4 months ago (1 children)

ChatGPT is not an information repository.

The correct answer to this problem is not "we can't correct it"; it is "this class of task is completely out of scope for ChatGPT, and we will do everything we can to make sure users understand that". Unfortunately, OpenAI knows damn well this is how the public perceives and uses its product and seems happy to let this misconception persist.

We do need laws to curb this, but it's really more a marketing issue than a technological issue. The underlying technology is amazing; the applications built around it are mostly garbage. What we have here is a hype trainwreck.

[+] gedaliyah@lemmy.world -30 points 4 months ago (3 children)

Yet, LLMs are trained on data - an information repository. They are capable of accessing and recalling the contents of that information repository, and relaying information from that repository to an end user. It may not be an information repository functionally, but it legally seems to have the capabilities to be classified as one. (I am neither a lawyer nor a programmer, and I am not in the EU.)

The software breaks the law, and the people who built it knew that this was likely the case. It was developed as a research project, which has very different legal requirements from a consumer product. They might not outright ban the software, but they might issue some hefty fines, etc. Banning a product is not the only recourse of the courts.

[–] CarbonatedPastaSauce@lemmy.world 40 points 4 months ago (2 children)

They are capable of accessing and recalling the contents of that information repository, and relaying information from that repository to an end user.

This is not correct based on my understanding of LLMs, but I am certainly not an expert. As I understand it, it's basically a statistics exercise in how they determine what order to put words into. They don't 'look stuff up' in their training data. They probably don't even have access to their training data once the model is complete. These models are trained on terabytes of data but are small enough to fit in memory, so it's impossible for them to still have access to all that. But it wouldn't matter if they did, because that's not how they work.

[–] UnpluggedFridge@lemmy.world 2 points 4 months ago (1 children)

LLMs do not look stuff up (except when they have an API that allows them to), but I think OP's point still stands. The statistical next token predictor metaphor is useful , but in many regards that's what text and language are. If you can understand that certain words are linked to certain other words, then you should be able to appreciate that certain groups of words can be associated in a way that is functionally the same as data.

I have not memorized the pytorch documentation, but I can use what I understand about pytorch and other libraries to infer specific aspects of the library that I am not familiar with. Functionally, this is no different than if I accessed the documentation directly. If I communicate this information to others I have functioned as a data repository. The repository works on a more abstract and error-prone level, but it works nonetheless.

Here is another very concrete example: LLMs know George Washington's birthday. Not because they look up that information, but because of the learned associations between George Washington, birthday, and his actual date of birth.

[–] CarbonatedPastaSauce@lemmy.world 11 points 4 months ago (1 children)

I can use what I understand about pytorch and other libraries to infer specific aspects of the library that I am not familiar with.

This is what LLM's can't do though. They can't use what they understand because they don't understand anything. They can't infer, they can't reason, they can't evaluate or compare. They can spit out words that make it look like they did those things, but they didn't.

[–] UnpluggedFridge@lemmy.world 0 points 4 months ago* (last edited 4 months ago) (1 children)

Here I think you are behind on the literature. LLMs can infer and reason, and there are whole series of papers that evaluate LLMs for these properties the exact same way we evaluate humans. So if you can't trust the metrics, then you cannot even assert that humans can reason and infer and understand.

[–] CarbonatedPastaSauce@lemmy.world 6 points 4 months ago (1 children)

https://arxiv.org/html/2403.04121v1

Good read from a group of computer scientists at Arizona State. Their conclusions are the same as mine but they illustrate the problems better than I ever could.

[–] UnpluggedFridge@lemmy.world 0 points 4 months ago (1 children)

You linked a paper on planning in LLMs. Planning is largely in the domain of reinforcement learning. The paper you linked conflates reasoning with planning, alongside the obviously biased prose, so the author really doesn't seem credible. I prefer nuanced and careful evaluations such as: https://www.sciencedirect.com/science/article/pii/S2949719123000298

[–] petrol_sniff_king@lemmy.blahaj.zone 2 points 4 months ago (1 children)

Without commenting on the content of the paper,

so the author really doesn't seem credible. I prefer nuanced and careful evaluations

Novelis

https://novelis.io/

Innovation in action: Optimizing Business Efficiency

Our mission is to revolutionize business efficiency through Innovation. Leveraging AI, automation and process intelligence, we transform operations, supercharge performance, and unlock growth for businesses worldwide

Hm. 🤔

[–] UnpluggedFridge@lemmy.world 1 points 4 months ago

Notice that there are methods, data, and peer reviews that I can freely scrutinize. All things your opinion piece lacks.

[+] gedaliyah@lemmy.world -22 points 4 months ago (4 children)

To me that sounds like a distinction without a difference. A jpeg is not an image, but a set of data that can be algorithmically processed and rendered as an image - which is why it can fit in a smaller space than a bmp. Despite the technical differences, a jpg and a bmp are legally equivalent. If something is illegal in a bmp, it's also illegal in a jpg. The same laws apply to EVs and gas vehicles. The same laws apply to vinyl records and cassette tapes. The law does not care about the mechanism.*

*for the most part

[–] CarbonatedPastaSauce@lemmy.world 31 points 4 months ago

You’re illustrating the issue so many people have with this technology. Without a fundamental understanding of how it works, people will attempt to use it in ways it shouldn’t be used, and won’t understand why it isn’t giving them correct information. It simply doesn’t have the ability to do anything but put words in an order that statistically will resemble how a human might answer the question.

LLMs don’t know anything. They can’t tell fact from fiction (and are incapable of even trying), and don’t understand concepts such as verifying info when requested. That’s the problem, they don’t ‘understand’ anything, including what they are telling you. But they do spit out words in a statistically probable order, even if the result is complete bullshit. They do it so well that they can fool most people into thinking the computer actually knows what it’s telling you.

[–] Grimy@lemmy.world 23 points 4 months ago

Here's a better metaphor because yours completely misses the mark when it comes to the difference between an LLM and an actual encyclopedia.

A painter will spend years honing his craft by studying other paintings as well as photos and real life. If you ask him to paint you a house from memory and try to build it with what he gives you, that just makes you an idiot, it doesn't make him a bad architect.

Chatgpt is not an encyclopedia and any thing it says that is remotely important to your personal or work life should be verified. They explicitly tell you it can and will give false responses.

[–] Xtallll@lemmy.blahaj.zone 18 points 4 months ago

LLM isn't a compilation of its training data, anymore than a cake is a pile of eggs, flower and sugar.

[–] kurwa@lemmy.world 12 points 4 months ago

Jpg is a lossy compression algorithm. Statistical probability of words occuring in sequence is not compression. That's like saying generative images are compression, they aren't. It's not producing blurry matches of images, it's producing something "novel". Otherwise, that would be considered over fitting the data.

[–] antidote101@lemmy.world 11 points 4 months ago

They don't recall information from a repository, the repository is translated into a set of topic based weighted probabilities of what words come next.

Those probabilities are then used to reconstruct a best-guess at what words are next when generating strings of language.

It's not recall, it's a form of "free" association, which is quite tightly bounded to the context, topic, and weightings of the training data.

This is not precise and is more likely to create average answers and sentences, rather than precise ones.

It's not recall, it's really convincing lies.

"He seems to know what he's talking about, and speaks with a certain kind of authority which makes sense and sounds knowledgeable".

[–] GenderNeutralBro@lemmy.sdf.org 4 points 4 months ago* (last edited 4 months ago)

To clarify, I mean to say that users should not consider it an information repository, because it does not function as one, by design. Whether it should be classified as such under the law is another matter, one on which I do not have enough knowledge to comment. I do think OpenAI is presenting ChatGPT inappropriately, and I hope they will be held accountable for that.

I'm sure in the future we will see true databases built on the same technology (and they will be awesome, if implemented properly). But that's not what ChatGPT is (or, as far as I know, any other existing LLM-based application). Any information it is able to "recall" is almost a coincidence of how it was trained. You can sort of think of it like lossy compression. The LLM gets all of its information from its training set, but it is not designed to retain any specific information from the training set in full. In cases where it does, that usually means one of two things:

The information appeared many times in the training set, enough prevent it from being washed out.
The model is far bigger than it should be, and is overfitted to its training data.

[–] guyrocket@kbin.social 24 points 4 months ago (3 children)

OpenAI openly admits that it is unable to correct incorrect information on ChatGPT. Furthermore, the company cannot say where the data comes from or what data ChatGPT stores about individual people. The company is well aware of this problem, but doesn’t seem to care.

Wow. Where are all the news stories about THIS?

[–] vrighter@discuss.tchncs.de 44 points 4 months ago (1 children)

If you try to start learning how they work, the first thing you realize is that hallucinations are fundamental to how the technology works. Of course they are unfixable. That's literally how they work.

They're broken clocks that happen to be right more than just twice a day, but still broken nonetheless.

[–] DdCno1@kbin.social 10 points 4 months ago (1 children)

It's an inherent issue with deep learning. Awareness of this among people who are regularly using these tools is very low, which is troubling.

https://umdearborn.edu/news/ais-mysterious-black-box-problem-explained

[–] guyrocket@kbin.social 1 points 4 months ago

That article explains the issues well and clearly. Thanks for sharing.

I think it should be shared more broadly.

[–] FaceDeer@fedia.io 9 points 4 months ago

You're reading one right now?

[–] mansfield@lemmy.world 11 points 4 months ago (1 children)

This stuff is literally a bullshit(1) machine. How can you fix it without making something else entirely?

(1)

[–] tinwhiskers@lemmy.world 1 points 4 months ago* (last edited 4 months ago)

When they hallucinate, they don't do it consistently, so one option is running the same query through multiple times (with different "expert" base prompts), or through different LLMs and then rejecting it as "I don't know" if there's too much disagreement between them. The Q* approach is similar, but baked in. This should dramatically reduce hallucinations.

Edit: added bit about different experts

[–] FaceDeer@fedia.io 10 points 4 months ago (4 children)

The technology has to follow the legal requirements, not the other way around.

Given the possibility that this is a general problem of AI that simply cannot be corrected, the law could end up meaning that LLMs are outright forbidden in the EU. If that's true then the legal requirements will have to be changed, there's no way the EU would actually ban them. It'd be like opting out of the internal combustion engine due to some detail of an old law that they happened to violate.

[–] vrighter@discuss.tchncs.de 14 points 4 months ago (2 children)

they would not be banned outright. They just can't be used to process data about customers.

But an ai furry porn generator doesn't necessarily process customer data

[–] gregorum@lemm.ee 3 points 4 months ago (1 children)

Not unless you want furry porn about your… taxes?

[–] Grimy@lemmy.world 1 points 4 months ago

That would get it banned in the US, not the eu.

[–] FaceDeer@fedia.io -2 points 4 months ago

The economic effects would still be enormous. You can amend my analogy to "banning internal combustion engines when their services are being sold to customers", leaving them free for individuals to use to carry themselves around, and it'd still have a massive impact.

Europe's not going to kneecap themselves over this.

[–] xhieron@lemmy.world 10 points 4 months ago (1 children)

The technology has to follow the legal requirements, not the other way around.

This is something that really needs to be taught better, at least in the US.

GDPR doesn't mean that LLMs are forbidden in the EU, but it does mean that the companies that create them may be liable for damages. That said, the damages must be real. Actual damages is somewhat cut and dry (e.g., ChatGPT publishes defamatory information about you, and someone relies on it to your detriment), but GDPR also contemplates damages for distress (e.g., emotional).

If that’s true then the legal requirements will have to be changed ...

I think this position needs to be rejected in the strongest possible terms. Our response to any emerging technology should not be "It's too good not to have, so who cares if people lose their rights?" The right to privacy and with it the right to control one's likeness, name, and personal data is a much easier right to conceptually trade away than, say, the right to bodily integrity, but I think we've seen enough dystopian sci-fi at this point to understand where the intersections might lie between other rights and correspondingly miraculous technologies. [And after all, without the combustion engine we probably wouldn't be staring down the barrel of climate change right now.]

Should we, for instance, do away with the right to bodily integrity if it means everyone gets chipped shortly after birth? [The analogy to circumcision is unintentional but not lost on me.] After all, the chips mean that we can locate missing and abducted children easily and at trivial cost. They also mean that we no longer need to carry money or proxies for money. Crime is at an all-time low. Worth it, right? After all, the procedure is "minimally invasive."

The point is, rights have to be sacrosanct. They need to be the first consideration, and they need to be non-negotiable. If a technology needs those rights to bend or give way in order to exist, then it should not exist. If it's of sufficient benefit to society, then it can be made to exist in a way that preserves those rights, and those who are unwilling to create it in such a way should suffer the sanction of law.

[–] General_Effort@lemmy.world -4 points 4 months ago (1 children)

This post kinda shows the problem I have with the GDPR. It creates this pseudo property right in information about yourself. It's not about a right to privacy but about Data Protection; rhymes with copyright protection. GDPR fans are worried about "their data" being "stolen", not about being spied on. It's about property.

It's not something that has traditionally existed. People always gossiped; maybe had a little black book. That's still allowed, because the GDPR has an exemption for that. Strictly, it's a violation of other's rights.

Privacy means that some areas of life are simply off-limits. For example, you mustn't read other people's mail. The GDPR isn't concerned with that. In fact, there is an implied contradiction. GDPR rights are concerned with controlling the storage and exchange of information as an intellectual property. Enforcing that requires surveillance of communication. Only the exemptions prevent that from being an issue.

No right exists in isolation. You mentioned the right to bodily integrity. What if someone is injured and needs medical care. Maybe they need surgery or they lose a limb, but they can't pay the surgery. You'd have to take someone else's money; their property. Even in the US, this is done to some degree. Your argument about rights being "sacrosanct" is against that. If you can't take someone's precious data, then certainly not their money either.

Something about the GDPR turns people into right libertarians / conservatives /neoliberals. Call it what you will. It's: Fuck you, I got mine. It's not about what's best for everyone, society, human progress, or anything beyond the individual.

[–] xhieron@lemmy.world 3 points 4 months ago* (last edited 4 months ago) (1 children)

I'm not sure where to start here. Privacy and copyright aren't the same thing. If you don't understand that, I doubt we're actually able to debate this issue. People have been gossiping for as long as there has been language, and they've also been making shit up about each other, and as a result the law against slander is older than the Norman conquest, and HIPAA is just the latest statutory framework enshrining the very old rule that the ladies can gossip about whatever they hear in town, but the priests, doctors, and lawyers absolutely cannot. This isn't a novel, 21st century selfish gotcha. It's a very old, very simple principle: that some things are none of your fucking business, and if you run your mouth about people there will be consequences for it. That idea doesn't belong to a political ideology, and it's not an author's monopoly. The GDPR is absolutely concerned with those rights, and it rightly sanctions violators.

OpenAI is not society or human progress. It's a corporation trying to make money for its shareholders who care not one whit about the future of mankind of any lofty ideal. This isn't rugged individualism versus progress. It's natural persons versus a corporation that wants to trod roughshod over their rights to be left alone, to be free from the publication of lies about them, and to keep from invasion by the prying eyes of robber barons, governments, and newsmen their private business. That's best for everyone and what's best for society and human progress: If I want you to know something about me, I'll tell you. Otherwise, mind your own business, and that goes for OpenAI same as it goes for you.

Edit: Also your bodily integrity example is rubbish. A surgeon who interferes with your body without your informed consent has committed a battery against you. Even in an emergency the standard is frequently that you would have consented to the surgery if you were able.

[–] General_Effort@lemmy.world -1 points 4 months ago

I’m not sure where to start here. Privacy and copyright aren’t the same thing.

I never claimed it was. In fact, I pointed out the implied contradiction. If you could point out what gave you the false impression, I can hopefully be more clear in the future.

[–] TheOneCurly@lemm.ee 10 points 4 months ago (1 children)

If the world had opted out of the ICE early, maybe we wouldn't be in quite the global warming situation we're in.
LLMs are still a novelty product that can barely perform their novelty. Comparing them to the wildly useful and game changing ICE is not terribly accurate.

[–] FaceDeer@fedia.io 0 points 4 months ago (1 children)

It's not the world that would be opting out of the internal combustion engine in this analogy, it would be Europe. There rest of the world would go on industrializing while Europe remains in the 19th century. It would be an insane act of self-destruction.

[–] TheOneCurly@lemm.ee 7 points 4 months ago

Or European websites will suddenly be the only ones worth visiting because they aren't buried under mountains of LLM garbage text.

[–] gedaliyah@lemmy.world 7 points 4 months ago (1 children)

Or on the other hand, maybe we have to admit that these technologies were released before they were finished, and that was a dangerous decision. It's now been well documented that chat gpt and similar technologies were rushed to the public against the advice of some of their developers.

The developers will need to devise ways for the LLMs to understand their own training data.

[–] db0@lemmy.dbzer0.com 13 points 4 months ago

Llm tech is not rushed. The models are not for accurate information and trying to use them this way is out of their scope. What's rushed is corpos trying to use them for searches

[–] uranibaba@lemmy.world 2 points 4 months ago

I read the article and I read the comments. Is there something I am missing here? I thought they were discussing OpenAI gathering data on it's users (those using ChatGPT) and not giving that data back. Based on the comments, the article is upset that OpenAI can give back data that ChatGPT was trained on.

Does the second case fall under GDPR? Could not OpenAI just claim that they removed any information that makes it identifiable and call is a day?

[–] HubertManne@kbin.social 1 points 4 months ago

It so lol as it say hubert manne is vampire lizard technomancer from alpha centauri. so much I laugh because it is so not truth. fun funny it is.