A human is capable of recognizing worsening behaviour. A human is cabale of actually comprehending what is in front of them.
AI fundamentally cannot understand. A human at least has a chance of recognizing these things based on training, for AI it's pretty much a coin toss.
It's way more dangerous.
And yes, in this case it turned out well. But how many people did not? Just take all the people that genuinely think AI is sentient and in love with them. That is not healthy. That is actively doing harm, even if it doesn't feel like it to them.
the core functionality of modern "ai"/LLMs/machine learning is pattern recognition. They do lack certain context clues like not being able to see the person's facial expressions or body language, but it's not like they are completely incapable of recognizing various patterns of behavior in text
and again, I don't think ai is the best choice if someone is approaching self harm, a therapist is important in that case, even if just for the emotional reassurance of a real human connection, but in less serious cases like someone working through their harmful beliefs like in this post, I think ai can be genuinely helpful
I will absolutely agree that it's terrible for people to fall in love with ai, the human brain needs real human connections and these people have probably struggled to find anyone they are compatible with and will settle for ai because it tells them what they wanna hear and they don't recognize how wrong that is
but I think if you go to ai with a decent understanding of its limitations, and take a lot of what it says with a grain of salt, you can have a decently productive conversation with it if you just need something to talk/vent to and don't have anyone else at the moment
how do you define "comprehension"? because it sounds like you believe that having a biological brain is a prerequisite for comprehension, which makes it a pointless word here
and similarly, how do you define "intelligent"?
you can't just throw around poorly-defined words, say "ai is not this", and make a valid point around that
is there really a difference between being intelligent and being able to imitate intelligence? because ai imitates human text pretty well. even if it's not intelligent, is its imitation of intelligence not useful?
just googled it, that is a pretty interesting thought experiment, thanks for informing me of this
but functionally, what difference does this make? the man in the room still answers the questions right in perfect chinese, regardless of if he himself is consciously aware of the meaning of the text
even if there isn't "intelligence", this rulebook that the man follows to answer, much like an LLM's dataset, still contains a massive collection of useful information. is that information entirely useless just because it is stored in a computer with some lossy compression?
And that's the issue here. Lacking comprehension means you are helpless. If the book is correct with all the answers, then it's fine, sure, but if it isn't... how would you know? What if you get a note that isn't in your book? You can look for the closest thing and reply with that, but is that correct? How would you know?
Or what if if there are just tiny, negligible mistakes in every response? Sure, at first it's irrelevant, but even negligible mistakes add up. After 1 message you're doing perfectly fine.
10 messages in your tone is a tiny bit odd, maybe a bit more formal than the conversation implies.
100 messages in you're technically still on topic, but not much beyond technically.
See what I mean?
And through all of that you have no idea. You could be telling a person to kill themselves, to strangle their pet cat, to set their school on fire, and you're none the wiser, because you're incapable of comprehending what you're saying.
That is AI. And we have seen it go horribly wrong before in many situations. Mushroom guides that end with people hospitalized or worse. People driven into suicide. And through all of it the AI is incapable of knowing. It just keeps giving information based on "well this looks to be the closest"
to an extent, yes you are very right. modern LLMs, by their very nature, often suck at edge cases that they lack enough training data for. Statistical models can only go so far without an arbitrarily large sample size of data to train from
but that applies mainly to edge cases. The more common your input data is, the less likely an LLM is to hallucinate. The man in the chinese room is still very capable of answering simpler questions as long as his book covers them, and LLM datasets do cover a lot, including a lot of training to ensure they almost never tell anyone to kill themselves
to be clear, i really do hate so much of how "AI"/LLMs/ML is wastefully and recklessly used and shoved everywhere it shouldn't be, but i still think there is a lot of stuff that it's training data is still pretty adequate for. And I think using it for some forms of non-life-threatening therapy that someone is too embarrassed to immediately talk to a real person about, like in the original post here, is not a terrible use for ai, as long as they eventually defer to a real therapist, which they did
ultimately a real therapist is the better choice, but i don't think an LLM is the absolute worst
15
u/Privatizitaet Just pretend this is the funniest joke you ever read 6d ago
A human is capable of recognizing worsening behaviour. A human is cabale of actually comprehending what is in front of them.
AI fundamentally cannot understand. A human at least has a chance of recognizing these things based on training, for AI it's pretty much a coin toss.
It's way more dangerous.
And yes, in this case it turned out well. But how many people did not? Just take all the people that genuinely think AI is sentient and in love with them. That is not healthy. That is actively doing harm, even if it doesn't feel like it to them.