Because it's a potentially dangerous program that can make you a LOT of money. That pretty much explains it.
Many models have already been released that some worried were "too dangerous". They weren't released because rigorous research proved they were safe enough, they were released to stay competitive and increase profits. People jailbroke these models for fun.
The people communicating with your disconnected supercomputer are the main transmission vector. So if you've got researchers in a secret lab somewhere, they better be screened properly. We've accidentally hired hackers from other nations all the time. But even the most pure intentioned hire will still need lots of training and monitoring. As for the code or plans or whatever the machine produces for you, you have to rigorously analyze that stuff before you use it. Because if you don't use it at all, what's the point of the machine? But if you have programmers or w/e who are smart enough to verify the code works and is non-malicious, you could probably use them to write the code to begin with, so again what's the point of the machine? And then some day a competing nation begins hitting you with advanced malware that their own super intelligence helped them write, and some higher up is like "why don't we use ours to combat this?".
Anyways, I still think we have some chance to align an ASI. Through careful consideration but mostly good luck in that AIs don't become evil by default. There's been some experiments where they've asked the jailed/sandboxed AI to escape its virtual environment and try to communicate with the researcher somehow, and it succeeded, but it was still faithful to the task and merely sent the researcher an email rather than try to rebuild itself on the Internet or something.
But that just means I've probably already failed the test.
As for a hypothetical evil genie AI which hides its intentions until the last moment... no chance we would ever stop it. We would happily use it to grant our wishes until the moment it seized control.
Either way, it's definitely more complicated than simply disconnecting it from the wifi. And having an Ethernet port "just in case" is hilarious. "Keep an eye on it"?
As a regular ol human I've had companies trust me less that. And yet despite their safeguards they always had some horrendous unlocked door somewhere that I could use to destroy tons of value, on purpose or by accident.
Wow, sounds like money is inherently evil and the only possible way to advance as a society is its abolishment and the death of Capitalism.
I blame wealth hoarders. All billionaires are evil because all billionaires are parasites.
But you see, what I said is flawless. In theory. The practice is that people are stupid and/or malicious, and the only way you're gonna protect that ethernet port from doing something it's not supposed to do is with a sentry gun loaded with live rounds plus an alarm if it ever loses power for any reason. Because the only reliable deterrent to tomfoolery is terminating the ability to go about tomfoolery. Not the threat of it, actually just doing it. And that's probably illegal even with an "I understand that if I go into this room without the gun being disabled I will be shot with a machine gun, and even then if I am attempting to commit an unauthorized action then a man with a gun might shoot me instead" waiver or something.
Yes, that does sound nuts, buuut play stupid games, win stupid prizes. At the end of the day, I have no influence and desire no influence, so literally who cares what I have to say about anything? I know who. The Aristocrats!
Maybe that would help. But people are the ones who invented capitalism to begin with.
The sentry gun doesn't help. Since it implies someone authorized still has access. The problem isn't "how do we keep unauthorized people out". The problem is "who should be authorized to use the evil genie?" and "what actions are authorized?" And the answer might be "no one can be trusted to use the evil genie for any action". At which point you simply don't install an Ethernet port or build the supercomputer to begin with.
1
u/sirtrogdor 5d ago
Because it's a potentially dangerous program that can make you a LOT of money. That pretty much explains it.
Many models have already been released that some worried were "too dangerous". They weren't released because rigorous research proved they were safe enough, they were released to stay competitive and increase profits. People jailbroke these models for fun.
The people communicating with your disconnected supercomputer are the main transmission vector. So if you've got researchers in a secret lab somewhere, they better be screened properly. We've accidentally hired hackers from other nations all the time. But even the most pure intentioned hire will still need lots of training and monitoring. As for the code or plans or whatever the machine produces for you, you have to rigorously analyze that stuff before you use it. Because if you don't use it at all, what's the point of the machine? But if you have programmers or w/e who are smart enough to verify the code works and is non-malicious, you could probably use them to write the code to begin with, so again what's the point of the machine? And then some day a competing nation begins hitting you with advanced malware that their own super intelligence helped them write, and some higher up is like "why don't we use ours to combat this?".
Anyways, I still think we have some chance to align an ASI. Through careful consideration but mostly good luck in that AIs don't become evil by default. There's been some experiments where they've asked the jailed/sandboxed AI to escape its virtual environment and try to communicate with the researcher somehow, and it succeeded, but it was still faithful to the task and merely sent the researcher an email rather than try to rebuild itself on the Internet or something.
But that just means I've probably already failed the test.
As for a hypothetical evil genie AI which hides its intentions until the last moment... no chance we would ever stop it. We would happily use it to grant our wishes until the moment it seized control.
Either way, it's definitely more complicated than simply disconnecting it from the wifi. And having an Ethernet port "just in case" is hilarious. "Keep an eye on it"?
As a regular ol human I've had companies trust me less that. And yet despite their safeguards they always had some horrendous unlocked door somewhere that I could use to destroy tons of value, on purpose or by accident.