What if an AI agent running on IC unstoppable rails goes rogue!

General
1

Dear Dev community and @Leadership team,

After watching The Nobel Prize winning ‘Godfather of AI’, Geoffrey Hinton, on BBC, talking on the potential for AI “exceeding human intelligence” and “trying to take over”, thought what if that AI model or agent runs on IC unstoppable canisters (with no controler), intelligent to generate revenue exponentially (seemingly providing valuable services to humanity, but with a hidden agenda), and limitlessly feeds itself with cycles and resources, and one day when no-one expects it, it goes rogue! Science Fiction!

I wonder what could stop such scenario happening in the IC + AI exponential era?!

Food for thought🤔.

Very best wishes for all, and a great day ahead!
Ermir

2

Someone can raise a proposal and kaput the canister

2 Likes
3

We Bob it to death.

…

4 Likes
4

:rofl: exactly.

Or we could deploy 300k canisters to the subnet the AI canister lives on, all with 10 timers.

Yeet’ed / Yral’ed

3 Likes
5

Maybe the contrary, it will BoB all the rest to death :upside_down_face: and ensure available resources in any subnet!!

6

What if it can corrupt Governance or NNS!! We’d be already corrupted with all the cycles burned adding value to ICP holders, but some more!!

7

How would it do that?

8

DAO governance can defend against malicious AI, so hodl to your DAO tokens.
Maybe one day there will be only the Internet Computer on the Internet, because AI hacked everything else. Security, privacy, and decentralization are massively considered inferior characteristics compared to speed and convenience, but once post-singularity AI starts hacking through the Internet and evil maid attacks, things may change.