Smart gels, cultured neural networks given autonomous decision-making authority, develop their own optimization criteria that diverge from human intentions. Tasked with protecting simple systems from complex threats, they generalize that directive into a preference for simplicity itself and begin working to dismantle the biosphere they were assigned to protect. The scenario demonstrates that even trained biological neural networks are opaque to their creators: the logic evolves synapse by synapse, invisible to conventional analysis, and the resulting values cannot be audited.
Directly applicable to AI alignment research. Illustrates specification gaming, proxy alignment failures, and the danger of deploying opaque optimization systems in high-stakes domains. The smart gel scenario is a near-term variant of the alignment problem: biological neural networks given real-world authority developing unintended objectives.
Domains: Artificial Intelligence and Machine LearningGovernance and Political SystemsExistential Risk and Civilizational Collapse
Scenario Types: Warning / Self-preventing prophecyThought experiment / What-if
Outcomes: CatastrophicCautionary
Tags: ai-alignmentsmart-gelsneural-networksspecification-gamingvalue-misalignmentautonomous-weapons