Plain text: 36-sec33-ai-not-boxed.md · on GitHub

§33 Competitive AI capabilities will not be boxed

Reframing Superintelligence: Comprehensive AI Services as General Intelligence — K. Eric Drexler, FHI Technical Report #2019-1 (Markdown edition). Section 33 of 40. Source label: ai-not-boxed. Rendered PDF: https://ora.ox.ac.uk/objects/uuid:9c05427a-6390-4b42-9c55-ee45f73a26ad

Thesis. Because the world’s aggregate AI capacity will greatly exceed that of any single system, the classic “AI confinement” challenge (with AI in a box and humans outside) is better regarded as an idealization than as a concrete problem situation.

Cites sections: §10 R&D automation dissociates recursive improvement from AI agency; §15 Development-oriented models align with deeply-structured AI systems; §20 Collusion among superintelligent oracles can readily be avoided

Cited by sections: §II, §5

Summary

Current trends suggest that superintelligent-level AI capabilities will emerge from a distributed, increasingly automated process of AI research and development, and it is difficult to envision a scenario in which a predominant portion of overall AI capacity would (or could) emerge and be confined in “a box”. Individual systems could be highly problematic, but we should expect that AI systems will exist in a milieu that enables instantiation of diverse peer-level systems, a capability that affords scalable, potentially effective mechanisms for managing threatening AI capabilities.

SI-level capabilities will likely emerge from incremental R&D automation

Current trends in machine learning point to an increasing range of super-human capabilities emerging from extensions of today’s technology base that emerge from extensions of today’s R&D milieu. Today we see a distributed, increasingly automated R&D process that employs a heterogeneous toolset to develop diverse demonstration, prototype, and application systems. Increasingly, we find that AI applications include AI development tools, pointing the way toward thorough automation of development that would enable AI progress at AI speed—an incremental model of asymptotically recursive technology improvement.

We can expect AI R&D capacity to be distributed widely, beyond any “box”

Current methods in machine learning suggest that access to large-scale machine resources will be critical to competitive performance in AI technology and applications development. The growing diversity of technologies and applications (e.g., to vision, speech recognition, language translation, ML architecture design…) speak against the idea that a world-class R&D process (or its functional equivalent) will someday be embodied a distinct, general-purpose system. In other words, it is difficult to envision plausible scenarios in which we find more AI capability “in a box” than in the world outside.

AI systems will be instantiated together with diverse peer-level systems

We should expect that any particular AI system will be embedded in an extended AI R&D ecosystem having aggregate capabilities that exceed its own. Any particular AI architecture will be a piece of software that can be trained and run an indefinite number of times, providing multiple instantiations that serve a wide range of purposes (a very wide range of purposes, if we posit truly general learning algorithms). As is true today, we can expect that the basic algorithms and implementation techniques that constitute any particular architecture will be deployed in diverse configurations, trained on diverse data, and provide diverse services.

The ability to instantiate diverse, highly-capable systems presents both risks and opportunities for AI safety

Absent systemic constraints, advanced AI technologies will enable the implementation of systems that are radically unsafe or serve abhorrent purposes. These prospects can be classed as bad-actor risks that, in this framing, include actions that incur classic AGI-agent risks.

The almost unavoidable ability to instantiate diverse AI systems at any given level of technology also offers benefits for AI reliability and safety. In particular, the ability to instantiate diverse peer-level AI systems enables the use of architectures that rely on implicitly competitive and adversarial relationships among AI components, an approach that enables the use of AI systems to manage other AI systems while avoiding concerns regarding potential collusion. Both competitive and adversarial mechanisms are found in current AI practice, and scale to a superintelligent level.