Plain text: 16-sec13-agi-complexity.md · on GitHub

§13 AGI-agent models entail greater complexity than CAIS

Reframing Superintelligence: Comprehensive AI Services as General Intelligence — K. Eric Drexler, FHI Technical Report #2019-1 (Markdown edition). Section 13 of 40. Source label: agi-complexity. Rendered PDF: https://ora.ox.ac.uk/objects/uuid:9c05427a-6390-4b42-9c55-ee45f73a26ad

Thesis. Relative to comprehensive AI services (CAIS), and contrary to widespread intuitions, the classic AGI-agent model implicitly increases (while obscuring) the complexity and challenges of self-improvement, general functionality, and AI goal alignment.

Summary

Recent discussions suggest that it would be useful to compare the relative complexities of AGI-agent and comprehensive AI services (CAIS) models of general intelligence. The functional requirements for open-ended self-improvement and general AI capabilities are the same in both instances, but made more difficult in classic AGI models, which require that fully-general functionality be internal to an autonomous, utility-directed agent. The rewards for accomplishing this compression of functionality are difficult to see. To attempt to encompass general human goals within the utility function of a single, powerful agent would reduce none of the challenges of aligning concrete AI behaviors with concrete human goals, yet would increase the scope for problematic outcomes. This extreme compression and its attendant problems are unnecessary: Task-oriented AI systems within the CAIS framework could apply high-level reasoning and broad understanding to a full spectrum of goals, coordinating open-ended, collectively-general AI capabilities to provide services that, though seamlessly integrated, need not individually or collectively behave as a unitary AGI agent.

Classic AGI models neither simplify nor explain self improvement

The classic AGI agent model posits open-ended self improvement, but this simple concept hides what by nature must be functionally equivalent to fully-automated and open-ended AI research and development. Hiding the complexity of AI development in a conceptual box provides only the illusion of simplicity. Discussion within the classic AGI model typically assumes an unexplained breakthrough in machine learning capabilities. For simplicity, an AI-services model could arbitrarily assume equivalent capabilities (perhaps based on the same hypothetical breakthrough), a deeper model offers a framework for considering their implementation. To be comprehensive, AI services must of course include the service of developing new services, and current research practice shows that expanding the scope of AI services can be both incremental and increasingly automated.

Classic AGI models neither simplify nor explain general AI capabilities

Similarly, the classic AGI agent model posits systems that could provide general, fluidly-integrated AI capabilities, but this seemingly simple concept hides what by nature must be functionally equivalent to a comprehensive range of AI services and coordination mechanisms. The classic model assumes these capabilities without explaining how they might work; for simplicity, an AI-services model could arbitrarily assume equivalent capabilities, but a deeper model offers a framework for considering how diverse, increasingly comprehensive capabilities could be developed and integrated by increasingly automated means.

Note that, by intended definition, the “C” in CAIS is effectively equivalent to the “G” in AGI. Accordingly, to propose that an AGI agent could provide services beyond the scope of CAIS is either to misunderstand the CAIS model, or to reject it, e.g., on grounds of feasibility or coherence. To be clear, fully realized CAIS services would include the service of coordinating and providing a seamless interface to other services, modeling behaviors one might have attributed to aligned AGI agents. The CAIS model of course extends to the provision of potentially dangerous services, including the service of building unaligned AGI agents.

Classic AGI models increase the challenges of AI goal alignment

The classic AGI model posits the construction of a powerful, utility-directed, superintelligent agent, a conceptual move that both engenders the problems of aligning a superintelligent agent’s overall goals with human values and amalgamates and abstracts away the concrete problems that arise in aligning specific, useful behaviors with diverse and changing human goals. Although a simplified AI-services model could arbitrarily assume aligned systems with bounded goals and action spaces, a deeper model offers a framework for considering how such systems could be developed and how their development might go wrong—for example, by indirectly and inadvertently giving rise to agents with problematic goals and capabilities. Many of the questions first framed as problems of AGI safety still arise, but in a different and perhaps more tractable systemic context.

The CAIS model addresses a range of problems without sacrificing efficiency or generality

To summarize, in each of the areas outlined above, the classic AGI model both obscures and increases complexity: In order for general learning and capabilities to fit a classic AGI model, they must not only exist, but must be integrated into a single, autonomous, self-modifying agent. Further, achieving this kind of integration would increase, not reduce, the challenges of aligning AI behaviors with human goals: These challenges become more difficult when the goals of a single agent must motivate all (and only) useful tasks.

Agent-services that are artificial, intelligent, and general are surely useful, both conceptually and in practice, but fall within the scope of comprehensive agent (and non-agent) AI services. The key contribution of the CAIS model is to show how integrated, fully-general AI capabilities could be provided within an open-ended architecture that is natural, efficient, relatively transparent, and quite unlike a willful, uniquely-powerful agent.