Plain text: 18-sec15-development-oriented.md · on GitHub

§15 Development-oriented models align with deeply-structured AI systems

Reframing Superintelligence: Comprehensive AI Services as General Intelligence — K. Eric Drexler, FHI Technical Report #2019-1 (Markdown edition). Section 15 of 40. Source label: development-oriented. Rendered PDF: https://ora.ox.ac.uk/objects/uuid:9c05427a-6390-4b42-9c55-ee45f73a26ad

Thesis. Unitary, unstructured models of superintelligent systems are natural objects of theoretical study, but development-oriented models suggest that advanced AI systems will in practice comprise deeply structured compositions of differentiated components.

Cited by sections: §II, §5, §11, §12, §17, §18, §21, §23, §33, §36

Summary

AI safety research has focused on unitary, relatively unstructured models of superintelligent systems. Development-oriented models, however, suggest that advanced AI systems will in practice comprise or employ structured systems of task-oriented components. Prospects for heterogeneous, deeply structured systems are best understood by considering development processes in which structure results from composing components, not from partitioning a hypothetical unitary functionality. A focus on development processes that lead to structured products links AI safety to ongoing AI research practice and suggests a range of topics for further inquiry.

AI safety research has often focused on unstructured rational-agent models

Research in AI safety has been motivated by prospects for recursive improvement that enables the rapid development of systems with general, superhuman problem-solving capabilities. Research working within this paradigm has centered not on the process of recursive improvement, but on its potential products, and these products have typically been modeled as discrete, relatively unstructured, general-purpose AI systems.

Structured systems are products of structured development

In an alternative, potentially complementary model of high-level AI, the products of recursive improvement are deeply structured, task-focused systems that collectively deliver a comprehensive range of superintelligent task capabilities, yet need not be (or become) discrete entities that individually span a full range of capabilities. This model has been criticized as potentially incurring high development costs, hence prospects for deploying differentiated (but not necessarily narrow) AI systems with a collectively comprehensive range of task capabilities may best be approached through an explicit consideration of potential AI research and development processes.

AI development naturally produces structured AI systems

Today, the AI research community is developing a growing range of relatively narrow, strongly differentiated AI systems and composing them to build systems that embrace broader domains of competence. A system that requires, for example, both vision and planning will contain vision and planning components; a system that interprets voice input will contain interacting yet distinct speech recognition and semantic interpretation components. A self-driving car with a conversational interface would include components with all of the above functionalities, and more. The principle of composing differentiated competencies to implement broader task-performance naturally generalizes to potential systems that would perform high-order tasks such as the human-directed design and management of space transportation systems, or AI research and development.

Structure arises from composing components, not partitioning unitary systems

If one begins with unitary systems as a reference model, the task of implementing structured, broadly competent AI systems may appear to be a problem of imposing structure by functional decomposition, rather than one of building structures by composing functional components. In other words, taking a unitary-system model as a reference model focuses attention on how a hypothetical system with unitary, universal competence might be divided into parts. While this framing may be useful in conceptual design, it can easily lead to confusion regarding the nature of structured AI system development and products.

A development-oriented approach to deeply structured systems suggests a broad range of topics for further inquiry

A focus on AI development links AI safety studies to current R&D practice. In particular, the prospective ability to deliver deeply-structured, task-focused AI systems offers rich affordances for the study and potential implementation of safe superintelligent systems.

The joint consideration of structured AI development and products invites inquiry into a range of topics, including:

Further Reading