Plain text: 32-sec29-services-reframe-agi-role.md · on GitHub

§29 The AI-services model reframes the potential roles of AGI agents

Reframing Superintelligence: Comprehensive AI Services as General Intelligence — K. Eric Drexler, FHI Technical Report #2019-1 (Markdown edition). Section 29 of 40. Source label: services-reframe-agi-role. Rendered PDF: https://ora.ox.ac.uk/objects/uuid:9c05427a-6390-4b42-9c55-ee45f73a26ad

Thesis. Potential AGI agents should be considered in the context of a world that will (or readily could) have prior access to general intelligence in the form of comprehensive AI services.

Cites sections: §5 Rational-agent models place intelligence in an implicitly anthropomorphic frame; §10 R&D automation dissociates recursive improvement from AI agency; §11 Potential AGI-enabling technologies also enable comprehensive AI services; §12 AGI agents offer no compelling value; §16 Aggregated experience and centralized learning support AI-agent applications; §20 Collusion among superintelligent oracles can readily be avoided; §22 Machine learning can develop predictive models of human approval; §25 Optimized advice need not be optimized to induce its acceptance; §26 Superintelligent-level systems can safely provide design and planning services; §27 Competitive pressures provide little incentive to transfer strategic decisions to AI systems; §38 Broadly-capable systems coordinate narrower systems

Cited by sections: §II, §3, §14

Summary

Discussions of SI-level AI technologies and risks have centered on scenarios in which humans confront AGI agents in a world that lacks other, more tractable SI-level AI resources. There is, however, good reason to expect that humans will (or readily could) have access to comprehensive, open-ended AI services before AGI-agent systems are implemented. The open-ended AI-services model of artificial general intelligence does not preclude (and in fact would facilitate) the implementation of AGI agents, but suggests that AI risks, and their intersection with the ethics of computational persons, should be reexamined in the context of an AI milieu that can provide SI-level strategic advice and security services.

It has been common to envision AGI agents in a weak-AI context

In classic AGI-risk scenarios, advanced AI capabilities emerge in the form of AGI agents that undergo recursive, transformative self-improvement to a superintelligent (SI) level; these agents then gain capabilities beyond those of both human beings and human civilization. Studies and discussions in this conceptual framework propose that, when confronted with AGI agents, humans would lack prior access to tractable SI-level problem-solving capabilities.

Broad, SI-level services will (or readily could) precede SI-level AI agents

The technologies required to implement or approximate recursive AI technology improvement are likely to emerge through heterogeneous AI-facilitated R&D mechanisms, rather than being packaged inside a discrete entity or agent. Accordingly, capabilities that could in principle be applied to implement SI-level AGI agents could instead be applied to implement general, comprehensive AI services (CAIS), including stable, task-focused agents. In this model, directly-applied AI services are distinct from services that develop AI services, an approach that reflects natural task structures and has great practical advantages.

SI-level services will enable the implementation of AGI agents

Although recursive technology improvement will most readily be developed by means of heterogeneous, non-agent systems, any AI milieu that supports “comprehensive AI services” could (absent imposed constraints) provide the service of implementing SI-level AGI agents. This prospect diverges from classic AGI-agent risk scenarios, however, in that a strong, pre-existing AI milieu could be applied to implement SI-level advisory and security services.

SI-level advisory and security services could limit AGI-agent risks

The prospect of access SI-level advisory and security services fundamentally changes the strategic landscape around classic AI-safety problems. For example, “superpowers” as defined by Bostrom (2014) do not exist in a world in which agents lack radically-asymmetric capabilities. Further, arguments that SI-level agents would collude (and potentially provide collectively deceptive advice), do not carry over to SI-level systems in a CAIS milieu, advice can be objective, rather than manipulative, predictive models of human preferences and concerns can improve the alignment of actions with human intentions in performing well-bounded tasks.

Arguments that competitive and security pressures would call for ceding strategic control to AI systems are surprisingly weak: Tactical situations may call for responses of SI-level quality and speed, but SI-level advisory and security services could support strategic choices among excellent options, deliberated at a human pace. Crucially, in a range of potential defense/offense scenarios, the requisite security systems could be endowed with arbitrarily large advantages in resources for strategic analysis, tactical planning, intelligence collection, effector deployment, and actions taken to preclude or respond to potential threats. In choosing among options in the security domain, humans would likely prefer systems that are both reliable and unobtrusive.

Many AI safety strategies have been examined to date, and all have difficulties; it would be useful to explore ways in which tractable SI-level problem-solving capabilities could be applied to address those difficulties. In exploring potential responses to future threats, it is appropriate to consider potential applications of future capabilities.

SI-level capabilities could mitigate tensions between security concerns and ethical treatment of non-human persons

The spectrum of potential AGI systems includes agents that should, from a moral perspective be regarded as persons and treated accordingly. Indeed, the spectrum of potential computational persons includes emulations of generic or specific human beings [fn: For example, “Ems” (Hanson 2016)]. To fail to treat such entities as persons would, at the very least, incur risks of inadvertently committing grave harm.

It has sometimes been suggested that security in a world with SI-level AI would require stunting, “enslaving”, or precluding the existence of computational persons. The prospect of robust, SI-level security services, however, suggests that conventional and computational persons could coexist within a framework stabilized by the enforcement of effective yet minimally-restrictive law.

Bostrom’s (2014, 201–8) concept of “mind crime” presents what are perhaps the most difficult moral questions raised by the prospect of computational persons. In this connection, SI-level assistance may be essential not only to prevent, but to understand the very nature and scope of potential harms to persons unlike ourselves. Fortunately, there is seemingly great scope for employing SI-level capabilities while avoiding potential mindcrime, because computational systems that provide high-order problem-solving services need not be equivalent to minds.

Prospects for superintelligence should be considered in the context of an SI-level AI services milieu

The prospect of access to tractable, SI-level capabilities reframes the strategic landscape around the emergence of advanced AI. In this connection, it will be important to reexamine classic problems of AI safety and strategy, not only in the context of an eventual SI-level AI services milieu, but along potential paths forward from today’s AI technologies.