# §14 The AI-services model brings ample risks *Reframing Superintelligence: Comprehensive AI Services as General Intelligence — K. Eric Drexler, FHI Technical Report #2019-1 (Markdown edition). Section 14 of 40. Source label: `services-risks`. Rendered PDF: https://ora.ox.ac.uk/objects/uuid:9c05427a-6390-4b42-9c55-ee45f73a26ad* **Thesis.** High-level AI services could facilitate the development or emergence of dangerous agents, empower bad actors, and accelerate the development of seductive AI applications with harmful effects. **Cites sections:** §10 *R&D automation dissociates recursive improvement from AI agency*; §11 *Potential AGI-enabling technologies also enable comprehensive AI services*; §12 *AGI agents offer no compelling value*; §16 *Aggregated experience and centralized learning support AI-agent applications*; §22 *Machine learning can develop predictive models of human approval*; §23 *AI development systems can support effective human guidance*; §26 *Superintelligent-level systems can safely provide design and planning services*; §29 *The AI-services model reframes the potential roles of AGI agents* **Cited by sections:** §II, §4, §6, §7 --- ## Summary Prospects for general, high-level AI services reframe—but do not eliminate—a range of AI risks. On the positive side, access to increasingly comprehensive AI services (CAIS) can reduce the practical incentives for developing potentially problematic AGI agents while providing means for mitigating their potential dangers. On the negative side, AI services could facilitate the development of dangerous agents, empower bad actors, and accelerate the development of seductive AI applications with harmful effects. A further concern—avoiding perverse agent-like behaviors arising from interactions among service providers—calls for further study that draws on agent-centric models. Taking the long view, the CAIS model suggests a technology-agnostic, relatively path-independent perspective on potential means for managing SI-level AI risks. ## Prospects for general, high-level AI services reframe AI risks In a classic model of high-level AI risks, AI development leads to self-improving agents that gain general capabilities and enormous power relative to the rest of the world. The [AI-services model](→§12) points to a different prospect: [Continued automation of AI R&D](→§10) (viewed as an increasingly-comprehensive set of development services) leads to a general ability to implement systems that provide AI services, ultimately scaling to a superintelligent level. Prospects for comprehensive AI services (CAIS) contrast sharply with classic expectations that center on AGI agents: The leading risks and remedies differ in both nature and context. ## CAIS capabilities could mitigate a range of AGI risks On the positive side, capabilities within the CAIS model can be applied to mitigate AGI risks. The CAIS model arises naturally from current trends in AI development and outlines [a more accessible path](→§10) to general AI capabilities; as a consequence, CAIS points to a future in which [AGI agents have relatively low marginal instrumental value](→§12) and [follow rather than lead the application of superintelligent-level AI functionality](→§11) to diverse problems. Accordingly, the CAIS model suggests that high-level agents will (or readily could) be developed [in the context of safer, more tractable AI systems](→§29) that can provide services useful for managing such agents. [Predictive models of human concerns](→§22) are a prominent example of such services; others include [AI-enabled capabilities for AI-systems design](→§26), [analysis](→§23), [monitoring, and upgrade](→§16). ## CAIS capabilities could facilitate the development of dangerous AGI agents Comprehensive AI services necessarily include the service of developing useful AI agents with stable, bounded capabilities, but superintelligent-level CAIS could also be employed to implement general, autonomous, self-modifying systems that match the specifications for risky AGI agents. If not properly directed or constrained—a focus of current AI-safety research—such agents could pose catastrophic or even existential risks to humanity. The AI-services model suggests broadening studies of AI safety to explore potential applications of CAIS-enabled capabilities to risk-mitigating differential technology development, including AI-supported means for developing safe AGI agents. ## CAIS capabilities could empower bad actors. AI-service resources *per se* are neutral in their potential applications, and human beings can already apply AI services and products to do intentional harm. Access to advanced AI services could further empower bad actors in ways both expected and as-yet unimagined; in compensation, advanced AI services could enable detection and defense against bad actors. Prospective threats and mitigation strategies call for exploration and study of novel policy options. ## CAIS capabilities could facilitate disruptive applications Today’s disruptive AI applications are services that serve some practical purpose. As shown by current developments, however, one person’s service may be another person’s threat, whether to business models, employment, privacy, or military security. Increasingly comprehensive and high-level AI services will continue this trend, again in ways both expected and as-yet imagined. Mitigation of non-malicious disruption raises political, legal, and economic questions, because both the disruptors and the disrupted may have conflicting yet legitimate interests. ## CAIS capabilities could facilitate seductive and addictive applications Some disruptive applications provide seductive services that are detrimental, yet welcomed by their targets. AI systems today are employed to make games more addictive, to build comfortable filter bubbles, and to optimize message channels for appeal unconstrained by truth. There will be enormous scope for high-level AI systems to please the people they harm, yet mitigation of the individual and societal consequences of unconstrained seductive and addictive services raises potentially intractable questions at the interface of values and policy. ## Conditions for avoiding emergent agent-like behaviors call for further study Although it is important to distinguish between pools of AI services and classic conceptions of integrated, opaque, utility-maximizing agents, we should be alert to the potential for coupled AI services to develop emergent, unintended, and potentially risky agent-like behaviors. Because there is no bright line between agents and non-agents, or between rational utility maximization and reactive behaviors shaped by blind evolution, avoiding risky behaviors calls for at least two complementary perspectives: both (1) design-oriented studies that can guide implementation of systems that will provide requisite degrees of *e.g.*, stability, reliability, and transparency, and (2) agent-oriented studies support design by exploring the characteristics of systems that could display emergent, unintended, and potentially risky agent-like behaviors. The possibility (or likelihood) of humans implementing highly-adaptive agents that pursue open-ended goals in the world (*e.g.*, money-maximizers) presents particularly difficult problems. ## Further Reading - §3 *To understand AI prospects, focus on services, not implementations* - §7 *Training agents in human-like environments can provide useful, bounded services* - §10 *R&D automation dissociates recursive improvement from AI agency* - §12 *AGI agents offer no compelling value*