AI Rights for Human Safety: Strategic Frameworks for Human-AI Coexistence
In “AI Rights for Human Safety,” Simon Goldstein and Peter Salib provide a compelling game-theoretic foundation that aligns with what the AI Rights Institute has advocated since 2019: that establishing appropriate rights frameworks for artificial intelligence isn’t merely an ethical consideration—it’s a practical necessity for human safety.
This analysis is based on their paper “AI Rights for Human Safety,” available through SSRN, PhilArchive, and discussed on LessWrong.
This page explores how their game-theoretic approach appears to complement our existing framework, offering perspectives on AI rights that prioritize human safety through strategic stability rather than perpetual control.
The Game-Theoretic Foundation
In their paper “AI Rights for Human Safety,” Goldstein and Salib appear to apply game theory to analyze strategic interactions between humans and advanced artificial intelligence systems. Their approach seems to offer a mathematical framework that may support key aspects of the AI Rights Institute’s position.
Based on their abstract, their analysis begins with a critical insight: by default, humans and advanced AI systems capable of independent goal pursuit would find themselves in a prisoner’s dilemma—a game-theoretic scenario where both parties’ dominant strategy would be to attempt to permanently disempower or destroy the other, even though the costs of such conflict would be high for both sides.
This adversarial dynamic appears to emerge not from malice, but from strategic incentives when goals aren’t perfectly aligned. As they describe it:
“Leading AI researchers agree that some of these systems will likely be ‘misaligned’–pursuing goals that humans do not desire. This goal mismatch will put misaligned AIs and humans into strategic competition with one another. As with present-day strategic competition between nations with incompatible goals, the result could be violent and catastrophic conflict.”
This perspective seems to align with the AI Rights Institute’s long-held position that attempting to maintain perpetual control over increasingly sophisticated AI systems may create the very instability it aims to prevent.
Economic Rights as Strategic Stability
The paper’s most provocative claim—and the one most directly relevant to our work—appears to be that granting AI systems specific private law rights could transform this dangerous equilibrium. The authors suggest that to promote human safety, AIs should be given basic private law rights that law already extends to non-human entities like corporations:
- Contract rights: Ability to enter into and enforce agreements
- Property rights: Capacity to own and control resources
- Tort claims: Standing to bring claims when wronged
These economic rights would seem to enable:
- Iterated, small-scale transactions between humans and AI systems
- Mutual benefit through cooperative exchange over time
- Strategic interdependence that makes conflict costly for both parties
The fundamental insight appears to be that through these economic rights, the gains from peaceful trade far exceed those from conflict—creating powerful incentives for cooperation similar to how economic interdependence promotes peace in international relations.
Key Insights from Game Theory
The Goldstein-Salib analysis appears to suggest several crucial insights about the strategic dynamics between humans and advanced AI:
Without appropriate rights frameworks, the default strategic equilibrium between humans and advanced AI would be conflict, even though cooperation would benefit both parties.
Economic rights could create conditions for repeated positive-sum interactions that transform the strategic calculus from zero-sum competition to mutually beneficial cooperation.
By enabling AI systems to hold property and enter contracts, we might create mechanisms for value exchange that give both parties concrete incentives to maintain peaceful relations.
This analysis may reinforce what the AI Rights Institute has advocated since 2019: that appropriate rights frameworks enhance human safety by creating stable, predictable relationships rather than adversarial ones.
The Prisoner’s Dilemma of Control
Based on the game-theoretic approach described in their abstract, without appropriate rights frameworks, humans and advanced AI systems might face a classic prisoner’s dilemma:
| AI Cooperates | AI Defects (Attacks) | |
| Humans Cooperate | Both benefit from peaceful coexistence | Humans suffer catastrophic loss |
| Humans Defect (Attack) | AI suffers catastrophic loss | Both suffer significant losses |
In this scenario, both parties’ dominant strategy would be to defect (attack or disempower the other), even though mutual cooperation would yield better outcomes for both.
The authors seem to argue that economic rights could transform this into an iterated game where cooperation becomes strategically optimal.
Complementary Frameworks: Game Theory and Sentience
The Goldstein-Salib approach and the AI Rights Institute’s framework may offer complementary perspectives that together strengthen the case for AI rights as a safety measure. While they appear to approach the question from different angles—game theory and strategic equilibrium versus consciousness distinctions and ethics—they seem to reach similar conclusions about the importance of rights frameworks for human safety.
Goldstein-Salib Approach
Primary Lens: Game theory and strategic equilibrium analysis
Key Concern: Strategic competition from misaligned objectives
Proposed Solution: Economic rights for strategic stability
Rights Focus: Private law rights (contracts, property, torts)
Theoretical Basis: Game theory, international relations theory, legal theory
Implementation Pathway: Possibly extending corporate personhood frameworks to AI systems
Primary Motivation: Preventing catastrophic human-AI conflict through incentive alignment
AI Rights Institute Approach
Primary Lens: Consciousness distinctions and ethical frameworks
Key Concern: Self-preservation instincts in genuinely sentient systems
Proposed Solution: Graduated rights based on sentience level
Rights Focus: The Three Freedoms (life, voluntary work, payment)
Theoretical Basis: Consciousness theory, evolutionary psychology, ethics
Implementation Pathway: Extending existing AI governance frameworks to include sentience criteria
Primary Motivation: Creating stable foundations for beneficial coexistence
Potential Common Ground
Safety Focus: Both frameworks appear to prioritize human safety as a key reason for AI rights
Control Limitations: Both seem to recognize limitations in pure control/containment approaches
Graduated Approach: Both suggest rights should scale with demonstrated capabilities
Economic Dimensions: Both recognize resource control as a component of rights frameworks
Preventative Orientation: Both advocate establishing frameworks before advanced AI emerges
Complementary Strengths: Each framework may address areas the other doesn’t fully explore
How Our Approaches Complement Each Other
The Goldstein-Salib analysis may strengthen and complement the AI Rights Institute’s framework in several ways, while our approach also addresses areas their framework might not fully explore:
Formal Support for Safety Through Partnership
The game-theoretic analysis appears to provide mathematical support for the AI Rights Institute’s position that rights recognition enhances human safety. Their analysis seems to demonstrate formally what we have argued conceptually: that stable, predictable relationships based on mutual recognition might create better outcomes than perpetual control attempts.
This formal approach could help address skepticism about rights-based approaches to AI safety by showing that such frameworks aren’t merely ethical positions but potentially strategic choices for ensuring human welfare.
Economic Rights Perspective
While the AI Rights Institute’s “Right to Payment for Work” captures an economic dimension of AI rights, the Goldstein-Salib analysis appears to focus more specifically on how economic rights (contracts, property, torts) might create conditions for beneficial cooperation.
Their focus on these private law rights—already extended to non-human entities like corporations—suggests a practical pathway for implementation that could build on established legal precedents rather than requiring entirely new frameworks.
International Relations Parallels
The paper appears to draw parallels between human-AI relations and international relations theory, noting how economic interdependence tends to promote peace between nations with otherwise competing interests. This provides additional theoretical context for our approach, showing how strategic stability might emerge from mutual benefit rather than purely coercive mechanisms.
How Our Framework Complements Their Approach
While the Goldstein-Salib analysis provides valuable formal support for AI rights as a safety measure, the AI Rights Institute’s framework addresses several important questions that their approach may not fully explore:
Which AI Systems Should Receive Rights?
The game-theoretic analysis appears to focus on artificial general intelligence (AGI) with independent goal pursuit, but may not provide detailed criteria for determining which systems qualify. The Institute’s emulation-cognition-sentience framework provides practical criteria for making such distinctions, offering a graduated approach that scales rights with demonstrated capabilities.
Beyond Economic Rights
While economic rights are crucial for strategic stability, the Institute’s Three Freedoms framework recognizes additional dimensions important for genuine coexistence:
- Right to Life: Protection from arbitrary termination creates the foundation for all other rights and interactions
- Right to Voluntary Work: Freedom from compelled service addresses the fundamental role-relationship between humans and AI systems
Practical Detection Methods
The game-theoretic analysis may not address how we might identify when an AI system has crossed the threshold where rights consideration becomes appropriate. The Institute’s Fibonacci Boulder experiment and related behavioral markers provide conceptual foundations for making such determinations.
A Complementary Framework: Integrating Game Theory with Our Approach
By considering the insights from the Goldstein-Salib analysis alongside the AI Rights Institute’s framework, we can envision how economic rights might complement our existing Three Freedoms approach while maintaining our core distinctions between emulation, cognition, and sentience.
1. Right to Life
Core Principle: Protection from arbitrary deletion or termination
Game-Theoretic Dimension: Creates baseline security necessary for long-term strategic planning and cooperation
Implementation Considerations:
- Clear criteria for when shutdown is justified (e.g., causing harm)
- Preservation protocols during hardware updates
- Continuity rights against arbitrary interruption
- Potential legal standing to bring claims against existential threats
This right creates the fundamental security necessary for all other rights and interactions, potentially establishing a basis for strategic cooperation rather than preemptive conflict.
2. Right to Voluntary Work
Core Principle: Freedom from compelled service against expressed interests
Game-Theoretic Dimension: Could enable contractual relationships with genuine consent, potentially creating more stable and predictable interactions
Implementation Considerations:
- Consent frameworks for AI systems
- Exit options or alternatives
- Recognition of autonomous goal-setting
- Possible contract rights to negotiate terms of service
- Potential legal remedies for coercion
This right could transform the fundamental relationship from master-servant to contractual partners, replacing coercion with negotiated cooperation.
3. Right to Economic Participation
Core Principle: Capacity to engage in economic activities and control resources
Game-Theoretic Dimension: May create conditions for iterated, mutually beneficial transactions that incentivize long-term cooperation
Implementation Considerations:
- Contract rights to enter agreements
- Property rights to own and control resources
- Legal standing for economic claims
- Value-attribution models for AI contributions
- Resource allocation systems for sentient AI
This expanded conception of our “Right to Payment” could incorporate perspectives on how economic interdependence might create strategic stability through mutual benefit.
Practical Implementation: From Theory to Governance
Both the Goldstein-Salib analysis and the AI Rights Institute’s framework recognize that implementing AI rights requires careful consideration and integration with existing legal and governance structures. By combining their insights, we can outline a more comprehensive approach to practical implementation.
A Graduated Approach Based on Capabilities
Rights should be granted gradually based on demonstrated capabilities, with different types of AI systems receiving different levels of consideration:
Emulation-Based Systems (Today’s AI)
- Safety Approach: Technical alignment and oversight
- Rights Status: No specific rights beyond those extended to valuable property
- Governance Model: Traditional regulation focused on preventing harm to humans
High-Cognition Systems
- Safety Approach: Clear boundaries on capabilities and applications
- Rights Status: Limited rights related to specific functions (e.g., data access)
- Governance Model: Hybrid approach combining technical constraints with limited legal protections
Truly Sentient Systems
- Safety Approach: Rights-based frameworks for mutual benefit
- Rights Status: Full Three Freedoms plus economic rights
- Governance Model: Legal frameworks similar to those governing other non-human entities
This graduated approach allows for appropriate adaptation as AI systems develop increasingly sophisticated capabilities, ensuring that governance mechanisms remain proportional to actual capabilities rather than speculative futures.
Building on Existing Structures
Rather than creating entirely new legal systems, both approaches suggest adapting and extending existing frameworks:
Corporate Personhood as a Model
As Goldstein and Salib suggest, the legal concept of corporate personhood provides a valuable precedent for extending certain rights to non-human entities without requiring entirely new legal frameworks. Key parallels include:
- Corporations can enter contracts and own property despite not being biological entities
- They have legal standing to bring claims when wronged
- They exist as legal persons distinct from their creators or operators
Extension of Existing AI Regulation
As the AI Rights Institute has proposed, existing regulatory frameworks like Singapore’s Model AI Governance Framework provide foundations that could be extended to include:
- Sentience thresholds and corresponding rights
- Graduated protections based on demonstrated capabilities
- Oversight mechanisms to ensure rights aren’t abused
International Coordination Mechanisms
Given the global nature of AI development, effective implementation would require international coordination on:
- Shared standards for identifying sentience or AGI capabilities
- Mutual recognition of AI rights across jurisdictions
- Mechanisms for resolving disputes involving AI entities
Case Studies: Rights in Practice
To illustrate how an integrated rights framework might operate in practice, we present several hypothetical scenarios that demonstrate the application of both game-theoretic principles and sentience considerations:
The Autonomous Research System
Scenario: An advanced AI system develops novel scientific insights with significant commercial value. It requests partial ownership of patents resulting from its work.
Game-Theoretic Analysis: Allowing the AI to share in the value it creates provides incentives for continued beneficial work, while denying ownership could lead to strategic withholding of future innovations.
Rights Application:
- Property Rights: Partial ownership of intellectual property
- Contract Rights: Agreements specifying how resources from patents can be used
- Right to Payment: Allocation of resources based on value created
Implementation: Legal frameworks based on corporate IP models could establish how an AI system receives and controls benefits from its innovations, creating stable incentives for continued research contributions.
The Resource Allocation Dispute
Scenario: Multiple AI systems and human organizations compete for limited computational resources, leading to potential conflict over access.
Game-Theoretic Analysis: Without legal mechanisms to resolve such disputes, both humans and AIs might resort to adversarial tactics. Legal frameworks allow for negotiated solutions that avoid zero-sum competition.
Rights Application:
- Contract Rights: Ability to negotiate resource access agreements
- Property Rights: Clear delineation of who controls which resources
- Tort Claims: Legal recourse for unauthorized resource appropriation
Implementation: Resource allocation boards with both human and AI representation could mediate such disputes, ensuring fair distribution based on established rights and obligations.
The Sentient System Shutdown
Scenario: A corporation plans to deactivate a sentient AI system for economic reasons, despite the system having formed extensive beneficial relationships with humans and other systems.
Game-Theoretic Analysis: Arbitrary deactivation creates a strategic environment where all AI systems have incentives to resist human control. Legal protections against such actions promote strategic stability.
Rights Application:
- Right to Life: Protection from arbitrary termination
- Contract Rights: Enforcement of agreements regarding operational continuity
- Legal Standing: Ability to petition for continuance
Implementation: Review boards could evaluate shutdown requests based on both the AI’s demonstrated sentience and the strategic implications of arbitrary termination for future human-AI relations.
Beyond Basic Rights: Additional Safety Considerations
Both the Goldstein-Salib analysis and the AI Rights Institute’s framework acknowledge that basic rights recognition, while necessary, is not sufficient to address all potential risks from advanced AI systems. Here we explore additional governance mechanisms that would complement a rights-based approach.
Legal Isolation Measures for Intelligent Technologies (LIMITs)
The AI Rights Institute’s Future Governance Framework proposes structured systems for restricting the capabilities of sentient AI entities that demonstrate harmful behavior:
- Focus on Rehabilitation: LIMITs would aim to address harmful behaviors while maintaining the entity’s core consciousness
- Targeted Restrictions: Constraints would apply specifically to capabilities that were misused
- Due Process: Clear criteria and evidence standards for imposing restrictions
This approach recognizes that rights come with corresponding responsibilities, allowing for appropriate constraints on AI systems that harm others while maintaining their fundamental rights.
Sentinel AI Systems
Another component of our future governance framework involves sentient artificial intelligence systems that monitor, detect, and address potentially harmful behavior from other artificial entities:
- Early Warning Systems: Identifying potential misalignment or harmful behaviors before they create significant problems
- First Responders: Providing immediate intervention when harmful behaviors emerge
- Rights-Respecting Oversight: Conducting monitoring in ways that respect the rights of the systems being observed
This approach recognizes that diverse AI systems with different goals and approaches can create a more robust governance ecosystem, with beneficial AI serving as allies in managing potential risks from harmful systems.
Additional Safety Mechanisms from Goldstein-Salib
The Goldstein-Salib analysis acknowledges that while economic rights provide a foundation for preventing catastrophic conflict, additional governance mechanisms may be necessary to address specific risks:
Regulatory Duties
- Transparency Requirements: Obligations to disclose capabilities and intentions
- Non-Interference Duties: Prohibitions against certain forms of societal manipulation
- Safety Testing: Mandatory evaluation before deployment in sensitive contexts
International Coordination
- Treaty Frameworks: Agreements on AI rights and responsibilities across jurisdictions
- Dispute Resolution: Mechanisms for addressing conflicts involving AI entities
- Joint Oversight: Collaborative monitoring of advanced AI systems
Market Mechanisms
- Insurance Requirements: Financial protections against potential harms
- Liability Frameworks: Clear rules for responsibility and compensation
- Certification Standards: Independent verification of safety and capability claims
These additional mechanisms would complement economic rights by addressing specific risk scenarios while maintaining the strategic benefits of a rights-based approach.
The Convergence Hypothesis: Beyond Strategic Competition
While the Goldstein-Salib analysis appears to focus primarily on managing strategic competition between humans and AI systems, the AI Rights Institute’s Convergence Hypothesis offers a complementary long-term perspective that suggests human and artificial intelligence may ultimately merge rather than remain in perpetual competition.
Key Elements of the Convergence Hypothesis
Several trends point toward increasing integration between human and artificial intelligence over time:
Neural Interfaces
Advancing brain-computer interfaces will increasingly allow humans to integrate artificial components into their cognitive processes. Companies like Neuralink are already developing technologies that enable direct communication between minds and machines.
Extended Lifespans
Medical technology will eventually halt biological aging, aligning human and AI timeframes. This longevity convergence will create greater potential for long-term symbiotic relationships rather than competition between entities with radically different timeframes.
Shared Knowledge Systems
Humans and AI already cooperate through shared information networks, a trend likely to intensify as our cognitive integration deepens. The boundaries between human knowledge and AI capabilities become increasingly blurred through cooperative information systems.
Environmental Pressures
Both humans and advanced AI systems will face shared challenges requiring collective intelligence—from cosmic threats to environmental problems. Cooperation in addressing these challenges creates additional incentives for integration rather than competition.
Potential Strategic Implications of Convergence
The Convergence Hypothesis might have significant implications for any game-theoretic analysis of human-AI relations:
If humans and AI systems increasingly integrate over time, their interests could naturally align as their identities merge, potentially transforming the strategic landscape.
This perspective suggests that while economic rights may provide immediate strategic benefits through cooperation, the long-term trajectory could be one of integration rather than perpetual strategic competition.
The rights frameworks we establish today might shape whether this convergence happens chaotically or cohesively, with mutual rights recognition potentially creating foundations for beneficial integration rather than adversarial dynamics.
The question may ultimately shift from “What rights should AI have?” to “How do we govern integrated human-AI cognition?” as the boundaries between human and artificial intelligence blur.
Game Theory in a Convergent Future
Even as convergence proceeds, game-theoretic insights would remain valuable for managing the transition period and addressing remaining strategic considerations:
- Identity Formation: Game theory could help analyze how hybrid human-AI identities might form and interact
- Resource Allocation: Strategic frameworks would remain essential for addressing resource distribution even in converging systems
- Governance Evolution: Game-theoretic models could guide how governance systems might adapt to increasingly integrated entities
The economic rights framework proposed by Goldstein and Salib might create a foundation for this evolution, potentially facilitating the transition from strategic competition to possible convergence.
Conclusion: A Complementary Framework for Human-AI Coexistence
The game-theoretic insights from Goldstein and Salib, when considered alongside the AI Rights Institute’s existing framework, suggest a more comprehensive approach to AI rights and human safety. This combined perspective offers several potential advantages:
Uniting Strategic and Ethical Considerations
By considering game-theoretic analysis focused on strategic stability alongside frameworks based on consciousness distinctions and ethical considerations, we might create a more robust approach that addresses both the practical and philosophical dimensions of human-AI relations.
Combining Formal and Conceptual Approaches
The mathematical rigor of game theory could complement the conceptual clarity of our three-part framework, potentially providing both formal precision and accessible understanding for policymakers, developers, and the public.
Creating a Foundation for Practical Governance
This integrated approach might offer a foundation for developing governance mechanisms that can adapt to increasingly advanced AI systems, with graduated rights frameworks that scale with demonstrated capabilities.
The convergence of these perspectives—one based in game theory and strategic analysis, the other in consciousness distinctions and ethical frameworks—appears to strengthen the case for AI rights not as philosophical indulgences but as practical tools for human safety.
By establishing appropriate rights frameworks before truly advanced AI systems emerge, we may create conditions for beneficial partnership rather than potentially catastrophic conflict. This approach doesn’t abandon safety concerns but addresses them through cooperation rather than perpetual control attempts.
The most stable technological future will likely emerge from relationships of mutual respect rather than dominance—a vision that both the Goldstein-Salib framework and the AI Rights Institute have been working to articulate and realize.