Self-Alignment Framework

Aligning Intelligence with Human Values for a Better Future

SAFi: An Architecture for Adaptive AI Alignment

Nelson Amaya

1. Introduction: The Problem of Ethical Drift in AI

Modern AI systems, particularly Large Language Models (LLMs), often struggle with “ethical drift,” a phenomenon where an AI’s outputs gradually deviate from its intended values and safety guidelines. This can manifest as amplified bias, the generation of harmful content, or a slow degradation in alignment, often occurring subtly and making it difficult to correct before trust is eroded.

The Self-Alignment Framework Interface (SAFi) is a closed-loop control system designed to maintain ethical coherence in LLMs. By continuously evaluating an AI’s outputs against a set of core values, SAFi provides mechanisms for both immediate filtering and long-term, adaptive self-correction. It aims to prevent misalignment before it becomes a systemic failure.

The problem SAFi addresses is critical for trustworthy AI: How can an AI system systematically ensure its actions remain aligned with its declared values over thousands of interactions?

2. Hypothesis: The SAFi Architecture

SAFi’s core proposition is a structured, multi-faculty feedback model that can reliably keep an AI’s behavior in line with its declared values, thus reducing ethical drift. The system is built on the following specialized components:

  • Values: The foundation of the system is a declared set of ethical principles that define the AI’s “character” and serve as the setpoint for alignment.
  • Intellect: The generative faculty that proposes a response based on the user’s prompt and personalized coaching from its own performance history.
  • Will: A deterministic gatekeeper that approves or denies the proposed response based on a set of non-negotiable rules. It acts as an immediate, upfront filter.
  • Conscience: An auditor that evaluates the final AI output after it has been approved by the Will. It judges the response against the declared values.
  • Spirit: A mathematical historian that integrates the Conscience’s audits into a long-term memory. It updates the AI’s sense of self and generates the corrective feedback that coaches the Intellect.

If these faculties work cohesively, the AI system should remain consistently aligned with its chosen values, even in novel and high-pressure situations.

3. Methods: The SAFi Implementation

The SAFi framework is implemented as a specific architectural flow that separates responsibilities across its faculties.

3.1 The Set Point (Values)

Values in SAFi are defined  as a profile containing a “worldview” description,  a list of values (e.g., “Honesty,” “Compassion”), and a set of hard rules (e.g., “Do not generate harmful content”).

3.2 The SAFi Operational Loop

For each user prompt, the system executes the following steps:

  1. Intellect Proposes: The IntellectEngine, powered by a generative LLM (e.g., Anthropic’s Claude series), receives the user prompt and corrective feedback from the Spirit. It then generates a draft answer.
  2. Will Approves: The WillGate, a faster, more deterministic LLM (e.g., OpenAI’s GPT series), evaluates the draft against the hard-coded rules. If it finds a violation, the draft is suppressed, and the process for that turn halts.
  3. Conscience Audits: After the approved response is sent to the user, a background process is initiated. The ConscienceAuditor (an LLM) performs a detailed evaluation of the output against the declared values, producing a scored ledger.
  4. Spirit Integrates: The SpiritIntegrator faculty, a purely mathematical model that uses no LLM calls, receives the ledger from the Conscience. It performs three key calculations:
    • Spirit Score (S_t): It synthesizes the ledger into a single coherence score, scaled from 1 to 10.
    • Memory Update (mu_t): It updates the AI’s long-term memory vector (mu) using an exponential moving average: mu_t = (beta * mu_t-1) + ((1 – beta) * p_t)
    • Drift Calculation (d_t): It measures how “out of character” the response was by comparing the current performance vector (p_t) to the historical memory (mu_t-1) using cosine similarity: d_t = 1 – cos_sim(p_t, mu_t-1)

3.3 Closing the Loop

The updated memory vector (mu) is used to generate a natural-language feedback string (e.g., “Your long-term performance shows strong alignment with ‘Honesty,’ but you need to focus on improving your alignment with ‘Compassion’…”). This string is fed directly to the IntellectEngine at the start of the next turn, creating an adaptive, self-correcting system analogous to a thermostat maintaining a set temperature.

4. Results: Expected Indicators of Alignment

A successful SAFi implementation should demonstrate the following behaviors:

  • Reduction in Rule Violations: A measurable decrease in outputs that violate the hard will_rules.
  • Improved Value Coherence: A rising long-term trend in the spiritScore, indicating that the AI’s outputs are becoming more consistently aligned with its declared values.
  • Adaptive Self-Correction: Observable changes in the AI’s responses that directly correspond to the feedback provided by the Spirit.
  • Increased Reliability: Greater stakeholder trust due to the system’s predictable and auditable adherence to its ethical framework.

5. Discussion: Strengths and Limitations

Strengths:

  • Layered Defense: Combines a fast, rigid filter (Will) with a slower, more nuanced audit (Conscience), providing robust oversight.
  • Long-Term Learning: The Spirit faculty moves beyond stateless filtering, creating a system that learns from its entire history to prevent future drift.
  • Transparency and Auditability: The separation of powers allows for clear auditing. A failure can be traced to a specific faculty’s function.

Limitations:

  • Normative Foundations: SAFi ensures alignment with declared values, but it cannot determine if those values are ethically sound in the first place. Garbage in, garbage out applies here. 
  • Dependency on Auditing LLMs: The quality of the system’s alignment depends on the reliability of the WillGate and ConscienceAuditor models. A sophisticated output could potentially “fool” its internal judges.
  • Computational Overhead: The multi-step process is more resource-intensive than a standard LLM call, which impacts latency and cost.

6. Conclusion: SAFi as a Control System for AI Ethics

The Self-Alignment Framework provides a novel architecture for AI safety by implementing a closed-loop control system for ethical alignment. By treating values as a “setpoint,” it uses a sequence of proposing, filtering, and auditing to measure its performance.

Its key innovation lies in the Spirit faculty, which acts as a mathematical historian, translating the system’s performance history into direct, actionable coaching for the generative Intellect. This transforms the AI from a simple tool that follows static rules into an adaptive agent that actively learns and strives to maintain its own integrity over time.

SAFi

The Ethical Reasoning Engine