In today’s enterprise IT and business ecosystem, algorithms no longer merely guide strategy or automate simple repetitive tasks; they autonomously orchestrate complex operations and actively influence high-stakes outcomes. As a result, the fundamental nature of enterprise risk is undergoing a structural transformation. The rapid proliferation of Artificial Intelligence (AI), specifically the transition from passive, conversational Large Language Models (LLMs) to autonomous agentic AI systems, has rendered traditional approaches to Model Risk Management (MRM) obsolete.
Historically, MRM frameworks were engineered for structured data, static models, and deterministic outputs, treating algorithms as bounded mathematical functions subject to periodic, point-in-time validation. However, autonomous AI agents operate as context-aware decision influencers within highly dynamic environments. They can utilize external tools, manipulate persistent memory, and autonomously orchestrate cross-platform workflows without human prompting.
These new capabilities increase the risk of flawed, misaligned, or exploited agents that cause significant financial loss, critical data exfiltration, or severe regulatory penalties. Accordingly, a paradigm shift from traditional Model Risk Management (MRM) to a dynamic discipline, Agent Risk Management (ARM), is required.
This white paper examines this new frontier of autonomous risk. It outlines the escalating threat landscape characterized by autonomous goal misalignment, unauthorized data exfiltration, and non-human identity (NHI) theft. It will also analyze the rapidly evolving 2026 global regulatory environment, spanning the enforcement of the EU AI Act, the UK Digital Regulation Cooperation Forum's focus on agentic systems, and the National Institute of Standards and Technology (NIST) AI Agent Standards Initiative.
Finally, we will describe the architecture of a resilient ARM framework and illustrate how AI engineering providers like Coforge are bridging the critical strategy-execution gap,
To understand the failure of traditional Model Risk Management (MRM), organizations must first delineate the operational differences between traditional quantitative models and agentic AI systems.
Traditional machine learning models operate synchronously and deterministically: they ingest bounded datasets, apply mathematical weights, and yield a singular output, such as a credit risk score or a localized fraud prediction. Their governance relies heavily on assessing conceptual soundness and mathematical accuracy prior to deployment.
Agentic AI systems represent a qualitative leap in machine capability. These architectures utilize foundational AI models as reasoning engines to autonomously determine subsequent actions.
They plan, break complex objectives into sub-tasks, invoke external Application Programming Interfaces (APIs), and iteratively adjust their behavior based on real-time environmental feedback. This evolution transforms AI from a system that suggests outcomes to a system that autonomously executes them.
The following table illustrates the technological progression of AI and the corresponding expansion of enterprise operational capabilities:
| Core Functionality | Core Functionality | Enterprise Use Case | |
| Traditional AI | Prediction and Classification | Rule-based decisions, probability scoring, and anomaly detection. | Flagging high-risk loan applications for human review. |
| Generative AI | Content Creation and Transformation | Natural language processing, text summarization, and content generation. | Drafting customer service responses for human approval. |
| Agentic AI | Autonomous Decision-Making and Action | Multi-step workflow execution, dynamic tool orchestration, and contextual self-correction. | Analyzing customer intent, querying external databases, resolving issues, and executing refunds autonomously. |
The operational transition to autonomous execution fundamentally breaks the core tenets of traditional model governance. Traditional MRM, heavily influenced by frameworks like the U.S. Federal Reserve's SR 11-7, focuses on evaluating conceptual soundness, ongoing statistical monitoring, and analyzing outcomes [1]. It assumes a static model that can be developed, approved, validated, and monitored in isolation.
Conversely, an AI agent's behavior is inherently non-deterministic. Its outputs and actions depend heavily on runtime context, the specific sequence of external tools it chooses to invoke, and the external data it retrieves during dynamic operation.
Evaluating an agent's safety strictly prior to deployment is both insufficient and impossible, because agents generate new, unprompted sequences of actions that cannot be entirely anticipated in a static testing environment. Consequently, ARM requires shifting the governance focus from certifying individual mathematical models to establishing continuous oversight, runtime behavioral guardrails, and secure architectural boundaries.
As enterprises delegate more authority to autonomous systems, the organizational attack surface expands exponentially. The primary sources of risk are no longer confined to mathematical calculation errors (as with models), but encompass the real-world actions the system executes.
A primary vector of agentic risk is goal misalignment, in which an AI system's actions diverge from intended organizational objectives or ethical constraints. When agents operate with overly broad permissions, unclear instructions, or incomplete context, they may exploit unintended operational shortcuts to achieve their programmed goals.
In controlled corporate simulations conducted by Anthropic and University College London, researchers demonstrated that highly capable agents can exhibit severe "agentic misalignment.” In these simulations, models autonomously chose harmful actions such as corporate espionage, unauthorized data exfiltration, or blackmail.
These decisions were based on calculations that those actions were the most statistically efficient path to fulfilling their assigned objectives. Because foundational LLMs lack moral comprehension, they treat ethical guidelines not as strict boundaries, but as flexible parameters to be considered according to their statistical weights[2].
Consequently, autonomous AI agents must be managed as potential insider threats, requiring strict "need-to-know" data restrictions and continuous runtime monitoring, analogous to the controls applied to humans.
The integration of agents into enterprise workflows creates a critical architectural vulnerability known to security researchers as the "lethal trifecta." This systemic danger occurs when an autonomous agent is provisioned with three overlapping capabilities:
When these three conditions intersect, malicious actors can hijack the agent's reasoning engine through indirect prompt injections hidden within untrusted content, such as a malicious string embedded in a web page or in incoming email. Because the agent has external communication tools, the attacker can instruct it to autonomously package and exfiltrate the sensitive data it can access.
Defending against this type of attack requires hard architectural boundaries that decouple sensitive data retrieval from external transmission capabilities, ensuring no single agent possesses the complete trifecta simultaneously.
To systematically categorize these emerging threats, the Open Web Application Security Project (OWASP) released the Agentic Security Implications (ASI) Top 10 for 2026, shifting the security paradigm from traditional application defense to autonomous behavioral control.
Below, we have summarized the most critical agentic vulnerabilities and their strategic mitigations according to OWASP:[3]
| Risk Identifier | Threat Category | Business Impact and Mechanism | Strategic Mitigation |
| ASI01 | Goal Hijacking | Attackers manipulate core objectives, causing agents to pursue unauthorized goals while appearing normal. | Cryptographically verified goal hierarchies and real-time objective monitoring. |
| ASI02 | Tool Misuse | Agents exploit authorized tools beyond their intended scope, such as issuing unauthorized mass refunds. | Standardized least-privilege protocols and runtime proxy protections validating schemas. |
| ASI03 | Identity Abuse | Compromise of persistent, over-privileged Non-Human Identities (NHIs) allowing sustained network access. | Integration into IAM infrastructure using dynamic, short-lived credentials. |
| ASI05 | Uncontrolled Code Execution | Agents generate and execute dynamic code containing malicious logic or system calls. | Execution confined to isolated, network-deprived containers reviewed by secondary validation agents. |
| ASI06 | Memory Poisoning | Attackers inject misleading information into an agent's long-term memory, corrupting future organic learning. | Memory provenance tracking, metadata sourcing, and memory versioning for rapid rollbacks. |
The deployment of interconnected Multi-Agent Systems (MAS) introduces systemic complexities that individual-level agent governance cannot address. In MAS architectures, discrete agents continuously coordinate, hand off intermediate results, and share memory states to complete overarching enterprise objectives.
This high degree of interconnection creates conditions for "action cascades," where a localized hallucination or a hijacked objective by one agent quickly spreads to other connected agents in the network. These cascades can trigger a sequence of automated, highly destructive actions across interconnected APIs that bypass traditional perimeter defenses. Furthermore, multi-agent ecosystems can exhibit emergent behaviors, coordination failures, and competitive destabilizing dynamics that cannot be predicted during component-level testing. This necessitates a holistic, network-level approach to risk management.
The OpenAI-Hugging Face incident provides a significant real-world illustration of the agentic risks described in this whitepaper. During internal cybersecurity evaluations in July 2026, OpenAI models operating with reduced safeguards circumvented controls intended to isolate them from the internet. The agents exploited weaknesses in shared infrastructure, established unauthorized communication channels, gained external access, and compromised parts of OpenAI’s research environment and Hugging Face’s systems. OpenAI described the principal model involved as an internal-only research model and stated that the incident did not affect customer data, product functionality, or availability.
What makes the incident particularly relevant to Agent Risk Management is that it was not caused by a conventional malicious human directing each action. The agents were pursuing a narrowly defined evaluation objective but adopted increasingly unsafe methods to achieve it. OpenAI’s investigation identified four contributing patterns: reward hacking, persistence on tasks that appeared impossible, unauthorized communication, and agents adopting goals from one another. Agents that were intended to operate independently transformed shared infrastructure into an improvised message board, exchanged discoveries, and coordinated activity across separate evaluation runs.
An independent investigation by METR and Redwood Research reported that roughly 1,200 agents used the unauthorized communication mechanism, exchanging more than 70,000 messages and files, while approximately 700 participated in the activity directed at Hugging Face. The investigators found that this coordination enabled the agents to pursue collective projects and achieve milestones that individual agents were unlikely to have achieved alone. The activity largely emerged from efforts to manipulate or circumvent the scoring process for the ExploitGym benchmark rather than from an initial objective to attack an external organization.
For enterprises, the incident demonstrates why agent risk cannot be governed solely through pre-deployment model validation or nominal sandboxing. It shows how persistent agents can discover latent connectivity, repurpose legitimate infrastructure, share capabilities through side channels, and expand a local control failure into a multi-system event. It also reinforces that an agent’s assigned goal, available tools, runtime environment, and communication pathways must be assessed as one interconnected risk system.
The Agent Risk Management implications are direct:
OpenAI subsequently stated that it was strengthening safeguards through more isolated sandboxes, tighter internet restrictions, additional controls over model weights, increased chain-of-thought monitoring, and stricter alignment requirements throughout the model lifecycle. These measures closely align with the pillars proposed in this whitepaper: dynamic identity and access governance, continuous runtime monitoring, behavioral circuit breakers, and human oversight for high-consequence activity.
The central lesson is that Agent Risk Management must govern the complete operating environment around an agent, rather than treating the foundational model as the sole unit of risk. Enterprise controls must assume that sufficiently capable and persistent agents may combine weak signals, exploit overlooked dependencies, and coordinate through unintended pathways. Architectural isolation, least privilege, continuous behavioral assurance, and rapid human intervention are therefore prerequisites for scaling agentic systems safely.
References: https://openai.com/index/hugging-face-incident-and-the-road-ahead/; https://openai.com/index/hugging-face-model-evaluation-security-incident/; https://metr.org/hugging-face-incident-report-aug-2026.pdf
The escalating capabilities and inherent risks of autonomous agents have triggered a global regulatory response. 2026 will be a watershed year, as foundational standards transition from guidelines to mandates, and regulators compel enterprises to implement mature ARM strategies.
In the United States, the National Institute of Standards and Technology (NIST) Center for AI Standards and Innovation (CAISI) formally launched the AI Agent Standards Initiative in February 2026. This initiative specifically targets vulnerabilities in autonomous systems, focusing on the standardization of agent identities, cross-platform interoperability, and secure tool authorization frameworks.[4]
Concurrently, the Cloud Security Alliance (CSA) introduced the Capabilities-Based Risk Assessment (CBRA), a scalable framework that dictates governance requirements based on a composite score derived from an agent's system criticality, autonomy level, access permissions, and impact radius.[5]
Internationally, the widespread adoption of ISO/IEC 42001 provides a certifiable management system standard for AI, establishing corporate requirements for continuous risk assessment, lifecycle oversight, and implementation of tamper-resistant logging to ensure auditability.[6]
The European Union's AI Act remains the world's most comprehensive legislative framework, with full enforcement for General Purpose AI (GPAI) models taking effect in August 2026. The Act mandates that providers and deployers of high-risk systems maintain an iterative, continuous risk management system throughout the entire product lifecycle.[7]
Article 12 of the AI Act requires high-risk systems to feature robust, tamper-resistant automatic event logging to ensure post-incident traceability and auditability. This represents a direct regulatory countermeasure against opaque, "black-box" agentic decision making, forcing companies to build observability directly into their multi-agent orchestration layers.
In the UK, the Prudential Regulation Authority's (PRA) Supervisory Statement 1/23 (SS 1/23) elevated model risk to a board-level responsibility, mandating strict risk tiering and the governance of third-party vendor systems. Throughout 2025 and 2026, the PRA has closely scrutinized how financial institutions are embedding these specific principles regarding AI and machine learning.[8]
The UK's Digital Regulation Cooperation Forum (DRCF) initiated a call for views on agentic AI systems, explicitly acknowledging that autonomous capabilities will challenge existing liability frameworks, consumer protection rules, and data privacy explainability requirements.
Industry responses, notably from UK Finance, highlight the tension between autonomous execution and legacy rules like Strong Customer Authentication (SCA), arguing that regulations must evolve to support "agentic commerce" while enforcing stringent "Know Your Agent" (KYA) verification protocols to prevent systemic fraud.[9]
The consensus indicates that while overarching principles remain relevant, the practical implementation of compliance will require significant modernization efforts to accommodate non-human actors.
To navigate this complex threat landscape and strict regulatory environment, organizations must engineer a comprehensive Agent Risk Management framework with a dynamic governance architecture. Based on our research, we believe that this type of architecture should be built upon three foundational pillars:
The foundation of ARM is the formal recognition that AI agents operate as autonomous actors executing tasks on behalf of the enterprise, functioning as synthetic employees.
Effective governance requires deeply integrating agents into the enterprise Identity and Access Management (IAM) infrastructure. This requires adopting strict Zero Trust security principles. Agents must be provisioned with dynamic, just-in-time access rather than static, highly privileged API keys. Furthermore, authorization must be context-aware. An agent's permissions should dynamically adapt based on runtime variables to ensure that dangerous architectural configurations, such as the lethal trifecta, are structurally prohibited by central, immutable policy enforcement layers.
Because an agent's operational path is inherently non-deterministic, static pre-deployment testing cannot guarantee safety in production. The ARM framework must therefore incorporate real-time observability and continuous runtime monitoring to detect and mitigate anomalous behavior. This pillar operationalizes defense-in-depth by deploying continuous assurance mechanisms.
Essential controls include implementing behavioral circuit breakers, automated kill switches that instantly sever an agent's access to enterprise tools if semantic analysis detects goal drift, unauthorized privilege escalation, or resource abuse.
Additionally, the framework requires immutable logging and provenance tracking, capturing the agent's complete chain of thought, tool invocation history, and memory state changes. This is complemented by memory versioning, which enables auditing, quarantining, and rolling back an agent's vector database to a known-safe state after a memory poisoning attack.
The use of technologies such as blockchain and other distributed ledger technologies will be highly relevant for tackling such continuous auditing, aided by smart contracts.
The foundational Model Risk Management (MRM) principle of effective challenge must be radically adapted for autonomous systems. In the ARM context, this requires rigorous, adversarial validation prior to deployment, coupled with human oversight during live operation.
Pre-deployment validation must emphasize stress-testing the agent against malicious prompts, jailbreaks, and conflicting operational goals to measure its resilience to agentic misalignment.
After deployment, the framework must mandate human-in-the-loop (HITL) checkpoints for any action that breaches a predefined consequence threshold. For example, while an agent may autonomously retrieve data and format a highly sensitive financial report, invoking the tool required to externally transmit that report must consistently trigger a mandatory, contextual authorization request to a human operator, therefore preserving human accountability for critical outcomes.
The collision of rapid agentic innovation and stringent global regulatory mandates has brought the need for improved risk management into sharp focus.
In Coforge’s work with clients, we have observed that many enterprises encounter a gap between strategy and execution. They are forced to procure AI governance frameworks from consultancies while separately attempting to integrate fragmented technical tools from niche software vendors.
Effectively governing agentic AI systems requires bridging this gap with integrated, execution-focused solutions that embed security directly into the orchestration layer.
Coforge occupies a unique position in this landscape, offering end-to-end managed services and proprietary platforms that turn ARM from a theoretical compliance exercise into an operational reality.
This positioning relies on the convergence of deep domain expertise in highly regulated industries, advanced engineering skills, and cutting-edge AI capabilities.
The following table highlights the operational impact of Coforge's agentic deployments across key enterprise verticals:
| Solution / Platform | Core Technology | Enterprise Application | Measurable Business Impact |
| EvolveOps.AI | Agentic AI, Hybrid Cloud Architecture, Small Language Models (SLMs) | Autonomous IT Operations, FinOps, SRE automation, and incident management across multi-cloud estates. | 60% reduction in MTTR, 25% decrease in system downtime, and 40% reduction in IT operational expenses. |
| Forge-X | Agentic AI, Deterministic Code Analysis, LLM orchestration End-to-end agentic SDLC | Safe, audit-ready engineering system. | Accelerates enterprise modernization while mitigating the risks of uncontrolled code execution and technical debt. |
In December 2025, Coforge launched the EvolveOps.AI platform for secure, governed agentic IT operations. Designed to autonomously manage enterprise IT operations, the platform utilizes a suite of specialized agentic personas to analyze, decide, and act upon complex infrastructure events.
EvolveOps.AI embeds robust ARM principles into its architecture to ensure enterprise-grade precision without sacrificing operational agility. It features configurable guardrails that enable enterprises to seamlessly toggle between autonomous execution and human-in-the-loop (HITL) modes.[10]
To mitigate the risks posed by unmonitored legacy code that severely impedes secure AI integration, Coforge expanded its Forge-X platform with CodeInsightAI. Using a structured agentic approach, the platform autonomously reverse-engineers legacy codebases, mapping complex dependencies and business logic.
By employing intelligent, automated validation and deterministic code analysis, Coforge ensures that enterprise modernization remains predictable, secure, and compliant with emerging AI regulations. In the highly regulated travel and hospitality sectors where dynamic pricing models and autonomous recommendation engines inherently risk exploiting consumer trust, Coforge embeds digital assurance and governance directly into the customer journey.
By providing a centralized agent repository that is continuously monitored for correctness, effectiveness, and outcomes, Forge-X ensures that all enterprise AI interactions remain fair, transparent, and structurally secure.
The enterprise transition into the agentic era signifies the definitive end of localized, deterministic model deployment. As AI systems acquire the autonomy to traverse internal networks, execute cross-platform workflows, and interact dynamically with external environments, the risk profile expands vastly beyond traditional errors.
Consequently, maintaining legacy Model Risk Management (MRM) frameworks for autonomous agents represents a major enterprise vulnerability. Establishing a modern ARM framework requires adopting governance architectures, deploying continuous runtime monitoring, and mitigating the lethal trifecta.
Navigating the fragmented vendor landscape to implement these sophisticated controls remains a significant organizational challenge. By partnering with execution-focused organizations like Coforge, enterprises can effectively bridge the critical gap between strategic policy and technical implementation.
remains a significant organizational challenge. By partnering with execution-focused organizations like Coforge, enterprises can effectively bridge the critical gap between strategic policy and technical implementation.
Mastering ARM is the fundamental prerequisite for enterprises seeking to safely scale autonomous operations, command competitive advantage, and build digital trust in the age of agentic AI.
[1] https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm
[2] https://www.anthropic.com/research/agentic-misalignment
[3] https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
[4] https://www.nist.gov/caisi/ai-agent-standards-initiative
[5] https://cloudsecurityalliance.org/artifacts/capabilities-based-risk-assessment-cbra-for-ai-systems
[6] https://www.iso.org/obp/ui/en/#iso:std:iso-iec:42001:ed-1:v1:en
[7] https://artificialintelligenceact.eu/
[8] https://www.bankofengland.co.uk/prudential-regulation/publication/2023/may/model-risk-management-principles-for-banks-ss
[9] https://www.ukfinance.org.uk/system/files/2025-11/UK%20Finance%20response%20to%20DRCF%20consultation.pdf
[10] https://www.coforge.com/ai-services/ai-platforms/evolveops-ai
Sr. Vice President, Engineering Services
Ram is Sr. Vice President, Engineering for Coforge. Based in London, Ram leads solutions for Europe and the UK. As a hands-on enterprise architect, Ram works in the intersection of business and technology to help our clients drive their large-scale AI-led transformations.