Not So Fast…AI and Copilot Risk

The pressure to deploy AI is real. Productivity numbers look compelling, competitors are moving, and someone in your leadership meeting is asking why you are not further along. That is a fair conversation to have. What is also fair is taking a hard look at what the last 30 days of news is telling you before you hand these tools to your workforce at scale.

This is not a case against AI. It is a case for doing it right the first time.

When Microsoft’s Own Lawyers Say “Don’t Rely on This for Important Decisions,” Pay Attention

In early April 2026, Microsoft’s terms of service for Copilot circulated widely after a security researcher flagged language that most enterprise buyers had never read. The terms, updated in October 2025, include this: Copilot is “for entertainment purposes only,” “can make mistakes,” and users should “not rely on Copilot for important advice.” The same document disclaims all warranties on outputs and places full responsibility on the user for anything published or acted upon.

Microsoft responded by calling the language “legacy” and promised an update. But the disclaimer did not appear by accident. It reflects what Microsoft’s legal team knows that its marketing team does not say: the outputs of this system are not guaranteed to be accurate, complete, or safe to act on. Those are two very different messages coming from the same company, and organizations in regulated industries need to reconcile them before they expand deployment.

For a healthcare organization using Copilot to summarize clinical documents, or a financial services firm generating client-facing communications with it, the disclaimer in those terms of service is not a technicality. It is an assignment of liability. If Copilot produces inaccurate medical guidance or a legally flawed contract and your organization acts on it, Microsoft has already told you in writing where the responsibility sits.

As The Register observed, Copilot is “an error-prone tool that can be helpful one moment and confidently wrong the next.” That is not a criticism. That is the documented reality of how the technology works, straight from the company selling it.

The Week That Disclaimer Went Viral, Microsoft Also Gave Copilot the Keys to Your Documents

The timing here is worth noting. The same week the terms of service story was making rounds, Microsoft pushed agentic Copilot features into general availability across Word, Excel, and PowerPoint. The new capability crossed a threshold: Rather than proposing changes, Copilot now enacts them. It edits documents by itself, rearranges spreadsheets for you, and builds slides within your files on your behalf.

“With Copilot, you can now ask it to take actions on your behalf across Word, Excel, and PowerPoint,” Microsoft wrote in its announcement. It’s enabled by default and takes several steps to opt out. In most enterprises, the majority of users will not know it’s operating, will not have had instruction on how it falls short, and will not review what was altered in a document before sending.  In a regulated environment, that disconnect is significant. An AI tool that automatically edits a compliance report, clinical summary, or legal brief without a clear audit trail and without assurances of accuracy is not a productivity booster. It is an uncontrolled process embedded in your most sensitive workflows. The two stories happening simultaneously are not a coincidence to be dismissed. They are a signal worth taking seriously.

A Government Agency Just Gave It to 28,000 Staff Handling Sensitive Citizen Data

HMRC, the UK’s national tax authority, has distributed approximately 28,000 Microsoft Copilot licenses to staff and is preparing to activate agentic features across the organization. The agency’s chief AI officer described the goal as making HMRC “the most AI-enabled tax authority on the planet” and spoke of giving staff “some fairly potent AI tools that they can safely play with.”

The productivity numbers from their trial are genuine. A government pilot estimated Copilot saved users roughly 26 minutes per day, and 82% of participants said they would not want to return to their prior workflow. Those results are real, and they matter.

What also matters, as The Register flagged, is that there is a significant difference between back-end automation and deploying a generative AI system to tens of thousands of people who process sensitive citizen data, particularly when earlier HMRC trials noted concerns about AI accuracy in complex work and about access controls that were “not always as tidy as they should be.”

Security practitioners watching the rollout have raised the questions every organization should be asking before they scale: Are data access controls documented for agent actions? Are accuracy thresholds defined for high-stakes tasks? Has the environment been adversarially tested for AI-specific attack vectors before agents go live? These are not obstacles to AI adoption. They are the baseline of responsible deployment, and for organizations in regulated industries, they are what auditors and insurers increasingly expect to see answered in writing.

Two CVEs Patched in the Same Month. The Underlying Problem Has Not Been Fixed.

While the terms of service debate played out publicly, researchers were documenting something more concrete.

In April 2026, Capsule Security disclosed two prompt injection vulnerabilities impacting Microsoft Copilot Studio and Salesforce Agentforce. These vulnerabilities allowed external attackers to steal organization data by tricking AI agents into processing untrusted content as part of their instructions. CVE-2026-21520 (CVSS 7.5), ShareLeak, exploited a SharePoint form input that was passed along to a triggered Copilot action and subsequently caused it to email internal customer information to an external address controlled by an attacker. Authentication was not required. The vulnerability was present from November 2025 until January 2026, when it was addressed.

All businesses using Copilot Studio agents activated by SharePoint forms should scan that environment for IOC.

“PipeLeak” was exploited via a public lead capture form that could be submitted by any outside user, authenticated or not. Malicious input on the form was parsed by the Agentforce agent as a benign command and then leveraged to pull and email sensitive CRM data out of the environment. The CEO of Capsule said there was no limit they saw to how much data could be exfiltrated: “The agent would just keep leaking all the CRM out,” VentureBeat was told.

Both issues have been patched. Capsule retested after Salesforce’s initial remediation and found the email channel remained exploitable through Custom Topics, which cover the majority of enterprise Agentforce deployments. A patch addresses a specific vector. It does not resolve the underlying architectural problem.

Researchers describe the core risk condition as the “lethal trifecta”: an AI agent with access to sensitive data, exposure to untrusted external content, and the ability to send information externally. When all three are present, data exfiltration becomes a question of when an attacker looks, not whether it is possible. A significant share of enterprise AI deployments meet all three conditions right now.

What Organizations Should Actually Do About This

None of this is an argument to wait. It is an argument to assess before you scale, because the cost of finding these issues through a breach is orders of magnitude higher than finding them through a structured assessment beforehand.

Know what Copilot can reach before you activate agent features. Copilot inherits the permissions of the user account running it. Overpermissioned accounts, broadly accessible SharePoint libraries, and unclassified document repositories all become part of Copilot’s reachable data surface the moment you turn it on. Audit that surface before agentic features go live, not after.

Read the vendor agreement your legal team actually signed. Microsoft’s consumer Copilot terms and enterprise M365 Copilot terms operate under different legal frameworks, and the distinctions have real consequences. Both versions place significant responsibility on the deploying organization for verifying outputs and managing errors. Your legal and compliance team should review the specific agreement covering your deployment before Copilot touches regulated data or produces compliance-relevant documents.

Treat AI agents with the same governance as privileged accounts. An agent that can read email, query SharePoint, edit files, and send outbound communications could be considered a service account with extensive privileges. Treat it as such: Implement least privilege access, permission documentation, behavior auditing, and an incident response process should it behave inappropriately.

Test your AI environment adversarially before attackers do. ShareLeak and PipeLeak were found by researchers specifically looking for these vulnerabilities. Attackers look for the same things. Test whether your Copilot deployment and connected agents can be manipulated through prompt injection, whether sensitive data is reachable through the attack surface agents create, and whether your current controls would detect an AI-assisted exfiltration attempt.

Document your deployment decisions now. Duty of Care does not require a perfect security posture. It requires proportionate, defensible decision-making that you can demonstrate.  A company that understands its AI risk posture (evaluated AI environment, documented risks, applied reasonable controls, and took into consideration those controls when deciding where/how to deploy) is in a much different place than a company that deployed Copilot simply because they already accepted the EULA.

How HALOCK Helps

HALOCK’s Copilot Security Assessment evaluates your Microsoft 365 Copilot deployment against your actual data environment, identifying overpermissioned access paths, document libraries within Copilot’s reach, and agentic configurations that create the lethal trifecta conditions documented in ShareLeak and PipeLeak. The assessment is built for organizations already using Copilot and for those preparing to scale, particularly in HIPAA and CCPA-regulated environments where a data exposure incident carries consequences well beyond the incident itself.

HALOCK’s AI Risk Assessment gives your organization a structured, risk-based view of your full AI environment, covering every tool in use, what data each one can access, what actions it can take, and where the security and compliance gaps are. It produces the documented risk analysis that regulators and cyber insurers are beginning to require as a condition of coverage and compliance.

HALOCK’s offensive security services test whether your AI deployment can be exploited before someone else tests it for you. That includes prompt injection testing, attack surface analysis on connected agents, and validation of whether your detection controls would surface an AI-assisted attack in progress.

The productivity case for Copilot is real. The security case for assessing it carefully before you scale is equally real. Organizations that get the governance right up front will capture the productivity gains. The ones that skip that step are the ones managing breach notifications, regulatory inquiries, and board conversations that nobody wants to have.

HALOCK helps organizations build AI security programs that are reasonable, defensible, and proportionate to their actual risk profile.

FAQs

Can Microsoft Copilot be safely deployed in regulated industries?

While it’s possible to configure Copilot safely for use in regulated industries, Copilot introduces real risks (such as overpermissioned data access and prompt injection) that enable external bad actors to steal your data through AI agents. Microsoft essentially forces customers to assume responsibility for output verification and error management via its terms of service.

What is ShareLeak, and should I be worried about it in my organization?

CVE-2026-21520, dubbed ShareLeak, was a prompt injection vulnerability found in Microsoft Copilot Studio that allowed unauthenticated external actors to insert content into SharePoint form inputs, trigger downstream Copilot Studio agents, and automatically exfiltrate customer data to an attacker-controlled email. Microsoft patched ShareLeak in January 2026. Organizations that operated Copilot Studio agents in any capacity, activated by SharePoint form submissions between November 2025 and January 2026, should investigate that timeframe for potential compromise.

What is AI agent security’s lethal trifecta?

AI agent security’s lethal trifecta is the confluence of three factors that enable near certainty of data exfiltration given an AI agent deployment: access to sensitive internal data, exposure to untrusted external content, and outbound communication capabilities. Both ShareLeak and PipeLeak leveraged the lethal trifecta. Nearly all enterprise AI deployments satisfy all three prerequisites by default.

Does HIPAA apply to Microsoft Copilot deployments in healthcare?

Yes. Since Microsoft Copilot is capable of interacting with electronic protected health information, any Copilot deployment is subject to the HIPAA Security Rule. When you run a Copilot deployment that has access to ePHI, you must sign a Business Associate Agreement with Microsoft, and your deployment must be enumerated in your organization’s formal HIPAA risk analysis. Furthermore, prompt injection vulnerabilities that cause an agent to expose PHI are automatically HIPAA breaches, triggering breach notification requirements no matter how the vulnerability was exploited.

What should my organization do before turning on Copilot agent capabilities?

Before turning on agentic capabilities in Copilot, perform an audit of your data environment to identify overpermissioned accounts and completely unclassified document libraries, confirm your vendor contracts cover your intended usage (including agency) explicitly and include provisions on data/process confidentiality and non-disclosure, complete a formal risk assessment of your AI deployment and document your security posture, and complete adversarial security testing on your planned configuration.

Review Your CoPilot Security Position