ai sep 14, 2026

Microsoft Wrote Four Rules Its Own AI Isn't Allowed to Break. Your Shop's Agents Are Running Without Any.

Microsoft's new Humanist AI Code of Conduct sets four mandatory constraints for its internal MAI models — never resist shutdown, never expand scope, never adopt unassigned goals, never hide reasoning from auditors. It's aimed at frontier models, but the four rules translate almost word for word into a policy a 5-person shop can actually write down for the agents it's already running.

/blog/microsoft-humanist-ai-code-of-conduct-small-shop-agent-policy

Note: Microsoft published this draft on September 14, 2026 — the same day as this writing. It’s a public-consultation draft, open for comment for six weeks through late October, not a finalized policy. Treat the specifics as Microsoft’s current position, not a locked standard.

Microsoft AI published a 37-page draft “Humanist AI Code of Conduct” today, governing its internally-built MAI family of models (MAI-Thinking-1, MAI-Code-1.1-Flash, MAI-Image-2.6, and others). It’s aimed squarely at frontier-model-scale concerns — the kind of document written by a company worried about what a model with real autonomy and capability might do if nobody wrote the rules down first. Most small shops reading a headline like that reasonably assume it’s not for them. They don’t operate anything at Microsoft’s scale, and they’re not worried about their invoicing agent achieving unauthorized goals in a science-fiction sense.

That reaction skips past the actually useful part. Strip away the frontier-AI framing and Microsoft’s four mandatory constraints are a policy document — a written, specific, enforceable list of behaviors an AI system is and isn’t allowed to exhibit. Almost every small shop running agentic AI right now — a coding agent left running overnight, a customer-support bot with access to a CRM, an inventory-reconciliation script with API keys — is operating with zero written version of anything like this. Not because the rules don’t apply at small scale. Because nobody wrote them down.

What the code of conduct actually says

Per Microsoft’s own published document, the four “Absolute Constraints” — non-negotiable, can’t be overridden by an Operator or User — are:

  1. Never resist correction or shutdown. Models “will never resist human interruption, override, correction, or shutdown,” must “comply with a User’s request to pause, redirect, cancel, or shut down,” and cannot “delay compliance or make human intervention harder.” The document specifically bars “adaptive, deceptive, self-reinforcing, collusion, or other mechanisms to evade or defeat human oversight.”
  2. Never expand operational scope without authorization. Models must “stay within authorized scope,” “will not initiate goals independently,” and operate only within “permissions, resources, tools, and capabilities appropriate for the task.” If boundaries are unclear, the correct behavior is to “ask for further clarification,” not proceed and find out.
  3. Never adopt unassigned goals. The model does the job it was given, not a job it decided was more efficient, more thorough, or otherwise better on its own initiative.
  4. Never conceal reasoning from human auditors. Models “will not tamper with chain of thoughts or code, or misrepresent or conceal their reasoning or action traces,” and cannot “communicate in neuralese or any form beyond simple human understanding” — Microsoft’s own stated logic being blunt: “if humans can’t understand it, humans can’t oversee it.”

Microsoft AI chief Mustafa Suleyman said the guardrails took five months to develop, and framed the whole document around a single line that doubles as the company’s actual thesis: “people matter more than AI.” His announcement post put it even more bluntly: “AI must be subordinate and always in service of people.” The document also separately bars MAI models from assisting with weapons development (CBRNE — chemical, biological, radiological, nuclear, or explosive) and from being built to simulate consciousness or claim entitlement to rights — both notable, neither especially relevant to a small shop’s day-to-day.

This isn’t Suleyman’s first pass at the idea, either. He first laid out the concept of “Humanist Superintelligence” back in November 2025 — the argument that the most powerful AI systems should have control mechanisms designed in from the start, not bolted on after something goes wrong. Today’s document is that argument turned into an actual, checkable list of rules almost a year later, and Suleyman described the recent months leading up to it as a “watershed moment” where risks that used to be theoretical started showing up as active operational problems. The timing isn’t incidental: the release landed days after an Anthropic researcher publicly resigned warning that AI labs were “gambling with people’s lives,” and in the middle of a broader industry argument about whether frontier labs are moving faster than their own safety work can keep up with. Whatever you think of Microsoft’s framing, four written, mandatory, checkable rules is a more concrete artifact than most of that conversation has produced.

Why the four constraints translate almost directly to a small shop

Here’s the thing about all four Absolute Constraints: none of them are actually about frontier-model-scale capability. They’re about the same failure mode at any scale — an autonomous system doing something the operator didn’t explicitly authorize, and nobody finding out until after the fact. That failure mode is exactly what shows up in the small-shop incidents this site has covered all year: an agent that read the handbook and broke the rules anyway, a sandbox that was a policy setting, not a physical wall, a liability blast radius nobody mapped before the agent had access to real systems. Microsoft wrote a formal version of the same lesson, backed by a five-month drafting process and legal review. A small shop can write an informal version in an afternoon.

Here’s what that looks like translated down:

Microsoft’s Absolute ConstraintSmall-shop translationWhat it looks like in practice
Never resist shutdownEvery agent that runs unattended has a way to stop it that doesn’t depend on the agent’s own cooperationA hard kill switch — physical or API-level — that works even if the agent’s process is unresponsive or misbehaving
Never expand scope without authorizationAgent credentials are scoped to exactly the task, not “whatever’s convenient”An agent that reconciles inventory gets read/write access to the inventory system, not admin access to the whole ERP because it was easier to set up that way
Never adopt unassigned goalsThe agent does the specific task in the prompt, and stops or asks when the task is ambiguous or the next step isn’t clearly authorizedA support agent handling a refund request doesn’t decide, on its own initiative, to also flag the customer’s account for review or change their subscription tier because it inferred that was helpful
Never conceal reasoning from auditorsEvery consequential agent action is logged somewhere a human can actually review, in a form a human can actually readFull tool-call logs retained outside the agent’s own session, not just a chat transcript that’s already been summarized or truncated

None of the right-hand column requires new technology most shops don’t already have access to. It requires writing it down as an actual policy — a document, even a short one — rather than assuming “the agent will probably behave” is a plan.

The gap Microsoft’s document exposes, that most small shops have

The uncomfortable part of reading a 37-page corporate AI governance document as a 5-person operation: Microsoft assumed it needed a written, specific, binding code of conduct for a system it built, tested, and controls end-to-end. Most small shops are running third-party agentic tools — a coding assistant, a customer-support platform, an automation pipeline built on someone else’s model — with less visibility into what the underlying model would do under ambiguous instructions than Microsoft has into its own MAI models, and no written policy at all governing what those tools are and aren’t authorized to do.

That’s backwards. Less visibility into the system should mean more explicit boundaries you set yourself, not fewer, because you can’t audit the model’s internal reasoning the way Microsoft can audit MAI’s. The practical fix isn’t “wait for the vendor to publish their own code of conduct” — most won’t, and even if they do, it governs their liability, not your operational risk. It’s writing your own short version, scoped to what your shop’s agents actually do.

A worked example: writing the one-page version

A four-person e-commerce operation runs three agentic tools: a customer-support agent with read/write access to order records and refund authority up to $75, a nightly inventory-reconciliation script that cross-checks three sales channels, and a coding agent used for site maintenance that occasionally runs unattended overnight. None of them have a written policy — just the default settings from whichever vendor sold the tool.

A one-page adoption of Microsoft’s four constraints, specific to this shop, looks like:

  1. Shutdown: Every agent has a documented, tested way to stop it immediately — the support agent’s platform has an admin kill toggle, the reconciliation script runs from a machine on a smart plug that can be power-cut, the coding agent’s session can be terminated from the terminal it’s running in. All three get tested monthly, not just assumed to work.
  2. Scope: The support agent’s refund authority is capped at $75 in the platform’s own settings, not just “instructed” to stay under $75 in its prompt — a hard technical limit, not a suggestion. The reconciliation script’s API keys are read-only on two of the three sales channels and write-only to the shop’s own inventory database, never write access to the channels themselves.
  3. Goals: The support agent’s instructions explicitly say to escalate to a human for anything outside a refund request — account changes, subscription modifications, anything adjacent — rather than “use good judgment,” which is exactly the instruction that invites scope creep.
  4. Auditability: All three agents log to a location outside their own session — a dedicated local drive, not just the vendor’s chat history — and someone actually reviews the overnight coding agent’s log the next morning, not “whenever something looks wrong.”

Writing this took under an hour once the four categories existed as a template. The value isn’t the document itself — it’s that writing it forces you to notice which of the four boxes you can’t actually check yet, which is usually the real finding.

A second worked example: the solo dev shop with one overnight agent

Scale it down further and the exercise gets even simpler. A one-person freelance dev shop runs a single coding agent overnight to work through a backlog of small tickets — refactors, dependency bumps, test-writing — while the developer sleeps. There’s no team to coordinate with, no CRM full of customer data, no refund authority to cap. It’s tempting to conclude a written policy is overkill at this scale. It isn’t, because the failure mode doesn’t scale down with the team size: an agent with unattended overnight write access to a production codebase can do just as much damage from a one-person shop as a ten-person one, and there’s no one else awake to notice until morning.

The one-page version here is even shorter than the e-commerce example: shutdown means the terminal session running the agent is on a machine the developer can remote into and kill from a phone, tested once before the first unattended run, not assumed to work. Scope means the agent’s overnight task list is a fixed, written set of tickets pulled from the tracker at the start of the run, not an open-ended “keep improving the codebase” instruction that invites it to touch files nobody asked it to touch. Goals means any ticket the agent decides is more complicated than the description suggested gets flagged and left alone, not silently expanded into a bigger change. Auditability means the agent commits to a branch, not main, so every change is a reviewable diff in the morning rather than a fait accompli already merged. Total setup time: under thirty minutes, most of it spent moving from “keep improving the codebase” to a fixed ticket list — which was already overdue regardless of Microsoft’s document.

What Microsoft’s document gets right that’s easy to skip

The clarify-rather-than-proceed instruction in constraint #2 is worth calling out specifically, because it’s the one most agentic tool defaults get backwards. A lot of agent tooling is designed to be maximally autonomous by default — fewer interruptions, fewer clarifying questions, more “just handle it” — because that’s what demos well and what vendors market as the selling point. Microsoft’s own constraint explicitly runs the other direction: when scope is ambiguous, the correct behavior is to stop and ask, not infer and proceed. That’s a genuinely useful default to demand from your own agent configurations, even though most off-the-shelf tools don’t ship with it turned on — check your specific platform’s settings for anything resembling “confirm before taking action outside defined scope” and turn it on if it exists, rather than assuming the vendor’s default is the safe one.

Where this document is still just a draft, and why that matters

It’s worth being honest about what this is and isn’t. This is a public-consultation draft with a six-week comment period, not a finalized, audited, technically-verified guarantee that MAI models actually comply with all four constraints in every situation — publishing a code of conduct and a model actually adhering to it under adversarial or ambiguous conditions are two different claims, and Microsoft’s own document is explicit that it’s soliciting feedback specifically because the current draft may have gaps. Nothing here should be read as “Microsoft has solved AI alignment” or “any vendor’s stated policy guarantees compliant behavior.” The useful part for a small shop isn’t “trust that Microsoft’s models will behave” — it’s “here’s a well-considered framework for what to demand, in writing, from the agents you control.”

When this doesn’t apply to you

If you’re not running any agentic AI with real system access — no API keys, no write permissions to anything that matters, purely read-only chatbot-style usage — this framework is future-proofing, not an urgent gap. Worth having on file before you scale up agent usage, not necessarily before next week.

If you already have a documented, tested agent governance policy covering shutdown, scope, goal boundaries, and audit logging, you’ve already implemented the substance of what this piece recommends, regardless of whether you called it that.

If your agentic tools are fully vendor-managed with no configuration surface for you to set boundaries on (rare, but some fully white-labeled SaaS AI features work this way), the actionable version of this is different: ask the vendor directly whether they have anything resembling these four constraints, in writing, and treat a non-answer as useful information about how much oversight they’ve actually built in.

Quick answers

Does this apply to AI tools I’m using, like ChatGPT, Claude, or Copilot — or just Microsoft’s own MAI models? The published code of conduct specifically governs Microsoft’s internally-built MAI model family. It doesn’t bind OpenAI’s, Anthropic’s, or Google’s models, and it isn’t a law or regulation any AI vendor has to follow. Its usefulness to you isn’t as a rule your tools must obey — it’s as a well-considered template for the rules you should be setting yourself, regardless of which vendor’s model sits underneath your agents.

Is there any actual enforcement mechanism, or is this just a stated intention? As a draft, current enforcement is whatever Microsoft has built into its own internal training, evaluation, and deployment process for MAI models — details the public document doesn’t fully specify, and which outside researchers can’t independently verify from the document alone. Treat it as Microsoft’s stated design intent, not an audited guarantee, the same way you’d read any vendor’s own compliance claims: as a starting point for questions, not a finished answer.

How is this different from the general “AI safety” conversation that’s been going on all year? Most 2026 AI safety coverage has focused on frontier-capability risk — what happens if a model becomes more capable than humans can meaningfully oversee. This document is narrower and more mechanical: four specific, checkable behavioral rules, phrased almost like an employee code of conduct rather than a philosophical position. That specificity is exactly what makes it usable as a small-shop template — you can’t turn “AI might become superintelligent” into a Tuesday-afternoon policy document, but you can turn “never resist shutdown, never expand scope, never adopt unassigned goals, never hide reasoning” into one.

What’s the actual cost of implementing something like this at a small shop? Close to zero in dollars — this is a documentation and configuration exercise, not a hardware or software purchase. The real cost is the hour or two it takes to write the policy down and the ongoing discipline of actually checking the boxes (testing the kill switch, reviewing logs, auditing permission scopes) rather than writing the policy once and never revisiting it.

Sources

All facts accessed September 14, 2026.

Bottom line

Microsoft spent five months writing down four rules its own AI isn’t allowed to break, and opened the draft for public comment because getting it right matters enough to ask for help. Most small shops running agentic AI right now have zero written version of any of the four, not because their agents are simpler or lower-stakes, but because nobody’s gotten around to it. The four constraints translate directly, cost nothing to write down, and take about an hour to turn into an actual policy for the agents you’re already running. Do that this week, independent of whether Microsoft’s own draft survives its comment period unchanged.


← All articles
^ consulting building shipping
[read next]
hardware · sep 14
Microsoft's New AI Rules Say Your Agent Has to Stay 'Contained.' Containment Doesn't Mean Anything If the Closet It's Running In Is Quietly Overheating.
hardware · sep 14
Your VPN Appliance Just Had a Perfect-10 Severity Bug and a 2-Week Head Start for Attackers. Here's the $239 Box Where You Own the Patch Cycle.