According to recent reporting from the Financial Times, Amazon’s internal AI coding agent was involved in multiple AWS outages, including one that lasted 13 hours.

Engineers reportedly allowed the agent to make changes that resulted in deleting and recreating a production environment.

Amazon says this was user error, not AI error.

That distinction is interesting, but it misses the bigger point.

When an AI agent is given operator-level permissions, autonomy becomes a product decision, not just a technical one.

As I’ve previously shared, AI systems do not “know” anything. They predict likely outputs. In many workflows, that is incredibly powerful. In production infrastructure, probability carries risk.

Now, this is not an argument against AI agents. It is a reminder that autonomy without guardrails is a business choice.

If you’re a founder integrating AI into your product, ask yourself:

• Where does human review remain mandatory?
• What permissions are truly necessary?
• What is your acceptable failure threshold?
• Who holds accountability when the agent acts?

AI can accelerate execution. But humans must design the boundaries.

If you’re considering intentionally integrating AI, check out my AI Product Clarity Session, a paid working session to help teams decide what to build, what to avoid, and how AI should fit into an existing product or workflow.