Charles Lamanna (@clamanna) header
Charles Lamanna (@clamanna) profile picture

EVP of Copilot, Agents, and Platform at Microsoft, building the next generation of AI-powered tools for work. Notre Dame alum.

Seattle, WA
Joined April 2009
Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)Photo by Charles Lamanna (@clamanna)
The most important thing my team will do this year won’t be a single product or feature we ship - it will be changing how we work. I wrote about the 3 things that have mattered most as we’ve changed how we build – and there’s more we’re still figuring out.
17
47
18
255
115,267
More details on the root cause can be found in the Microsoft 365 Admin center, and an abridged version can be found below. We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business & operations. The incident on Thursday, November 12th, 2026, was caused by three main factors that together caused a global outage for the networking gateway that fronts many Microsoft 365 services. First, a latent defect caused servers to become overwhelmed and unresponsive under certain load conditions. This was due to lock contention inside an internal process which caused the machine to become unavailable. This defect was the result of a fix that was prepared specifically for Microsoft 365 networking gateway servers; neither the underlying defect nor the fix applies to broader Windows deployments. The initial change in behavior first started deployment in July of this year. The update went through safe deployment and saturated globally over several weeks, during which time no issues were observed. For more than three months, the change was deployed worldwide without impact. Our normal load testing did not detect the latent issue because the different traffic and SKU mix used in our test labs. The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on our weaker hardware SKUs with fewer cores. Second, a surge in load increased the likelihood for the networking gateways to encounter the contention. Within Microsoft 365, we maintain two independent and heterogenous network gateway systems designed for isolation and high availability. This allows many Microsoft 365 services to fail to/from them during outages or disruptions. Approximately fifteen hours before the outage, Teams failed over from the alternative networking gateway to the ‘primary’ gateway that also supports Exchange Online. This increased the load on the networking gateway and changed its traffic patterns. This type of failover is part of normal operations and is integral to standard Microsoft resiliency efforts. Third, at the same time, nearly 14% of server capacity for the networking gateway was out of service due to an unrelated issue from the previous day. This server capacity was on track to be returned to active service through routine maintenance and health processes. The service had sufficient capacity headroom, so the reduction in server capacity had no impact on its own. Taken together, the increased load and the reduced server capacity, caused the network gateway to encounter the latent bug from July. This caused requests to build up until servers ultimately stopped responding. This occurred during the peak window of traffic in the United States. When servers lose connectivity, our traffic management systems automatically route users to other healthy endpoints. This happens often and is critical to avoiding any interruption due to hardware, software, or network defects and is normally what makes the service resilient. However, the redirected traffic increased the likelihood for the same problem to occur on the remaining endpoints. This caused a cascading failure from server to server as the same amount of load was spread across fewer servers. In less than two hours, the previously unknown defect was encountered across our fleet and rendered them unresponsive. At its lowest, more than 80% of servers were unresponsive. Normally, the network gateway is composed of thousands of ‘capacity units’ which represent shards of capacity, hardware, and networking infrastructure. These ‘capacity units’ are then organized into regions. In this case, due to a separate issue, the normal bulkheads between these different partitions did not stop the cascading failure. This is what allowed the impact to spread across our network gateway infrastructure globally. Several services like Teams, OneDrive, SharePoint, and Office apps failed out of the faulty network gateway infrastructure. That is why those services were restored earlier, with Teams chats and meetings functioning without issue for most of the outage. But not all services, like Exchange Online, can fail out of this network gateway. Recovery took longer than usual because as servers returned to service, they were immediately overwhelmed by waiting traffic and became unresponsive again. Our normally automated recovery processes were not designed to restore service from this broad unresponsive state. Additionally, since the main code defect was a latent bug from several months previous, our normal rollback, flighting, and feature flags were unable to quickly revert the change. After engineers confirmed the defect as the cause, we developed an update that returned the code path to the previously known good state. To effectively apply this fix, and to slowly bring the service online again, we took groups of servers out of service, applied the change, and restarted them before returning them to service. This process was carefully applied to large quantities of infrastructure at a time (e.g. 50%+ of a region). The final Post-Incident Review (PIR) will provide more details later this week.
Replying to @MSFT365Status
We have released the Preliminary Post-Incident Report of last week’s outage, which is accessible via the Admin center under MO1550821 and EX1550543.
214
508
138
3,411
1,064,976
I want to share what we know so far. This was not a cyberattack or an issue caused by AI tooling. Our amazing team was able to work through a very challenging situation to restore service. Our investigation points to technical challenges and failure modes (cascading failures + thundering herds at our network perimeter) that made recovery more difficult than normal. I'm working closely with the engineering team to complete the full post-mortem. We will publish the post-mortem shortly covering the step-by-step timeline and our many repair items to avoid similar outages in the future. We take our responsibility to our customers seriously and will do everything we can to learn from this and prevent it going forward.
We are deeply sorry for the disruption this incident has caused our customers and partners. We know the trust they place in our services and take that responsibility extremely seriously. We are seeing broad recovery and stability across our services. This was a technical issue not related to cyber or AI / agents. The incident was introduced by an issue during an earlier network capacity upgrade that surfaced following a recent network change. We have addressed the underlying issue and continue to restore service to remaining customers and regions. Given the scale of demand, we restored services gradually, validating stability at each stage. While this took longer, it helped ensure a stable restoration. We will provide a full post-incident report once our review is complete.
610
266
79
2,247
505,506
Charles Lamanna retweeted
We’re seeing encouraging recovery progress, with service availability and email delivery performance improving across a growing portion of affected infrastructure. We know our customers rely on our services to get important work done every day, and regret the impact these outages had on those who depend on us. We will continue to share more information as we have it, and for now we can confirm that there is no indication this is the result of malicious or cyber activity. Our teams are reviewing what happened and how we can prevent similar issues going forward.
1,054
435
6,018
755,201
Organizations across Europe have told us they want the latest AI without trading off the data processing/residency commitments they rely on. Today, we’re announcing that Claude models by Anthropic are coming to Microsoft Copilot for customers in the EU and EFTA, with processing within the EU Data Boundary by the end of 2026 – all in one governed experience & part of our 'auto' model router. Read the blog to learn more: aka.ms/Anthropic-EUDB-Blog
15
32
5
192
25,949
There isn’t one model that’s best at everything anymore. Choosing the right one for each task (without sacrificing quality or paying more than you need to) is a hard engineering problem. That’s what we’re working on with Auto in Copilot. Today, more than 90% of Copilot queries use Auto to dynamically route work across models and optimize for quality, speed, and cost. In this video, I share how it works and chooses the right model for the job...
21
62
17
430
85,861
What a way to kick-off day 1 of the 5th annual Power Platform Community Conference! We sold out this year, with over 9,000 people joining from all around the world. There’s never been a better time to be a maker. The tools are getting more powerful, more people can build than ever before, and the kinds of problems we can solve keep expanding. My Autopilot (Apollo) even made a cameo in the keynote. Loved seeing what this community is building and sharing some of what we’ve been working on. And the week is just getting started here at #PPCC26. 🙌
7
8
73
9,384
Charles Lamanna retweeted
Big ideas are on the calendar. 📅 Join us at #MSIgnite, November 17–21: msft.it/6010axkOj
73
54
10
386
89,850
Persistent agents open up a whole new way to think about business processes... It is not just about automating the steps we have today but rethinking how work gets done. Excited to see what people build with this in Copilot Studio.
New in Copilot Studio: Agent accounts + agent memory + agent triggers = a big step forward for teams automating business processes. Now agents built with Copilot Studio can get their own identity, use it to interact across many surfaces, remember context and improve across those interactions, and kick off work based on events and schedules, not just chat messages. This goes way beyond a personal productivity assistant. For example: A Budget Coordinator agent can now review every new cost line item overnight, tag the right owner in the spreadsheet on transactions that look like they need more scrutiny, drop a summary into your group Teams chat and respond to three emails about invoices while you're on the train into work. BudgetCoordinator has its own secured access to data and interacts and responds as itself (not a maker or end user). It remembers what it was told and the preferences of people who interact with it. A Project Manager agent can collect status from multiple signals and people, remember it all and update the report accurately, be mentioned in a comment and respond to it, and know to send the right updates to stakeholders every day at 5pm. "ProjectManager are we on track for milestone 3?" "ProjectManager mentioned you in a comment" Check out details and demos here: microsoft.com/en-us/copilot/…
6
5
2
70
8,783
Charles Lamanna retweeted
Announcing Today - a team knowledge base, curated by Copilot and agents - powered by SharePoint! Finally, the tech is here to build a feature I've dreamed of for a long time! Excited to see early adopter feedback. techcommunity.microsoft.com/…
35
101
16
762
155,998
Charles Lamanna retweeted
Drive revenue and customer satisfaction with Dynamics 365 CRM and the next wave of our headless platform. Adopt a more intuitive user experience with AI in the flow of work in Copilot, Teams and agents. Join a growing number of organizations moving from Salesforce with Dynamics 365 Activate. aka.ms/cx-blog
1
7
1
26
3,490
The model is not enough - it doesn't come with an understanding of how your business or team works. That context is spread across the work happening every day: files, conversations, emails, databases, and apps. Figuring out what matters for a particular task, and what the model actually needs, is a hard & unsolved problem. That's why Work IQ + Copilot is special... so I white boarded a bit how it works under the hood.
9
47
5
266
48,663
Charles Lamanna retweeted
Today we announced the preview of Microsoft Autopilot is rolling out to our first customers - "Autopilot is a persistent, proactive and personal agent that keeps working even when you’re not" It's been a while since I updated you all on what we are building. First Scout is now Autopilot, and my team leads building this personal agent. We are building Autopilot on @openclaw, working with @steipete and the OpenClaw Foundation to make it a fantastic enterprise grade runtime. blogs.microsoft.com/blog/202… Re-introducing Sebastian, my Autopilot - and he is as excited as I am!
107
110
53
858
905,763
People increasingly want to hand off more work to AI and turn their ideas into automations or apps. That shaped what we’re building with Copilot. Autopilot and Managed Apps in Code / Cowork have been two of my favorites to use internally these past couple of months. You can give Autopilot a role and a goal and it keeps working across days or weeks. Mine is named Apollo. :) And when you want to build an app with Copilot, Managed Apps let you share it with your team. More here: aka.ms/AA13g7uq
We’re building Copilot as a new OS for work that spans every model, every form factor, and every task. Today, we’re announcing our biggest update to Copilot to date, bringing four things together: · Autopilot: proactive and long-running agent built for the enterprise · Code: build apps with Copilot, hosted inside your company’s tenant · Home: Chat + Cowork together · Office: now fully embedded in Copilot (and Copilot embedded in Office, of course!) Plus, you can invoke Copilot in Teams, and we’re introducing Today, a proactive experience that surfaces the most important information from across M365 without needing to ask for it. The way we work is changing and so are our workflows. This update brings AI into that flow, from answering a question, to building an app, to getting work done on your behalf.
19
23
169
17,873
Charles Lamanna retweeted
Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing. We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive. And for firms, it’s imperative that they retain full control over their unique and tacit knowledge. Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control. So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like "embedded evaluators" and the broader efforts to develop the mechanisms to make this more than just talk. The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia. This is the approach we are taking: broad access and choice at every layer of the AI stack; enterprise control of learning loops and models; and the “Code of Conduct” that underlies our own first party MAI models that we’ll publish tomorrow for public consultation.
1,288
1,913
575
15,097
3,279,045
Charles Lamanna retweeted
Now you can build apps in Cowork and Copilot Studio. Just another vibecoding play? Not quite. Many agents can write a decent front end today. The important part is what happens before and after. Now anyone can build in the flow of work, in the context of Work IQ. "Let's create a plan for this next product launch" can naturally evolve to "let's build an app based on the discussion in yesterday's meeting and the email and Teams threads about the project." And, critically, the apps you create are hosted, secured, and fully managed by Microsoft, connecting natively to the places you already keep business data. So no more "check out my cool app at localhost:3000" or "great, now I need to spin up a bunch of services and jump through a bunch of approval hoops to deploy this thing." Just click 'share' and send a link, we'll take it from there. More details about the new /app skill on the Cowork blog: techcommunity.microsoft.com/… And the same capabilities coming to Studio here: microsoft.com/en-us/microsof…
9
22
6
120
33,525
Charles Lamanna retweeted
With the @NFL back tonight, love seeing @Seahawks analyst Brian Eayrs and coaches across the league using new Copilot and Excel tools to help with decision making in the booths and on the sidelines.
65
83
12
716
335,079
Charles Lamanna retweeted
Customers are accelerating the move to the agentic business platform in Dynamics 365 to drive growth, efficiency, and employee satisfaction. Dynamics 365 Activate uses AI to migrate you off Salesforce (ERP next) faster, with less risk — not just lift-and-shift, but a smarter platform for what's next. Public preview live now 👇 microsoft.com/en-us/dynamics…
2
7
2
25
9,160
Our goal is to deliver the latest frontier AI models to customers the moment they’re ready. Today, I’m excited to share that OpenAI’s GPT-6 Astra is available on day one across Microsoft Copilot, Copilot Studio, GitHub Copilot, and Microsoft Foundry. Astra represents an important step forward in the kind of complex, multi-step work AI helps people accomplish. As models continue to improve, people are moving from using AI for individual tasks to delegating larger pieces of work. Astra can complete long running and highly complex tasks in a way we haven’t seen before. At Microsoft, we’re bringing those capabilities into the products and workflows our customers already use every day. Astra is paired with Work IQ in Microsoft Copilot to ground intelligence in an organization’s files, meetings, chats, and business data. Looking forward to seeing what customers build with it. Read more in the blog: aka.ms/astraincopilot
33
43
15
281
97,970
Charles Lamanna retweeted
Software engineering will look "nothing like it has in the past." Microsoft’s Charles Lamanna explains why AI coding means developers will need more editorial judgment bloom.bg/3Sqr3bZ
3
13
1
44
16,508