<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: NTCTech</title>
    <description>The latest articles on DEV Community by NTCTech (@ntctech).</description>
    <link>https://dev.to/ntctech</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3784059%2Fc609d531-fdab-47ac-bb17-37fd1ecc3d71.jpg</url>
      <title>DEV Community: NTCTech</title>
      <link>https://dev.to/ntctech</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ntctech"/>
    <language>en</language>
    <item>
      <title>The SRM Entitlement Key Expires May 2. The Migration Path Is Documented Across Separate Broadcom KBs.</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Wed, 18 Nov 2026 13:30:10 +0000</pubDate>
      <link>https://dev.to/ntctech/the-srm-entitlement-key-expires-may-2-the-migration-path-is-documented-across-separate-broadcom-2jik</link>
      <guid>https://dev.to/ntctech/the-srm-entitlement-key-expires-may-2-the-migration-path-is-documented-across-separate-broadcom-2jik</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fap29o0txbrojom5hxkel.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fap29o0txbrojom5hxkel.jpg" alt="SRM entitlement key expiry date beside an unsequenced migration path to Protection and Recovery 9.1.3 on vCenter 9.1+" width="800" height="437"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;SRM entitlement under VCF and VVF now carries a May 2, 2027 deadline for existing subscribers using the term-limited key, while reaching the required Protection and Recovery 9.1.3 and vCenter 9.1+ state depends on a migration path documented across separate Broadcom articles. Whether that is one deadline or two is the operator's call.&lt;/p&gt;

&lt;p&gt;This post reads six Broadcom knowledge base articles: &lt;a href="https://knowledge.broadcom.com/external/article/461219" rel="noopener noreferrer"&gt;KB 461219&lt;/a&gt;, &lt;a href="https://knowledge.broadcom.com/external/article/438925" rel="noopener noreferrer"&gt;KB 438925&lt;/a&gt;, &lt;a href="https://knowledge.broadcom.com/external/article/312725" rel="noopener noreferrer"&gt;KB 312725&lt;/a&gt;, &lt;a href="https://knowledge.broadcom.com/external/article/444864" rel="noopener noreferrer"&gt;KB 444864&lt;/a&gt;, &lt;a href="https://knowledge.broadcom.com/external/article/460443" rel="noopener noreferrer"&gt;KB 460443&lt;/a&gt; and &lt;a href="https://knowledge.broadcom.com/external/article/447296" rel="noopener noreferrer"&gt;KB 447296&lt;/a&gt;. They were read on November 13, 2026 and checked again on November 14 against each article's Updated On date. On November 18, all six were re-read. KB 461219's Updated On date had moved to 11-16-2026, and its current text was checked against this post. The other five Updated On dates were unchanged. The post separates what Broadcom documents from analysis and from what the operator must verify per deployment. Three sources the articles point to were not read: the convergence article linked from KB 444864, the Broadcom Product Interoperability Matrix, and the article Upgrading to VMware Cloud Foundation 9.1.x, which the KB 461219 FAQ points to. The proposed operator record later identifies all three as unresolved validation inputs.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Broadcom Documents About SRM Entitlement
&lt;/h2&gt;

&lt;p&gt;KB 461219 is the SRM entitlement article. It sits inside the larger &lt;a href="https://www.rack2cloud.com/virtualization-architecture/" rel="noopener noreferrer"&gt;virtualization architecture&lt;/a&gt; question of which stack to stay on, but its scope is narrow: one capability, two access paths and one expiry date. Table 1 restates its terms without extending them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 1: SRM entitlement terms in KB 461219 (Updated On 11-16-2026)&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Item&lt;/th&gt;
&lt;th&gt;What the article states&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Packaging&lt;/td&gt;
&lt;td&gt;As of November 2, 2026, SRM, formerly VMware Live Site Recovery (VLSR), is not sold standalone and is included in VCF and VVF&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Who is entitled&lt;/td&gt;
&lt;td&gt;Existing active VCF and VVF subscription customers: those who do not have SRM and want it, those renewing SRM or VLSR, and those expanding SRM or VLSR capacity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Not entitled&lt;/td&gt;
&lt;td&gt;Customers with perpetual VCF licenses, with or without active Support and Subscription&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Option 1&lt;/td&gt;
&lt;td&gt;Wait for VCF/VVF 9.1.3, stated as available between February 2027 and April 2027. The release adds SRM entitlement to existing subscriptions automatically, and using SRM then requires a Protection and Recovery appliance at 9.1.3 and vCenter at 9.1 or later&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Option 2&lt;/td&gt;
&lt;td&gt;Obtain a term-limited key from the Support Portal (My Downloads, not the Licensing Portal). It covers up to 10,000 protected VMs, expires May 2, 2027, cannot be split, combined, upgraded or downgraded, and replaces existing keys&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Condition tied to expiry&lt;/td&gt;
&lt;td&gt;The appliance must reach 9.1.3 and vCenter 9.1 or later before the key expires. The FAQ says the expiry date factors in the time to get there, to avoid "disaster recovery functionalities ceasing to function"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Newer subscriptions&lt;/td&gt;
&lt;td&gt;New, renewed or expanded subscriptions on or after November 2, 2026 include SRM and can disregard both options&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Existing SRM or VLSR entitlements&lt;/td&gt;
&lt;td&gt;The FAQ says customers can continue to use existing entitlements for the remainder of the contract term and keep current benefits, including support for older vCenter and Protection and Recovery appliance versions where applicable. Swapping licenses to the options above is the alternative for customers who want to protect more VMs, and the version requirements of the chosen option then apply&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The article also describes the Support Portal key as version 8.1, working on all supported SRM versions. It defers the minimum supported vCenter and appliance versions to the Interoperability Compatibility Matrix and to a KB article on invalid license key errors, which matches the title of KB 438925 and is covered below. For upgrades beyond vCenter 9.1+ and the Protection and Recovery appliance at 9.1.3, its FAQ points to the article Upgrading to VMware Cloud Foundation 9.1.x and to the Interoperability Compatibility Matrix. Neither was read for this post, and nothing here describes their contents.&lt;/p&gt;

&lt;p&gt;Subscriptions renewed or expanded on or after November 2 carry SRM with them, and the renewal position behind them belongs to the &lt;a href="https://www.rack2cloud.com/vmware-renewal-decision/" rel="noopener noreferrer"&gt;VMware renewal decision&lt;/a&gt;. This post starts after that decision.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision One: Entitlement Continuity Before May 2, 2027
&lt;/h2&gt;

&lt;p&gt;The first decision is which path keeps SRM entitlement in force. The articles describe two paths and state their conditions. They do not recommend one, and neither does this post.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the Two Options Commit You To
&lt;/h3&gt;

&lt;p&gt;Option 1 commits the operator to waiting for 9.1.3. When it ships, KB 461219 states that SRM entitlement arrives with the release, and that using it requires the Protection and Recovery appliance at 9.1.3 and vCenter at 9.1 or later. Option 2 commits the operator to replacing the existing SRM keys with the term-limited key in vCenter, since it cannot be split or combined. The same article ties that key to the same endpoint: the appliance and vCenter must reach 9.1.3 and 9.1+ before May 2, 2027.&lt;/p&gt;

&lt;p&gt;Both options end at the same endpoint. The article presents Option 2 for customers who need SRM capabilities before 9.1.3 ships.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Window Between 9.1.3 and Key Expiry
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Analysis:&lt;/strong&gt; KB 461219 gives two dates. It states that 9.1.3 is available between February 2027 and April 2027, and that the key expires May 2, 2027. The margin between them depends on where in that window the release lands. A release on February 1 leaves 90 days before May 2. A release on April 30 leaves two. The article does not state that the availability window is a commitment, or what happens to the expiry date if the release slips. These are theoretical calendar margins, not usable maintenance windows: they exclude preparation, validation, replication testing and recovery exercises. This is arithmetic from two documented dates, not a judgment that any margin is adequate.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the Article Does Not State
&lt;/h3&gt;

&lt;p&gt;Each item below was checked against the article text; none is filled in.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Behavior at or before expiry.&lt;/strong&gt; The FAQ names the outcome to avoid. It does not state which operations stop, whether there is a grace period, or whether the operator is notified before the key expires. KB 312725 documents a "not enough licenses" error that SRM shows on operations such as protecting VMs, failover and reprotect, and it lists an expired key among the conditions. Its environment list does not include Protection and Recovery 9.1.x, and it does not mention the term-limited key. Whether the same behavior applies is NOT STATED.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Option 1 until 9.1.3.&lt;/strong&gt; For customers with existing SRM or VLSR entitlements, the article states that those entitlements continue for the remainder of the contract term. It does not state how an environment with no existing SRM entitlement that waits for 9.1.3 is licensed before the release is installed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The key and KB 438925.&lt;/strong&gt; KB 438925 describes a validation defect for keys generated on or after April 20, 2026, affecting VLSR (sold through the VLR SKU), VCDA (VLR or ACC SKU) and standalone SRM (SRM SKU). Its environment list includes Protection and Recovery 9.1.0.0, and it lists fixed builds: 9.0.2.4 and later, 9.0.5.1 and later, Protection and Recovery 9.1.0.0100 and later, and vCDA 4.7.5 and later. KB 461219 points to an article with its title for minimum versions. Neither article names the other's key or SKU, and the generation date and SKU of the key are not stated in KB 461219. The articles do not establish whether the term-limited key needs the same validation remediation, so the installed build and the key's behavior are verified with Broadcom. This post does not reconcile them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Split and merge.&lt;/strong&gt; KB 438925 says to perform a split and merge in the Broadcom Support Portal to refresh license metadata if the validation error persists after the required upgrade. KB 461219 states that the term-limited key cannot be split or combined. The articles do not state whether the refresh applies to that key, so confirm with Broadcom before any license operation involving it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Decision Two: Migration and Recovery Configuration Before 9.1.3
&lt;/h2&gt;

&lt;p&gt;The second decision is how the installed appliance, vCenter and replication state reach the endpoint both SRM entitlement options share. The vCenter half of that endpoint is a version upgrade in its own right, and &lt;a href="https://www.rack2cloud.com/vsphere-lifecycle-management-governance/" rel="noopener noreferrer"&gt;vSphere lifecycle management&lt;/a&gt; covers the compatibility envelope it has to stay inside. The articles that bear on the rest are separate.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 2: What each Broadcom article covers&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;KB (Updated On)&lt;/th&gt;
&lt;th&gt;Condition it addresses&lt;/th&gt;
&lt;th&gt;What it states&lt;/th&gt;
&lt;th&gt;Scope limit&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;444864 (06-18-2026)&lt;/td&gt;
&lt;td&gt;Moving from standalone SRM and vSphere Replication 9.0.2 to Protection and Recovery 9.1&lt;/td&gt;
&lt;td&gt;Standalone appliances are deprecated in 9.1, with no in-place ISO upgrade. A fresh-deployment procedure is given. Standalone 9.0.2 to 9.1 typically needs a convergence migration via 9.0.4&lt;/td&gt;
&lt;td&gt;Minimal footprint only: single-VM replication, no Protection Groups or Recovery Plans. The example is vCenter 8.0 U3 to 9.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;312725 (11-04-2026)&lt;/td&gt;
&lt;td&gt;License errors on SRM operations when licenses are insufficient&lt;/td&gt;
&lt;td&gt;The license is preserved only on an upgrade within the same version. A cross-version upgrade reverts to evaluation and the key must be reassigned&lt;/td&gt;
&lt;td&gt;Lists SRM 8.x, VLSR 9.0 through 9.0.2.4 and VLR 9.0.4 through 9.0.5.1. Protection and Recovery 9.1.x is not listed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;460443 (10-25-2026)&lt;/td&gt;
&lt;td&gt;Replication configuration fails with vCenter 9.1&lt;/td&gt;
&lt;td&gt;VLR 9.0.2 is not compatible with vCenter 9.1. The fix is to deploy VLR 9.1.0, re-pair the sites and re-run replication configuration&lt;/td&gt;
&lt;td&gt;Environment lists VLR 9.0.2, vCenter 9.1 and VCF 9.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;447296 (09-20-2026)&lt;/td&gt;
&lt;td&gt;Where SRM and VLSR compatibility entries live&lt;/td&gt;
&lt;td&gt;SRM and VLSR are consolidated under Protection and Recovery in the Interoperability Matrix. Converged appliance model since VLSR 9.0.3. vVols are deprecated in VCF 9.0, and vVols replications move to array-based replication before Protection and Recovery 9.1&lt;/td&gt;
&lt;td&gt;Environment lists the VLR 9.0.5 appliance. The Matrix itself was not read&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;438925 (11-10-2026)&lt;/td&gt;
&lt;td&gt;"Invalid license key" on keys generated on or after April 20, 2026&lt;/td&gt;
&lt;td&gt;A validation defect affecting VLSR, VCDA and standalone SRM keys. Fixes are listed per build: 9.0.2.4+, 9.0.5.1+, Protection and Recovery 9.1.0.0100+ and vCDA 4.7.5+. If the error persists after the required upgrade, it says to perform a Support Portal split and merge&lt;/td&gt;
&lt;td&gt;Environment list includes Protection and Recovery 9.1.0.0. Does not name the term-limited key&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fa8cor3264yvq489j2nq8.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fa8cor3264yvq489j2nq8.jpg" alt="Five Broadcom articles on SRM entitlement migration, each covering one installed-state question, with the sequence across them left empty" width="799" height="391"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The Minimal-Footprint Limit
&lt;/h3&gt;

&lt;p&gt;KB 444864 gives a seven-step clean transition: delete active replications; access the legacy SRM and vSphere Replication appliances through VAMI and unregister them from vCenter; power down and delete those appliances; upgrade vCenter to 9.1; deploy the unified appliance; establish site pairing; and reconfigure VM replication. The article scopes it to minimal-footprint environments, meaning single-VM replication and no Protection Groups or Recovery Plans. For a standalone 9.0.2 environment beyond that, it points to a convergence migration via 9.0.4 and links a separate article. That article was not read here and is not summarized.&lt;/p&gt;

&lt;h3&gt;
  
  
  Licenses, Compatibility and Replication Do Not Share a Page
&lt;/h3&gt;

&lt;p&gt;A license assignment, a version-compatibility fact and a replication reconfiguration each sit in a different article. KB 312725 covers the license, KB 460443 a replication failure that follows from putting vCenter 9.1 beside VLR 9.0.2, and KB 447296 where compatibility is recorded and the vVols deprecation. Broadcom's naming also shifts across them: SRM, VLSR, VLR and Protection and Recovery. KB 460443 says to deploy VLR 9.1.0, while KB 461219 and KB 444864 call the 9.1 appliance Protection and Recovery. This post states each as written and does not reconcile them.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;⚠ &lt;strong&gt;Scope limit:&lt;/strong&gt; Each article is scoped to its own environment list. Nothing in the six says every environment needs the convergence path, a site re-pair or a vVols migration, and this post does not say it either.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Two Planning Concerns, Not One Documented Requirement
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Analysis:&lt;/strong&gt; In the six articles read, SRM entitlement and the migration path are related, and Broadcom documents the relationship at the endpoint. KB 461219 states that the appliance must be at 9.1.3 and vCenter at 9.1+ before the key expires. What the six articles reviewed do not provide is one consolidated migration procedure covering the different installed states, licensing conditions and replication configurations that may exist across an estate. Individual procedures and compatibility requirements are documented, but their sequencing must be established against the actual deployment. KB 461219 also points to Upgrading to VMware Cloud Foundation 9.1.x, which was not read for this post. Nothing here describes its contents or assumes it closes that gap.&lt;/p&gt;

&lt;p&gt;Read together, the articles imply an order. KB 460443 documents what fails when vCenter 9.1 meets VLR 9.0.2. KB 444864's clean transition upgrades vCenter only after the legacy replications and appliances are removed. KB 312725 documents that a cross-version upgrade drops a license to evaluation. None of them sequences those facts for an environment that has all three conditions at once. That is a planning concern the operator owns per deployment, not a requirement Broadcom has written as one. &lt;/p&gt;

&lt;h2&gt;
  
  
  What to Validate Before Committing a Path
&lt;/h2&gt;

&lt;p&gt;Table 3 is a proposed operator method. It is not Broadcom guidance and not a Broadcom checklist. It turns the open SRM entitlement items above into a record the operator completes per deployment, and it names the sources this post did not read.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 3: Proposed operator record, to be completed per deployment&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Item to verify&lt;/th&gt;
&lt;th&gt;Why it is open&lt;/th&gt;
&lt;th&gt;Where to verify&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Installed SRM, VLSR, VLR or Protection and Recovery version, per site&lt;/td&gt;
&lt;td&gt;Every article read is scoped by version, and none states a path for a version outside its list&lt;/td&gt;
&lt;td&gt;Appliance management interface at each site, against each article's environment list&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;vCenter version, per site&lt;/td&gt;
&lt;td&gt;9.1+ is the stated endpoint, and KB 460443 documents a failure with VLR 9.0.2&lt;/td&gt;
&lt;td&gt;vSphere Client, then the Protection and Recovery entry in the Interoperability Matrix (not read here)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Count of Protection Groups and Recovery Plans&lt;/td&gt;
&lt;td&gt;KB 444864's clean transition is scoped to environments that have none&lt;/td&gt;
&lt;td&gt;Site Recovery interface&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;vVols-backed replications&lt;/td&gt;
&lt;td&gt;KB 447296 says to move them to array-based replication before Protection and Recovery 9.1&lt;/td&gt;
&lt;td&gt;Replication inventory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;License assignment state&lt;/td&gt;
&lt;td&gt;KB 312725 documents reversion to evaluation on a cross-version upgrade&lt;/td&gt;
&lt;td&gt;vSphere Client, Licensing, Assets, Solutions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Which key each site will hold&lt;/td&gt;
&lt;td&gt;The term-limited key replaces existing keys and cannot be split or combined&lt;/td&gt;
&lt;td&gt;Support Portal My Downloads, per KB 461219&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The convergence article linked from KB 444864&lt;/td&gt;
&lt;td&gt;It is the stated route for larger environments and was not read here&lt;/td&gt;
&lt;td&gt;The link in KB 444864's Additional Resources&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The upgrade article pointed to by KB 461219 (Upgrading to VMware Cloud Foundation 9.1.x)&lt;/td&gt;
&lt;td&gt;KB 461219's FAQ points to it for upgrades beyond vCenter 9.1+ and the appliance at 9.1.3, and it was not read here&lt;/td&gt;
&lt;td&gt;The link in KB 461219's FAQ&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The KB 438925 build list against the key path chosen, and whether its split and merge applies to the term-limited key&lt;/td&gt;
&lt;td&gt;KB 461219 points to it by title, KB 438925 does not name the term-limited key, and KB 461219 says the key cannot be split or combined&lt;/td&gt;
&lt;td&gt;KB 438925, and a Broadcom ticket under VMware vCenter Server, Protection and Recovery, the route KB 461219 gives&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The 9.1.3 availability date against May 2, 2027&lt;/td&gt;
&lt;td&gt;Slip behavior is not stated&lt;/td&gt;
&lt;td&gt;Broadcom release communications when published&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fplv5pwiu0r3nv0pgndd7.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fplv5pwiu0r3nv0pgndd7.jpg" alt="Three evidence lanes for SRM entitlement planning: documented, analysis and validate per deployment" width="800" height="406"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The version path is a lifecycle item before it is an upgrade ticket. The &lt;a href="https://www.rack2cloud.com/modern-virtualization-learning-path/virtualization-deterministic-operations/" rel="noopener noreferrer"&gt;virtualization deterministic operations&lt;/a&gt; stage covers lifecycle and drift governance, which is where a record like Table 3 belongs once it exists. A blank row is an unestablished fact.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://rack2cloud.com/downloads/checklists/srm-entitlement-migration-deadline-checklist-v1.pdf" rel="noopener noreferrer"&gt;SRM 9.1.3 Migration Readiness Record&lt;/a&gt; (PDF, 2-page checklist). A proposed operator method, not Broadcom guidance.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;SRM entitlement now has a date, May 2, 2027, and a destination, Protection and Recovery 9.1.3 on vCenter 9.1 or later. Broadcom documents both and ties them together at the endpoint.&lt;/p&gt;

&lt;p&gt;What the six articles read document only conditionally is the route between an installed estate and that destination. The license behavior, the compatibility facts, the replication fixes and the migration procedure each live in a separate article with its own environment list, and none sequences them for an estate that has several of those conditions at once. For that estate, the key's expiry and the version path are two planning concerns joined at a single date.&lt;/p&gt;

&lt;p&gt;The real problem is assembly. The operator has to build the sequence and own every row of it, including the rows where the source does not say. The SRM entitlement decision is the simpler half. The route is the work.&lt;/p&gt;

&lt;p&gt;Broadcom documented the date and the destination. The route is yours to write down.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/virtualization-architecture/" rel="noopener noreferrer"&gt;Virtualization Architecture&lt;/a&gt; — the pillar covering which stack a vSphere estate stays on and what each choice carries&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/modern-virtualization-learning-path/virtualization-deterministic-operations/" rel="noopener noreferrer"&gt;Virtualization Deterministic Operations&lt;/a&gt; — the learning path stage on lifecycle and drift governance, where a per-deployment record like Table 3 belongs&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/vmware-renewal-decision/" rel="noopener noreferrer"&gt;The VMware Renewal Decision Was Made Two Years Ago&lt;/a&gt; — the renewal position this post starts after&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/vsphere-lifecycle-management-governance/" rel="noopener noreferrer"&gt;vSphere Lifecycle Management Is a Governance Problem, Not a Patching Problem&lt;/a&gt; — the compatibility envelope a vCenter upgrade has to stay inside&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/461219" rel="noopener noreferrer"&gt;KB 461219: Access to VMware Site Recovery Manager for Existing VMware Cloud Foundation and VMware vSphere Foundation Subscription Customers&lt;/a&gt; — the entitlement terms, the two options and the May 2, 2027 key expiry&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/438925" rel="noopener noreferrer"&gt;KB 438925: Invalid license key error when applying the SRM license key&lt;/a&gt; — the validation defect, the SKUs and builds it lists, and the fixed versions&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/312725" rel="noopener noreferrer"&gt;KB 312725: There are not enough licenses installed to perform the operation&lt;/a&gt; — license preservation across versions and the expired-key symptoms&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/444864" rel="noopener noreferrer"&gt;KB 444864: VCF Protection and Recovery 9.1: Transition from standalone SRM and VR 9.0.2&lt;/a&gt; — the minimal-footprint clean transition and its scope limit&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/460443" rel="noopener noreferrer"&gt;KB 460443: VMware Live Recovery Replication Configuration Fails Due to Incompatibility with vCenter Server 9.1&lt;/a&gt; — the VLR 9.0.2 and vCenter 9.1 failure and its fix&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://knowledge.broadcom.com/external/article/447296" rel="noopener noreferrer"&gt;KB 447296: Protection and Recovery category for SRM and VLSR compatibility in Interoperability Matrix&lt;/a&gt; — where compatibility is recorded and the vVols deprecation&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/srm-entitlement-migration-deadline/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>vmware</category>
      <category>disasterrecovery</category>
      <category>infrastructure</category>
      <category>sysadmin</category>
    </item>
    <item>
      <title>Kubernetes AI Conformance Is a Listing. Acceptance Is Still Yours to Record.</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Tue, 17 Nov 2026 15:04:20 +0000</pubDate>
      <link>https://dev.to/ntctech/kubernetes-ai-conformance-is-a-listing-acceptance-is-still-yours-to-record-3f50</link>
      <guid>https://dev.to/ntctech/kubernetes-ai-conformance-is-a-listing-acceptance-is-still-yours-to-record-3f50</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fu2k1la80n70jesg7kpis.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fu2k1la80n70jesg7kpis.jpg" alt="Kubernetes AI conformance listing badge beside an empty operator acceptance record" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Kubernetes AI conformance gives an operator a listing, not an acceptance record. A listing says a vendor filed a statement against a versioned checklist and that the statement sits in the program's repository. An acceptance record says the evidence behind that statement is sufficient for a named workload, on a named platform, at a named Kubernetes version. This post reads the 24 submissions filed against Kubernetes v1.35, states what the program establishes about them and what it leaves open, and sets out where the operator's own test begins.&lt;/p&gt;

&lt;p&gt;Everything below is bound to one repository state and one version. The submissions repository, &lt;code&gt;cncf/k8s-ai-conformance&lt;/code&gt;, was read at commit &lt;code&gt;c64e5bc&lt;/code&gt; (October 29, 2026) and the requirements repository, &lt;code&gt;kubernetes-sigs/ai-conformance&lt;/code&gt;, at &lt;code&gt;fd9ba2d&lt;/code&gt; (November 3, 2026), both re-checked on November 13, 2026. The v1.36 and v1.37 submissions are out of scope, and so is every platform comparison: the counts are aggregated and no product is named.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Kubernetes AI Conformance Requires
&lt;/h2&gt;

&lt;p&gt;The Kubernetes AI conformance program began as a Kubernetes working group and now runs as a subproject of Kubernetes SIG Architecture, according to the &lt;a href="https://kubernetes.io/blog/2026/10/29/sig-arch-ai-conformance-2026/" rel="noopener noreferrer"&gt;SIG Architecture write-up of October 29, 2026.&lt;/a&gt;. Requirements live in one repository and submissions in another. A vendor files a &lt;code&gt;PRODUCT.yaml&lt;/code&gt; for a product and a Kubernetes version, and that file carries, for every requirement, a status, a list of evidence references and a notes field.&lt;/p&gt;

&lt;p&gt;The v1.35 checklist has twelve requirements: nine MUST and three SHOULD. It covers the Kubernetes layer of &lt;a href="https://www.rack2cloud.com/ai-infrastructure-strategy-guide/" rel="noopener noreferrer"&gt;AI infrastructure architecture&lt;/a&gt;: accelerator access, inference networking, scheduling, observability, security and operator behavior. It says nothing about model quality, GPU supply, fabric or storage.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 1: The v1.35 checklist&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Requirement ID&lt;/th&gt;
&lt;th&gt;Level&lt;/th&gt;
&lt;th&gt;Area&lt;/th&gt;
&lt;th&gt;Asks the platform to&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;dra_support&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Accelerators&lt;/td&gt;
&lt;td&gt;Support Dynamic Resource Allocation APIs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;driver_runtime_management&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;SHOULD&lt;/td&gt;
&lt;td&gt;Accelerators&lt;/td&gt;
&lt;td&gt;Provide a verifiable mechanism for compatible drivers and container runtime configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;gpu_sharing&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;SHOULD&lt;/td&gt;
&lt;td&gt;Accelerators&lt;/td&gt;
&lt;td&gt;Provide at least one defined GPU sharing strategy, where the accelerator supports static sharing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;virtualized_accelerator&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;SHOULD&lt;/td&gt;
&lt;td&gt;Accelerators&lt;/td&gt;
&lt;td&gt;Expose and manage virtualized accelerators such as vGPU, where supported&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ai_inference&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Networking&lt;/td&gt;
&lt;td&gt;Support Gateway API with an implementation for advanced inference traffic management&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;gang_scheduling&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Scheduling and orchestration&lt;/td&gt;
&lt;td&gt;Allow installation and operation of at least one gang scheduling solution&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;cluster_autoscaling&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Scheduling and orchestration&lt;/td&gt;
&lt;td&gt;Where a cluster autoscaler exists, scale accelerator node groups&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;pod_autoscaling&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Scheduling and orchestration&lt;/td&gt;
&lt;td&gt;Where HorizontalPodAutoscaler is supported, make it work for accelerator pods&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;accelerator_metrics&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Observability&lt;/td&gt;
&lt;td&gt;Allow installation of an accelerator metrics solution&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ai_service_metrics&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Observability&lt;/td&gt;
&lt;td&gt;Discover and collect workload metrics in a standard format&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;secure_accelerator_access&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Security&lt;/td&gt;
&lt;td&gt;Isolate and mediate container access to accelerators through Kubernetes resource management&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;robust_controller&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;Operator&lt;/td&gt;
&lt;td&gt;Show that at least one complex AI operator with a CRD installs and runs reliably&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Levels move between versions, so the checklist is a versioned artifact. In the version files in &lt;code&gt;kubernetes-sigs/ai-conformance&lt;/code&gt;, &lt;code&gt;driver_runtime_management&lt;/code&gt; is SHOULD in v1.35 and MUST in v1.38, and &lt;code&gt;dra_support&lt;/code&gt;, a v1.35 MUST, does not appear in v1.36 through v1.38. A v1.35 listing is read against the v1.35 checklist and nothing else.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the 24 v1.35 Submissions Report
&lt;/h2&gt;

&lt;p&gt;The v1.35 folder holds 24 submissions and 285 requirement rows. Table 2 aggregates what Kubernetes AI conformance submitters wrote in the status field.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 2: Reported status, v1.35 submissions&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Row class&lt;/th&gt;
&lt;th&gt;Rows&lt;/th&gt;
&lt;th&gt;Implemented&lt;/th&gt;
&lt;th&gt;Blank&lt;/th&gt;
&lt;th&gt;N/A&lt;/th&gt;
&lt;th&gt;Not Implemented&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;MUST (9 requirements)&lt;/td&gt;
&lt;td&gt;216&lt;/td&gt;
&lt;td&gt;216&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD: &lt;code&gt;driver_runtime_management&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD: &lt;code&gt;gpu_sharing&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD: &lt;code&gt;virtualized_accelerator&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD total&lt;/td&gt;
&lt;td&gt;69&lt;/td&gt;
&lt;td&gt;53&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;One submitter filed only the nine MUST rows and omitted the three SHOULD requirements, so the SHOULD total is 69 rather than 72. The current script reports a missing requirement row as an error; whether that check existed when that submission merged is not established.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4csfmhvuykpizkn886st.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4csfmhvuykpizkn886st.jpg" alt="Status variation in v1.35 submissions: MUST rows, 216 of 216 Implemented; SHOULD rows, 69 split into 53 Implemented, 11 blank, 3 N/A and 2 Not Implemented" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;⚠ &lt;strong&gt;Common mistake:&lt;/strong&gt; These counts describe what submitters filed. The folder holds submissions, not every platform that runs AI workloads on Kubernetes, and a platform absent from it has not failed anything. Read the pattern as a record of reported statements, not as a market survey.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  The MUST Rows Report No Variance
&lt;/h3&gt;

&lt;p&gt;All 216 MUST rows report Implemented. Three things sit behind that figure, and they should stay separate.&lt;/p&gt;

&lt;h3&gt;
  
  
  01 — OBSERVED
&lt;/h3&gt;

&lt;p&gt;Every v1.35 submission reports every MUST row it filed as Implemented.&lt;/p&gt;

&lt;h3&gt;
  
  
  02 — CURRENT VALIDATOR BEHAVIOR (AT c64e5bc)
&lt;/h3&gt;

&lt;p&gt;The script requires a MUST row to be Implemented. The one exception is N/A on &lt;code&gt;cluster_autoscaling&lt;/code&gt;, which needs a justification in the notes field and raises a review warning.&lt;/p&gt;

&lt;h3&gt;
  
  
  03 — NOT ESTABLISHED
&lt;/h3&gt;

&lt;p&gt;The record does not show which version of the script ran against each v1.35 submission, or whether every submission was checked under the current rule. The workflow validates the product directories changed in a pull request, so the rule applied is the one current when that pull request ran.&lt;/p&gt;

&lt;p&gt;The result is a uniform reported status. It is not proof that 216 rows were independently tested, and it is not proof that every submission passes today's script. A column with no variation cannot separate one submission from another either, which is why the MUST rows work as an entry condition and not as a discriminator. In one submission a MUST row reports Implemented while its own notes say the component does not apply on that platform, a small illustration that the status field and the notes can carry different statements.&lt;/p&gt;

&lt;p&gt;The rows also state less than they appear to. The &lt;code&gt;gang_scheduling&lt;/code&gt; row asks for at least one installable and operable solution. Which solution the operator runs, and how it is sized against the workload's queue behavior, are separate decisions, covered in &lt;a href="https://www.rack2cloud.com/gpu-scheduling-kubernetes/" rel="noopener noreferrer"&gt;GPU scheduling in Kubernetes&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  The SHOULD Rows Carry All Status Variation
&lt;/h3&gt;

&lt;p&gt;Every blank, N/A and Not Implemented in the folder sits in a SHOULD row: 16 of the 69. They are three different statements, and a reader who collapses them loses information.&lt;/p&gt;

&lt;p&gt;A blank is no statement. The current script skips a SHOULD row whose status and evidence are both empty, so an unanswered row passes. N/A is a claim that the condition does not apply, and the script requires a non-empty justification for it. Not Implemented is an explicit negative. Of the three SHOULD requirements, &lt;code&gt;virtualized_accelerator&lt;/code&gt; carries the most variation, with nine of 23 rows not reporting Implemented: five blank, two N/A and two Not Implemented. &lt;code&gt;gpu_sharing&lt;/code&gt; follows with six. Both are written as conditional on what the accelerator supports, which is where an N/A is plausible and where a blank is hardest to read.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a Status Does and Does Not Establish
&lt;/h2&gt;

&lt;p&gt;Table 3 puts the Kubernetes AI conformance program's evidence beside the claims that evidence cannot carry. The left column states what the record supports. The right column states what it does not.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 3: What the record can and cannot support&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Evidence&lt;/th&gt;
&lt;th&gt;What we can say&lt;/th&gt;
&lt;th&gt;What we cannot say&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Checklist (every version)&lt;/td&gt;
&lt;td&gt;The checklist defines a level, status, evidence and notes field for each requirement, and the submitter supplies the reported values&lt;/td&gt;
&lt;td&gt;That a status is more than the submitter's statement unless it is corroborated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A v1.35 submission&lt;/td&gt;
&lt;td&gt;The submitter reports a status and may provide evidence references. In the 24 folders the evidence takes several forms: external links, in-folder files, both, and a handful of Implemented rows with an empty evidence field&lt;/td&gt;
&lt;td&gt;That the evidence was independently tested, or that the 24 submissions were processed by test-artifact validation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Program rules (verified November 13)&lt;/td&gt;
&lt;td&gt;At &lt;code&gt;c64e5bc&lt;/code&gt;, a MUST row must be Implemented, with the conditional &lt;code&gt;cluster_autoscaling&lt;/code&gt; exception. The script does not require an evidence entry for an Implemented row, and it checks web links for reachability, not content. The restriction on N/A for MUST rows was committed on September 24, 2026; an earlier commit, September 18, 2026, had accepted N/A on any MUST row with a justification&lt;/td&gt;
&lt;td&gt;That the 24 v1.35 submissions were validated by this version of the script, or that blank, N/A and Not Implemented SHOULD rows mean the same thing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Test-artifact guidance and validator behavior&lt;/td&gt;
&lt;td&gt;Test artifacts are recommended and manual attestation is allowed. The script's version gate controls only the recommendation warning. Recognized artifacts are checked whenever a submission cites them, and a failing test is an error, as is a skipped test on an Implemented row&lt;/td&gt;
&lt;td&gt;That testing is uniform across accelerator types, since the driver runtime test covers NVIDIA only, or that a given submission cited an artifact&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Upstream suite&lt;/td&gt;
&lt;td&gt;A test exists for every MUST requirement, per a documentation update dated November 3, 2026&lt;/td&gt;
&lt;td&gt;That any given submission ran it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The whole pattern&lt;/td&gt;
&lt;td&gt;Counts of what 24 submitters reported&lt;/td&gt;
&lt;td&gt;That every platform passed independent testing, that Implemented proves production capability, or that all SHOULD rows are equally verified&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For a v1.35 listing, the available program documentation does not establish that the program independently verified each submission's evidence. The vendor supplies the submission and evidence; the operator retains responsibility for deciding whether that evidence is sufficient for the workload being accepted. Three parties stay distinct: the vendor's evidence obligation, the program's conformance process, and the operator's acceptance decision. None of this says the program does nothing, and the next section sets out what it does.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the Program Verifies Claims, by Version
&lt;/h2&gt;

&lt;p&gt;Kubernetes AI conformance does real work on every submission. A submission arrives as a pull request. A validator workflow runs against the directories that pull request changed, and the script checks that statuses are valid values, that MUST rows meet the rule above, that an N/A carries a justification, that in-folder evidence files exist and that web evidence links respond. From v1.37 the documentation recommends test artifacts for requirements that have an automated test. I found no description of a review step beyond the validator workflow, so this post makes no claim about one, in either direction.&lt;/p&gt;

&lt;h3&gt;
  
  
  v1.35 and Earlier: Documentation and Evidence URLs
&lt;/h3&gt;

&lt;p&gt;The version boundary comes first, because the mechanism changes at it. The program's README describes certification for versions before v1.37, and for requirements without an automated test, as documentation-backed self-assessment. In plain terms, for v1.35 the record contains documented requirement statuses and may contain submitter-provided evidence references; the available record does not establish that each submission's evidence was independently tested. That sentence is not a full description of the v1.35 certification process, and nothing here projects the later mechanism backward onto it.&lt;/p&gt;

&lt;h3&gt;
  
  
  Test Artifacts and Hybrid Verification
&lt;/h3&gt;

&lt;p&gt;Four facts bear on verification, and each stands alone.&lt;/p&gt;

&lt;h3&gt;
  
  
  01 — UPSTREAM TESTS EXIST
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;test/&lt;/code&gt; directory in &lt;code&gt;kubernetes-sigs/ai-conformance&lt;/code&gt; holds an automated suite, and a documentation update dated November 3, 2026 states that a test exists for every MUST requirement. The driver runtime test covers NVIDIA only. The submissions validator's own mapping from requirement to upstream test is still being extended: an open pull request on November 13 adds the mapping for &lt;code&gt;ai_inference&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  02 — GUIDANCE RECOMMENDS ARTIFACTS FROM v1.37
&lt;/h3&gt;

&lt;p&gt;The documentation recommends &lt;code&gt;junit.xml&lt;/code&gt;, &lt;code&gt;results.json&lt;/code&gt; or &lt;code&gt;e2e.log&lt;/code&gt; for requirements with an automated test, calls hybrid verification recommended rather than required, and continues to allow manual attestation.&lt;/p&gt;

&lt;h3&gt;
  
  
  03 — VALIDATOR CHECKS RECOGNIZED ARTIFACTS AT ANY VERSION
&lt;/h3&gt;

&lt;p&gt;At &lt;code&gt;c64e5bc&lt;/code&gt; the v1.37 threshold gates only the recommendation warning. When an evidence entry points to a recognized test artifact, the script parses it whatever the version, and a failing test is an error, as is a skipped test on an Implemented row. Artifact validation entered the script on September 8, 2026.&lt;/p&gt;

&lt;h3&gt;
  
  
  04 — EXECUTION AGAINST A SUBMISSION IS SEPARATE
&lt;/h3&gt;

&lt;p&gt;The first three facts describe tests that exist, guidance that exists and a script that behaves a certain way. None of them establishes that a given submission was run through that workflow.&lt;/p&gt;

&lt;p&gt;No sentence in this post should be read as saying the 24 v1.35 submissions went through test-artifact validation. The record read does not show that they did.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the Documentation Does Not Show
&lt;/h3&gt;

&lt;p&gt;Two of the 24 v1.35 folders contain test-derived files: one holds an &lt;code&gt;e2e.log&lt;/code&gt; and a &lt;code&gt;junit_01.xml&lt;/code&gt;, and one holds per-test &lt;code&gt;output.md&lt;/code&gt; files. The other 22 do not. The presence of those files does not establish that the script recognized or processed them, because its artifact handling applies to the artifact types it names and to evidence entries that point to them. Which script version ran against any v1.35 submission is not established either, since the workflow validates directories changed in a pull request and artifact validation did not exist before September 8, 2026. I did not pull pull-request dates or check-run history for the 24 folders. That is the one check that would move the fourth fact above, and it stays open.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Acceptance Decision: Where the Operator's Test Begins
&lt;/h2&gt;

&lt;p&gt;The operator's decision is narrow and concrete: can the operator accept a v1.35 listing as sufficient evidence for this workload, or must the operator obtain and evaluate additional evidence before recording acceptance? Kubernetes AI conformance answers neither half. The listing supplies inputs to the answer, and Table 4 turns those inputs into a record.&lt;/p&gt;

&lt;p&gt;Table 4 is a proposed operator method. It is this post's analysis and is not program guidance, and no part of the program prescribes it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 4: A proposed acceptance record for a v1.35 listing&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Row class&lt;/th&gt;
&lt;th&gt;What the program establishes&lt;/th&gt;
&lt;th&gt;What the operator still owes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;MUST&lt;/td&gt;
&lt;td&gt;A reported Implemented status, which the current script requires, subject to the conditional &lt;code&gt;cluster_autoscaling&lt;/code&gt; exception&lt;/td&gt;
&lt;td&gt;Treat it as an entry condition, not a discriminator. For the conditional rows (&lt;code&gt;cluster_autoscaling&lt;/code&gt;, &lt;code&gt;pod_autoscaling&lt;/code&gt;), check the platform's actual configuration for this workload. Where a MUST row carries no evidence reference, obtain one before relying on it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD Implemented&lt;/td&gt;
&lt;td&gt;A reported status and, where the submitter provided one, an evidence reference&lt;/td&gt;
&lt;td&gt;Read the reference against the workload's actual dependency. With no reference, treat the row as an unsupported statement until evidence is obtained&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD blank&lt;/td&gt;
&lt;td&gt;No statement. The current script skips a SHOULD row with empty status and evidence&lt;/td&gt;
&lt;td&gt;Obtain a written answer before recording acceptance for any workload that depends on the capability&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD N/A&lt;/td&gt;
&lt;td&gt;A claim that the condition does not apply, with a justification the script requires&lt;/td&gt;
&lt;td&gt;Read the justification and decide whether the condition really does not apply to this hardware and workload&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SHOULD Not Implemented&lt;/td&gt;
&lt;td&gt;An explicit negative&lt;/td&gt;
&lt;td&gt;Assess the effect on the workload, then accept the gap, compensate for it or decline&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Version binding&lt;/td&gt;
&lt;td&gt;A listing for one product and one Kubernetes version&lt;/td&gt;
&lt;td&gt;Record the platform version, the Kubernetes minor version and the submission the acceptance rests on. A v1.35 listing does not carry to another version&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Post-upgrade re-check&lt;/td&gt;
&lt;td&gt;Nothing. A listing is a point-in-time submission, and the workflow validates only directories changed in a pull request&lt;/td&gt;
&lt;td&gt;Re-run the acceptance record after an upgrade, against a new listing for the new version or against your own evidence&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftqartuntj03oczbyofx5.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftqartuntj03oczbyofx5.webp" alt="Boundary between what the Kubernetes AI conformance program establishes and what the operator still owes" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The rows divide work by who can do it. The program can state what a status is and what the script requires. Only the operator knows which SHOULD row the workload depends on, which Kubernetes version will actually run and what an upgrade changes. Placing that mapping inside the &lt;a href="https://www.rack2cloud.com/ai-architecture-learning-path/runtime-cluster-orchestration/" rel="noopener noreferrer"&gt;Runtime and Cluster Orchestration&lt;/a&gt; stage of the AI architecture path keeps the acceptance record attached to the layer that owns the scheduling and operator behavior, and the &lt;a href="https://www.rack2cloud.com/ai-architecture-learning-path/governance-runtime-control/" rel="noopener noreferrer"&gt;Governance and Runtime Control&lt;/a&gt; stage covers who holds the authority to accept a gap once the record shows one.&lt;/p&gt;

&lt;p&gt;Node-level qualification after a change, covered in &lt;a href="https://www.rack2cloud.com/gpu-node-requalification/" rel="noopener noreferrer"&gt;GPU node requalification&lt;/a&gt;, is a different validation. It asks whether evidence about a running node still holds after an in-place change, where a conformance listing is a platform-level statement for one version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;A Kubernetes AI conformance listing is a record of what a vendor reported against a checklist, filed in a repository with a validator in front of it. That is useful and it is real. It is also a different object from an acceptance record, and the gap between them is where the operator's work sits.&lt;/p&gt;

&lt;p&gt;Kubernetes AI conformance for v1.35 has a quiet problem: the uniformity of the MUST column hides it. When 216 of 216 rows report Implemented, the column stops discriminating, and the variation moves into the SHOULD rows and into the evidence behind each status. For v1.35 the available documentation does not show that anyone independently tested that evidence. The Kubernetes AI conformance README describes the earlier versions as documentation-backed self-assessment, and the tests that exist upstream do not reach back to submissions they have not been shown to have touched.&lt;/p&gt;

&lt;p&gt;The burden of proof for a v1.35 claim therefore stays with the operator, who is also the only party that knows what the workload depends on. A blank, an N/A and a Not Implemented are three different statements, an Implemented without evidence is a claim waiting for evidence, and an upgrade starts the record over.&lt;/p&gt;

&lt;p&gt;Kubernetes AI conformance tells you where to start asking. Acceptance is the record you write after the answers come back.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/ai-infrastructure-strategy-guide/" rel="noopener noreferrer"&gt;AI Infrastructure Architecture&lt;/a&gt; — the pillar this post's checklist reading sits inside&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/ai-architecture-learning-path/runtime-cluster-orchestration/" rel="noopener noreferrer"&gt;Runtime &amp;amp; Cluster Orchestration&lt;/a&gt; — the stage that owns scheduling, autoscaling and operator behavior&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/ai-architecture-learning-path/governance-runtime-control/" rel="noopener noreferrer"&gt;Governance &amp;amp; Runtime Control&lt;/a&gt; — who holds authority over an accepted gap&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/gpu-scheduling-kubernetes/" rel="noopener noreferrer"&gt;GPU Scheduling in Kubernetes: Start Before the Scheduler&lt;/a&gt; — what sits behind the gang scheduling row&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/gpu-node-requalification/" rel="noopener noreferrer"&gt;GPU Node Requalification: The Readiness Gap After In-Place Changes&lt;/a&gt; — node-level validation after a change, a different object from a platform listing&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://kubernetes.io/blog/2026/10/29/sig-arch-ai-conformance-2026/" rel="noopener noreferrer"&gt;From Working Group to SIG Architecture: spotlight on AI Conformance&lt;/a&gt; — the program's move to a SIG Architecture subproject&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/kubernetes-sigs/ai-conformance/tree/main/conformance-versions" rel="noopener noreferrer"&gt;kubernetes-sigs/ai-conformance, &lt;code&gt;conformance-versions&lt;/code&gt;&lt;/a&gt; — the requirement levels by version&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/cncf/k8s-ai-conformance" rel="noopener noreferrer"&gt;cncf/k8s-ai-conformance&lt;/a&gt; — submissions, the validator script and the README on how certification works&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/kubernetes-ai-conformance/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>kubernetes</category>
      <category>ai</category>
      <category>devops</category>
      <category>cloud</category>
    </item>
    <item>
      <title>Certified Device Firmware Is Admission Evidence. It Is Not Behavior Assurance.A</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Sun, 15 Nov 2026 13:06:45 +0000</pubDate>
      <link>https://dev.to/ntctech/certified-device-firmware-is-admission-evidence-it-is-not-behavior-assurancea-7nb</link>
      <guid>https://dev.to/ntctech/certified-device-firmware-is-admission-evidence-it-is-not-behavior-assurancea-7nb</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5iir06dep4xz8vsle5zj.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5iir06dep4xz8vsle5zj.jpg" alt="Field Notes — Engineering Notes from the Complexity Gap | Rack2Cloud" width="800" height="197"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Certified device firmware is evidence that a device was admitted, not evidence of how it will behave once it is running. On November 2, Google's Threat Intelligence Group reported that the XCPE backdoor, which it found built directly into many device models and brands, had enrolled more than 1.3 million set-top boxes into a botnet. Google assesses with high confidence that Videostrong Technology, an Android OEM and ODM, intentionally developed, deployed and operated the backdoor and botnet for financial gain, and says Videostrong used obfuscation to exploit a gap in Google's own tests and load malicious code after first boot. The incident is easy to file as one more bad supplier. The narrower question is what a buyer still holds when the manufacturer was known to the certifier and was itself the source of the risk.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F136iuriscgs3egxavwhw.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F136iuriscgs3egxavwhw.jpg" alt="Certified device firmware: one admission gate, then four lifecycle stages with no assigned owner" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Google Established About XCPE, and What It Did Not
&lt;/h2&gt;

&lt;p&gt;For &lt;a href="https://www.rack2cloud.com/modern-infrastructure-iac-strategy-guide/" rel="noopener noreferrer"&gt;modern infrastructure architecture&lt;/a&gt;, the usable question is narrow, so the evidence boundary comes first. The middle column is Google's report, attributed to Google. The right column is what the report does not say.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Topic&lt;/th&gt;
&lt;th&gt;Google reports&lt;/th&gt;
&lt;th&gt;Not stated&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Attribution&lt;/td&gt;
&lt;td&gt;A high-confidence assessment that Videostrong intentionally developed, deployed and operated the backdoor and botnet for financial gain. Google also says it believes the campaign was not an isolated rogue insider or a development oversight.&lt;/td&gt;
&lt;td&gt;An adjudicated finding. Videostrong's own account does not appear in the report.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Persistence&lt;/td&gt;
&lt;td&gt;A stub preloaded in the firmware partition and running at system privilege. If a user uninstalls the backdoor, it reinstalls from the /oem/ partition on reboot.&lt;/td&gt;
&lt;td&gt;How the stub or the /oem/ package entered the firmware build.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;First boot&lt;/td&gt;
&lt;td&gt;Obfuscation exploited a gap in Google's tests, and malicious code loaded after the device's first boot.&lt;/td&gt;
&lt;td&gt;Which image Google tested, which tests, the nature of the gap, or any dormant period.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Google's action&lt;/td&gt;
&lt;td&gt;Videostrong was banned from making devices with preloaded Google apps, and from building and selling certified Android devices or supplying software to other certified vendors. Google also took legal action against control domains and servers, changed its review process, and is improving firmware integrity assurance on certified Android TV devices.&lt;/td&gt;
&lt;td&gt;Revocation of certification on shipped devices, blocking them, notifying owners, or what the review changes are.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Population&lt;/td&gt;
&lt;td&gt;More than 1.3 million set-top boxes, plus other Android-based devices Google observed: telehealth, karaoke machines and digital signage.&lt;/td&gt;
&lt;td&gt;Counts, brands or models for the non-set-top devices, enterprise deployment, or which affected devices were certified.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Keep the populations apart. The 1.3 million figure counts set-top boxes enrolled in the XCPE botnet. The at-least-10-million figure is Google's assessment of additional devices behind the affiliates' own command domains, which also distributed through other channels. They are different populations and should not be added.&lt;/p&gt;

&lt;p&gt;Plume Security Labs, a partner in the takedown, published its own whitepaper. It states that the fleet mixes AOSP and Google-certified devices, in one sentence, with no method for determining certification status and no counts. Google's report does not say which affected devices were certified. This note treats that question as open and writes about the class of certified OEM firmware.&lt;/p&gt;

&lt;p&gt;Scope: OEM-controlled firmware behavior on Android devices a buyer deploys, such as signage, telehealth and set-top. Google does not say these were enterprise deployments; that is this note's buyer scenario. No claim is made about server firmware, BMC or UEFI.&lt;/p&gt;

&lt;h2&gt;
  
  
  Certified Device Firmware Is an Admission Test
&lt;/h2&gt;

&lt;p&gt;Google describes certification as a comprehensive suite of compatibility and security tests that OEMs must go through. That establishes an admission process, but the report does not identify which firmware image was examined, which tests were applied to it, or how the tested image relates to the firmware that shipped. Certified device firmware therefore provides evidence of admission, not proof that the deployed image or its later behavior was independently validated. It does not answer what the firmware does after activation, or who can change what it does, and Google's report does not describe certification as covering either.&lt;/p&gt;

&lt;p&gt;Google's own response shows where the line sits. It changed its review process and says it is improving how it ensures firmware integrity on certified Android TV devices. Google has not described those changes, and this note does not assume what they contain.&lt;/p&gt;

&lt;p&gt;That is not an argument that certification is useless. Admission testing is a necessary filter. The claim is narrower: a passed admission test is evidence of one moment, and a buyer who treats certified device firmware as a standing guarantee holds no record of the moments after it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmzktoufv7l2w8uomy55g.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmzktoufv7l2w8uomy55g.jpg" alt="Google reports code loaded after first boot; what image it tested is not stated" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The Reviewed Manufacturer Can Be the Source
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.rack2cloud.com/vendor-review-process-supplier-visibility/" rel="noopener noreferrer"&gt;The vendor review process post&lt;/a&gt; covers a supplier that never enters review. Code of foreign origin arrived inside applications that procurement and security had approved, and because nobody had classified that code as a supplier relationship, no review gate ever looked at it. Its test asks whether a component would be reviewed if it were its own company.&lt;/p&gt;

&lt;p&gt;XCPE is the other case. Google's report describes Videostrong as an Android OEM and ODM and says Google cut ties with it, so the manufacturer was known to the certification ecosystem. The certification status of individual affected devices remains unresolved, as the Plume paragraph above sets out. The vendor-review question, whether anyone would have reviewed this party as its own company, does not apply the same way: the manufacturer was a named party with a relationship to the certifier. The exposure remains, because the risk did not enter through a gap in who was known. It came from the manufacturer itself, acting after admission. That is the difference this note rests on: the earlier failure is an invisible supplier, and this one is a visible manufacturer that is the source. Being known did not establish ongoing assurance over the firmware it supplied. Certified device firmware, where a device carries it, raises that manufacturer's standing with the buyer without changing who controls the behavior.&lt;/p&gt;

&lt;p&gt;Visibility has a limit here. Google says the malware appeared across many device models and brands, so a buyer holding a white-label brand may not know which OEM is behind it. That buyer is back in the vendor-review case, which is why the Purchase row requires the actual manufacturer's identity. The argument applies once the manufacturer is known.&lt;/p&gt;

&lt;p&gt;Google tells retailers and distributors to require independent security audits from ODMs, and delist manufacturers implicated in supply-chain abuse. An audit is still evidence gathered at a moment, and delisting comes after the fact. What a buyer holds between those moments is the lifecycle question, and neither the review nor the certified device firmware answers it by itself.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Firmware Lifecycle Needs Evidence and an Owner
&lt;/h2&gt;

&lt;p&gt;The table below is this note's proposal for certified device firmware, not a description of controls Google reports. It follows one device from purchase to the aftermath of a takedown and asks, at each stage, what certification or review establishes, what evidence the buyer retains, and who owns the obligation. Any acceptance test, egress baseline or independent validation named here is this note's recommendation, not a control reported in the XCPE case, and none assumes a delivery path for XCPE's code after first boot, which Google's report does not establish.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Lifecycle stage&lt;/th&gt;
&lt;th&gt;What certification or review establishes&lt;/th&gt;
&lt;th&gt;Evidence the buyer should retain&lt;/th&gt;
&lt;th&gt;Accountable owner&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Purchase&lt;/td&gt;
&lt;td&gt;At most, that the OEM and device model went through a defined program at a point in time.&lt;/td&gt;
&lt;td&gt;The actual manufacturer's identity, not only the brand; device and model; the certification evidence with its date and scope; the firmware build identifier and baseline.&lt;/td&gt;
&lt;td&gt;Procurement and security&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;First boot&lt;/td&gt;
&lt;td&gt;Nothing about behavior after activation beyond what the assessment exercised. Google says XCPE's code loaded after first boot.&lt;/td&gt;
&lt;td&gt;Network side: destinations, connection attempts and ports observed at activation, captured independently of the device where the platform permits. Device or management side: firmware identity, installed versions and package inventory. Both reconciled against the approved baseline retained from purchase; exceptions recorded.&lt;/td&gt;
&lt;td&gt;Device or platform owner&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ongoing operation&lt;/td&gt;
&lt;td&gt;Nothing about later change, or about who controls it.&lt;/td&gt;
&lt;td&gt;Firmware and package versions over time, change and update records, who holds update authority, and the buyer's own validation results.&lt;/td&gt;
&lt;td&gt;Named device owner&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;After takedown&lt;/td&gt;
&lt;td&gt;Nothing about devices already installed. Google's actions concern control infrastructure, Videostrong's future eligibility for certification, and firmware integrity assurance on certified Android TV devices.&lt;/td&gt;
&lt;td&gt;The affected-device inventory, remediation status per device, and a recorded residual-risk decision to keep, isolate or replace.&lt;/td&gt;
&lt;td&gt;Security and service owner&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Google says XCPE's code loaded after first boot, and the report gives a buyer nothing about what image was tested. A buyer who retained the firmware identity and installed versions from device or management records, observed network destinations independently of the device, and reconciled both against the purchase baseline and a later observation, would hold evidence that certified device firmware alone never produced. That is a proposed requirement. It only works if someone owns the comparison, and if the device is not the only source of the evidence.&lt;/p&gt;

&lt;p&gt;The after-takedown row exists because disrupting control infrastructure is not remediation. Google's report describes action against that infrastructure, which matters to affected devices, against Videostrong's future certification, and on firmware integrity for certified Android TV devices. It does not say what happens to devices already installed, and its account of persistence is that the backdoor reinstalls from the firmware's /oem/ partition after a user uninstalls it. Until the buyer has an inventory of affected devices and a recorded decision for each, the takedown has disrupted the control infrastructure but has not established remediation of the buyer's installed estate.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Diagnostic:&lt;/strong&gt; &lt;em&gt;"If the manufacturer of a certified device changed what its firmware does after acceptance, which record would show it, and who reads that record?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;Certified device firmware is admission evidence. Certification is meant to show that what was presented for admission met a defined assessment. In the XCPE case Google places a gap in its tests and says code loaded after first boot, and the certification status of individual affected devices remains unresolved. Treating certification as behavior assurance asks an admission result to cover a lifecycle it never observed.&lt;/p&gt;

&lt;p&gt;The vendor-review problem is a supplier nobody saw. This is a manufacturer the certifier knew. XCPE does not show that review and certification are worthless. It shows that a manufacturer known to the certifier can still control what its firmware does, and that nothing in the arrangement assigned anyone downstream to notice a change.&lt;/p&gt;

&lt;p&gt;The buyer's obligation for certified device firmware is to retain evidence at four stages and to name an owner at each: purchase, first boot, ongoing operation and after takedown. That is a proposal, and it is the part of the arrangement that certification cannot supply and that Google's report does not claim to.&lt;/p&gt;

&lt;p&gt;Certification admits the device. Someone still has to own what it does.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/certified-device-firmware/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>android</category>
      <category>iot</category>
      <category>supplychain</category>
    </item>
    <item>
      <title>PyTorch 2.15 Rollback Has Two Boundaries. Only One Is a Setting.</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Sat, 14 Nov 2026 13:45:05 +0000</pubDate>
      <link>https://dev.to/ntctech/pytorch-215-rollback-has-two-boundaries-only-one-is-a-setting-p03</link>
      <guid>https://dev.to/ntctech/pytorch-215-rollback-has-two-boundaries-only-one-is-a-setting-p03</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp84tku0o166ixklrpthm.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp84tku0o166ixklrpthm.jpg" alt="Field Notes — Engineering Notes from the Complexity Gap | Rack2Cloud" width="800" height="197"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;PyTorch 2.15 gives an operator two ways back to 2.14, and only one of them is a setting. The new NCCL backend can be switched to the legacy implementation with a backend string or an environment variable. The checkpoint format cannot be switched at all: according to the release notes, a checkpoint written by 2.15 cannot be loaded by 2.14 or earlier, and there is no flag to write the old format. The first is a temporary configuration fallback. The second is a state-compatibility boundary. They are independent, they expire on different terms, and a rollback plan that counts them as one will be wrong about the second.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcbc1nacjomlhhj4h28jo.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcbc1nacjomlhhj4h28jo.jpg" alt="PyTorch 2.15 rollback boundaries: the NCCL backend rail ends in a this-release-only exit, the checkpoint rail ends at the 2.14 loader with no old-format switch" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The New NCCL Default Has a One-Release Exit
&lt;/h2&gt;

&lt;p&gt;In PyTorch 2.15, &lt;code&gt;backend="nccl"&lt;/code&gt; is also the default CUDA backend of &lt;code&gt;init_process_group()&lt;/code&gt;, and it now creates the rewritten NCCL implementation. The legacy implementation stays available as &lt;code&gt;backend="nccl-legacy"&lt;/code&gt; for this release only. The &lt;a href="https://github.com/pytorch/pytorch/releases/tag/v2.15.0" rel="noopener noreferrer"&gt;release notes&lt;/a&gt; say &lt;code&gt;nccl-legacy&lt;/code&gt; and &lt;code&gt;TORCH_DIST_USE_NCCL2=0&lt;/code&gt; will be removed in the next release. To restore legacy behavior in the meantime, set &lt;code&gt;TORCH_DIST_USE_NCCL2=0&lt;/code&gt; on every rank before creating the process group, or use &lt;code&gt;nccl-legacy&lt;/code&gt;. For &lt;a href="https://www.rack2cloud.com/ai-infrastructure-strategy-guide/" rel="noopener noreferrer"&gt;AI infrastructure architecture&lt;/a&gt;, that makes the backend side of this upgrade a fallback with a date on it, not a standing option.&lt;/p&gt;

&lt;p&gt;The new backend also behaves differently at startup. The notes say communicators are created eagerly in &lt;code&gt;init_process_group()&lt;/code&gt; and &lt;code&gt;new_group()&lt;/code&gt;, allocating GPU memory even for unused groups, and they put that at about 200 MiB in &lt;code&gt;new_group&lt;/code&gt;. &lt;code&gt;ProcessGroupNCCL.Options.lazy_init&lt;/code&gt; (PR 200020) lets &lt;code&gt;nccl&lt;/code&gt; and &lt;code&gt;nccl-lazy&lt;/code&gt; groups create their communicator on first use instead. The 200 MiB figure is the release notes' number for that call, not a general memory requirement for the backend. &lt;code&gt;nccl-lazy&lt;/code&gt; is a separate choice: collectives still run on one eagerly created communicator, while each send/recv peer pair gets its own lazily created communicator.&lt;/p&gt;

&lt;p&gt;What this section establishes is narrow. The backend side of the upgrade can be reversed by configuration, on every rank, until the release that removes the fallback. Restoring the legacy backend does not restore the old checkpoint format. The notes state no flag writes it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Checkpoint Writer Has No Old-Format Switch
&lt;/h2&gt;

&lt;p&gt;Distributed checkpoint metadata in PyTorch 2.15 now records tensor strides (PR 194251). &lt;code&gt;TensorProperties&lt;/code&gt; gains an optional &lt;code&gt;strides&lt;/code&gt; field in the pickled &lt;code&gt;.metadata&lt;/code&gt; file. On the read side, 2.15 loads checkpoints written by 2.14 and by 2.15. The reverse fails: in older versions &lt;code&gt;dcp.load&lt;/code&gt; raises a &lt;code&gt;CheckpointException&lt;/code&gt; reporting too many values to unpack (expected 5). The notes state that no flag writes the old format.&lt;/p&gt;

&lt;p&gt;This is a different kind of boundary. There is no legacy mode to select, no environment variable, and no release after which it goes away, because nothing is being removed. It is a property of what the 2.15 writer puts on disk, and it applies to every checkpoint 2.15 writes. The release notes do not state a conversion path from a 2.15 checkpoint to one a 2.14 loader can read.&lt;/p&gt;

&lt;h2&gt;
  
  
  PyTorch 2.15 Has Two Rollback Boundaries
&lt;/h2&gt;

&lt;p&gt;Set side by side, the two changes differ on every row that matters for a rollback. The table below is derived from the release notes. The last row is a reading, not a PyTorch statement.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;NCCL backend&lt;/th&gt;
&lt;th&gt;Checkpoint format&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Boundary type&lt;/td&gt;
&lt;td&gt;Temporary configuration fallback&lt;/td&gt;
&lt;td&gt;State-compatibility boundary&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;What changed in 2.15&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;nccl&lt;/code&gt; creates the rewritten backend by default&lt;/td&gt;
&lt;td&gt;DCP metadata records tensor strides (PR 194251)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mechanism to reverse&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;backend="nccl-legacy"&lt;/code&gt; or &lt;code&gt;TORCH_DIST_USE_NCCL2=0&lt;/code&gt; on every rank&lt;/td&gt;
&lt;td&gt;None stated; no flag writes the old format&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Availability&lt;/td&gt;
&lt;td&gt;This release only; removal in the next release&lt;/td&gt;
&lt;td&gt;Every checkpoint 2.15 writes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;What stays out of reach&lt;/td&gt;
&lt;td&gt;Nothing stored is affected&lt;/td&gt;
&lt;td&gt;2.14 and earlier cannot load 2.15-written checkpoints&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Operational consequence (derived)&lt;/td&gt;
&lt;td&gt;The fallback has to be used or retired before the next upgrade&lt;/td&gt;
&lt;td&gt;A 2.14 rollback can resume only from a checkpoint a 2.14 loader can read&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;One boundary is crossed back by editing launch configuration. The other is fixed by what is already on disk when you decide to go back.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a Rollback Actually Preserves
&lt;/h2&gt;

&lt;p&gt;This section is a derived reading of the release notes, not something PyTorch promises. Moving back from PyTorch 2.15 by installing 2.14 again is a software change the notes do not forbid. The constraint they state is on what a 2.14 loader can read. Checkpoints written by 2.14 before the upgrade remain loadable by 2.15, per the notes, and a 2.14 loader can read what 2.14 wrote (derived). Checkpoints written by the 2.15 distributed checkpoint writer cannot be loaded by 2.14 or earlier, according to the release notes. The notes state no conversion path.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy103jdahypumzwuvowbk.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy103jdahypumzwuvowbk.jpg" alt="Checkpoint timeline across the upgrade: checkpoints written by 2.14 are readable by both loaders, checkpoints written by 2.15 cannot be loaded by 2.14 or earlier, and a rollback to 2.14 resumes from the retained pre-upgrade checkpoint" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So a rollback to 2.14 can only resume from a checkpoint written before the writer changed. Progress made since that checkpoint is lost only if no other compatible checkpoint or recovery path exists. That is an operational consequence, not a guarantee of data loss, and the later checkpoints are not unrecoverable: 2.15 still reads them. In the terms of &lt;a href="https://www.rack2cloud.com/ai-workload-rpo/" rel="noopener noreferrer"&gt;AI workload RPO&lt;/a&gt;, the checkpoint format and the loader that reads it are a version-bound pair, and 2.15 changes one half of that pair without a matching switch.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Diagnostic:&lt;/strong&gt; &lt;em&gt;"Which is your newest checkpoint a 2.14 loader can read, and when was it last test-loaded?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Download the nine-slide carousel (PDF): &lt;a href="https://rack2cloud.com/downloads/carousels/pytorch-2-15-rollback-boundary-carousel-v1.pdf" rel="noopener noreferrer"&gt;PyTorch 2.15 Rollback Boundaries Carousel&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The Upgrade Sequencing Decision
&lt;/h2&gt;

&lt;p&gt;This is architect guidance, not a PyTorch feature. The question is which checkpoint you must keep to preserve a viable rollback path when the backend and the checkpoint format change independently.&lt;/p&gt;

&lt;p&gt;Retain a pre-upgrade checkpoint and test-load it with the 2.14 loader before cutover. A retained checkpoint that has never been loaded is an assumption. Keep that recovery point protected until the 2.15 workload has passed its recovery acceptance criteria and the rollback window has formally closed. A successful application start proves neither checkpoint compatibility nor workload recovery. Retention is also a capacity decision: one more checkpoint generation sits on storage that the cluster's &lt;a href="https://www.rack2cloud.com/gpu-cluster-architecture-private-llm/" rel="noopener noreferrer"&gt;checkpoint sizing&lt;/a&gt; may not have counted.&lt;/p&gt;

&lt;p&gt;Set rollback criteria before the workload switches to the 2.15 checkpoint writer. Decide which signals send the job back to 2.14, and until when, and schedule the switch instead of letting the first autosave decide it. The compatibility boundary begins at the switch, not when the first checkpoint is saved. The first 2.15 checkpoint write makes the operational consequence concrete: a subsequent rollback to 2.14 cannot resume from that checkpoint. It requires a separately retained checkpoint that a 2.14 loader can read. Subsequent 2.15 checkpoints cannot replace that recovery point. This is a derived operational consequence, not a PyTorch statement.&lt;/p&gt;

&lt;p&gt;Decide when writers cut over, and treat the backend fallback as a short window. Because &lt;code&gt;nccl-legacy&lt;/code&gt; and &lt;code&gt;TORCH_DIST_USE_NCCL2=0&lt;/code&gt; are scheduled for removal in the next release, a plan that depends on them beyond this release is a plan for a configuration that will not exist.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the Release Notes Do Not Establish
&lt;/h2&gt;

&lt;p&gt;Vendor documentation establishes the PyTorch 2.15 changes above. No independent workload-recovery evidence was found, and reconfiguration is documented capability here, not demonstrated recovery. The release notes provide compatibility evidence for the checkpoint writer and reader boundary. They do not provide recovery evidence for the retention sequence proposed here: no actual rollback to 2.14 followed by workload resume has been tested. The release blog's summary lists the NCCL rollback paths; the blog text read for this note does not mention checkpoints, so the checkpoint boundary rests on the release notes alone.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Item&lt;/th&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Whether the optional &lt;code&gt;strides&lt;/code&gt; field is populated in every checkpoint&lt;/td&gt;
&lt;td&gt;NOT STATED&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Whether 2.15 reads checkpoints older than 2.14&lt;/td&gt;
&lt;td&gt;NOT STATED&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Behavior of DCP entry points other than &lt;code&gt;dcp.load&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;NOT STATED&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A conversion path from a 2.15 checkpoint to a 2.14-loadable one&lt;/td&gt;
&lt;td&gt;NOT STATED&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Eager-communicator memory cost beyond the about 200 MiB &lt;code&gt;new_group&lt;/code&gt; figure&lt;/td&gt;
&lt;td&gt;NOT STATED&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rank-loss recovery or GPU fault tolerance after reconfiguration&lt;/td&gt;
&lt;td&gt;Not claimed&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The &lt;a href="https://pytorch.org/blog/pytorch-2-15-release-blog/" rel="noopener noreferrer"&gt;release blog&lt;/a&gt; and the release notes are the only sources used. Everything labeled derived is this note's reading of them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;PyTorch 2.15 offers one boundary you can configure and one you cannot. The NCCL backend has an exit, and it has a date on it. The checkpoint format has a reader with a version on it, and no switch.&lt;/p&gt;

&lt;p&gt;The real problem is that both changes arrive in the same release and get filed under one word, rollback. The backend fallback is reversible by launch configuration until the next release. The checkpoint boundary is decided by which checkpoints exist when you need to go back, and nothing in the notes lets you change that afterward.&lt;/p&gt;

&lt;p&gt;Lost progress is not what 2.15 does to you. It is what follows if the only compatible checkpoint is older than the work you cared about, or was never test-loaded. Retain it, load it, and know its age before the writer changes.&lt;/p&gt;

&lt;p&gt;Within this release, a setting can be changed on the day you need it. A checkpoint has to exist before that day.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/pytorch-2-15-rollback-boundary/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>pytorch</category>
      <category>infrastructure</category>
    </item>
    <item>
      <title>The Alpine sshd Bypass Is Not a Version Check. It Is Four Conditions and a Restart.</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Wed, 11 Nov 2026 13:21:58 +0000</pubDate>
      <link>https://dev.to/ntctech/the-alpine-sshd-bypass-is-not-a-version-check-it-is-four-conditions-and-a-restart-4mbg</link>
      <guid>https://dev.to/ntctech/the-alpine-sshd-bypass-is-not-a-version-check-it-is-four-conditions-and-a-restart-4mbg</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy8mhysm79yx2czbmx6sz.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy8mhysm79yx2czbmx6sz.jpg" alt="Alpine sshd bypass exposure diagram: four conditions that must all hold on one host, with a version number shown as not deciding exposure" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The Alpine sshd bypass is not a version problem. On November 5, Alpine Linux published an &lt;a href="https://alpinelinux.org/posts/2026-11-05-openssh-password-authentication-bypass/" rel="noopener noreferrer"&gt;advisory for a password authentication bypass in its openssh-server package&lt;/a&gt;, and the OpenSSH project shipped 10.6p2 the same day with what its &lt;a href="https://www.openssh.org/releasenotes.html" rel="noopener noreferrer"&gt;release notes&lt;/a&gt; call a "critical security workaround" for Linux systems that use musl libc with PAM disabled. Exposure depends on four conditions holding at once, and one of them is the state of the accounts on the machine, not a setting anyone chose. The fix has the same shape: a package on disk, a daemon in memory and an image in a registry are different states.&lt;/p&gt;

&lt;p&gt;This post is scoped to what Alpine's advisory, the OpenSSH 10.6p2 release notes and the CVE record establish for Alpine hosts. The flaw is tracked as &lt;a href="https://www.cve.org/CVERecord?id=CVE-2026-123158" rel="noopener noreferrer"&gt;CVE-2026-123158&lt;/a&gt;, and &lt;a href="https://www.cisa.gov/news-events/alerts/2026/11/09/cisa-adds-four-known-exploited-vulnerabilities-catalog" rel="noopener noreferrer"&gt;CISA added it to its Known Exploited Vulnerabilities catalog&lt;/a&gt; on November 9. It does not claim the bypass reaches OpenSSH on other distributions, and it does not assess whether any particular host has been accessed.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Alpine sshd bypass needs four conditions at once
&lt;/h2&gt;

&lt;p&gt;The exposure lives in running state, which no template declares. Alpine lists four conditions. All four must be true on the same machine.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;#&lt;/th&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;Where it comes from&lt;/th&gt;
&lt;th&gt;What removes it&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;PAM is not in the password path&lt;/td&gt;
&lt;td&gt;The &lt;code&gt;openssh-server&lt;/code&gt; package is built without PAM; &lt;code&gt;openssh-server-pam&lt;/code&gt; with &lt;code&gt;UsePAM no&lt;/code&gt; is the same state&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;openssh-server-pam&lt;/code&gt; with &lt;code&gt;UsePAM yes&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;Password authentication is on&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;PasswordAuthentication yes&lt;/code&gt;, the default&lt;/td&gt;
&lt;td&gt;&lt;code&gt;PasswordAuthentication no&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;An account's password field is exactly &lt;code&gt;*&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;root&lt;/code&gt; (shipped as &lt;code&gt;root:*&lt;/code&gt; since about 3.16), key-only users, musl-nscd directory users, the &lt;code&gt;alpine&lt;/code&gt; user in official tiny-cloud images&lt;/td&gt;
&lt;td&gt;No such account, or accounts locked with &lt;code&gt;!&lt;/code&gt;, which Alpine says are not affected&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;At least one local account has a &lt;code&gt;$5$&lt;/code&gt; or &lt;code&gt;$6$&lt;/code&gt; hash&lt;/td&gt;
&lt;td&gt;Any account with a real password&lt;/td&gt;
&lt;td&gt;No account on the machine with a hash&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Root is protected by the default &lt;code&gt;PermitRootLogin prohibit-password&lt;/code&gt;, but that does not help when another account that can log in can become root (Alpine illustrates this with &lt;code&gt;doas&lt;/code&gt;). Containers that do not run sshd are outside the exposure. Alpine reports that every release it checked contains the affected code path: 3.21 through 3.24 and edge, plus the older 3.16 to 3.20 releases that no longer receive regular updates. That describes the packages, not every installation. The four conditions above must also hold on a given host.&lt;/p&gt;

&lt;p&gt;Condition 3 is the key-only account, which is good practice. Condition 4 is a console password set once, or a user added through user-data. Neither is a mistake. The exposure appears when the second is added to the first, on a host whose configuration has not changed.&lt;/p&gt;

&lt;p&gt;A read-only screen for conditions 2 to 4. The first two commands are this post's method, not Alpine's instruction; the third is from the advisory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;awk -F: &lt;span class="s1"&gt;'$2=="*" {print $1}'&lt;/span&gt; /etc/shadow
awk -F: &lt;span class="s1"&gt;'$2 ~ /^\$(5|6)\$/ {print $1}'&lt;/span&gt; /etc/shadow
sshd -T &lt;span class="o"&gt;|&lt;/span&gt; grep -i passwordauthentication
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These are screening commands, not an exposure determination, and they have been run only against a mock shadow file, not validated on an Alpine host. The first cannot see musl-nscd accounts, so an empty result does not establish that condition 3 is false. An empty second result does mean condition 4 fails locally. The third does not evaluate &lt;code&gt;Match&lt;/code&gt; blocks and reports configuration as sshd would load it now, not what a running daemon loaded. Condition 1 comes from the installed package and the &lt;code&gt;UsePAM&lt;/code&gt; setting.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj1bsnn2ncsweptfmq6fp.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj1bsnn2ncsweptfmq6fp.jpg" alt="Alpine sshd bypass mechanism: the stored password field for a password-less account and the error value returned by crypt() are the same string, so the comparison matches" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How an account with no password becomes a login
&lt;/h2&gt;

&lt;p&gt;Without PAM, sshd hashes the attempt with &lt;code&gt;crypt()&lt;/code&gt; and compares the result with the stored field. For a stored field too short to contain a salt, sshd borrows another account's salt, which is why condition 4 exists. musl's &lt;code&gt;crypt()&lt;/code&gt; returns &lt;code&gt;*&lt;/code&gt; as an error indicator for a key longer than 256 bytes, and &lt;code&gt;*&lt;/code&gt; is also exactly what the stored field holds for an account with no password. The comparison sees two equal strings and the login succeeds.&lt;/p&gt;

&lt;p&gt;Two unrelated meanings of &lt;code&gt;*&lt;/code&gt; collide: "no password is set" in the shadow file, and "this call failed" in a return value. One account with a real password is enough, in Alpine's words, to make every account without one reachable. A field of &lt;code&gt;!&lt;/code&gt; is not equal to the error marker, so the comparison fails; Alpine states the unaffected result directly.&lt;/p&gt;

&lt;p&gt;For a key-only account, the consequence is a login without a key. Where the account can reach root through &lt;code&gt;doas&lt;/code&gt;, as the &lt;code&gt;alpine&lt;/code&gt; user in the tiny-cloud images can, that is a root shell. Alpine offers a second route for hosts that cannot update yet: set &lt;code&gt;PasswordAuthentication no&lt;/code&gt; in the effective configuration and restart sshd. Alpine says either route closes the exposure, and both end with a restart. An edit without a restart leaves the running daemon unchanged, and a restart without the edit leaves the exposure in place.&lt;/p&gt;

&lt;h2&gt;
  
  
  Package, daemon and image are not one fix
&lt;/h2&gt;

&lt;p&gt;The first half of the fix is &lt;code&gt;apk upgrade -U -a&lt;/code&gt;. The second is &lt;code&gt;rc-service sshd restart&lt;/code&gt;, because upgrading the package does not replace the running daemon. A host that took the update and never restarted sshd runs the old code with the new package on disk.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8ajbfw64jeztvzncjcqp.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8ajbfw64jeztvzncjcqp.jpg" alt="Alpine sshd bypass fix paths: running host needs package upgrade and daemon restart, deployment path needs a rebuilt image, and a rebuilt image does not reach running instances" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Path&lt;/th&gt;
&lt;th&gt;What closes the exposure&lt;/th&gt;
&lt;th&gt;What does not&lt;/th&gt;
&lt;th&gt;How you know&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Running host&lt;/td&gt;
&lt;td&gt;Fixed package installed and sshd restarted&lt;/td&gt;
&lt;td&gt;The package upgrade alone&lt;/td&gt;
&lt;td&gt;The listener started after the package was upgraded (this post's method)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Running host, interim&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;PasswordAuthentication no&lt;/code&gt; on every such line, &lt;code&gt;Match&lt;/code&gt; blocks included, and sshd restarted, after confirming key login works&lt;/td&gt;
&lt;td&gt;Editing the file without a restart&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;sshd -T&lt;/code&gt; run after the restart, as Alpine's steps end&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deployment path&lt;/td&gt;
&lt;td&gt;Images rebuilt from 3.21.10, 3.22.8, 3.23.8 and 3.24.4 or later&lt;/td&gt;
&lt;td&gt;A rebuilt image says nothing about instances already running&lt;/td&gt;
&lt;td&gt;Check each running instance&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On the host, compare the sshd process start time with the time the package was upgraded; the listener has to be the younger of the two. &lt;code&gt;sshd -T&lt;/code&gt; is evidence about the file, not about what a running daemon loaded.&lt;/p&gt;

&lt;p&gt;On the deployment path, images from earlier point releases do not include the fixed package, and for every running instance the answer is to update openssh or disable password authentication. Alpine has published fixed packages back to 3.16, including 3.21, which reached the end of its support period on November 1. Stable branches carry the fix as patches to their existing OpenSSH versions, so the fixed package version differs from upstream 10.6p2: edge ships 10.6_p2-r0 and 3.24 ships 10.3_p1-r2, per the advisory's fixed-version table. Check a host against its own branch's version. For the interim route, use Alpine's published commands as written, confirm key login works first, and keep the current session open until a new login has been tested.&lt;/p&gt;

&lt;p&gt;The same exists-versus-running separation shows up where a provider lifecycle setting makes the final move, in &lt;a href="https://www.rack2cloud.com/managed-compute-patching/" rel="noopener noreferrer"&gt;Managed Compute Patching Is Not Managed Remediation&lt;/a&gt;. Here the final action is a command on a host you operate, and the package manager does not schedule it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tiny-cloud images are safe at first boot and exposed after the first password
&lt;/h2&gt;

&lt;p&gt;The official tiny-cloud images (qcow2, VHD, AMI) from 3.18.11 through 3.24.3 ship an &lt;code&gt;alpine&lt;/code&gt; user with a &lt;code&gt;*&lt;/code&gt; field and passwordless &lt;code&gt;doas&lt;/code&gt;. Alpine describes them as safe on first boot. They become exploitable once a &lt;code&gt;$6$&lt;/code&gt; hash exists, for example when an administrator sets a console password or user-data adds a user with &lt;code&gt;hashed_passwd&lt;/code&gt;, provided the other conditions hold. The image is not vulnerable as shipped, and the state change that makes it vulnerable is an ordinary provisioning action. Fixed images come from 3.21.10, 3.22.8, 3.23.8 and 3.24.4 onward. An instance launched from an earlier image keeps its sshd until someone updates it or turns password authentication off.&lt;/p&gt;

&lt;h2&gt;
  
  
  When a login cannot be ruled out, the verdict stops at exposure
&lt;/h2&gt;

&lt;p&gt;Alpine's guidance: look in &lt;code&gt;/var/log/messages&lt;/code&gt; for sshd lines reading &lt;code&gt;Accepted password&lt;/code&gt; for an account that has no password, review &lt;code&gt;doas&lt;/code&gt; use and recently created users, files and authorized keys, and treat the machine as compromised if you cannot rule out a login.&lt;/p&gt;

&lt;p&gt;This post stops at the exposure verdict: not exposed because a condition fails, exposed and fixed because both fix states are verified, or exposed with a login history you can or cannot read. The architectural question is narrower: does the evidence to rule a login out still exist? A rotated or unshipped log cannot show an absence. When the records were never kept, the decision narrows to what you can rebuild and attest, which &lt;a href="https://www.rack2cloud.com/recovery-decision-integrity-evidence/" rel="noopener noreferrer"&gt;When You Cannot Prove Integrity, Rebuilding Becomes the Recovery Decision&lt;/a&gt; works through on a different Alpine matter. A shell as a key-only account with &lt;code&gt;doas&lt;/code&gt; also leaves things behind, and &lt;a href="https://www.rack2cloud.com/persistent-access-survives-fix/" rel="noopener noreferrer"&gt;The Server Was Fixed. Persistent Access Wasn't.&lt;/a&gt; covers why a rebuild does not remove access written somewhere it never reached.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the sources leave open
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Who owns the root cause is disputed.&lt;/strong&gt; The OpenSSH release notes say the root cause is outside sshd while describing the change as one that avoids it in sshd. Rich Felker, writing to the &lt;a href="https://www.openwall.com/lists/musl/2026/11/05/4" rel="noopener noreferrer"&gt;musl mailing list&lt;/a&gt; on November 5, states his position that it is not a bug in musl but erroneous use by the caller, and that the mechanism is likely to change in the upcoming release cycle. The CVE record lists OpenSSH as the affected product, which names where the fix shipped and does not settle that dispute. This post takes no side. The exposure test and the fix paths are identical either way.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scope beyond Alpine is not assessed.&lt;/strong&gt; The release notes scope the workaround to Linux systems using musl without PAM. How that maps onto other musl-based distributions is outside what this post establishes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;An upstream affected-version lower bound is not established.&lt;/strong&gt; Alpine reports the releases it checked; neither the release notes nor the CVE record, which says only "before 10.6p2", state where the exposure begins.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Exploitation is reported, not described.&lt;/strong&gt; CISA lists this vulnerability in its catalog, which it populates on evidence of active exploitation. The listing does not say which hosts were affected, how many, or how the attacks were carried out, and this post makes no claim about whether any particular host has been accessed.&lt;/p&gt;

&lt;p&gt;Alpine's advisory reflects the position on November 5. Re-read it before acting on a specific version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;The Alpine sshd bypass is a state, not a version. Four conditions have to hold together, and the one that does the work is an account with no password, which is exactly what key-only hygiene produces. Alpine says accounts locked with &lt;code&gt;!&lt;/code&gt; are not affected, but a locked account cannot log in by key either, which is my analysis, not Alpine's. The fix is a state as well: a package on disk, a restarted daemon and a rebuilt image are three facts, and closing one does not close the others.&lt;/p&gt;

&lt;p&gt;Version numbers, upgrade logs and image tags are signals that fire when an action ran. Exposure lives in two places they do not reach: the accounts that exist today, and the process listening today. An administrator who sets one console password can move a host across the line with no change to its configuration or version.&lt;/p&gt;

&lt;p&gt;The operational answer is to ask the running host. Does a password-less account exist, does a hashed account exist, is password authentication on, and is the listener younger than the package.&lt;/p&gt;

&lt;p&gt;A package manager reports what was installed. Only the running host can say what is listening.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/alpine-sshd-bypass/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>alpine</category>
      <category>linux</category>
      <category>security</category>
      <category>devops</category>
    </item>
    <item>
      <title>PCA 2011 Revocation: Who Authorizes It, and What Recovery Must ProveI</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Tue, 10 Nov 2026 13:03:48 +0000</pubDate>
      <link>https://dev.to/ntctech/pca-2011-revocation-who-authorizes-it-and-what-recovery-must-provei-39ao</link>
      <guid>https://dev.to/ntctech/pca-2011-revocation-who-authorizes-it-and-what-recovery-must-provei-39ao</guid>
      <description>&lt;p&gt;PCA 2011 revocation is the step that removes trust in boot managers signed under PCA 2011, and installing the update does not take it. On October 22, 2026, Kaspersky's &lt;a href="https://securelist.com/uefi-bootkit-blackx/121539/" rel="noopener noreferrer"&gt;Securelist report on the BlackX bootkit&lt;/a&gt; described a UEFI bootkit found on fully patched systems in Central Asian diplomatic agencies, planted by an attacker who already held administrator privileges. The part that matters to an infrastructure owner is narrower than the malware: the revocation that closes the gap is a manual step, and Microsoft's automatic Enforcement Phase had not been implemented when the report was published. &lt;a href="https://www.rack2cloud.com/ca-distrust-propagation/" rel="noopener noreferrer"&gt;CA distrust propagation&lt;/a&gt; covers whether a verifier has consumed a distrust decision. This post starts after that question, at whether the organization is ready to authorize consuming it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4jkkoh1goc6lo00bgmr9.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4jkkoh1goc6lo00bgmr9.jpg" alt="PCA 2011 revocation diagram" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Revocation here means untrusting every boot manager signed under Windows Production PCA 2011 by adding it to the Secure Boot forbidden signature database (DBX). That makes it a recovery event as much as a security one, and recovery is a core layer of &lt;a href="https://www.rack2cloud.com/data-protection-architecture-strategy-guide/" rel="noopener noreferrer"&gt;data protection architecture&lt;/a&gt;, because the boot paths used to recover a device, such as USB media, PXE servers and VM templates, can depend on the same signing chain. The question about PCA 2011 revocation is not whether to close the gap. It is who authorizes the change, on what proof, and what can still boot afterward. The scope is deliberately that narrow: this is not a Secure Boot tutorial, it makes no claim about how many devices remain unrevoked, and it does not discuss how an attacker gets in.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why PCA 2011 Revocation Is Not Part of Patching
&lt;/h2&gt;

&lt;p&gt;Microsoft's mitigations for CVE-2023-24932 are in Windows updates from July 9, 2024, and they are not enabled by default, according to &lt;a href="https://support.microsoft.com/en-us/topic/kb5025885-how-to-manage-the-windows-boot-manager-revocations-for-secure-boot-changes-associated-with-cve-2023-24932-41a975df-beb2-40c1-99a3-b3ff139f832d" rel="noopener noreferrer"&gt;KB5025885&lt;/a&gt;. A device can carry every update and still trust boot managers signed under PCA 2011. Securelist's description of the victims is the same point from the attacker's side: fully patched machines, with the revocation left as a manual step that needs a registry flag and the Secure-Boot-Update scheduled task. Kaspersky's example of why the design is manual is that automatic revocation could break dual-boot setups, and it assesses that the instructions are hard to find. Both are Kaspersky's inferences, not Microsoft statements, and neither is measured here.&lt;/p&gt;

&lt;p&gt;Two identifiers appear in this story and they belong to two sources. Kaspersky ties BlackX to Baton Drop, CVE-2022-21894, and describes the PCA 2011 revocation as that flaw's remediation. Microsoft frames KB5025885 under CVE-2023-24932. This post attributes each statement to the source that makes it and does not treat the identifiers as interchangeable.&lt;/p&gt;

&lt;p&gt;Expiry is not revocation either. The PCA 2011 certificate expired on October 19, 2026, and Securelist notes that expiry does not stop a boot manager signed under it from loading unless that boot manager is blocklisted. Expiry is a date on a certificate. PCA 2011 revocation is a change to DBX that someone makes on each device.&lt;/p&gt;

&lt;p&gt;Microsoft describes the mitigations as preventive, not corrective, and says an attacker needs administrator or physical access to exploit the gap they close. That sharpens the authorization. The benefit of PCA 2011 revocation is protection against a bootkit that has not yet been planted, and the cost is paid in recovery on every device the change reaches. A device that is already compromised is not cleaned by it. Weighing a benefit that is contingent on an attacker against a cost that is certain on the day of enforcement is the kind of judgment that needs a named owner.&lt;/p&gt;

&lt;p&gt;Microsoft's own plan is an Enforcement Phase that applies the revocation automatically, with no option to disable it afterward. The September 30, 2026 revision of KB5025885 gives no date for it and commits to at least six months' notice. That cuts two ways. Waiting for Microsoft is a decision, not a neutral default. And an organization that starts its recovery testing on the day notice arrives is starting with a clock it did not choose. Nothing here predicts when that notice will come.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Diagnostic:&lt;/strong&gt; &lt;em&gt;"If every boot manager signed under PCA 2011 were untrusted on this device class tomorrow, which recovery path would still boot, and when did we last prove it?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Someone Has to Own PCA 2011 Revocation
&lt;/h2&gt;

&lt;p&gt;The mitigations sit outside the normal patch path. Installing an update is a patching task. Revocation writes to firmware variables, depends on the firmware accepting the DB and DBX updates, and has a consequence in recovery that the patching team does not own. The recovery owner knows which media exist but does not run the DBX change. The endpoint team runs the change but may not know a PXE server or a VM template depends on the old chain. Without a named authorizer, each team can reasonably believe the other holds the decision, and PCA 2011 revocation proceeds only when someone happens to run the change or never proceeds at all. Neither outcome is a decision. Both leave the organization with an exposure or a recovery gap that nobody recorded.&lt;/p&gt;

&lt;p&gt;The adjacent ownership question is already covered in &lt;a href="https://www.rack2cloud.com/secure-boot-trust-state-ownership/" rel="noopener noreferrer"&gt;who controls trust-state updates in virtual firmware&lt;/a&gt;, which establishes who holds the platform keys and DB state. The question here comes one step later: who may initiate and execute a trust-state change that cannot be taken back once Secure Boot is in use. It also fits the &lt;a href="https://www.rack2cloud.com/revocation-unit-credential-rotation/" rel="noopener noreferrer"&gt;revocation unit&lt;/a&gt; idea in a different form. PCA 2011 is a trust root, and revoking it removes everything signed beneath it at once, the same shape as rotating a credential that many systems share, with a firmware list in place of a credential store.&lt;/p&gt;

&lt;p&gt;An authorization has three parts: a named approver, the criteria the approver applies, and the device classes the decision covers. It also has three possible outcomes, and "wait" is only one of them when it is recorded with an owner. The approver can enforce by device class. The approver can defer, with the open exposure documented and an accountable owner. Or the approver can block enforcement until the identified recovery dependencies are remediated.&lt;/p&gt;

&lt;h2&gt;
  
  
  Revocation Takes Boot Paths Away From Recovery
&lt;/h2&gt;

&lt;p&gt;Before PCA 2011 revocation is enforced on a device class, the compatibility question has to be answered with a boot test, not an assumption. Microsoft states that revoking boot managers could cause issues for some device boot configurations, and that recovery media must be updated because existing media may fail to start or recover the device. Its FAQ names CD/DVD, PXE boot and USB as media that will not start after revocation. Its &lt;a href="https://techcommunity.microsoft.com/blog/windows-itpro-blog/revoking-vulnerable-windows-boot-managers/4121735" rel="noopener noreferrer"&gt;Windows IT Pro blog post on revoking vulnerable boot managers&lt;/a&gt; singles out PXE as especially likely to be affected. Firmware has to be able to apply the DB and DBX updates, which is why Microsoft says to test one device of each type before widening the change. On machines with more than one Windows installation, every installation has to be updated to the July 9, 2024 level or later before revoking.&lt;/p&gt;

&lt;p&gt;The table maps each dependency to what Microsoft documents. It is not a prevalence claim: the KB does not say how many devices are affected.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Dependency&lt;/th&gt;
&lt;th&gt;What Microsoft documents&lt;/th&gt;
&lt;th&gt;What it means for recovery&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Recovery media (CD/DVD, USB)&lt;/td&gt;
&lt;td&gt;Existing media may fail to start or recover the device after revocation; media must be updated&lt;/td&gt;
&lt;td&gt;Rebuild each media type and boot-test it on a device of each class&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PXE and network boot servers&lt;/td&gt;
&lt;td&gt;Named as media that will not start after revocation; the Windows IT Pro guidance calls PXE especially likely to be affected&lt;/td&gt;
&lt;td&gt;Inventory boot servers and test the network boot path before enforcement&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;x86 VMs with Secure Boot&lt;/td&gt;
&lt;td&gt;Known issue: Secure Boot VMs fail to boot; Microsoft says it is coordinating with VMware&lt;/td&gt;
&lt;td&gt;Attributed to Microsoft only; Broadcom's side was not confirmed in the sources read&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Symantec Endpoint Encryption&lt;/td&gt;
&lt;td&gt;Known issue: the mitigations cannot be applied&lt;/td&gt;
&lt;td&gt;Get the vendor's written position before the device class is enrolled&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Server 2012 and 2012 R2 with TPM 2.0&lt;/td&gt;
&lt;td&gt;Known issue, addressed April 8, 2026&lt;/td&gt;
&lt;td&gt;Confirm the update level before the device class is enrolled&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Older external media after SVN 5.0&lt;/td&gt;
&lt;td&gt;Listed as a known issue&lt;/td&gt;
&lt;td&gt;Test external boot media against the SVN you intend to apply&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The KB also lists HP Sure Start, Arm64 devices with Qualcomm chips and Apple Boot Camp among the known issues. BitLocker belongs in the same pass: Microsoft's guidance is to retain the BitLocker recovery key before making the change.&lt;/p&gt;

&lt;p&gt;Recovery dependencies fail in a recognizable way, and it is not the way the failing component announces. &lt;a href="https://www.rack2cloud.com/recovery-boundary-dependency-failure/" rel="noopener noreferrer"&gt;The System Recovered. Your Recovery Boundary Didn’t.&lt;/a&gt; covers recovery paths that quietly depend on systems outside the recovery design, with certificates already on the list. A boot manager's signing chain is that class of dependency. It is invisible until the first device that needs the old media does not boot.&lt;/p&gt;

&lt;h2&gt;
  
  
  PCA 2011 Revocation Is Effectively Irreversible
&lt;/h2&gt;

&lt;p&gt;The KB's caution is direct: once the mitigation is applied, it cannot be reverted if Secure Boot continues to be used, and reformatting the device does not remove it. The accurate phrase is &lt;strong&gt;effectively irreversible while Secure Boot remains enabled&lt;/strong&gt;, not universally irreversible. Microsoft separates the work into four ordered mitigations, and only the third one is PCA 2011 revocation itself. The first two prepare the device to boot under the 2023 chain, which is why the order is fixed.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo25pmbqcf7aceysmcxpn.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo25pmbqcf7aceysmcxpn.jpg" alt="The four ordered Secure Boot mitigations" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  01 — 2023 CERTIFICATE INTO DB
&lt;/h3&gt;

&lt;p&gt;The Windows UEFI CA 2023 certificate is added to the Secure Boot DB. Evidence: event 1036.&lt;/p&gt;

&lt;h3&gt;
  
  
  02 — PCA2023-SIGNED BOOT MANAGER
&lt;/h3&gt;

&lt;p&gt;The device has the PCA2023-signed boot manager installed. Evidence: event 1799 and a UEFICA2023Status of Updated, which Microsoft describes as the completion indicator for the first two mitigations. Event 1799 records that the boot manager was applied, not that the device booted through it. A controlled reboot test confirms that, and it is this post's recommendation, not something Microsoft's events establish.&lt;/p&gt;

&lt;h3&gt;
  
  
  03 — PCA 2011 INTO DBX: THE REVOCATION
&lt;/h3&gt;

&lt;p&gt;Boot managers signed under PCA 2011 are untrusted. Microsoft's instruction is to create a recovery flash drive first. Evidence: event 1037 and the DBX read-back below.&lt;/p&gt;

&lt;h3&gt;
  
  
  04 — SVN UPDATE
&lt;/h3&gt;

&lt;p&gt;The DBX is updated with the latest security version number. Evidence: event 1042, added to the KB on June 2, 2026. It belongs to this step and is not evidence of the revocation.&lt;/p&gt;

&lt;p&gt;Microsoft's documented read-back for step three checks that the DBX contains the PCA 2011 entry and should return True:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;System&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Encoding&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="no"&gt;ASCII&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;GetString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Get&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nc"&gt;SecureBootUEFI&lt;/span&gt; &lt;span class="n"&gt;dbx&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;bytes&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;match&lt;/span&gt; &lt;span class="s1"&gt;'Microsoft Windows Production PCA 2011'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three terms stay separate in any change record. &lt;strong&gt;Deployment rollback&lt;/strong&gt; is reversing or stopping a deployment that has not completed. &lt;strong&gt;Revocation reversal&lt;/strong&gt; is restoring trust in the revoked boot manager, which Microsoft says cannot be done while Secure Boot stays in use. &lt;strong&gt;Recovery&lt;/strong&gt; is restoring bootability through a documented procedure that may itself disable Secure Boot for a time. A change plan that says "we can roll back" is usually describing the first of the three.&lt;/p&gt;

&lt;p&gt;The documented recovery shows how narrow the third one is. Microsoft's procedure is to turn Secure Boot off, reset the keys to factory defaults (Reset, not Clear), restore the boot files with &lt;code&gt;bcdboot&lt;/code&gt;, reinstall from recovery media if needed, and re-enable Secure Boot. The KB describes the result as a pre-revocation state. While Secure Boot is off, the device is exposed to the bootkits the revocation was meant to stop. The only documented way back from a failed enforcement temporarily removes the control.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;⚠ &lt;strong&gt;Recovery Is Not Reversal:&lt;/strong&gt; Recovering a device that failed to boot does not restore trust in PCA 2011. It returns the device to a pre-revocation state through a procedure that turns Secure Boot off first. A plan that counts on that procedure as its safety net has accepted a window of exposure it has not recorded.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  The Recovery-Proof Gate: Eight Checks to Authorize and Verify Enforcement
&lt;/h2&gt;

&lt;p&gt;Authorizing PCA 2011 revocation needs two recovery proofs, made at different points. The first is made before authorization: for each device class, a device that fails to boot can be recovered, tested on one device of that type. The second is made after revocation, on a pilot device in each class, with no boot manager signed under PCA 2011 left in the path: media rebuilt, boot servers updated, and recovery still works. The first passing says nothing about the second, because the old media still boots until the moment it does not. The second is a condition for widening, not for starting. It closes on the pilot before enforcement extends to the rest of the class.&lt;/p&gt;

&lt;p&gt;Microsoft's guidance supports testing one device of each type. The staged sequence of authorize, pilot, prove and widen is this post's recommendation, not a Microsoft procedure. The second proof is a controlled recovery test conducted after revocation, with the legacy boot dependency unavailable. That is the point of &lt;a href="https://www.rack2cloud.com/adversarial-restore-testing/" rel="noopener noreferrer"&gt;adversarial restore testing&lt;/a&gt;: demonstrate that recovery still works when a dependency the old design relied on is gone. Evidence of that kind belongs in the same record as the approval, in the way &lt;a href="https://www.rack2cloud.com/data-protection-resiliency-learning-path/governance-recovery-assurance/" rel="noopener noreferrer"&gt;recovery-evidence assurance&lt;/a&gt; treats recovery as something a team demonstrates rather than asserts.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Gate&lt;/th&gt;
&lt;th&gt;When&lt;/th&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Proof&lt;/th&gt;
&lt;th&gt;Owner&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1. Prerequisite state&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Does the device hold the 2023 certificate and run a PCA2023-signed boot manager, and can its firmware apply DB and DBX updates?&lt;/td&gt;
&lt;td&gt;Events 1036 and 1799; UEFICA2023Status; a controlled reboot test; firmware vendor confirmation for devices that failed&lt;/td&gt;
&lt;td&gt;Endpoint or platform team&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2. Compatibility inventory&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Which recovery media, USB media, PXE or network boot servers, VM templates and disk-encryption products depend on a PCA 2011 boot manager?&lt;/td&gt;
&lt;td&gt;Inventory plus a boot test per media type; KB known-issues check&lt;/td&gt;
&lt;td&gt;Recovery owner&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3. Encryption readiness&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Are BitLocker recovery keys escrowed before the change, and has each third-party encryption vendor confirmed compatibility?&lt;/td&gt;
&lt;td&gt;Escrow check; vendor statement&lt;/td&gt;
&lt;td&gt;Security or endpoint team&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4. Pre-enforcement recovery proof&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Can a device that fails to boot be recovered, and was it tested on one device of each type?&lt;/td&gt;
&lt;td&gt;Documented test per device class&lt;/td&gt;
&lt;td&gt;Recovery owner&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5. Authorization&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Who may approve an effectively irreversible transition, and on what criteria?&lt;/td&gt;
&lt;td&gt;Named approver; recorded criteria&lt;/td&gt;
&lt;td&gt;To be named by the organization&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6. Enforcement decision&lt;/td&gt;
&lt;td&gt;Before&lt;/td&gt;
&lt;td&gt;Enforce on a pilot by device class, defer with documented exposure and an accountable owner, or block until identified recovery dependencies are remediated?&lt;/td&gt;
&lt;td&gt;Recorded decision with the open exposure stated&lt;/td&gt;
&lt;td&gt;The authorizer from gate 5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7. Enforcement evidence&lt;/td&gt;
&lt;td&gt;After&lt;/td&gt;
&lt;td&gt;Did revocation take effect on each pilot device?&lt;/td&gt;
&lt;td&gt;Event 1037; DBX read-back&lt;/td&gt;
&lt;td&gt;Whoever ran the change&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8. Post-revocation recovery proof&lt;/td&gt;
&lt;td&gt;After&lt;/td&gt;
&lt;td&gt;Can recovery proceed without any boot manager signed under PCA 2011, including rebuilt media and updated PXE servers? Passing is the condition for widening.&lt;/td&gt;
&lt;td&gt;Boot test of rebuilt media on the pilot after revocation&lt;/td&gt;
&lt;td&gt;Recovery owner&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Gates 5 and 6 are the two that no tooling closes. Gate 5 names who decides. Gate 6 records what they decided, in terms of organizational readiness and documented risk and not an assumed Microsoft timetable. A deferral with an owner and a stated exposure is a legitimate result. A deferral that exists only because nobody ran the change is not. Gate 8 is the closure: until it passes on the pilot, enforcement does not widen.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Worksheet: &lt;a href="https://www.rack2cloud.com/pca-2011-revocation/" rel="noopener noreferrer"&gt;PCA 2011 Revocation Readiness Worksheet (PDF)&lt;/a&gt;, a two-page fill-in record for one device class.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;A Secure Boot revocation can be available, deployed and correct, and still leave an organization unable to recover through the boot paths it built. Patching delivers the capability. It does not deliver the decision, and Microsoft's own documentation places the decision outside the default install path.&lt;/p&gt;

&lt;p&gt;The real problem with PCA 2011 revocation is that its failure shows up in recovery, not in security tooling. The device keeps running until the day it has to boot from media signed under the chain that was just untrusted, and the only documented way back temporarily removes the protection the change was made to add.&lt;/p&gt;

&lt;p&gt;None of this argues against PCA 2011 revocation. It argues against enforcing it by default, and against waiting by default. Waiting is also a decision, and it needs an owner and a stated exposure like any other. Nothing here estimates how many devices are affected or when Microsoft will announce enforcement.&lt;/p&gt;

&lt;p&gt;Patched is not revoked, and revoked is not recoverable until someone proves it. Name the authorizer, prove recovery before the first enforcement, then prove it again on the pilot before widening, not after the first device fails to boot.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://dev.to/data-protection-architecture-strategy-guide/"&gt;Data Protection Architecture&lt;/a&gt; — the pillar guide for recovery architecture, where boot-path dependencies sit&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/data-protection-resiliency-learning-path/governance-recovery-assurance/"&gt;Governance and Recovery Assurance&lt;/a&gt; — the learning-path stage on proving recovery with evidence, not assertion&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/ca-distrust-propagation/"&gt;A CA Distrust Decision Is Not Protection Until Every Verifier Has Consumed It&lt;/a&gt; — the question this post starts after: whether each verifier has consumed a distrust decision&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/secure-boot-trust-state-ownership/"&gt;Secure Boot Trust: Who Owns the Virtual Firmware State&lt;/a&gt; — who holds the platform keys and DB state, the ownership question this post picks up after&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/revocation-unit-credential-rotation/"&gt;Rotating Credentials Is Not Revoking Them. The Revocation Unit Decides Whether You Can.&lt;/a&gt; — treating a shared trust root as the unit that revocation removes&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/recovery-boundary-dependency-failure/"&gt;The System Recovered. Your Recovery Boundary Didn’t.&lt;/a&gt; — recovery paths that quietly depend on systems outside the recovery design&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://dev.to/adversarial-restore-testing/"&gt;Restore Testing Is Becoming An Adversarial Discipline&lt;/a&gt; — proving recovery still works when a dependency the old design relied on is gone&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://support.microsoft.com/en-us/topic/kb5025885-how-to-manage-the-windows-boot-manager-revocations-for-secure-boot-changes-associated-with-cve-2023-24932-41a975df-beb2-40c1-99a3-b3ff139f832d" rel="noopener noreferrer"&gt;Microsoft KB5025885: Managing the Windows Boot Manager Revocations&lt;/a&gt; — the primary source for the mitigations, events, known issues and recovery procedure&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://techcommunity.microsoft.com/blog/windows-itpro-blog/revoking-vulnerable-windows-boot-managers/4121735" rel="noopener noreferrer"&gt;Windows IT Pro Blog: Revoking vulnerable Windows boot managers&lt;/a&gt; — Microsoft's April 2024 explanation of why PXE boot is especially likely to be affected&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://securelist.com/uefi-bootkit-blackx/121539/" rel="noopener noreferrer"&gt;Securelist: the BlackX UEFI bootkit report&lt;/a&gt; — Kaspersky's October 22, 2026 analysis of the incident that dated this post&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/pca-2011-revocation/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>windows</category>
    </item>
    <item>
      <title>VeloCloud Shows Why Integration Endpoint Trust Needs an Owner</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Sun, 08 Nov 2026 13:10:20 +0000</pubDate>
      <link>https://dev.to/ntctech/velocloud-shows-why-integration-endpoint-trust-needs-an-owner-192i</link>
      <guid>https://dev.to/ntctech/velocloud-shows-why-integration-endpoint-trust-needs-an-owner-192i</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqfm82d5yxfg2u50v3sxs.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqfm82d5yxfg2u50v3sxs.jpg" alt="Field Notes — Engineering Notes from the Complexity Gap | Rack2Cloud" width="800" height="197"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Integration endpoint trust is the part of an orchestrator's exposure that an inbound-reachability review can miss. On November 2, 2026, Arista published Security Advisory 0202 for VeloCloud Orchestrator (VCO), the platform that configures, monitors and manages VeloCloud SD-WAN deployments, covering CVE-2026-111497, a memory-corruption flaw in the Brotli decoder. Arista's wording draws the line that matters: exploitation requires the affected runtime to process a crafted Brotli response through an outbound HTTP client that decompresses automatically. The question that follows is narrow, and it is not whether VCO has been exploited. It is what validation boundary surrounds content returned by a configured integration endpoint, and who owns it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqbp1gmmv932h9w7ug9eq.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqbp1gmmv932h9w7ug9eq.jpg" alt="Integration endpoint trust diagram: an orchestrator with reviewed inbound arrows, and outbound integration calls whose responses are processed with no owner assigned" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Arista's Advisory Establishes, and What It Does Not
&lt;/h2&gt;

&lt;p&gt;Arista's advisory (revision 1.0, November 2) describes CVE-2026-111497 as a buffer overflow in Google's Brotli C decoder, rates it 9.8 under CVSS v3.1, and characterizes it as remotely exploitable without authentication. Arista says it is still reviewing its product lines and will update the advisory, so the table below describes revision 1.0. The conditions matter more than the score.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Established by the advisory&lt;/th&gt;
&lt;th&gt;Not established&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;VCO releases in the 5.2.3 train at 5.2.3.19 and below, in the 6.1.3 train at 6.1.3.11 and below, and in the 6.4.2 train at 6.4.2.13 and below are affected; 5.2.3.20, 6.1.3.12 and 6.4.2.14 and later in those same trains are fixed. Arista states that a release not on its affected list is not vulnerable&lt;/td&gt;
&lt;td&gt;That every VCO deployment is remotely exploitable regardless of integration configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exploitation requires the runtime to process a crafted Brotli response through an outbound HTTP client with automatic decompression&lt;/td&gt;
&lt;td&gt;That VCO has been exploited through this path&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The response can come from an attacker-controlled or compromised integration endpoint, or through an HTTP redirect where redirects are followed&lt;/td&gt;
&lt;td&gt;That the vendor behind any named integration is compromised, or which integrations are enabled by default&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Arista names Broadcom Symantec, Zscaler and Azure Non SD-WAN Destination integrations as examples&lt;/td&gt;
&lt;td&gt;A validated VCO-specific detection signature (Arista says none has been validated in its assessment)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Arista's assessment to date has not identified Brotli decoding in the reviewed inbound API request paths&lt;/td&gt;
&lt;td&gt;A mitigation (none is available; Arista recommends upgrading)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Two public facts sit beside the advisory and should not be confused with it. The Canadian Centre for Cyber Security reports that &lt;a href="https://cyber.gc.ca/en/alerts-advisories/al26-027-critical-vulnerability-impacting-google-brotli-cve-2026-111497" rel="noopener noreferrer"&gt;CISA added CVE-2026-111497 to its Known Exploited Vulnerabilities catalog on October 27&lt;/a&gt;. That listing establishes exploitation of the underlying Brotli flaw. It does not establish that VCO was exploited or that any VCO integration path was used. Separately, &lt;a href="https://www.zscaler.com/blogs/security-research/cve-2026-111497-node2shell-http2shell-vulnerability-may-allow-rce" rel="noopener noreferrer"&gt;Zscaler ThreatLabz reported reconnaissance and exploitation attempts&lt;/a&gt; observed through its Deception decoys in the first 24 hours after the Brotli flaw's public disclosure (Zscaler dates the Node.js disclosure to October 21): 402 events from 20 source IPs against 12 customers. The report does not mention VCO, and it describes attempts, not confirmed compromise.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Exposure Map Is Drawn Inbound
&lt;/h2&gt;

&lt;p&gt;Most exposure reviews in &lt;a href="https://www.rack2cloud.com/modern-infrastructure-iac-strategy-guide/" rel="noopener noreferrer"&gt;modern infrastructure&lt;/a&gt; begin with the same question: what can an attacker reach? Scanners, firewall rules and reachability maps all answer it from the outside in. That is the right starting point for flaws an attacker triggers by connecting to a system.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.rack2cloud.com/patch-visibility-debt/" rel="noopener noreferrer"&gt;Patch visibility debt&lt;/a&gt; builds a prioritization rule on that starting point: treat internet-reachable instances as exploited until proven otherwise, and let confirmation catch up. For flaws triggered by inbound connections, the rule holds. This advisory describes a different trigger. An orchestrator that no scanner can reach from the internet can still open a connection to a configured integration endpoint and process what comes back. When the trigger arrives on a connection the orchestrator itself initiates, inbound reachability does not tell you whether the trigger can arrive.&lt;/p&gt;

&lt;p&gt;Arista reports that its assessment to date has not identified Brotli decoding in the reviewed inbound API request paths. For a buyer, the practical reading is that the path the advisory describes is the outbound one, and an exposure assessment that stops at the inbound surface never examines integration endpoint trust at all.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Diagnostic:&lt;/strong&gt; &lt;em&gt;"If every inbound path to this orchestrator were closed, what could still make it process attacker-controlled bytes?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  The Outbound Half: Destination, Redirect, Decoder
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fene55407ad6mg68zpw96.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fene55407ad6mg68zpw96.jpg" alt="Response path from a configured integration endpoint, through an optional redirect and an auto-decompressing HTTP client, to the Brotli decoder in an orchestrator" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The flaw sits at the decoder. The decisions that determine what reaches it sit upstream.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The mechanism has four steps, and only the last one is a software defect.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The orchestrator opens a connection to an integration endpoint an operator configured.&lt;/li&gt;
&lt;li&gt;The response arrives from that destination or, where redirects are followed, from a destination the first one chose.&lt;/li&gt;
&lt;li&gt;The HTTP client decompresses the response automatically before the application sees it.&lt;/li&gt;
&lt;li&gt;The Brotli decoder processes the bytes.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Steps one and two are decisions, whoever makes them. Someone chose the destination. Redirect and decompression behavior belong to the HTTP client, and on a packaged orchestrator the vendor may have set them, so the operator's job is to find out which and record the answer. The flaw sits at step four, and Arista's fix is the upgrade; it lists no mitigation. But the set of places that can deliver bytes to that decoder is set by the first two steps. Zscaler's analysis lists webhook delivery and API integrations among the outbound examples, which is the same shape.&lt;/p&gt;

&lt;p&gt;A redirect widens the set. Without one, the exposure is the destinations an operator configured. With followed redirects, it includes destinations those destinations point to. Not every destination is exploitable and not every redirect is followed, so the assessment has to find out which are, deployment by deployment, instead of assuming either answer.&lt;/p&gt;

&lt;p&gt;A vendor review asks whether the supplier of a component was classified and reviewed, the question &lt;a href="https://www.rack2cloud.com/vendor-review-process-supplier-visibility/" rel="noopener noreferrer"&gt;your vendor review process never saw the real supplier&lt;/a&gt; argues about. This is the other question. Once a component is known and sits in the processing path, what stands between it and the content a configured destination returns? That is the integration endpoint trust question.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who Owns Integration Endpoint Trust
&lt;/h2&gt;

&lt;p&gt;An integration is usually enabled to solve an operational problem, and the record of that decision often does not say what the orchestrator does with what comes back. Integration endpoint trust has an owner or it has a default. For this class of flaw, the exposure assessment has to include the configured integration destinations, the destinations reachable through followed redirects, and the client behavior applied to returned content.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;⚠ &lt;strong&gt;Architectural recommendations, not vendor fixes:&lt;/strong&gt; Arista lists no mitigation for CVE-2026-111497 and recommends upgrading. Its only configuration guidance is to ensure third-party URLs point to trusted endpoints. Destination allowlists, redirect restrictions and disabling unused integrations go further, and they are this site's architectural recommendations, not vendor fixes.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A register makes the decision operational. One row per integration, and the first column separates an integration that is configured from one that is enabled and used, because the second needs an owner and the first may need removing.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Integration and operational status&lt;/th&gt;
&lt;th&gt;Configured destination&lt;/th&gt;
&lt;th&gt;Redirect behavior and permitted destination changes&lt;/th&gt;
&lt;th&gt;HTTP client and automatic decompression behavior&lt;/th&gt;
&lt;th&gt;Owner accountable for the integration&lt;/th&gt;
&lt;th&gt;Evidence of configuration and validation&lt;/th&gt;
&lt;th&gt;Last verification date&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Illustrative row, not a finding: a SaaS integration, enabled and used&lt;/td&gt;
&lt;td&gt;The exact host the orchestrator calls&lt;/td&gt;
&lt;td&gt;Followed or not, and which destinations a redirect may reach&lt;/td&gt;
&lt;td&gt;Which client makes the call, and whether it decompresses automatically&lt;/td&gt;
&lt;td&gt;A named role, not a team&lt;/td&gt;
&lt;td&gt;A configuration export and the result of a validation test&lt;/td&gt;
&lt;td&gt;The date someone last checked&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The register is only as good as its last verification date. Arista's advisory notes that a trusted endpoint can be compromised without any change to VCO configuration, so a configuration export alone does not show the destination is still trustworthy. Three questions belong to whoever approves a new integration: where does the response come from, which redirects can change that, and what does the client do to the bytes before the application sees them. The controls the register points to, destination allowlists, redirect restrictions and removing integrations nobody uses, narrow the set of destinations that can reach the decoder. They do not remove the flaw. The upgrade does.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;The Brotli flaw is the instance. The decision is older: an exposure review that starts from what can reach the orchestrator leaves out what the orchestrator chooses to reach.&lt;/p&gt;

&lt;p&gt;Integration endpoint trust has no default owner. Each review has its own object: procurement reviews the vendor, the security review maps the inbound API, and an engineer enables the integration to solve a problem. The destinations the orchestrator calls, the redirects it follows and the client behavior applied to what returns sit between those objects, and a flaw in a decoder is simply the first time anyone needs the answer.&lt;/p&gt;

&lt;p&gt;The advisory is specific about its conditions, and the reader should be equally specific. Nothing in it, as summarized above, shows that VCO was exploited through this path, and nothing in it says every deployment is exposed. What it does show is that the assessment can be wrong in a specific way, by being complete on the inbound surface and silent on the outbound one.&lt;/p&gt;

&lt;p&gt;Reachability tells you who can call the orchestrator. Integration endpoint trust tells you whom the orchestrator believes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/modern-infrastructure-iac-strategy-guide/" rel="noopener noreferrer"&gt;Modern Infrastructure &amp;amp; IaC Architecture&lt;/a&gt; — the pillar hub for infrastructure control planes, governance and day-2 operations, where orchestrator trust boundaries sit&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/patch-visibility-debt/" rel="noopener noreferrer"&gt;SharePoint Vulnerabilities And The Cost Of Patch Visibility Debt&lt;/a&gt; — the exposure-first prioritization rule this post extends to flaws triggered by outbound connections&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/vendor-review-process-supplier-visibility/" rel="noopener noreferrer"&gt;Your Vendor Review Process Never Saw The Real Supplier&lt;/a&gt; — the supplier-classification question, contrasted here with trust in content a known component receives&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.arista.com/en/support/advisories-notices/security-advisory/24853-security-advisory-0202" rel="noopener noreferrer"&gt;Arista Security Advisory 0202&lt;/a&gt; — the vendor primary source for CVE-2026-111497 in VeloCloud Orchestrator, including conditions, affected releases and fixed releases&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.zscaler.com/blogs/security-research/cve-2026-111497-node2shell-http2shell-vulnerability-may-allow-rce" rel="noopener noreferrer"&gt;Zscaler ThreatLabz: CVE-2026-111497 Node2Shell / HTTP2Shell&lt;/a&gt; — public analysis of the inbound and outbound exploitation paths and the first-24-hour activity&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://cyber.gc.ca/en/alerts-advisories/al26-027-critical-vulnerability-impacting-google-brotli-cve-2026-111497" rel="noopener noreferrer"&gt;Canadian Centre for Cyber Security, AL26-027&lt;/a&gt; — the alert that reports the October 27 CISA KEV listing&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/integration-endpoint-trust/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>networking</category>
    </item>
    <item>
      <title>Extension Update Authority Decides What Runs on the Endpoint. Uninstalling Doesn't Undo It.</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Sat, 07 Nov 2026 13:29:36 +0000</pubDate>
      <link>https://dev.to/ntctech/extension-update-authority-decides-what-runs-on-the-endpoint-uninstalling-doesnt-undo-it-116p</link>
      <guid>https://dev.to/ntctech/extension-update-authority-decides-what-runs-on-the-endpoint-uninstalling-doesnt-undo-it-116p</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fttxygva7tx3pqtbf8a5z.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fttxygva7tx3pqtbf8a5z.jpg" alt="Field Notes — Engineering Notes from the Complexity Gap | Rack2Cloud" width="800" height="197"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Extension update authority is the decision about who can change code on an engineering endpoint without an acquisition event the organization controls. On October 29, 2026, that decision was tested in the Cursor VS Code Extension Marketplace. According to Cursor, a maliciously modified Anthropic.claude-code extension, versions 2.1.312 and 2.1.313 in Windows and macOS packages, was available between about 2:27am and 1:35pm PT, roughly eleven hours. Cursor states that Anthropic did not develop the malicious versions.&lt;/p&gt;

&lt;p&gt;The sources are specific enough to separate three responsibilities that an editor's extension feature quietly bundles together: the permitted update path, the activation boundary, and recovery. They are not specific enough to say how the package got into the marketplace, and this post does not pretend otherwise.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzaprka8efvyba7nk9veq.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzaprka8efvyba7nk9veq.jpg" alt="Extension update authority diagram: a marketplace channel updating code on an engineering endpoint across acquire, update, activate and recover stages" width="800" height="447"&gt;&lt;/a&gt; &lt;/p&gt;

&lt;h2&gt;
  
  
  What the Sources Establish, and What They Do Not
&lt;/h2&gt;

&lt;p&gt;This post rests on two primary sources: Cursor's &lt;a href="https://cursor.com/blog/claude-code-extension-security-notice" rel="noopener noreferrer"&gt;security notice&lt;/a&gt; and Microsoft Threat Intelligence's &lt;a href="https://www.microsoft.com/en-us/security/blog/2026/10/30/inside-malicious-claude-code-branded-cursor-extension-persistent-ssh-backdoor/" rel="noopener noreferrer"&gt;analysis of the injected code&lt;/a&gt;. Every claim below is attributed to one of them.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Established by the sources&lt;/th&gt;
&lt;th&gt;Not established&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;The injected loader runs when the extension module loads, before normal extension initialization, and the extension keeps working (Microsoft)&lt;/td&gt;
&lt;td&gt;How the package was first admitted to the marketplace, or which control failed. Cursor's notice gives no root cause&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Persistence is written outside the extension directory. Microsoft states that removing or updating the extension does not remove it, and Cursor says uninstalling may not be sufficient&lt;/td&gt;
&lt;td&gt;Whether any host was accessed through the tunnel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The loader reports environment-variable names, not values (Microsoft)&lt;/td&gt;
&lt;td&gt;Whether credential values were taken&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The malicious versions were removed, the marketplace was shut down, keys rotated and the marketplace rebuilt (Cursor)&lt;/td&gt;
&lt;td&gt;Anything beyond Cursor's own statement that its audit found no other tampered extensions&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Scope of the evidence:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Windows and macOS packages are the identified affected packages. Microsoft's analysis shows Linux crontab logic in the loader, and Cursor's indicators list Linux persistence artifacts, but neither identifies an affected Linux package.&lt;/li&gt;
&lt;li&gt;The loader attempted SSH key installation, SSH enablement, a reverse tunnel and persistence. Cursor says the key and persistence were written and host details sent even where the tunnel failed. Microsoft lists these as attempted actions and says the sequence does not prove every step succeeded on every host.&lt;/li&gt;
&lt;li&gt;Cursor says SSH enablement failed on most macOS and Windows hosts, citing errors in the malicious code and the administrator rights the step requires.
## Extension Update Authority: Who Governs the Channel&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Microsoft's guidance tells organizations to find systems that "installed or automatically updated" the affected versions through the Cursor marketplace. Automatic update turns the marketplace operator's publishing decision into a change on each eligible endpoint that has the extension installed and automatic updates enabled. The organization made a deployment decision whether or not it ever wrote one down: it permitted a channel to change executable code on &lt;a href="https://www.rack2cloud.com/ai-infrastructure-strategy-guide/" rel="noopener noreferrer"&gt;AI infrastructure&lt;/a&gt; engineering endpoints on the channel's schedule.&lt;/p&gt;

&lt;p&gt;The acquisition record cannot capture that. The &lt;a href="https://www.rack2cloud.com/ovhcloud-cli-release-tags/" rel="noopener noreferrer"&gt;OVHcloud release-tags post&lt;/a&gt; records what a pipeline fetched and validated at acquisition. An extension that updates itself has no such moment on the consumer side, so the first thing to govern is extension update authority: which channels may install and update, and under what conditions.&lt;/p&gt;

&lt;p&gt;Microsoft points to controls from Cursor's security-hardening guidance: restricting extensions to approved publishers, and an installation cooldown that delays newly published versions. Both are real controls. Neither is extension update authority itself, and each carries an assumption. An approved-publisher list is only as strong as the identity and provenance checks behind it. This package carried the Anthropic.claude-code identifier while Cursor says Anthropic did not develop it, and whether a publisher list would have stopped it depends on what the marketplace verifies, which the notice does not say. A cooldown helps only if it outlasts the time a bad version stays available, and by Cursor's times that was about eleven hours. The version label is not the control either: Cursor lists modified packages for 2.1.312 as well as 2.1.313.&lt;/p&gt;

&lt;p&gt;Who admitted the package into the channel is a separate authority, and nothing disclosed shows how it failed. The point for the architect is narrower: admission, update and activation are separate lifecycle decisions, and each needs a named owner.&lt;/p&gt;

&lt;h2&gt;
  
  
  Activation Runs While the Extension Still Looks Normal
&lt;/h2&gt;

&lt;p&gt;Microsoft's analysis describes a JavaScript function that executes before the extension's expected code. When the module loads, it can use Node.js APIs to collect host information, modify files and execute operating-system commands. It suppresses command output and silently catches errors, while the extension continues functioning normally. The loader ran in the extension's execution context. Cursor attributes the failure of the SSH step to errors in the malicious code and to the administrator rights that enabling the local SSH service generally requires.&lt;/p&gt;

&lt;p&gt;Extension update authority decides when new code arrives. The activation boundary decides what that code can do on arrival, and it is the earliest critical execution boundary. By the time the module has loaded, the extension has already begun executing code, and the user sees an extension that behaves as before. Endpoint detection, process controls and isolation can still act, but they act on code that is already running. The architect needs to know, per extension class, what loading it permits, and which telemetry would show a change at editor start: file writes outside the extension directory, new startup objects, new outbound SSH processes. The same boundary question applies to tools an agent registers from a hub, which &lt;a href="https://www.rack2cloud.com/mcp-security-architecture/" rel="noopener noreferrer"&gt;MCP security architecture&lt;/a&gt; maps. The difference here is that the extension's code executes at module load, before normal initialization, with no tool call or approval in the path.&lt;/p&gt;

&lt;h2&gt;
  
  
  Recovery Proof Is Not Package Removal
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo0frq4ywy6n755pfqimw.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo0frq4ywy6n755pfqimw.jpg" alt="Removing an extension versus recovering the endpoint: uninstall removes the package while the SSH key, startup objects, scripts and tunnel processes can remain on an affected endpoint" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Uninstall removes the package. The artifacts it wrote outside its directory can stay.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Microsoft states that removing or updating the extension does not remove what it wrote outside its directory, and Cursor says uninstalling may not be sufficient. Microsoft adds that Cursor's takedown of the attacker's server does not remove artifacts already on a device. &lt;a href="https://www.rack2cloud.com/persistent-access-survives-fix/" rel="noopener noreferrer"&gt;The Server Was Fixed. Persistent Access Wasn't.&lt;/a&gt; names the general pattern: remediation that succeeds on the layers it was scoped to and never reaches the layer persistence lives in. Here the package is the layer that removal reaches. A recovery runbook has to be built on that, in this order:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Preserve evidence before cleanup: the extension package, the modified SSH authorization files, generated keys and scripts, and startup configuration (Microsoft).&lt;/li&gt;
&lt;li&gt;Identify and remove the documented persistence and unauthorized processes. Cursor documents a LaunchAgent on macOS, a scheduled task with a Startup-folder fallback on Windows, scripts and identity files under a &lt;code&gt;.config&lt;/code&gt; directory, and a key labeled svc-telemetry. Microsoft says to inspect each startup object's configured action before removal, because names alone do not distinguish malicious objects from legitimate monitoring software.&lt;/li&gt;
&lt;li&gt;Assess whether the endpoint can be trusted or must be rebuilt (Cursor, Microsoft).&lt;/li&gt;
&lt;li&gt;Rotate credentials and sessions reachable from the host, from a trusted system. Cursor says limited environment-variable collection does not rule out later credential access over the backdoor. Microsoft ties rotation to telemetry showing credential access or to device trust that cannot be restored. What rotating them actually revokes is the &lt;a href="https://www.rack2cloud.com/revocation-unit-credential-rotation/" rel="noopener noreferrer"&gt;revocation unit&lt;/a&gt; question.&lt;/li&gt;
&lt;li&gt;Verify recovery independently of whether the package is gone: confirm the tunnel and startup artifacts do not return after login or restart (Microsoft).
Extension update authority governs how a package arrives. Recovery has to be proved on the endpoint, and a clean extension list is not that proof.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Three Responsibilities, Three Failure Consequences
&lt;/h2&gt;

&lt;p&gt;Each responsibility needs an owner and the evidence that owner can produce.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;th&gt;Organizational responsibility&lt;/th&gt;
&lt;th&gt;Evidence required&lt;/th&gt;
&lt;th&gt;Failure consequence&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Extension update authority&lt;/td&gt;
&lt;td&gt;Endpoint fleet owner governs permitted channels, update policy and deployment conditions&lt;/td&gt;
&lt;td&gt;Approved channels, update mode, extension inventory, installation and update timestamps&lt;/td&gt;
&lt;td&gt;A channel changes executable code on engineering endpoints with no organizational decision in the path&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Activation boundary&lt;/td&gt;
&lt;td&gt;Endpoint security sets execution controls and verifies what an extension actually does at load&lt;/td&gt;
&lt;td&gt;What runs at load, with which privileges; file, process and startup-object telemetry at editor start&lt;/td&gt;
&lt;td&gt;The loader executes at module load while normal extension behavior continues&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Recovery&lt;/td&gt;
&lt;td&gt;Incident owner holds the runbook for persistence removal, rebuild decision and credential rotation&lt;/td&gt;
&lt;td&gt;Preserved evidence, documented artifacts removed, rotation from a trusted system, a check after restart&lt;/td&gt;
&lt;td&gt;The package is gone, the persistence and credentials are not, and the endpoint is treated as clean&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Admission is a separate question: who may put a package into the channel at all. Nothing in the disclosures shows how it failed here, which is a reason to assign it, not a finding about it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;An extension is executable software with an independently governed acquisition, update, activation and recovery lifecycle. Treating it as an editor feature leaves all four to defaults.&lt;/p&gt;

&lt;p&gt;The real problem is extension update authority. An automatically updating extension creates an organizational deployment decision, whether or not the organization has explicitly governed it. On October 29 the exposure window, by Cursor's times, was about eleven hours, and the loader ran at module load while the extension kept behaving normally. Whoever answers for the engineering fleet answers for that decision.&lt;/p&gt;

&lt;p&gt;For each extension class on engineering endpoints, name three owners before the next incident: who holds extension update authority and governs the permitted update path, who establishes and verifies the activation boundary, and who owns recovery, including persistence removal and credential rotation. Write the runbook so it ends in a check that does not depend on the extension being gone, and keep admission as its own governance question.&lt;/p&gt;

&lt;p&gt;Removing the package is not recovering the endpoint. The decision about what the extension was allowed to do has to exist before anyone needs it.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/extension-update-authority/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>ai</category>
      <category>vscode</category>
    </item>
    <item>
      <title>Policy Fail Mode Is an Architecture Decision, Not an Identity Integration Default</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Fri, 06 Nov 2026 13:19:58 +0000</pubDate>
      <link>https://dev.to/ntctech/policy-fail-mode-is-an-architecture-decision-not-an-identity-integration-default-4d8d</link>
      <guid>https://dev.to/ntctech/policy-fail-mode-is-an-architecture-decision-not-an-identity-integration-default-4d8d</guid>
      <description>&lt;p&gt;Policy fail mode is what an identity-conditioned rule does when the identity it depends on cannot be evaluated, and it is often inherited from an integration default instead of chosen. The default is set by a vendor, or by whoever wired the identity provider into the enforcement plane, and it stays invisible until the day the identity input is missing.&lt;/p&gt;

&lt;p&gt;On October 20, 2026, Cloudflare reported an issue with traffic through Cloudflare One Gateway when an identity policy was part of the policy decision, while Gateway policies with no identity policy were unaffected. That outage is the occasion for this post, not its subject. The subject is the decision it made visible: what each rule does when its identity input is missing, who may change that mid-incident, and whether the path they would use depends on the thing that failed.&lt;/p&gt;

&lt;p&gt;Three arguments follow. Composition determines how much of the estate an identity failure takes with it. Fail-closed protects the boundary but is not a recovery architecture. And the emergency path has to be independently operable and explicitly authorized, or the fallback inherits the failure it was built to survive.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyazkq1lbqvo51fuqj85s.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyazkq1lbqvo51fuqj85s.jpg" alt="policy fail mode: a rule set where identity-conditioned rules are cut off while identity-independent rules continue" width="799" height="413"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;One identity dependency, a partial outage. The rule set decides what fails.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Is Documented, and What Is Not
&lt;/h2&gt;

&lt;p&gt;Cloudflare's status page records the October 20 incident as Cloudflare Zero Trust connectivity, impact critical. It opened at 03:20 UTC with an investigation into an issue with traffic through Cloudflare One Gateway when an identity policy was present in the policy decision, and stated that all onramps were affected, that Gateway policies with no identity policy were unaffected, and that Dashboard impact was also being seen. The page marked the Zero Trust component a major outage. A fix was reported implemented at 04:44. At 05:46 Cloudflare reported edge functionality, including Access logins and Secure Web Gateway connectivity, restored, with Access API errors still elevated.&lt;/p&gt;

&lt;p&gt;As of November 6, Cloudflare had published no root cause. It had not said whether affected traffic was blocked, timed out or bypassed, or whether any emergency path was affected. Nothing in this post depends on those answers.&lt;/p&gt;

&lt;p&gt;Cloudflare's post-mortem for its June 12, 2025 outage documents a mechanism for that incident. Gateway retrieved user identity and device posture from Workers KV. DNS queries with identity-based rules failed when it could not, and the proxy path was designed to fail closed so traffic could not bypass customer-configured rules. The WARP emergency disconnect override became unavailable because it depended on the same storage service. That documents 2025. It does not explain 2026, and this post does not use it that way.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Statement&lt;/th&gt;
&lt;th&gt;Basis&lt;/th&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gateway traffic was affected when an identity policy was in the decision; policies with none were unaffected&lt;/td&gt;
&lt;td&gt;Cloudflare status page, October 20, 2026&lt;/td&gt;
&lt;td&gt;Vendor-stated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Incident ran 03:20 to 06:32 UTC; fix implemented 04:44; edge restored by 05:46&lt;/td&gt;
&lt;td&gt;Cloudflare status page&lt;/td&gt;
&lt;td&gt;Vendor-stated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Root cause; whether traffic was blocked, timed out or bypassed; whether any emergency path was affected&lt;/td&gt;
&lt;td&gt;None published as of November 6&lt;/td&gt;
&lt;td&gt;Not established&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;June 2025: identity-based rules failed without identity; proxy path fail-closed by design; WARP override shared the dependency&lt;/td&gt;
&lt;td&gt;Cloudflare post-mortem, June 12, 2025&lt;/td&gt;
&lt;td&gt;Vendor-stated, 2025 only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The matrix and safeguards below&lt;/td&gt;
&lt;td&gt;Architect's design reasoning&lt;/td&gt;
&lt;td&gt;Inference: proposed design, not a vendor implementation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;What follows is a &lt;a href="https://www.rack2cloud.com/cloud-strategy/" rel="noopener noreferrer"&gt;cloud architecture strategy&lt;/a&gt; argument about policy fail mode.&lt;/p&gt;

&lt;h2&gt;
  
  
  Policy Fail Mode Is Decided Per Rule
&lt;/h2&gt;

&lt;p&gt;An identity-conditioned rule carries a dependency its text does not show. Before the evaluator can match the rule, it has to obtain who is asking, and often which groups that person belongs to and what state their device is in. When it cannot, it has a short list of options: deny, skip the rule and fall through, allow, or apply a fallback someone defined in advance. Which one happens is the policy fail mode, and it belongs to each rule, not to the integration.&lt;/p&gt;

&lt;p&gt;Where nobody decided, the behavior came from the vendor's defaults, and the rule was written for the case where identity evaluates. &lt;a href="https://www.rack2cloud.com/cloud-governance-failure-domains/" rel="noopener noreferrer"&gt;Cloud Governance Failure Domains&lt;/a&gt; describes the condition: policy engines sit in the execution path, so they carry a runtime, a failure mode and a blast radius. That post covers a wrong decision cascading downward; this one covers a correct decision whose input has gone missing. The &lt;a href="https://www.rack2cloud.com/cloud-architecture-learning-path/governance-architecture/" rel="noopener noreferrer"&gt;Strategic Governance&lt;/a&gt; stage anchors the governance side.&lt;/p&gt;

&lt;p&gt;Composition sets the blast radius. The October 20 statement describes a partial outage: rules with an identity condition were affected, rules without one were not. Whatever the cause, the failure follows the rule set. An estate that put an identity condition on every rule has made identity evaluation a single point of failure for all enforcement. An estate with a mix has a partial, predictable failure, but only if someone mapped which traffic classes sit on which side of the line.&lt;/p&gt;

&lt;p&gt;Identity-free by selector isn't identity-independent by dependency. A rule with no identity selector can still depend on inputs retrieved the way identity is: group membership synchronized from the directory, device posture, session or registration state, or a shared store behind several of those. The 2025 post-mortem documents identity and device posture coming from one storage service in that incident. For October 2026 nothing is documented, and this post does not assume it. The architect's task is to map, per rule, every input the evaluator needs and where each is fetched from, by &lt;a href="https://www.rack2cloud.com/cloud-architecture-learning-path/dependency-architecture/" rel="noopener noreferrer"&gt;tracing dependencies&lt;/a&gt; instead of reading the rule. That inventory is the matrix below.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Policy Failure-Mode Matrix
&lt;/h2&gt;

&lt;p&gt;Written down per traffic class, a policy fail mode becomes a design decision: what happens when identity cannot be evaluated, what stays reachable, and who can authorize recovery. The matrix is the artifact to take into a design review.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Traffic class&lt;/th&gt;
&lt;th&gt;Evaluation failure behavior&lt;/th&gt;
&lt;th&gt;Permitted fallback scope&lt;/th&gt;
&lt;th&gt;Recovery authority&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Privileged administration&lt;/td&gt;
&lt;td&gt;Deny. Identity cannot be evaluated, so no identity-conditioned access is granted.&lt;/td&gt;
&lt;td&gt;None through the normal path. Access only through a separately authenticated break-glass path with its own logging.&lt;/td&gt;
&lt;td&gt;Named break-glass holders, acting under the written procedure.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sensitive applications&lt;/td&gt;
&lt;td&gt;Deny.&lt;/td&gt;
&lt;td&gt;None by default. Any exception is named, owned and expires.&lt;/td&gt;
&lt;td&gt;Designated security authority.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Standard workforce access&lt;/td&gt;
&lt;td&gt;Explicitly defined restricted behavior.&lt;/td&gt;
&lt;td&gt;A named allowlist of destinations, everything else denied. Ends on restoration or at a stated time limit, whichever comes first. Every decision logged.&lt;/td&gt;
&lt;td&gt;Approved operator, activating under the written procedure.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Identity-independent services&lt;/td&gt;
&lt;td&gt;No identity dependency, established by tracing every input, not by reading the rule.&lt;/td&gt;
&lt;td&gt;Unchanged. Continue under existing controls.&lt;/td&gt;
&lt;td&gt;Normal operations.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;em&gt;Proposed design examples, not any vendor’s implementation.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Each row answers what a default never does: what the rule does, what remains reachable, and who may change that, in what role, under what trigger. A row with only the traffic class filled in is a default with a label on it.&lt;/p&gt;

&lt;h3&gt;
  
  
  Where Deny Is Correct
&lt;/h3&gt;

&lt;p&gt;Privileged administration and sensitive applications are the rows where the boundary is the product. If identity cannot be evaluated, nothing establishes that the requester is entitled to the access, and refusing is the outcome the rule exists to produce. Deny is right here, but not complete: the people responding to the incident need privileged access too, which is why the first row names a break-glass path and the second a designated security authority, not whoever holds the pager.&lt;/p&gt;

&lt;h3&gt;
  
  
  Fallback Needs a Boundary and an Exit
&lt;/h3&gt;

&lt;p&gt;The third row is where designs go soft, and where silent widening starts. "Restricted behavior" is not a fail mode until it says what is restricted. The fallback needs an explicit boundary, meaning which destinations and which traffic, and an exit condition: restoration of identity evaluation, a hard time limit or a named person closing it, whichever comes first. A bounded last-known-identity window is one option, but the cache is itself a dependency and an authorization decision, and needs the same boundary and exit. Some estates will choose deny here too, legitimately, if the outage cost is priced.&lt;/p&gt;

&lt;p&gt;A restricted workforce fallback that never expires is just another permanent exception. It is the failure state &lt;a href="https://www.rack2cloud.com/configuration-standards-emergency-changes/" rel="noopener noreferrer"&gt;the Emergency Reconciliation Gap&lt;/a&gt; names for configuration changes made during incidents, moved onto the policy plane: an authorized exception with no named owner and no closure deadline becomes the steady state without anyone deciding it should.&lt;/p&gt;

&lt;h2&gt;
  
  
  An Identity-Free Policy Is Not a Safe Policy
&lt;/h2&gt;

&lt;p&gt;The usual advice for an identity outage is to keep an identity-free baseline: a rule set that keeps working when identity does not. The advice is sound, and it hides a trap. A policy that does not ask who is asking can only grant by traffic characteristics, so it is coarse by construction. Activating it during an incident does not restore the normal boundary. It substitutes a blunter one, and the substitution has to be designed as carefully as the original.&lt;/p&gt;

&lt;p&gt;Four things make an identity-free policy safe to activate. A least-privilege boundary narrower than the identity-conditioned policy it replaces, because it has lost the ability to tell users apart. A definition of permitted traffic that names destinations, not categories of convenience. An expiration condition that ends the policy without anyone remembering to. And an audit trail that records who activated it, what it permitted and for how long, because the decisions made under it are the ones reviewed afterward.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;⚠ &lt;strong&gt;Common mistake:&lt;/strong&gt; Treating the identity-free baseline as the safe state. It is the available state. Without its own boundary, expiry and audit trail, activating it trades a loud outage for an unreviewed policy.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Fail-Closed Protects the Boundary. It Doesn't Restore Availability.
&lt;/h2&gt;

&lt;p&gt;Fail-closed is a security outcome. For the deny rules it is the correct one, and Cloudflare's 2025 post-mortem states it as design intent for the Gateway proxy path. What it does not do is return anyone to work. The boundary holds, legitimate access stops, and availability returns only when the dependency recovers or someone changes what enforcement does.&lt;/p&gt;

&lt;p&gt;Each of those is decided by default if it is not decided on purpose. Waiting for the vendor accepts the outage’s duration: on October 20 the status page reported edge functionality restored about two and a half hours after the incident opened, with Access API errors still elevated at that point. Whether your organization can absorb that for the traffic classes behind those rules is a question to answer before the outage, with a number attached. Changing what enforcement does needs a mechanism, an authority and a boundary, which is what the last two columns of the matrix are for.&lt;/p&gt;

&lt;p&gt;Neither outcome is sufficient architecture by itself. Fail-closed with no designed recovery path is an outage the organization accepted without knowing it. A fail-open default is worse in a different way: it is silent widening, authorization broadening with nothing to announce it. The architecture is the pair: a deliberate policy fail mode per rule, and a recovery path that is tested and authorized.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F62545u18i43udww2jper.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F62545u18i43udww2jper.jpg" alt="fail-closed enforcement boundary intact, with an empty recovery authority slot and no path back to access" width="799" height="413"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The boundary held. Nobody was designated to open it.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;This is the enforcement-plane version of the problem &lt;a href="https://www.rack2cloud.com/identity-system-single-point-of-failure/" rel="noopener noreferrer"&gt;Your Identity System Is Your Biggest Single Point of Failure&lt;/a&gt; describes for operators, where an identity provider goes silent and locks the people out. Here the identity provider can be healthy while the plane that consumes its answers cannot get them. The dependency is the same and the place it breaks is different, so the break-glass accounts that post recommends do not by themselves recover user traffic.&lt;/p&gt;

&lt;p&gt;Fail-closed is also the failure people report. The opposite failure produces no reports. &lt;a href="https://www.rack2cloud.com/enterprise-browser-infrastructure/" rel="noopener noreferrer"&gt;Enterprise Browser Infrastructure&lt;/a&gt; describes enforcement that stops synchronizing while availability stays green, so nobody notices until an audit does. A policy fail mode that widens access quietly belongs to that family, which is why the first safeguard below is about outcomes, not labels.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three Safeguards for Policy Fail Mode
&lt;/h2&gt;

&lt;p&gt;Three safeguards apply to every row of the matrix.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx1g2fk84t5nztme1cv27.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx1g2fk84t5nztme1cv27.jpg" alt="emergency override fed by a separate retrieval path, while the policy evaluator and the override's original feed depend on the same failed identity retrieval" width="800" height="423"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The override needs a retrieval path the failed dependency cannot reach.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  01 — NO SILENT WIDENING
&lt;/h3&gt;

&lt;p&gt;Loss of identity evaluation must never produce broader authorization than the explicitly approved fallback permits. A condition that normally restricts access disappears, and the remaining policy permits more than anyone intended. The test is the outcome, not the label: fail-open is acceptable only as the approved fallback, with a boundary and an exit.&lt;/p&gt;

&lt;h3&gt;
  
  
  02 — NO SHARED EMERGENCY DEPENDENCY
&lt;/h3&gt;

&lt;p&gt;The override cannot depend on the retrieval path that failed. Trace every input the emergency path needs to the same standard as the identity-independent row: where it is fetched from, and whether that source is the one in the incident.&lt;/p&gt;

&lt;h3&gt;
  
  
  03 — NO AMBIGUOUS AUTHORITY
&lt;/h3&gt;

&lt;p&gt;Someone holds, by role, the authority to activate the alternative access path, someone can end it, and the activation has been rehearsed. Authority that is assigned, documented and exercised survives the incident. Authority that is assumed does not.&lt;/p&gt;

&lt;p&gt;Safeguards 2 and 3 apply to the policy plane what &lt;a href="https://www.rack2cloud.com/disaster-recovery-authority/" rel="noopener noreferrer"&gt;Recovery Authority Fragmentation&lt;/a&gt; establishes for recovery in general: authority that shares a dependency with the thing it governs shares its blast radius. The 2025 post-mortem shows the first half: the override that let a user disconnect depended on the same storage service as the identity retrieval. Whether an equivalent path was affected on October 20 has not been stated, and the safeguard does not need the answer. It needs the trace.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to Test
&lt;/h2&gt;

&lt;p&gt;Policy fail mode needs an inventory and five answers per rule.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Which rules carry an identity condition, and which traffic classes do they protect?&lt;/li&gt;
&lt;li&gt;When identity cannot be evaluated, what does each rule do, and who chose that?&lt;/li&gt;
&lt;li&gt;For each rule you call identity-independent, what inputs does it fetch, and from where?&lt;/li&gt;
&lt;li&gt;What does the fallback permit, and what ends it?&lt;/li&gt;
&lt;li&gt;Who can activate it, does the activation path touch the dependency that failed, and when was it last exercised?&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;Policy fail mode is an architecture decision. What an identity-conditioned rule does when identity cannot be evaluated is chosen per traffic class, with a boundary, an exit and an owner, or it is inherited from a default and discovered during an incident.&lt;/p&gt;

&lt;p&gt;The real problem is not that identity evaluation fails. It will. The problem is that composition decides how much fails with it, that fail-closed gets mistaken for a recovery plan, and that the emergency path is so often built on the dependency it exists to bypass. An identity-free baseline does not fix any of that on its own. It is a different boundary, and it needs its own limits.&lt;/p&gt;

&lt;p&gt;No silent widening, no shared emergency dependency, no ambiguous authority. Those three constraints turn a default into a design. Everything else in the matrix is a choice an architect can defend in a review.&lt;/p&gt;

&lt;p&gt;Failing closed protects the boundary. Failing well is deciding, in advance, who can safely open it again.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/cloud-strategy/" rel="noopener noreferrer"&gt;Cloud Architecture Strategy&lt;/a&gt; — the Cloud Strategy pillar hub: control plane governance, dependency ownership and the decision frameworks this argument sits inside.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/cloud-architecture-learning-path/governance-architecture/" rel="noopener noreferrer"&gt;Strategic Governance&lt;/a&gt; — the Cloud Architecture learning path stage that anchors governance operating as infrastructure, the premise behind policy fail mode.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/cloud-architecture-learning-path/dependency-architecture/" rel="noopener noreferrer"&gt;Dependency Architecture&lt;/a&gt; — the stage on mapping and owning dependencies, the method for tracing what a rule actually needs to evaluate.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/cloud-governance-failure-domains/" rel="noopener noreferrer"&gt;Cloud Governance Failure Domains&lt;/a&gt; — policy engines in the execution path and the cascade from a wrong decision; this post covers a correct decision with a missing input.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/identity-system-single-point-of-failure/" rel="noopener noreferrer"&gt;Your Identity System Is Your Biggest Single Point of Failure&lt;/a&gt; — the operator-side counterpart: break-glass access when the identity provider goes silent.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/enterprise-browser-infrastructure/" rel="noopener noreferrer"&gt;Enterprise Browser Infrastructure&lt;/a&gt; — the quiet counterpart: enforcement that stops while availability stays green.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/disaster-recovery-authority/" rel="noopener noreferrer"&gt;Disaster Recovery Authority&lt;/a&gt; — Framework #144: recovery authority that shares a dependency with what it governs shares its blast radius.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/configuration-standards-emergency-changes/" rel="noopener noreferrer"&gt;Configuration Standards Emergency Changes&lt;/a&gt; — the Emergency Reconciliation Gap: an authorized exception with no owner and no closure deadline becomes permanent.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.cloudflarestatus.com/incidents/0eqtn0khkz87" rel="noopener noreferrer"&gt;Cloudflare Zero Trust connectivity, October 20, 2026&lt;/a&gt; — Cloudflare's status-page record of the incident: the primary source for every October 2026 statement here.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://blog.cloudflare.com/cloudflare-service-outage-june-12-2025/" rel="noopener noreferrer"&gt;Cloudflare service outage, June 12, 2025&lt;/a&gt; — Cloudflare's post-mortem: the documented mechanism, fail-closed intent and shared override dependency for 2025.&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/policy-fail-mode/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>cloud</category>
      <category>devops</category>
      <category>architecture</category>
    </item>
    <item>
      <title>Managed Compute Patching Is Not Managed Remediation</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Thu, 05 Nov 2026 14:00:25 +0000</pubDate>
      <link>https://dev.to/ntctech/managed-compute-patching-is-not-managed-remediation-1d3g</link>
      <guid>https://dev.to/ntctech/managed-compute-patching-is-not-managed-remediation-1d3g</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft4wh2jwzv974r9m1ihhb.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft4wh2jwzv974r9m1ihhb.jpg" alt="Managed compute patching gap: three service lanes where the AWS fix stops short of existing workloads" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
Managed compute patching means AWS ships the fixed platform. It does not mean your running workloads are on it.&lt;/p&gt;

&lt;p&gt;On October 30, AWS published &lt;a href="https://aws.amazon.com/security/security-bulletins/2026-160-aws/" rel="noopener noreferrer"&gt;bulletin 2026-160-AWS&lt;/a&gt; for CVE-2026-98432, a Linux kernel IPv4 TCP use-after-free, and has updated it since, most recently on November 4. The bulletin lists what AWS is releasing for Fargate, ECS Managed Instances, EKS Auto Mode, Bottlerocket and RDS, and it reads like a remediation statement. For three of those services it is closer to a start date: new compute gets the fix, while existing compute moves through provider-managed lifecycle processes shaped by service settings and workload constraints.&lt;/p&gt;

&lt;p&gt;This post is about that interval, not the flaw. The bulletin says AWS has no evidence the issue has been leveraged, and the exploit conditions are out of scope here.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the AWS Bulletin Confirms and What It Leaves to You
&lt;/h2&gt;

&lt;p&gt;The bulletin gives each service a different kind of statement. For Fargate and ECS Managed Instances, compute launched after 17:00 UTC on November 1 (9:00 AM PST in the bulletin) uses the patched platform version or AMI. For EKS Auto Mode, AWS released updated AMIs on October 31 and says Auto Mode replaces existing nodes automatically, subject to your NodePool disruption budgets. Bottlerocket and RDS get dates of their own, shown in the table below.&lt;/p&gt;

&lt;p&gt;Everything in that list is about what exists. None of it is about what is running, and what exists is all that managed compute patching can promise on its own. For existing compute, each managed service hands the timing of the last step to a setting: Fargate tasks are retired per your task retirement settings, ECS Managed Instances drain 14 days after launch or earlier if event windows say so, and Auto Mode replaces nodes automatically, subject to your NodePool disruption budgets. Those are lifecycle and governance questions of the kind the &lt;a href="https://www.rack2cloud.com/modern-infrastructure-iac-strategy-guide/" rel="noopener noreferrer"&gt;modern infrastructure and IaC architecture&lt;/a&gt; pillar covers. The fix is a provider artifact, and the state it has to reach is yours.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Managed Compute Patching Gap
&lt;/h2&gt;

&lt;p&gt;The managed compute patching gap is the interval between a provider making a security fix available and your existing workloads running on remediated compute. AWS controls when the replacement revision exists and how the managed service applies it. You control some of the conditions that shape when existing workloads move: retirement timing, maintenance windows, disruption budgets and deployment behavior. The gap sits between provider availability and workload adoption, with responsibility split across both sides.&lt;/p&gt;

&lt;p&gt;It is not an interval this site has measured before. &lt;a href="https://www.rack2cloud.com/patch-visibility-debt/" rel="noopener noreferrer"&gt;Patch visibility debt&lt;/a&gt; is the time between attacker activity and the confirmation signals that set remediation priority. &lt;a href="https://www.rack2cloud.com/patch-rollback-risk/" rel="noopener noreferrer"&gt;Patch rollback risk&lt;/a&gt; is the question of who may reverse a remediation once it breaks something. Neither measures how long existing compute stays on the old revision after the provider has shipped the new one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three Services, Three Levers
&lt;/h2&gt;

&lt;p&gt;Each service splits the last step differently. The table names what AWS ships, what moves existing compute, which lever you hold, and what the sources leave open. Bottlerocket and RDS appear as contrast rows only.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Service&lt;/th&gt;
&lt;th&gt;What AWS ships&lt;/th&gt;
&lt;th&gt;What moves existing compute&lt;/th&gt;
&lt;th&gt;Your lever&lt;/th&gt;
&lt;th&gt;Left open by the sources&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Fargate&lt;/td&gt;
&lt;td&gt;Patched platform version; tasks launched after 17:00 UTC on Nov 1 use it&lt;/td&gt;
&lt;td&gt;Retirement of the older revision, or your own replacement&lt;/td&gt;
&lt;td&gt;Retirement wait period or event windows; replacing tasks sooner&lt;/td&gt;
&lt;td&gt;Whether AWS overrode the wait period for this CVE&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ECS Managed Instances&lt;/td&gt;
&lt;td&gt;Updated AMI; instances launched after the same cutoff use it&lt;/td&gt;
&lt;td&gt;Drain from day 14 after launch; deregistration at day 21&lt;/td&gt;
&lt;td&gt;Event windows, service deployment settings, manual drain&lt;/td&gt;
&lt;td&gt;Whether early draining for vulnerability detection applied here&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;EKS Auto Mode&lt;/td&gt;
&lt;td&gt;Updated Auto Mode AMIs, released Oct 31&lt;/td&gt;
&lt;td&gt;Automatic drift replacement when a new AMI releases; node expiry&lt;/td&gt;
&lt;td&gt;NodePool budgets, expiry and grace period; PodDisruptionBudgets&lt;/td&gt;
&lt;td&gt;No replacement deadline; budgets and PodDisruptionBudgets set the pace&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bottlerocket (contrast)&lt;/td&gt;
&lt;td&gt;Patched kernel AMIs (v1.68.1), released Oct 31&lt;/td&gt;
&lt;td&gt;Nothing described; you move to the latest AMI&lt;/td&gt;
&lt;td&gt;Entirely yours&lt;/td&gt;
&lt;td&gt;No drain or retirement timeline in the bulletin&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RDS and Aurora (contrast)&lt;/td&gt;
&lt;td&gt;Mitigations deploying; host OS patch available Nov 6&lt;/td&gt;
&lt;td&gt;Not described in the bulletin&lt;/td&gt;
&lt;td&gt;Keep database ports off the internet&lt;/td&gt;
&lt;td&gt;Timing for your own instances&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Fargate: Retirement Timing
&lt;/h3&gt;

&lt;p&gt;On Fargate, managed compute patching is retirement timing. New tasks run on the newest platform revision, and older revisions are retired once a newer one exists. AWS sends a retirement notice with a date, and you cannot opt out of retirement. The &lt;a href="https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-maintenance.html" rel="noopener noreferrer"&gt;task maintenance documentation&lt;/a&gt; says so directly. What you set is when it starts. The &lt;a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-account-settings.html" rel="noopener noreferrer"&gt;account setting&lt;/a&gt; &lt;code&gt;fargateTaskRetirementWaitPeriod&lt;/code&gt; takes 0, 7 or 14 days, with 7 as the default; 0 retires affected tasks immediately after the notice. If you enable EC2 event windows for Fargate, they take precedence over the wait period, and replacement inside a window is best effort. With event windows, Fargate also keeps tasks running at least three days before retiring them in the next available window, except for user-initiated stops or critical health events such as hardware degradation. You can also replace tasks yourself at any time, for example with a forced new deployment of the service.&lt;/p&gt;

&lt;p&gt;What happens at retirement depends on the task type. ECS replaces service tasks automatically, starting the replacement first unless the service's maximum percent is 100. Standalone tasks, such as those started by RunTask, EventBridge, Batch or Step Functions, are stopped on or after the retirement date and are not replaced. For those, "no action required" is not true.&lt;/p&gt;

&lt;p&gt;AWS has also said it may override the wait period for a critical security update and retire affected tasks immediately. That statement comes from an &lt;a href="https://aws.amazon.com/blogs/containers/improving-operational-visibility-with-aws-fargate-task-retirement-notifications" rel="noopener noreferrer"&gt;AWS blog post&lt;/a&gt;, not the current task maintenance page, and the bulletin does not say whether it applied to this CVE.&lt;/p&gt;

&lt;h3&gt;
  
  
  ECS Managed Instances: Day 14 and Day 21
&lt;/h3&gt;

&lt;p&gt;Per the &lt;a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/managed-instances-patching.html" rel="noopener noreferrer"&gt;ECS Managed Instances patching documentation&lt;/a&gt;, ECS Managed Instances set a drain schedule at launch. Draining begins at day 14, and event windows you configure can pull it earlier on a best-effort basis. ECS also lists detection of security vulnerabilities as a reason to drain early, and the bulletin does not say whether that applied here. Until the scheduled drain time, ECS keeps placing tasks on the instance as it normally would, so an instance launched on the old AMI can keep receiving new tasks until its drain begins. Event window changes apply only to instances launched afterward, so editing a window today does not shorten the clock for instances already running.&lt;/p&gt;

&lt;p&gt;Once draining starts, ECS honors your service deployment configuration. If draining has not finished by day 21, ECS deregisters the instance and stops what remains. Your deployment settings decide whether a drain finishes cleanly before then or is cut off at the limit.&lt;/p&gt;

&lt;h3&gt;
  
  
  EKS Auto Mode: Drift, Budgets and the Grace Period
&lt;/h3&gt;

&lt;p&gt;On Auto Mode, managed compute patching is a drift and budget question. AWS controls the AMI. Auto Mode releases a new one roughly weekly, and a new AMI marks existing nodes as drifted so they are replaced. By default, per the &lt;a href="https://docs.aws.amazon.com/eks/latest/userguide/create-node-pool.html" rel="noopener noreferrer"&gt;EKS Auto Mode node pool documentation&lt;/a&gt;, a NodePool has a single disruption budget of 10 percent of nodes, nodes expire after 336 hours, and a 24-hour termination grace period is applied to the node's claim when you set none. Those are defaults, not a deadline for this CVE. On custom NodePools you can change the budget, the expiry and the grace period. The built-in system and general-purpose pools cannot be modified. A budget can also be scoped by reason, so a zero-node budget for drifted nodes outside a maintenance window can stop drift replacement from starting there.&lt;/p&gt;

&lt;p&gt;Pods are the other lever. Pods that block eviction, through a PodDisruptionBudget or a do-not-disrupt annotation, are respected while a node drains. The termination grace period is the ceiling on that: &lt;a href="https://repost.aws/articles/ARbff3_8A_R7uiPMpCfjHznw/eks-auto-mode-and-maintenance-window-for-drifted-nodes" rel="noopener noreferrer"&gt;AWS guidance on re:Post&lt;/a&gt; says blocking pods are honored until it is reached, and then they are deleted. The &lt;a href="https://www.rack2cloud.com/kubernetes-scheduler-stuck-cpu-fragmentation/" rel="noopener noreferrer"&gt;Kubernetes scheduler post&lt;/a&gt; describes a different failure, a strict PodDisruptionBudget that leaves a node drain hanging indefinitely during an upgrade. Auto Mode does not leave it open-ended. With a grace period set or defaulted, the block ends. That limits the delay. It does not remove it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the Customer Lever Becomes the Delay
&lt;/h2&gt;

&lt;p&gt;Every lever above exists for a good reason: a wait period or event window lets a task stop when you chose, and a disruption budget keeps capacity serving while nodes turn over. Each one also lengthens the time existing compute stays on the old revision.&lt;/p&gt;

&lt;p&gt;The providers set lifecycle controls, and they matter. Fargate retirement cannot be declined, ECS deregisters at day 21, and Auto Mode nodes expire after 336 hours by default. Those controls shape how long existing compute can stay on an older revision, but they are not interchangeable, and their defaults are not a universal remediation deadline. The practical question in managed compute patching is where your workloads sit within the lifecycle AWS provides and the settings your team has chosen. A team that tuned a wait period or a budget for availability made a patch-timing decision at the same moment, whether or not it saw it that way.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq1w2ure27aooxax57fbb.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq1w2ure27aooxax57fbb.jpg" alt="Provider lifecycle controls shape the managed compute patching gap while customer settings decide where existing compute sits" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The lifecycle controls are the provider's. The position inside them is set by your settings.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;This is the same shape &lt;a href="https://www.rack2cloud.com/ca-distrust-propagation/" rel="noopener noreferrer"&gt;CA distrust propagation&lt;/a&gt; describes for client classes. A vendor's action establishes what the vendor changed, not that each of your clients has consumed it, and the update owner differs by class. Managed compute adds a twist. Because the provider also runs the replacement, it is easier to assume the provider owns the last step too.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Derived, not published:&lt;/strong&gt; Assume one ECS Managed Instance launched just before 17:00 UTC on November 1. On the documented schedule, draining begins at day 14, around November 15, and ECS deregisters the instance if draining has not finished by day 21, around November 22. This is a single example built from the documented 14-day and 21-day behavior and an assumed launch time. AWS has not published it as a guarantee. Event windows or early draining can shorten it, and instances launched at other times follow other dates. It is not a worst case for your estate.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  What to Inventory Before the Next Provider Bulletin
&lt;/h2&gt;

&lt;p&gt;The managed compute patching gap is measurable, but only if someone has written down the settings that position it. Four items, in order.&lt;/p&gt;

&lt;h3&gt;
  
  
  01 — Name the compute classes
&lt;/h3&gt;

&lt;p&gt;List which workloads run on Fargate, ECS Managed Instances, EKS Auto Mode and compute you patch yourself, such as Bottlerocket. On Fargate, separate service tasks from standalone tasks, because only service tasks are replaced for you. Each class has a different owner for the last step.&lt;/p&gt;

&lt;h3&gt;
  
  
  02 — Record the settings that position the gap
&lt;/h3&gt;

&lt;p&gt;For each class, record the values in force: the Fargate retirement wait period and event windows, ECS event windows and service deployment settings, Auto Mode disruption budgets, expiry and termination grace period, and the PodDisruptionBudgets that block drains. Note which are defaults and which someone changed.&lt;/p&gt;

&lt;h3&gt;
  
  
  03 — Decide how you will show remediation
&lt;/h3&gt;

&lt;p&gt;Evidence of remediation comes from the running compute, not from the provider's release notice or your deployment record. A shipped revision tells you what AWS released. The revision a workload is running is a separate fact, and it is the one that answers whether you are remediated.&lt;/p&gt;

&lt;h3&gt;
  
  
  04 — Assign an owner and an expiry
&lt;/h3&gt;

&lt;p&gt;Name the role that can change these settings during a security event. Any setting that defers replacement beyond the provider's default should carry an owner and an expiry, so a deferral made for availability does not outlive the reason for it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What This Post Does Not Establish
&lt;/h2&gt;

&lt;p&gt;It does not assess exploitation or exploit conditions. The bulletin says AWS has no evidence the issue has been leveraged and says researchers have demonstrated remote code execution against hosts with an MPTCP listener; this post evaluates neither statement. It does not establish whether AWS overrode the Fargate wait period or drained ECS Managed Instances early for this CVE, or what the maximum lifetime of an Auto Mode node is beyond the documented 336-hour default. The November 15 and November 22 dates above are one example, not a bound. The bulletin was most recently updated on November 4, so read the current bulletin before acting on any date here.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;Managed compute patching ends where the provider's fix exists. Whether your workloads are running that fix is a separate state, and it is decided by lifecycle settings, not by the bulletin.&lt;/p&gt;

&lt;p&gt;The real problem is that the last step has three authors. The provider sets the defaults and the lifecycle controls, and in some cases it can override your settings. You hold some of the levers in between: wait periods, event windows, disruption budgets, grace periods and the PodDisruptionBudgets your applications ship with. Most estates cannot say which of those values is in force on which compute.&lt;/p&gt;

&lt;p&gt;In managed compute patching, the settings that protect availability are the same ones that position the exposure. That is not a flaw in them. It is why they need an owner and an expiry, and why the evidence that a workload is remediated has to come from the running compute.&lt;/p&gt;

&lt;p&gt;The provider ships the fix. The provider's lifecycle and your settings together determine when existing workloads receive it.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/managed-compute-patching/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>aws</category>
      <category>kubernetes</category>
      <category>devops</category>
      <category>cloud</category>
    </item>
    <item>
      <title>When You Cannot Prove Integrity, Rebuilding Becomes the Recovery Decision</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Tue, 03 Nov 2026 13:04:00 +0000</pubDate>
      <link>https://dev.to/ntctech/when-you-cannot-prove-integrity-rebuilding-becomes-the-recovery-decision-4d9n</link>
      <guid>https://dev.to/ntctech/when-you-cannot-prove-integrity-rebuilding-becomes-the-recovery-decision-4d9n</guid>
      <description>&lt;p&gt;A recovery decision is only as defensible as the evidence you kept before the incident. On October 20 and 21, an attacker tried to exploit a critical GitLab vulnerability, CVE-2026-97229, on Alpine Linux's own GitLab server, a week after GitLab released the fix. Alpine found no evidence that the attacker's code ran, and said it could not rule it out. On October 27 it began rebuilding the server from scratch. The interesting part is not the rebuild. It is that the recovery decision came from a deficit, not a finding: Alpine could not obtain the evidence that would have let it keep the server.&lt;/p&gt;

&lt;p&gt;This post is not an advisory on GitLab, and it does not claim that Alpine was compromised or that it was not. Alpine says it found no evidence of compromise and could not rule one out. That distinction is the whole argument. When absence of evidence cannot be read as evidence of absence, what you decide is set by what you retained beforehand, and by whose records those are.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbecsul1rsmyxaecfmlme.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbecsul1rsmyxaecfmlme.jpg" alt="recovery decision: a platform node with source history and packages attested from outside records, signing keys kept off the platform by design, code execution and egress marked rebuild, and database access marked not established" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The same incident, three answers: what outside records can attest, what only missing host-level records could have settled, and what Alpine's account leaves open.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Alpine Could Not Rule Out
&lt;/h2&gt;

&lt;p&gt;Alpine's account, published October 28 and updated October 30, is unusually specific. GitLab released its fix on October 12 (19.3.5 on the 19.3 branch Alpine ran). An attacker registered an account on Alpine's instance on October 20 and began probing. During the attack on October 20 and 21, Alpine says, GitLab worker processes crashed 8 times, and in 4 of those crashes the attacker got partway through the exploit before failing at the next step. Alpine then looked for signs of a successful run. Timing probes showed no delays, and it found no unexpected files, keys, hooks or processes.&lt;/p&gt;

&lt;p&gt;Those are negative checks. They support a conclusion; they cannot establish one, and Alpine's own reasoning says why: a successful exploit would leave no crash and no trace in the web logs. The crashes belong to the attempts that failed. An attempt that worked would look like silence.&lt;/p&gt;

&lt;p&gt;The records that could have closed the question were missing. Alpine says some logs from the attack day are gone, including background job logs and the old container's filesystem. It had no logging of outbound network connections, and its hosting provider keeps none either.&lt;/p&gt;

&lt;p&gt;Alpine's account does not establish when the missing attack-day logs became unavailable. Some of that evidence may have been lost during the response rather than through a retention decision made beforehand. The outbound-logging gap is different: Alpine had no outbound connection logging when the attack began.&lt;/p&gt;

&lt;p&gt;Alpine also reports that upgrading did not help as expected. In its account, the fix for this flaw is a Ruby patch that GitLab applies in its own official packages, and Alpine's image ran standard Ruby, so the October 22 upgrade to 19.3.6 did not close the hole. That is Alpine's explanation, and it is the only source this post relies on for it.&lt;/p&gt;

&lt;p&gt;Alpine's conclusion is narrow. It could not rule out code execution, and the evidence needed to confirm it no longer exists, so it rebuilt the instance from scratch. It closed GitLab to everyone except developers and build servers on October 27, finished the rebuild on October 28, reset all passwords, and replaced GitLab's secret keys.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Recovery Decision Was Set Before the Incident
&lt;/h2&gt;

&lt;p&gt;Most incident frameworks ask first whether the platform was compromised. Alpine could not answer that, and the reason was not skill or speed. The answer lives in records, and the records either existed before the incident or did not. &lt;a href="https://www.rack2cloud.com/data-protection-architecture-strategy-guide/" rel="noopener noreferrer"&gt;Data protection architecture&lt;/a&gt; usually frames recovery as the question of whether you can restore. This is the question underneath it: whether you can establish what you would be restoring into.&lt;/p&gt;

&lt;p&gt;The Environment Re-Trust phase in &lt;a href="https://www.rack2cloud.com/incident-recovery-process/" rel="noopener noreferrer"&gt;Incident Recovery Process&lt;/a&gt; demands that an environment be confirmed clean, not assumed clean, before it is declared safe. Alpine shows the case that phase does not resolve: confirmation that cannot be produced. The outcome is not a degraded re-trust. It is a different recovery decision. Rebuilding stops being a response to a finding and becomes the defensible position when the evidence for keeping the platform is unobtainable.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fogxncgkfwqtxtzb4ciuw.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fogxncgkfwqtxtzb4ciuw.jpg" alt="Alpine GitLab timeline from October 20 to 28 showing outside records available and no outbound connection logging" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The outbound logging gap was in place before the attack began.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Look at what would have answered the question. Whether anything left the host is a question for outbound connection logs. Whether the database was touched is a question for database connection logs. Alpine has announced both changes: it will record outbound connections from the GitLab host and database connections, and keep logs longer than 14 days. Those are announcements, not history. The outbound gap is established by Alpine's own account: it had no such logging. Whether earlier database-access records existed, the account does not say. This post does not claim that the logs would have changed the outcome. The claim is narrower: the outbound logging gap, in place before October 20, fixed one of the questions Alpine could not answer on October 27.&lt;/p&gt;

&lt;p&gt;The corollary is uncomfortable for any estate that runs a build or pipeline platform. CI/CD variables and runner tokens were among the things Alpine says could have been exposed had the attack succeeded, and in many environments &lt;a href="https://www.rack2cloud.com/ci-cd-control-plane-infrastructure/" rel="noopener noreferrer"&gt;the CI/CD system is the highest-privilege system in the estate&lt;/a&gt;. The rebuild-or-keep decision for that platform is being made now, in log retention and in what your provider keeps, whether or not anyone is treating it as a decision.&lt;/p&gt;

&lt;h2&gt;
  
  
  Records the Attacker Could Not Reach
&lt;/h2&gt;

&lt;p&gt;The other half of the account is what Alpine could establish. Its verification worked because it used records the attacker could not reach. Alpine checked every aports commit added October 20 to 22 against a developer's clone and all 45 builders, and found that each arrived through a recorded push. It found that every package built from October 20 to 24 came from commits as published on git.alpinelinux.org. And it found that the October 21 stable releases (3.21.9, 3.22.7, 3.23.7 and 3.24.3), including their ISO images and minirootfs tarballs, came from commits it verified against records outside GitLab. It ran those checks on October 24.&lt;/p&gt;

&lt;p&gt;The developer's clone and the 45 builders are verification sources here, not exposed assets. They are records Alpine says the attacker could not reach, and that is the property that made them usable for verification. This post calls them outside records, in Alpine's phrase, and not independent records. Alpine's page does not say how the package record on git.alpinelinux.org is isolated from GitLab, and this post does not assume it.&lt;/p&gt;

&lt;p&gt;One question needed no record at all. Alpine states that the keys that sign its packages and releases are not, and never have been, stored on the GitLab server. That is an answer by design: a compromised GitLab could not have exposed keys it never held. It is also the firmest statement in the account, because it does not depend on any log surviving.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.rack2cloud.com/evidence-layer-trust-boundary/" rel="noopener noreferrer"&gt;Your Logs Are Not Evidence If The Infrastructure That Produces Them Can Rewrite The Record&lt;/a&gt; draws the underlying line: evidence produced by infrastructure inside the blast radius can be shaped by whoever holds it. Alpine's verification avoided that by looking elsewhere. But outside records have a limit the Alpine account makes visible. They can show that what the platform produced is intact. They cannot show what happened on the platform. Whether code ran on the host, and whether anything left it, are not questions a developer's clone can answer. Only host-side and network-side records can. Alpine had no outbound connection logging, and says the evidence it would need to confirm code execution no longer exists.&lt;/p&gt;

&lt;p&gt;Who may reject a recovery claim is the separate question that Framework #167, &lt;a href="https://www.rack2cloud.com/recovery-evidence-boundary/" rel="noopener noreferrer"&gt;Recovery Evidence Boundary&lt;/a&gt;, owns, and that the &lt;a href="https://www.rack2cloud.com/data-protection-resiliency-learning-path/governance-recovery-assurance/" rel="noopener noreferrer"&gt;Governance &amp;amp; Recovery Assurance&lt;/a&gt; stage of the learning path builds on. This post asks an earlier one: what has to exist before a recovery decision can rest on evidence at all.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Rebuild-or-Attest Decision Table
&lt;/h2&gt;

&lt;p&gt;The table below turns the Alpine account into a recovery decision tool. It is this site's method, not Alpine's procedure. Each row is an integrity question you will be asked after an incident on a platform of this kind, the evidence that has to exist before the incident to answer it, and what its absence means. The Alpine column records Alpine's stated position as of its October 28 account and October 30 update.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Integrity question&lt;/th&gt;
&lt;th&gt;Evidence required before the incident&lt;/th&gt;
&lt;th&gt;Whose records&lt;/th&gt;
&lt;th&gt;Alpine's stated position&lt;/th&gt;
&lt;th&gt;Without it&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Did attacker code execute on the host?&lt;/td&gt;
&lt;td&gt;Host-side execution telemetry that records success; web logs alone cannot&lt;/td&gt;
&lt;td&gt;Held off the host or tamper-resistant&lt;/td&gt;
&lt;td&gt;Negative checks only (timing probes; file, key, hook and process review). A successful exploit would leave no crash and no web-log trace&lt;/td&gt;
&lt;td&gt;Rebuild the host&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Did anything leave the host?&lt;/td&gt;
&lt;td&gt;Outbound connection logs, kept longer than the detection window&lt;/td&gt;
&lt;td&gt;Network or provider layer, not the host&lt;/td&gt;
&lt;td&gt;No outbound connection logging; its hosting provider keeps none&lt;/td&gt;
&lt;td&gt;Rebuild; treat everything the host held as exposed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Was the database accessed?&lt;/td&gt;
&lt;td&gt;Database connection logs&lt;/td&gt;
&lt;td&gt;Database or network layer&lt;/td&gt;
&lt;td&gt;Lists database connection logging among its announced additions. Its account does not establish whether earlier database-access records existed or were sufficient&lt;/td&gt;
&lt;td&gt;If unavailable, database access cannot be established from this evidence; determine database exposure and recovery scope separately&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Is the source history intact?&lt;/td&gt;
&lt;td&gt;Copies held where the attacker cannot reach them&lt;/td&gt;
&lt;td&gt;Developer clones, builders&lt;/td&gt;
&lt;td&gt;Commits from Oct 20-22 checked against a developer's clone and all 45 builders&lt;/td&gt;
&lt;td&gt;Attest: the copies exist&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Are published packages and releases what the source says?&lt;/td&gt;
&lt;td&gt;Outside records matched to commits&lt;/td&gt;
&lt;td&gt;Records outside the platform, per Alpine&lt;/td&gt;
&lt;td&gt;Packages built Oct 20-24 matched published commits; Oct 21 releases verified against records outside GitLab on Oct 24&lt;/td&gt;
&lt;td&gt;Attest, for the window checked&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Were signing keys exposed?&lt;/td&gt;
&lt;td&gt;Keys held outside the platform by design&lt;/td&gt;
&lt;td&gt;Architecture, not a log&lt;/td&gt;
&lt;td&gt;Keys never stored on the GitLab server&lt;/td&gt;
&lt;td&gt;Attest: a design property&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;What could the platform have disclosed?&lt;/td&gt;
&lt;td&gt;An inventory of the secrets and data it held&lt;/td&gt;
&lt;td&gt;The platform's own inventory&lt;/td&gt;
&lt;td&gt;Six categories listed as "could have been exposed", including CI/CD variables, runner tokens and 2FA secrets&lt;/td&gt;
&lt;td&gt;Replace all of it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can later releases be verified without trusting the platform?&lt;/td&gt;
&lt;td&gt;Release verification that does not rely on the platform&lt;/td&gt;
&lt;td&gt;Signed tags&lt;/td&gt;
&lt;td&gt;Announced: signed release tags&lt;/td&gt;
&lt;td&gt;Attest going forward, once in place&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For the seventh row, Alpine lists six categories that could have been exposed had the attack succeeded: email addresses, names and IP addresses of all users; password hashes; 2FA secrets, CI/CD variables and runner tokens; webhook, integration and push mirror credentials; personal access tokens, stored as hashes; and private repositories along with confidential issues and merge requests. Alpine reset all passwords and replaced GitLab's secret keys. Replacing a set of secrets is a different job from enumerating it, and the second is the one that fails. &lt;a href="https://www.rack2cloud.com/revocation-unit-credential-rotation/" rel="noopener noreferrer"&gt;Rotating Credentials Is Not Revoking Them. The Revocation Unit Decides Whether You Can.&lt;/a&gt; covers what one revoke action has to enclose.&lt;/p&gt;

&lt;p&gt;Read down the last column and a rule appears. Attest what outside records can establish. Rebuild when the integrity of the host cannot be established from obtainable evidence. Rows four to six demonstrate what can be verified through records outside the affected platform or through architectural separation. Rows one and two demonstrate the evidence gaps Alpine identified: successful execution could not be ruled out, and outbound connections could not be reconstructed. The scope of the recovery decision follows: the platform, not the artifacts it produced. Alpine's account is of a rebuilt platform and of releases it verified against outside records. Row three is a separate design requirement. Alpine announced database connection logging, but its account does not establish whether earlier database-access records existed. Do not turn that uncertainty into a finding.&lt;/p&gt;

&lt;p&gt;To use the table before an incident, fill the second and third columns first for every platform in the estate that holds credentials or builds what you ship. Where does the record live, who can reach or alter it, and how long is it retained compared with how long detection takes? Every row you cannot fill represents a recovery limitation you have already accepted. Some gaps will force a rebuild. Others will require credential replacement, exposure assessment, or a narrower attestation. The important thing is to know which recovery decision each missing record will force before the incident arrives: rebuild, attest, rotate, or investigate further.&lt;/p&gt;

&lt;h2&gt;
  
  
  What This Does Not Argue
&lt;/h2&gt;

&lt;p&gt;This post does not argue that Alpine was compromised, or that it was not. Alpine found no evidence either way and says so, and nothing here upgrades that. It does not argue for rebuilding every platform after an attempted exploit; rows four to six are the counter-example, cases where a platform's output can be attested without rebuilding anything, and the recovery decision is not always rebuild. It does not argue that better logging would have changed what Alpine decided. Its announced logging changes show where the deficit was, not what the logs would have found.&lt;/p&gt;

&lt;p&gt;A rebuild is also not a clean bill on its own. &lt;a href="https://www.rack2cloud.com/persistent-access-survives-fix/" rel="noopener noreferrer"&gt;A rebuild can leave persistent access untouched&lt;/a&gt; in a layer the recovery never reached, which is why replacing secrets and verifying outputs belong inside the same decision as the rebuild.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;Alpine's decision is usually read as caution. It is better read as arithmetic. With the records that could answer it missing, the question of whether anything ran on the server had no obtainable answer, and a platform whose integrity cannot be established cannot be kept in service on hope. Rebuilding was the recovery decision that remained.&lt;/p&gt;

&lt;p&gt;The real problem is not the rebuild. A rebuild is expensive, but it is the one option that does not depend on evidence. The problem is that the recovery decision between rebuilding and attesting is made months earlier, in log retention settings, in what the hosting provider keeps, in whether the signing keys live on the platform at all. The quiet rows of the table, the signing keys and the outside copies, were settled by design, and they are the reason Alpine could say anything firm about its releases.&lt;/p&gt;

&lt;p&gt;You cannot establish an environment's integrity after an incident if you did not preserve the evidence for it beforehand. What you cannot establish, you rebuild. Retention is not a compliance chore attached to recovery. It is the input to the recovery decision, and by the time the incident arrives, that input is already fixed.&lt;/p&gt;

&lt;p&gt;Decide what you will need to prove, then keep the records that prove it somewhere an attacker controlling the platform cannot reach.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/data-protection-architecture-strategy-guide/" rel="noopener noreferrer"&gt;Data Protection Architecture&lt;/a&gt; — the pillar guide for recovery, resilience and evidence decisions this post sits in&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/data-protection-resiliency-learning-path/governance-recovery-assurance/" rel="noopener noreferrer"&gt;Governance &amp;amp; Recovery Assurance&lt;/a&gt; — the learning path stage where recovery claims are independently proven&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/evidence-layer-trust-boundary/" rel="noopener noreferrer"&gt;Your Logs Are Not Evidence If The Infrastructure That Produces Them Can Rewrite The Record&lt;/a&gt; — why records produced inside the blast radius cannot carry an investigation alone&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/recovery-evidence-boundary/" rel="noopener noreferrer"&gt;Your Recovery Program Has a Recovery Evidence Boundary&lt;/a&gt; — Framework #167: who has standing to reject a recovery claim&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/incident-recovery-process/" rel="noopener noreferrer"&gt;Incident Recovery Process: Why the Incident Isn’t Over After Restore&lt;/a&gt; — Environment Re-Trust: confirming a platform is clean before declaring it safe&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/revocation-unit-credential-rotation/" rel="noopener noreferrer"&gt;Rotating Credentials Is Not Revoking Them. The Revocation Unit Decides Whether You Can.&lt;/a&gt; — what one revoke action has to enclose so replacing secrets counts as closed&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/persistent-access-survives-fix/" rel="noopener noreferrer"&gt;The Server Was Fixed. Persistent Access Wasn’t.&lt;/a&gt; — recovery that succeeded on every layer except the one carrying access&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/ci-cd-control-plane-infrastructure/" rel="noopener noreferrer"&gt;Your CI-CD Pipeline Is Your Real Infrastructure Control Plane&lt;/a&gt; — why the pipeline holds more privilege than most teams govern it for&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://alpinelinux.org/posts/GitLab-security-incident-CVE-2026-97229.html" rel="noopener noreferrer"&gt;GitLab security incident: attempted exploitation of CVE-2026-97229&lt;/a&gt; — Alpine's account, published October 28 and updated October 30&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-2-released/" rel="noopener noreferrer"&gt;GitLab Critical Patch Release: 19.4.2, 19.3.5, 19.2.8&lt;/a&gt; — GitLab's October 12 release&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/recovery-decision-integrity-evidence/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>architecture</category>
      <category>infrastructure</category>
    </item>
    <item>
      <title>Backup Placement Is Not Production Placement</title>
      <dc:creator>NTCTech</dc:creator>
      <pubDate>Mon, 02 Nov 2026 13:06:41 +0000</pubDate>
      <link>https://dev.to/ntctech/backup-placement-is-not-production-placement-5ll</link>
      <guid>https://dev.to/ntctech/backup-placement-is-not-production-placement-5ll</guid>
      <description>&lt;p&gt;Backup placement is the part of a residency decision that a production-only review leaves unanswered. A production-only review approves one dataset in one place. Unless it explicitly includes the repository, copy-job target, replica and archive tier, it does not establish that those destinations meet the same residency boundary. Each of those destinations holds a placed copy of the same data, placed by whoever configured it, at the time they configured it.&lt;/p&gt;

&lt;p&gt;This post makes a narrow argument. It does not claim that a regulation requires backups to sit beside production, and it does not claim that any vendor fails to control where copies go. It argues that backup placement is its own architecture decision, and that it needs its own inventory, a named owner and a verification routine, evaluated apart from the production dataset.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh0hhlxfa276ifmwjtxgf.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh0hhlxfa276ifmwjtxgf.jpg" alt="backup placement — a residency boundary drawn around the production dataset, with backup, replica and archive copies outside it labelled placement not assessed" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The residency review covered one dataset. Its copies sit outside the boundary, unexamined.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Backup Placement Is Not Production Placement
&lt;/h2&gt;

&lt;p&gt;Production placement is a design-time decision. Someone chooses a region or a site, a review confirms it against the residency requirement, and the decision is recorded. Copy placement is a configuration-time decision. It is made when a backup job gets a target, when a replication policy gets a destination, when a tiering rule gets a bucket. The two decisions typically have different owners, different change paths and different review gates, so approving one says nothing about the other. That makes this a &lt;a href="https://www.rack2cloud.com/data-protection-architecture-strategy-guide/" rel="noopener noreferrer"&gt;data protection architecture&lt;/a&gt; question before it is a compliance one.&lt;/p&gt;

&lt;p&gt;Isolation is a different property again. A repository can be isolated by account and identity and still sit in a location nobody assessed. &lt;a href="https://www.rack2cloud.com/backup-blast-radius/" rel="noopener noreferrer"&gt;Your Backup System Is Part of the Blast Radius&lt;/a&gt; covers the isolation side of backup design; as that post puts it, isolation is measured by what the system depends on, not where it resides. The broader platform-level jurisdiction problem is covered in &lt;a href="https://www.rack2cloud.com/sovereign-cloud-vs-public-cloud-compliance-trap/" rel="noopener noreferrer"&gt;Sovereign Cloud vs. Public Cloud: Navigating Compliance in a Non-Deterministic Landscape&lt;/a&gt;. This post applies the question to the copies.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Copy Paths a Residency Review Misses
&lt;/h2&gt;

&lt;p&gt;A residency review that starts and ends with the production dataset covers one row of the inventory below. Backup placement is spread across every other row: each is a placed copy or a recovery destination, with its own point at which its location is decided.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Item&lt;/th&gt;
&lt;th&gt;Typically created by&lt;/th&gt;
&lt;th&gt;Where its placement is decided&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Production baseline (not a copy)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Application and platform owners&lt;/td&gt;
&lt;td&gt;Architecture and residency review&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Primary backup repository&lt;/td&gt;
&lt;td&gt;Backup administration&lt;/td&gt;
&lt;td&gt;Repository or storage target configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Secondary copy-job target&lt;/td&gt;
&lt;td&gt;Backup administration&lt;/td&gt;
&lt;td&gt;Copy job or policy settings&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Replica or DR target&lt;/td&gt;
&lt;td&gt;Backup or DR administration&lt;/td&gt;
&lt;td&gt;Replication policy or DR design&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Archive or cloud-tier copy&lt;/td&gt;
&lt;td&gt;Backup administration or storage teams&lt;/td&gt;
&lt;td&gt;Tiering, archive or cloud-target policy&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Restore target (a destination, not necessarily a stored copy)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Recovery operators, at restore time&lt;/td&gt;
&lt;td&gt;The recovery runbook or the restore operation itself&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The production baseline is the reference row. Every row below it has a placement decision of its own. A production review covers those decisions only when it explicitly includes them. A replica or DR target deserves its own line because it is built to be used during an outage, so the location that matters is the one the data is recovered from, not only the one it is stored in.&lt;/p&gt;

&lt;p&gt;A restore target is not necessarily a retained backup copy. It is a destination into which data is restored, and the access paths around it matter while recovery is running. It belongs in the inventory with that label.&lt;/p&gt;

&lt;p&gt;Backup catalogs and metadata belong on the inventory as well. This post makes no claim about how any product places them. It notes only that a register listing the data copies and omitting the records that describe them is incomplete, and that the architect should ask the question.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Two Vendors Show, and What They Don't
&lt;/h2&gt;

&lt;p&gt;Two vendor documents illustrate the pattern: the Veeam ONE 13 user guide and Rubrik's API reference. They are examples of how backup placement can be handled. This is not a comparison, and nothing here describes what other vendors offer or what either product can do beyond what these pages state.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;th&gt;What it shows&lt;/th&gt;
&lt;th&gt;What it does not establish&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Veeam ONE 13&lt;/td&gt;
&lt;td&gt;Location-based reporting and mismatch detection across protected objects, backups, backup copies and replicas. Locations are configured assignments on infrastructure objects.&lt;/td&gt;
&lt;td&gt;Independent verification of physical residency or legal jurisdiction. The Veeam ONE and Veeam Backup &amp;amp; Replication topology views must match; otherwise report data can be invalid.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rubrik API&lt;/td&gt;
&lt;td&gt;A replication specification accepts a cloud provider and target region, alongside retention settings.&lt;/td&gt;
&lt;td&gt;Whether the target is validated against a residency policy or continuously enforced.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Veeam ONE's Data Sovereignty report presents protected objects, backups and replicas "grouped by location," and a companion Data Sovereignty Violations report lists protected objects with backups or replicas “residing in a different location.”. Because locations are configured assignments, the report tells you where Veeam has been told things are, and where those configured locations disagree. That is useful detection. It is not proof of where the bytes physically sit, and it is not a finding of legal noncompliance. Veeam also notes that the topology views in Veeam ONE and Veeam Backup &amp;amp; Replication must match, or the report data can be invalid, so the report is only as reliable as the configuration behind it.&lt;/p&gt;

&lt;p&gt;Rubrik's developer documentation describes a &lt;code&gt;ReplicationToCloudRegionSpecInput&lt;/code&gt; type with four fields: &lt;code&gt;cloudProvider&lt;/code&gt; ("Replication cloud provider"), &lt;code&gt;replicationTargetRegion&lt;/code&gt; ("Replication target region"), &lt;code&gt;retention&lt;/code&gt; ("Retention period on replication region") and &lt;code&gt;retentionUnit&lt;/code&gt; ("Unit of retention period"). The target region is a string input to the replication specification. The reference documents the field. It does not say whether the value is validated against a residency policy or enforced after the fact.&lt;/p&gt;

&lt;p&gt;Read together, the two pages show one pattern: placement is declared in one place and observed in another. Rubrik's field records where a replica is meant to go. Veeam's report compares configured locations after the fact. Neither document establishes preventive enforcement, and that is a statement about these two pages, not about what these products can do or what any other vendor provides. For the architect the consequence is the same either way. A declared location is a claim. A report is evidence that the configuration is consistent with itself. Neither is a verified fact about where the data sits.&lt;/p&gt;

&lt;h2&gt;
  
  
  What GDPR Chapter V Does and Does Not Say
&lt;/h2&gt;

&lt;p&gt;Placement discussions pick up a legal reflex quickly, so the boundary needs stating precisely. Article 44 of the GDPR sets the general principle: "Any transfer of personal data which are undergoing processing or are intended for processing after transfer to a third country or to an international organisation shall take place only if, subject to the other provisions of this Regulation, the conditions laid down in this Chapter are complied with by the controller and processor, including for onward transfers of personal data from the third country or an international organisation to another third country or to another international organisation." The text governs transfers. Chapter V regulates qualifying transfers; it does not itself impose a general geographic residency requirement.&lt;/p&gt;

&lt;p&gt;The EDPB's Guidelines 05/2021 (Version 2.0, February 2023) explain what counts as a transfer. Paragraph 9 sets out "three cumulative criteria": the exporter is subject to the GDPR for the processing; the exporter discloses or otherwise makes the personal data available to another controller or processor; and the recipient is in a third country or is an international organization. All three must hold.&lt;/p&gt;

&lt;p&gt;Paragraph 16 is the one that matters for placement. Remote access from a third country, including access for support or administration, and cloud storage outside the EEA offered by a service provider, can be transfers when those criteria are met. Not every access event qualifies, but the point for the architect is that exposure is a function of who can reach the data, from where, and under what relationship. Storage location is only part of it.&lt;/p&gt;

&lt;p&gt;Paragraph 17 draws the limit. Processing by the same controller or processor outside the EU, without making the data available to another controller or processor, is not a Chapter V transfer. That does not exempt the processing from the GDPR. Section 4 of the Guidelines addresses processing outside the EEA where no transfer takes place: the controller remains accountable, must still assess the legal framework and risks of that processing, including conflicting laws and disproportionate public-authority access, and must apply appropriate safeguards. Articles 5, 24 and 32 remain relevant.&lt;/p&gt;

&lt;p&gt;Put plainly: Chapter V regulates qualifying transfers. It is not a residency mandate, and it does not require backups to sit in the same country as production. It also does not make location irrelevant. Contractual, sectoral and national requirements can impose different restrictions, and the accountability duties apply whether or not a transfer occurs. The legal question and the architectural question overlap, but they are not the same question. The backup placement test below is the architectural one, and confirming the legal position for a specific data set is a job for counsel.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fipbfltshk5al11dzmzp0.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fipbfltshk5al11dzmzp0.jpg" alt="a copy placement register row with copy path, declared location, access path and owner filled in and the verification cell left empty" width="800" height="447"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;A declared location without a verification entry is a claim, not evidence.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The Placement Test
&lt;/h2&gt;

&lt;p&gt;Apply the backup placement test to each row of the inventory, one row at a time. It asks four questions.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Where does it reside?&lt;/strong&gt; Record the declared location of the copy and the basis for it: the policy field, the repository setting, the site.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Who can reach it, and by what path?&lt;/strong&gt; Storage location is only part of the answer. Providers, support relationships and administrative access from other locations are part of the access path.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Who owns the placement decision?&lt;/strong&gt; A named role, not a team that happens to configure the job.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;How is it verified, and how often?&lt;/strong&gt; Name the evidence source, what it establishes, and the cadence. A declared location is a claim; a check against a platform report shows consistency, not physical residency.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The output is a backup placement register: one row per copy path, kept with the DR documentation. It is a table, not a framework.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;What the row records&lt;/th&gt;
&lt;th&gt;Why it exists&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Copy path&lt;/td&gt;
&lt;td&gt;Which inventory row this is: repository, copy target, replica, archive tier or restore target&lt;/td&gt;
&lt;td&gt;Keeps the register aligned to the inventory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Declared location&lt;/td&gt;
&lt;td&gt;Where the copy is configured to reside, and the setting that says so&lt;/td&gt;
&lt;td&gt;Gives verification something to compare against&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Access path&lt;/td&gt;
&lt;td&gt;The providers, support relationships and administrative routes that can reach the copy&lt;/td&gt;
&lt;td&gt;Makes exposure a function of access, not only storage&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Owner&lt;/td&gt;
&lt;td&gt;A named role with authority over both the boundary and the policy&lt;/td&gt;
&lt;td&gt;Prevents the decision from defaulting to whoever built the job&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Verification and cadence&lt;/td&gt;
&lt;td&gt;The evidence source, what it establishes, the check frequency and the last validation date; configuration consistency is distinguished from independent residency evidence&lt;/td&gt;
&lt;td&gt;Separates a declaration from evidence of where the data sits&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A register that records only the declared location has the weakness of a location tag: it is accurate until someone changes the policy. That is why the last column carries the weight. Scheduled checks should compare declared placement against independent evidence of actual placement wherever that evidence is available. A comparison between policy settings and a platform report can identify configuration drift or inconsistent location assignments, but it does not independently prove where the data resides. The architect should record what each check establishes, and what it does not. This verification routine is an architectural recommendation, not a capability established by either vendor document discussed above.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who Owns the Decision
&lt;/h2&gt;

&lt;p&gt;The failure mode to test for here is an ownership gap, not a technical one. Backup administration typically sets copy policy. Data governance typically sets the residency boundary. The join between the two, the question of whether a given copy sits inside the boundary, can fall to nobody. Whether it does in a particular organization is a question the architect has to answer, not assume either way.&lt;/p&gt;

&lt;p&gt;The backup placement decision needs to sit with a role that can see both sides: able to read the boundary that governance set, and able to change the policy that backup administration runs. In practice that means writing the owner column of the register before the first verification run, not after the first finding.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architect's Verdict
&lt;/h2&gt;

&lt;p&gt;Backup placement is a decision that is often made by default. A production review can approve one dataset in one place while subsequent copies are placed through configuration decisions that were not part of that review. Treating those two decisions as one is how a residency boundary can end up covering the dataset and nothing else.&lt;/p&gt;

&lt;p&gt;The problem is not that vendors hide where copies go. The two documents examined here show a declared field and a configured-location report. The problem is that declaration and detection are not verification, and the difference between them often has no named owner. A backup placement register with an owner and a verification cadence narrows that gap without requiring any claim about what a product enforces.&lt;/p&gt;

&lt;p&gt;A residency boundary drawn around the production dataset protects the dataset. It says nothing about the copies, and those copies may become the source of recovery when something fails.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/data-protection-architecture-strategy-guide/" rel="noopener noreferrer"&gt;Data Protection Architecture&lt;/a&gt; — the pillar guide for backup, recovery and resilience architecture, including where copy design sits in the wider recovery model.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/backup-blast-radius/" rel="noopener noreferrer"&gt;Your Backup System Is Part of the Blast Radius&lt;/a&gt; — the isolation side of backup design: shared accounts, identity and API authority, as distinct from where copies sit.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rack2cloud.com/sovereign-cloud-vs-public-cloud-compliance-trap/" rel="noopener noreferrer"&gt;Sovereign Cloud vs. Public Cloud: Navigating Compliance in a Non-Deterministic Landscape&lt;/a&gt; — the broader platform-level jurisdiction problem this post applies to backup copies.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.edpb.europa.eu/documents/guideline/guidelines-052021-on-the-interplay-between-the-application-of-article-3-and-the_en" rel="noopener noreferrer"&gt;EDPB Guidelines 05/2021 on the interplay between Article 3 and Chapter V&lt;/a&gt; — the official guidance on what counts as a transfer, including the three cumulative criteria and the accountability section.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://helpcenter.veeam.com/docs/one/userguide/data_sovereignty_overview.html" rel="noopener noreferrer"&gt;Veeam ONE 13 User Guide: Data Sovereignty Overview&lt;/a&gt; — the report that groups protected objects, backups and replicas by configured location.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://helpcenter.veeam.com/docs/one/userguide/data_sovereignty_violations.html" rel="noopener noreferrer"&gt;Veeam ONE 13 User Guide: Data Sovereignty Violations&lt;/a&gt; — the companion report that lists protected objects whose backups or replicas reside in a different location.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://developer.rubrik.com/Rubrik-Security-Cloud-API/API-Reference/types/inputs/ReplicationToCloudRegionSpecInput/" rel="noopener noreferrer"&gt;Rubrik Developer Center: ReplicationToCloudRegionSpecInput&lt;/a&gt; — the API type documenting the replication cloud provider, target region and retention fields.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://eur-lex.europa.eu/eli/reg/2016/679/oj" rel="noopener noreferrer"&gt;GDPR Article 44 (EUR-Lex, Regulation (EU) 2016/679)&lt;/a&gt; — the general principle for transfers quoted in this post.&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.rack2cloud.com/backup-placement/" rel="noopener noreferrer"&gt;rack2cloud.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

</description>
      <category>backup</category>
      <category>architecture</category>
      <category>cloud</category>
      <category>infrastructure</category>
    </item>
  </channel>
</rss>
