Deep Dive of the Day Who owns which password in VCF 9.1, and how to keep them from expiring
Password management in VCF 9.1 is split across two tools, and most of the pain comes from not knowing which one owns a given account. Broadcom's guidance is that VCF Operations is now the preferred place to manage passwords: it covers local, application and service accounts on VCF 5.2 and later, plus standalone vCenter and ESX 8.0 Update 3 and later, and it doesn't keep its own copy of passwords in a vault. SDDC Manager still manages credentials for the core components, but TechDocs says it "will be deprecated in a future VCF release" and that using it for passwords "is not advised" unless your environment needs it.
In practice that means two worlds. The vCenter, ESX, NSX and SDDC Manager root and admin accounts are rotated by SDDC Manager, which can rotate (generate a new password and apply it everywhere), remediate (tell SDDC Manager about a password that was changed directly on the component) and auto-rotate on a schedule. Broadcom's design recommendations say auto-rotation is limited to a 30, 60 or 90-day cycle. KB 435599 adds that some fleet components (Fleet Management, VCF Operations and its logs and networks appliances, and the identity broker) may currently sit outside that automated rotation engine and need managing by hand, so watch their expiry through VCF Operations.
VCF Operations works on policies rather than vaulted secrets. A policy sets length, complexity, expiry, lockout and history, and can apply to the whole fleet, the management components or a single instance, with the narrower one winning. The catch is that expiry is counted from the last password change: apply a 90-day policy to an account last changed 120 days ago and it expires the moment the policy lands. Complexity rules also don't touch existing passwords, which is why Broadcom recommends changing passwords straight after a policy goes live. Policy tasks depend on Salt, so give a fresh management deployment about 30 minutes before you push one.
The gotchas in production are mostly about stepping round the platform. Broadcom is explicit that you shouldn't change the passwords or permissions of the inter-component service accounts on the components themselves, because that breaks the integrations; KB 459311 from earlier this week, where stale lock entries in SDDC Manager kept an NSX service account locked, shows how fiddly that plumbing can be. There's no "never expire" option for core components, and using chage or passwd on the appliances to get round it creates drift that SDDC Manager or lifecycle operations will trip over. Cosmin Trif's write-up on expired accounts in Fleet password management adds a useful rule: VCF 9.1 has no "store a password I already changed" button, so if someone changed one by hand you supply that as the current password and rotate once more through the UI or API.
In a nested lab the temptation is to turn expiry off altogether, and that's exactly what bites you when you come back to a lab after three months and find root expired on half the appliances. A better habit is to turn on SDDC Manager auto-rotation for the core accounts, keep the VCF Operations password alerts switched on, and when something has expired, reset it from the console to a temporary value and immediately rotate it through VCF Operations or the SDDC Manager credentials API so the platform knows the final password. If you script any of this, watch for passwords containing !, which some shells will happily expand.
Based on Managing Passwords for VCF Components (TechDocs) Official Using SDDC Manager to Manage Passwords (TechDocs) Official Recommendations for VCF Password Management (TechDocs) Official KB 435599: Password expiration and rotation in VCF 9.x Official Cosmin Trif: Fixing expired password sync in VCF 9.1 Fleet Management