Using Thycotic Secret Server to Rotate VMware ESXi Root Passwords
One of my security related projects this coming year will be to reduce (remove?) the usage of identical passwords across similar devices. For example, how many of you utilize the same root password for ESXi servers or the same password for the local administrator account on servers and desktops? We do, and so had every place I’ve worked for previously. In a time of almost daily breach announcements, it’s no longer sufficient to maintain this concept moving forward. We fixed it using Thycotic’s Secret Server.
For us, Secret Server was originally looked at as a centralized password management tool for our team who, at the time, were all using different products installed on their local workstations. Although we got by, it wasn’t an ideal solution and proved to become an issue at times, sparking the search for a solution. We quickly found that Secret Server also has tons of other features, one of which is managing and remotely rotating credentials for both service accounts and a devices local accounts. This had been a goal of mine since I started and after a week long proof of concept, we were impressed and purchased the product.
The following demonstrates how to configure Secret Server to rotate a ESXi server’s root password.
1. Using the Discovery feature of Secret Server, import the ESXi servers you want to manage. For me, I’m going to choose both of the ESXi servers in our lab. You can find this under Admin –> Discovery.
- Create a new Discovery Source, choosing the type as “VMware ESX/ESXi Discovery Source” and follow through the wizard. Make sure to add a secret so Secret Server may authenticate properly and read the list of current users.
- Run Discovery by hitting the Run Now button.
- Enter the Discovery Network View page by clicking the Discovery Network View button.
- On the left, expand the Discovery Source you created and click on a server. In the right pane, you should see a list of user accounts available on the server. If the status displays “Not Scanned”, click the Rescan icon next to the server name.
- Select the root account and click Import – *Note: Do not change the VPXUSER account as it is the account vCenter uses to communicate with the host. This password is already random and is rotated by default.
- Follow the Import wizard and choose the desired actions. This will create a new secret in the selected folder in which you can manage and modify later as shown below.
2. Navigate to the newly created secret and choose Edit, opening the screen that you see above. Here you can see the details of the current account including the password and it’s expiration. In this case the VMware ESX/ESXi Secret Template has an expiration of 30 days which this secret has inherited.
- Click on the Remote Password Changing tab.
- To change the password immediately, click the Change Password Remotely button. Type in a new password or Generate a new one. Click Change.
- As shown by the vSphere Client’s Recent Tasks, Secret server has modified the root password as requested. The new password can be found by viewing it within the secret.
- You may also view Secret Server’s Remote Password Changing logs under Admin –> Remote Password Changing Log.
3. Once you’ve added all your ESXi hosts using the same Secret Template, they will automatically be changed when the password expires as configured. Scheduling the frequency of a Secret’s expiration is done within the Secret Template while the time/date of the actual password change can be configured using the AutoChange Schedule. If you leave the AutoChange Schedule set to none, the password change will be queued for a change immediately after the expiration, assuming AutoChange is enabled. For more information on definitions on Expiration, AutoChange, and the AutoChange Schedule, including examples, check out Thycotic’s KB article here.
- Secret Template – If you wish to modify the frequency of the expiration, you’ll want to change the value for Expiration Days in the template; navigate to Admin –> Secret Templates. Choose your template and modify it accordingly. This will apply to all secrets utilizing this template.
- AutoChange Schedule – If you wish to modify the actual time/day that the password will change, say Saturday at 2am, utilize the AutoChange Schedule feature.
- AutoChange – As noted in the graphic below, be sure to enable AutoChange for your secrets. This can be done per secret, as noted below, or in bulk by selecting all desired secrets in the browser widget and selecting “Enable AutoChange” under the bulk operation drop down menu below.
Last but not least, if you ever get in a bind with your ESXi’s root account and you’re unable to retrieve it’s password, you can use this post to safely reset the host’s root account. Also, be sure to check out one of our general vSphere security posts here.