‘The security database on the server does not have a computer account for this workstation trust relationship’ when trying to log on via RDP to a customized VM in Azure.

Last summer, our customized lab VM’s started having issues after updating from Windows Update and applying the latest fixes.

The error message received was ‘The security database on the server does not have a computer account for this workstation trust relationship’ when trying to log on via RDP.

After extensive troubleshooting we determined that this seems to have been caused by changes which renamed the VM internally, either in Azure Resource Manager, the Windows Update code or the version of the Azure IaaS agent running on the VM’s.

At any rate… as the server was a domain controller, when it renamed itself to match the external name of the VM it caused an identity crisis as it could no longer find itself.

Yes… the technically correct answer from the handbook is to Sysprep/Generalize any image/template VM that you deploy to Azure but for a lab scenario you sometimes want to have a specialized image that contains AD & Friends – and AD and Sysprep are just not friends as Sysprep will eat AD and not even leave a skeleton behind.

Prior to June this year, deploying a customized non-sysprepped VM from an image with a new name to Azure Resource Manager did not change the actual computer name inside the image if the VM was a domain controller.

This meant you could deploy the same lab image multiple times to Azure with a different external VM name but the name of the VM inside the OS would stay the same.

At some point between June and August this year there seems to have been a change to this which was affecting the image, after updating from Windows Update the name of the server inside the VM was now being changed to match the external name.

This in turn resulted in it being impossible to log on to the VM after rebooting, as the actual computer name of the domain controller VM in the AD that it hosts isn’t changed to the new name at the same time.

TLDR: Redeploy the VM using the original VM name, run Windows Update and update without problems.  Create a new image from the updated VHD as a new template VHD.

I’ve tested a couple of different deployments after updating the specialized image and it doesn’t look like there is an issue updating from Windows Update using the new template that contains the latest updates, even if deployed again using a different name.  Either the rename is dependent on a specific component being updated or possibly it’s related to the version of the Azure agent running on the VM.

Quickly undeleting users, groups or Organizational Units using the AD Recycle Bin feature

Note: if the AD Recycle Bin isn’t already enabled in your forest then you need to dig out your last system state backup and go through the restore process – enabling the ADRB does not help with recovering any objects deleted prior to it being enabled.  See this article for enabling it. 

A VM snapshot can potentially be used to restore from for DR purposes but only if *both* the virtualization host platform and the underlying guest OS supports it (W2k12 or later for the guest OS and Hyper-V W2k12 or vSphere/ESXi 5.0 Update 2 for the virtualization hosts). 

If your VM snapshot doesn’t fulfil the minimum OS and virtualization requirements then you may still be able to use it as a last resort but the operation will be a lengthy process if the snapshot DC was brought online before manually setting the “Database Restored from Backup” registry key on it to force it to generate a new invocation ID.

From the VMware article on VM-GenerationID support:

VM-Generation ID functionality requires the hypervisor vendor to create the virtual machine identifier and expose it to the guest. VMware has provided this functionality in the following releases of vSphere:

  • VMware vSphere 5.0 Update 2 (vCenter Server and ESXi must both be at 5.0 Update 2)
  • VMware vSphere 5.1 (ESXi must be at least 5.0 Update 2)

From the Technet article on Safe Domain Controller Virtualization:
Beginning with Windows Server 2012, AD DS virtual domain controllers hosted on hypervisor platforms that expose an identifier called VM-Generation ID can detect and employ necessary safety measures to protect the AD DS environment if the virtual machine is rolled back in time by the application of a VM snapshot. The VM-GenerationID design uses a hypervisor-vendor independent mechanism to expose this identifier in the address space of the guest virtual machine, so the safe virtualization experience is consistently available of any hypervisor that supports VM-GenerationID. This identifier can be sampled by services and applications running inside the virtual machine to detect if a virtual machine has been rolled back in time.
…………….
Virtualized domain controller cloning relies on the hypervisor platform to expose an identifier called VM-Generation ID to detect creation of a virtual machine. AD DS initially stores the value of this identifier in its database (NTDS.DIT) during domain controller promotion. When the virtual machine boots up, the current value of the VM-Generation ID from the virtual machine is compared against the value in the database. If the two values are different, the domain controller resets the Invocation ID and discards the RID pool, thereby preventing USN re-use or the potential creation of duplicate security-principals.

 

Let´s look at the AD Recycle Bin in action in a lab:

First, we check whether the AD Recycle Bin feature is enabled in my lab forest using the following Powershell command:

get-ADOptionalFeature “Recycle Bin Feature”

ADRB1

If the EnabledScopes part is empty it means the AD Recycle Bin still hasn’t been enabled in your AD forest, luckily I already have it enabled in my lab.

Now, I just deleted a few items from my AD lab, some of which shouldn’t have been deleted and now need to be restored quickly.  I know all the deleted objects had ‘Test’ in the name so let’s see what shows up in the AD Recycle Bin when searching for objects with the isDeleted AD attribute set to ‘true’ and that include Test somewhere in the name and then sorting the list by the AD attributes msds-lastKnownRdn and lastknownParent to see where they were originally deleted from:

ADRB2

Of these 6 items, I’m only interested in restoring the group objects (the other ones were intentionally deleted).  I’ll add a filter to only target groups and then pipe it to the Restore-ADObject Powershell command with the -WhatIf switch appended to determine what action would be performed on which objects without actually performing the restore:

ADRB3

Great, the -WhatIf option shows me the command would restore only the two groups that I want to restore and leave everything else we don’t want to restore still in a deleted state in the AD Recycle Bin.  We can now confidently run the same command again without the -WhatIf switch (Adding the -Verbose option instead to report the status for each restore):

ADRB4

Excellent! The two groups and their pre-deletion membership state are now back in business without having to bother with a lengthy restore process!  Let’s confirm the groups are really back in the correct OU in AD by running Get-ADObject again without the ‘isDeleted’ filter option:
ADRB5

The undeleted groups are back where they belong in AD with the correct membership in the minimum amount of time required to recover from the original error.

This will of course *not* replace your regular backup procedures but it should really be considered a mandatory precaution to enable it in any AD environment running Windows 2008 R2 DC’s as it really cuts down the time required to recover from accidental deletion of objects in AD.

In W2k12 R2 there is now also the option of using the Active Directory Administrative Center to undelete objects if you want a more graphical way of doing things:
ADAdminCenter1
Additional links:

AD Recycle Bin Best Practices
http://blogs.technet.com/b/askds/archive/2009/08/27/the-ad-recycle-bin-understanding-implementing-best-practices-and-troubleshooting.aspx

AD Recycle Bin behaviour
http://blogs.technet.com/b/instan/archive/2010/05/10/ad-recycle-bin-and-the-conspicuously-cloned-user-accounts-conundrum.aspx

Safe virtualization of Domain Controllers
https://technet.microsoft.com/en-us/library/hh831734.aspx#safe_virt_dc

vSphere versions required for VM-Generation ID support
https://blogs.vmware.com/apps/2013/01/windows-server-2012-vm-generation-id-support-in-vsphere.html

Fixing When Your Domain Traveled Back In Time, the Great System Time Rollback to the Year 2000
http://blogs.technet.com/b/askpfeplat/archive/2012/11/23/fixing-when-your-domain-traveled-back-in-time-the-great-system-time-rollback-to-the-year-2000.aspx

Virtualized Domain Controller Architecture
https://technet.microsoft.com/en-us/library/jj574118.aspx

Things to consider when enabling the AD Recycle Bin

Recycle Bin

#1 check if you have any applications that use the DirSync control and make sure you have applied the NTDS hotfix from KB979214 to Domain Controllers (if you are running Windows 2008 R2 DC’s) as this changes the default behaviour of the relevant LDAP search control when returning values for objects that reference recycled (soft deleted) objects.

#2 read the ASKDS article on best practices for implementing the AD Recycle Bin.

#3 figure out if you have any critical applications using the DirSync control that can’t handle the isRecycled attribute state for objects that have been soft deleted.

 

ILM 2007 will unfortunately not be able to handle the AD Recycle Bin but FIM 2010 with Update 1 will (ILM 2007 being out of support for more than a few years now should also really be reason enough to upgrade to at least FIM 2010 R2 or MIM 2016).

Applications that have the potential to be impacted negatively by enabling the AD Recycle Bin are most likely going to be older product versions that focus on directory synchronization or monitoring using the Dirsync control.  Those will either need to be retired or updated to a newer version (older GALSync and MIIS instances probably fall under the same unsupportable hat as ILM 2007).

 

Additional links:

Polling for AD Changes using the DirSync control
https://msdn.microsoft.com/en-us/library/windows/desktop/ms677626(v=vs.85).aspx

FIM 2010 Update 1
https://support.microsoft.com/en-us/kb/978864

The DirSync control search does not return the deactivated linked attributes from a modified object in a Windows Server 2008 R2-based domain
https://support.microsoft.com/en-us/kb/979214

Using the DirSync Control
http://blogs.technet.com/b/isrpfeplat/archive/2010/09/20/using-the-dirsync-control.aspx

PowerShell DirSync Sample
http://dloder.blogspot.is/2012/01/powershell-dirsync-sample.html