Rebuilding a corrupted WMI Repository


net stop winmgmt
c: 
cd %systemroot%\system32\wbem 
rd /S /Q repository 
regsvr32 /s %systemroot%\system32\scecli.dll 
regsvr32 /s %systemroot%\system32\userenv.dll 
mofcomp cimwin32.mof 
mofcomp cimwin32.mfl 
mofcomp rsop.mof 
mofcomp rsop.mfl 
for /f %%s in ('dir /b /s *.dll') do regsvr32 /s %%s 
for /f %%s in ('dir /b *.mof') do mofcomp %%s 
for /f %%s in ('dir /b *.mfl') do mofcomp %%s 
echo DONE reboot 
pause
-------------------------------------
More info
http://www.ibm.com/support/knowledgecenter/SSAV7B_6.3.7/com.ibm.director.tbs.helps.doc/fqm0_r_tbs_wmi_repair.html

Force replication to all Domain Controllers in Powershell


Import-Module ActiveDirectory

$DCs = (Get-ADForest).Domains | %{ Get-ADDomainController -Filter * -Server $_ } | select HostName

foreach ($DC in $DCs)
{
repadmin /syncall $DC.HostName
}

How to read the Windowsupdate.log file


Components

The following components can write to the Windowsupdate.log file:

  • AGENT– Windows Update agent
  • AU– Automatic Updates is performing this task
  • AUCLNT– Interaction by AU with the logged on user
  • CDM– Device Manager
  • CMPRESS– Compression agent
  • COMAPI– Windows Update API
  • DRIVER– Device driver information
  • DTASTOR– Handles database transactions
  • DWNLDMGR– Creates and monitors download jobs
  • EEHNDLER– Expression handler used to evaluate update applicability
  • HANDLER– Manages the update installers
  • MISC– General service information
  • OFFLSNC– Detect available updates when not connected to the network
  • PARSER– Parses expression information
  • PT– Synchronizes updates information to the local datastore
  • REPORT– Collects reporting information
  • SERVICE– Startup/Shutdown of the Automatic Updates service
  • SETUP– Installs new versions of the Windows Update client when available
  • SHUTDWN– Install at shutdown feature
  • WUREDIR– The Windows Update redirector files
  • WUWEB– The Windows Update ActiveX control

More information:

https://support.microsoft.com/en-us/kb/902093

Domain Controllers test:


Scenario

It’s a pleasant day and all is well with the world. Colleagues are skipping around the office with smiles on faces…until…duh duh daaa! One by one, services start failing:

  • Printers go offline:
    • First, for Win7 users
    • Then for all clients
    • Can still print from server though
  • File shares go offline
  • Active Directory replication fails
  • DNS console will not open

Basically, your main Domain Controller (DC) has just taken a dump…and so have you!

These are the steps I took to troubleshoot the issues and get everything back online.

Solution

Gather Information

Run the following commands to gather useful information:

ipconfig /all > c:\ipconfig.txt (from each DC/DNS Server)
dcdiag /v /c /d /e /s: > c:\dcdiag.txt
dcdiag /test:dns /s: /DnsBasic > c:\dcdiag-dnsbasic.txt
repadmin /showrepl dc* /verbose /all /intersite > c:\showrepl.txt (dc* is a placeholder for the starting name of the DCs if they all begin the same - if more then one DC exists)
repadmin /replsum > c:\replsum.txt

Pour through the txt files and note down the errors. Some of mine included:

  • repadmin /showrepl
    • Last error: 1256 (0x4e8): The remote system is not available.
    • Last error: 5 (0x5): Access is denied.
    • WARNING: KCC could not add this REPLICA LINK due to error.
    • result 1722 (0x6ba): The RPC server is unavailable.
  • repadmin /replsum
    • (1722) The RPC server is unavailable.
    • (5) Access is denied.
  • dcdiag /test:dns /s: /DnsBasic
    • The host <hostname> could not be resolved to an IP address. Check the DNS server, DHCP,server name, etc.
    • Got error while checking LDAP and RPC connectivity. Please check your firewall settings.
    • Error: No LDAP connectivity.
    • invalid DNS server:
    • No host records (A or AAAA) were found for this DC.
    • Warning: no DNS RPC connectivity (error or non Microsoft DNS server is running).
    • Name resolution is not functional.
  • dcdiag /v /c /d /e /s:
    • EventID: 0x40000004 – The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server.
    • EventID: 0xC00004B2 – The DFS Replication service failed to contact domain controller  to access configuration information.
    • EventID: 0xC000138A – The DFS Replication service encountered an error communicating with partnerfor replication group Domain System Volume.
    • The replication generated an error (-2146893022): The target principal name is incorrect.
    • Error: Detected circular loop trying to locate the ISTG.
  • repadmin /syncall
    • -2146893022 (0x80090322): The target principal name is incorrect.
    • SyncAll exited with fatal Win32 error: 8440 (0x20f8): The naming context specified for this replication operation is invalid.

Some information seemed to conflict as similar tests for certain services failed (like DNS) yet you could still ping by name and confirm using nslookup. Moving on.

Go through the errors one by one and search online for solutions. Here are some of the URLs I used to troubleshoot errors:

By now things might seem to snowball, but stay calm and keep trying recommended steps from Microsoft, recording your steps along the way:

To stop the KDC

  1. At a command prompt, type the following command and press ENTER:
  2. net stop KDC
  3. If the KDC cannot stop, set its startup state to disable and restart.

To purge the ticket cache

  1. At a command prompt, type the following command and press ENTER:
  2. klist purge
  3. Answer Yes for each ticket

To reset the computer account password on the PDC emulator

  1. At a command prompt, type the following command and press ENTER:
  2. netdom resetpwd /server:/userd:\administrator /passwordd:*

Some other commands I used included:

dcdiag /test:CheckSecurityError /s
dcdiag /testdomain:
nltest /logon_query
nltest /dclist:
nltest /domain_trusts
nltest /DSQUERYDNS
nltest /DSREGDNS
nltest /sc_verify:
nltest /dsgetdc: /force
net config rdr
dsquery * forestroot -scope subtree -filter "(serviceprincipalname=)" -attr * -s
<pre>

nltest /dsgetdc: /gc gave this error:
Getting DC name failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN

nltest /server: /sc_query: gave this error:
I_NetLogonControl failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN

Know when to quit

My troubleshooting ran on to a second day. By now, users were using a workaround to access printers and file shares, but the DC errors continued. At this point, I decided to demote the DC and just leave it as a file and print server; which is best practice anyway.

After taking a snapshot of the DC (via VMware vCenter), I proceeded to go through the standard steps to demote a DC:

  1. Transfer all FSMO roles to another DC – this failed with a generic error (http://social.technet.microsoft.com/Forums/en/winserverDS/thread/3f49ddbc-c948-43ac-af21-2f5a4f3dce9b).
  2. Run dcpromo to demote DC – this also failed.

Great. Now the only option was a forceful removal of the DC (http://technet.microsoft.com/en-us/library/cc731871(v=ws.10).aspx). I

dcpromo /forceremoval worked fine. I then removed the DC from Sites and Services, at which point the FSMO roles were transferred to another DC, so I didn’t need to seize them. You used to have to go through a Metadata Cleanup, after forcing a demotion, but now this is done for you when you remove the DC from Sites and Services. This can be confirmed by following the steps here: http://www.petri.co.il/delete_failed_dcs_from_ad.htm

Although this is much easier using 2008 R2, you will still need to tidy up a little in other areas:

  1. Remove all entries of failed DC in Name Server Tabs on all relevant DNS zone properties.
  2. Backup and restore DHCP database to another server.
  3. Tombstone WINs entries from failed DC:
    1. From another DC, go to WINS >Active Registrations > right-click > Delete Owner.
    2. Select failed DC.
    3. Replicate deletion to other servers (tombstone).
    4. The new DC will then take ownership of the records.
  4. Uninstall above roles from failed DC.
  5. Update DHCP and devices with static IPs to use the new DC’s IP Address for DNS and WINS. You did spin up a new DC right?!?!

Another great tip I found was from this thread on Spiceworks:

If we really want to be safe then open a command prompt with elevated privileges and run the following command
csvde –f C:\\ad_details.csv
This exports all contents of ASDIEdit to an excel file in the root of C drive called “ad_details.csv” Open this in Excel and do a find all for <FAILED DC>. If it finds any references then we have lingering objects and will need to perform a Metadata Cleanup.

Conclusion

Although this was a nightmare to troubleshoot – and I have a chip on my shoulder as I didn’t find the root-cause or fix the DC – I have more confidence in the steps to force the removal of a screwed up DC. Next time I’ll learn to let go a little faster.

Update: I’ve just found more notes on this that may be useful in future:

SC commands


Specfic services status:

sc \\10.23.4.188 qc FAX

List of all not running services:

sc \\10.23.4.188 query type= service state= inactive | find “SERVICE_NAME”

List of all services:

sc \\10.23.4.188 query type= service state= all | find “SERVICE_NAME”

List of all running services:

sc \\10.23.4.188 query type= service | find “SERVICE_NAME”

Troubleshoot WSUS clients and server


When you consider the issues of WSUS there are multiple thing to refer. These two parts are mainly the communication of WSUS server and communication of client computers. Let us consider the parts one by one.
Communication of WSUS server
1. When you have an issue with WSUS it is really necessary to make sure that WSUS server is working correctly, then only the clients will get the updates properly. In order to make sure that the WSUS is working correctly we have the in built MS utility called as ‘wsusutil’ which will help you to manage the server using command line. Since we need to make sure that WSUS is working correctly we can check the health of WSUS and make sure that it is working as expected.
a) Open command prompt and redirect to the below directory using the command ‘cd C:\Program Files\Update Services\Tools’
b) Now type the command ‘wsusutil.exe checkhealth’ and ENTER. It will take few seconds to complete and once it is completed open the application event viewer and make sure that it has generated ‘Event id 10000, source: Windows server update’ which indicates that WSUS is working correctly. Search for any error messages in the Microsoft Knowledge Base for more troubleshooting information or post a comment on this article.
Note: You may fail to perform check health by the error message update service is not running. In this case open ‘services.msc’ from ‘run’ and make sure that the service ‘update service’ is running and startup type as ‘automatic’. If you would like to more details about using ‘wsusutil’ follow the below link to get it.
2. If the initial step is success you can continue with remaining steps. WSUS server and its details are specified through GPO hence it is really necessary to make sure that there are no GPO errors reported in server. Review the application and system logs to make sure that there are no Active directory or GPO related errors. If you found anything resolve the same and continue with the investigation.
3. Check whether the server can reach the WSUS client by pingWSUSClient and make sure that the client is listed under ‘Computers’ in WSUS server console.
If you are unable to ping the clients make sure that the firewall or proxy servers does not prevent the communication. And make use of the ‘telnet’ command to trace the route.
         Communication of client computers
When we consider the WSUS clients there are multiple thing , perform the below steps to confirm the clients have got proper details of WSUS.
1. Make sure that there are no AD and GPO related errors in your domain controllers and affected clients(Check through System and application logs of event viewer). It is really necessary to do this since the clients are pulling the WSUS server details through group policy and any errors related to this may not allow those to get correct details.
2. Run a rsop and make sure that under Microsoft update service WSUS settings are defined properly.
Open Run-> Type ‘rsop.msc’ and ENTER. It will generate the group policy result which indicates the policies that are applied to this computer.
Expand Computer configurations->Administrative tools->Windows components and locate the policy ‘specify intranet Microsoft  update service  location’ is pointed to WSUS server.
That is the WSUS server will be specified as ‘http://WSUSservername:8530 ‘. If the settings are incorrect make the necessary changes in server GPO and make sure that it receives in clients.
 3. If the clients are receiving GPO correctly those details will be listed in registry as well. Make sure the server details are present on the registry as well.
Open run command and type ‘reg query HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate’ which should return the value with your server details and looks like this:
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate
WUServer    REG_SZ  http://WSUSServerName
WUStatusServer      REG_SZ  http://WSUSServerName
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU
Note: You will have the above output only if your clients are configured to get updates  from WSUS server.
Else you can manually locate the registry directory and view the information are correct.
Open registry editor and locate ‘HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate’ and see the WSUS server details are correct.
 4. Verify that clients can reach the WSUS server by performing the below steps:
Open the web browser and type’ http://<WSUSServerName>/iuident.cab’ if the results prompt you for downloading a file named as ‘iuident.cab’ you can safely cancel it and it shows the client is able to communicate with WSUS server and there is no connectivity issue. If the webpage fails to respond and does not ask for the file to download it indicates that may be a communication issue, name resolution or WSUS server is not configured properly. One of the useful link in this situation is(self update issues)
5. Determine the last time that the clients has updated. This can be done in two ways either through report of WSUS administration console or from the registry values present in clients. It is more convenient to use the second option if you have the direct access to affected clients.
a) If you want to get the details using report viewer follow these steps:
Open the Update Services console on the WSUS server. Click the Reports icon and then click Computer Detailed Status. Browse the computers to find the problematic computer and examine the updates that have been successfully installed, as well as those that have not yet been installed.
b) To get the details directly from the client computer open the registry editor and locate the directory,  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results now each folders determine the status of WSUS updates like ‘Detect’ shows the last time the client detected on server and checked for updates. ‘Download’ will show the last date and time the updates installed and ‘Install’ says the last date and time that it has successfully installed the updates.
6. Download and install the  windows update troubleshooter which can fix most of the common issues.
7. When clients are not receiving the updates, determine whether its because of a problem that is affected the entire functionalities of Windows updates on clients or because of WSUS. So manually locate the ‘Windows Updates’ and click on ‘Check for updates’ to make sure that it is reporting that there are updates to install. If it is listed as there are updates pending to install do not initiate to install because by these steps we are checking whether there is any cryptographic service provider errors or a file Windows Update requires (named catalog store) is corrupted.
If there are error reported for this follow the below links to find the solution for some of the error codes.
8. All the client windows updates action details are stored locally named as ‘WindowsUpdate.log’. Verify these logs and check whether there is any issue reported in logs. Follow these methods to get the latest logs.
The below link will help you to read the windows logs: http://support.microsoft.com/kb/902093
For server 2003\XP the – C:\winnt\WindowsUpdate.log
For 2008\Windows 7- C:\windows\WindowsUpdate.log
Else on Run command you can simply type ‘WindowsUpdate.log’ to open it.
9. Allow the affected client to reestablish the connection once again with WSUS . In order to achieve that, locate the affected client in the ‘Computers’ list of WSUS console  and delete it(Right click->Delete). Now on affected client open command prompt and type ‘wuauclt /detectnow’ wait for 30 minutes and check the logs that is available in client. Check the windows update logs from the time of doing these steps and see whether it reported any errors.
Search for any error messages in the Microsoft Knowledge Base for more troubleshooting information.
10. I have seen in may cases deleting  and regenerating the SUS client ID on affected clients will provide a solution for update errors.
On the affected client open command prompt and type the below commands,
net stop wuauserv
REG DELETE “HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate” /v SusClientId /f
REG DELETE “HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate” /v SusClientIdValidation /f
net start wuauserv
The first and last command will stop and start the Windows update service on this computer. The other two will delete the SUS client ID. Now wait for another 15 minutes and observe the status.
11. There are many WSUS troubleshooting tools available through MS. Please follow the below link to get those and check whether it help you to resolve the issue.

LGPO.exe – Local Group Policy Object Utility, v1.0


LGPO.exe is a new command-line utility to automate the management of local group policy. It replaces the no-longer-maintained LocalGPO tool that shipped with the Security Compliance Manager (SCM), and the Apply_LGPO_Delta and ImportRegPol tools.

Features:

  • Import settings into local group policy from GPO backups or from individual policy component files, including Registry Policy (registry.pol), security templates, and advanced auditing CSV files.
  • Export local policy to a GPO backup.
  • Parse a Registry Policy (registry.pol) file to readable “LGPO text” directly to the console or redirected to a file which can edited and imported into local policy.
  • Build a new Registry Policy (registry.pol) file from “LGPO text”.
  • Enable group policy client side extensions for local policy processing.

The zip file attached to this post includes LGPO.exe and full documentation. This is the command line syntax:

LGPO.exe v1.00 – Local Group Policy Object utility

LGPO.exe has four modes:
* Import and apply policy settings;
* Export local policy to a GPO backup;
* Parse a registry.pol file to “LGPO text” format;
* Build a registry.pol file from “LGPO text”.

To apply policy settings:

LGPO.exe command […]

where “command” is one or more of the following (each of which can be repeated):

/g path               import settings from one or more GPO backups under “path”
/m path\registry.pol  import settings from registry.pol into machine config
/u path\registry.pol  import settings from registry.pol into user config
/s path\GptTmpl.inf   apply security template
/a[c] path\Audit.csv  apply advanced auditing settings; /ac to clear policy first
/t path\lgpo.txt      apply registry commands from LGPO text
/e <name>|<guid>      enable GP extension for local policy processing; specify a
GUID, or one of these names:
* “zone” for IE zone mapping extension
* “mitigation” for mitigation options, including font blocking
* “audit” for advanced audit policy configuration
/boot                 reboot after applying policies
/v                    verbose output
/q                    quiet output (no headers)

To create a GPO backup from local policy:

LGPO.exe /b path [/n GPO-name]

/b path               Create GPO backup in “path”
/n GPO-name           Optional GPO display name (use quotes if it contains spaces)

To parse a Registry.pol file to LGPO text (stdout):

LGPO.exe /parse [/q] {/m|/u} path\registry.pol

/m path\registry.pol  parse registry.pol as machine config commands
/u path\registry.pol  parse registry.pol as user config commands
/q                    quiet output (no headers)

To build a Registry.pol file from LGPO text:

LGPO.exe /r path\lgpo.txt /w path\registry.pol [/v]

/r path\lgpo.txt      Read input from LGPO text file
/w path\registry.pol  Write new registry.pol file

(See the documentation for more information and examples.)

Follow

Get every new post delivered to your Inbox.

Join 40 other followers

%d bloggers like this: