Phantom DHCP servers

I recently came across this issue when working with the DHCP servers on the domain at work. Instead of 41 DHCP servers, one for each remote location, the DHCP administration snap-in was showing we had 93! That’s just a few too many.

In order to clean up the DHCP server list, I first opened Active Directory Sites & Services. You have to click View and then Show services node in order to see the available AD services. Then, I expanded Services and clicked on NetServices to see what DHCP servers (dHCPClass records) were listed. I removed all of the invalid records (but not the DhcpRoot record) and then checked the DHCP admin utility. All of the invalid servers were still listed.

The next step was to use ADSI Edit in order to edit the DhcpRoot record found in the NetServices container. Using ADSI Edit, I browsed to the CN=NetServices,CN=Services,CN=Configuration,DC=Domain,DC=Domain container, substitute your domain name after DC=, right clicked on DhcpRoot and chose Properties. I found another list of invalid DHCP servers when I opened the dhcpServers property of the DhcpRoot record. Once I deleted the invalid servers there, the DHCP admin utility only listed the valid DHCP servers.

It was odd that the list was wrong to begin with. Any time a DHCP server is taken off line it is unauthorized first. Only one or two of the invalid servers were servers that had been replaced due to a server crash that resulted in reloading Windows Server 2003. I may need to set up a DHCP server, authorize on the domain and then decommission it to see if AD is removing them properly. I hope it was just a minor glitch carried over from when the domain controllers were Windows 2000 based.

Patience is something you admire in the driver behind you and scorn in the one ahead. – Mac McCleary

23.Nov.09 Active Directory, DHCP, Networking, Server 2003, Windows Comments (0)

Finding the cause of the problem

Since implementing the Cisco NAC, I’ve had a few fun calls that could have been avoided with a little more troubleshooting.  The calls always start with, “Hey, we’re having some NAC issues here.” and go down hill from there.  Here are a few issues initially blamed on the NAC with the true cause of the issue in parentheses.

• User shared drives are not mapping at login. (Network cable was unplugged.  User was being authenticated with cached credentials.)
• No one can log on to the domain when using X computer in Y room. (There was no computer account on the domain.  Joined the computer to the domain and logins started working.)
• User cannot log on, no matter what computer they try.  (User account had been disabled due to administrative request.)
• Clicking noise when computer is turned on.  (Seriously?  A technician actually asked if the recent agent install had caused the issue.  A cookie to the correct answer in the comments.)
• A projector connected to the computer did not show the shared screen.  (It helps if the projector is actually turned on.)

There are a few more in the ever growing compilation list to be posted later.  I get at least two calls a weeks that are very face-palm worthy.  I’m thinking about setting up a wall-o’-shame at work.  Let the guilty party step forth.

Sometimes it’s more important to be human, than to have good taste. – Brecht

09.Dec.08 Humor, Networking, Security Comments (2)

The CAT matters

A network technician called me the other day about a network printer he was having a problem with. He said that he could ping the printer but printing was sporadic. When he tried to bring up the printer’s configuration web page, it would not load. It wouldn’t load for me either (I’m at another location). I asked if he had tried a different network port and he said he had but it still didn’t work.

I decided to check out the printer myself after a few other troubleshooting checks failed. It was nice to get out of the office for the short drive. When I got to the other location, the technician took me to the printer. The first thing I noticed was the color of the network cable. The cable was a dark blue. Those cables, I remembered, were pretty old and no longer being used because they are CAT 5. Almost all of our buildings are wired with CAT 6 for gigabit.

Before changing out the cable, I wanted to try a quick test. I went into the configuration of the printer via the front control panel and changed the network type from “Auto” to “100Mb Full-Duplex” (the printer supports gigabit). The printer was restarted and tested. We could now access the configuration web page and print jobs worked 10 out of 10 times.

The technician said he didn’t have any of the CAT 6 patch cables but would pick some up at our office later. I reminded him to change the network type back to “Auto” after he changed out the cable. Not that it would really matter because I doubt moving from 100MB to gigabit would spit out the prints any faster.

When prosperity comes, do not use all of it. – Confucius

25.Oct.08 Networking Comments (0)

It’s working? Holy crap! It’s working!

The Cisco NAC is finally working!  Here are the two main things that made it start working:

• A rule was needed to allow pings to all domain controllers. The Active Directory logon process involves pinging all known domain controllers to see if they are online. The pings were being blocked for Unauthenticated users. The blocked pings caused erratic logons, some would authenticate while others didn’t, and it all started working once the pings were allowed.
• Cisco is full of crap when it comes to setting up their NAC to work with their Wireless LAN Controllers. Their directions said to set up WLCs as VPN concentrators. I had it set up like they directed but wireless users still had problems logging on. When I removed all of the settings, in order to try setting it back up from scratch, I happened to see that wireless users were authenticating properly.

I have four remote locations set up with the site Cisco Clean Access Server (CAS) reporting back to a centralized Clean Access Manager (CAM). All four sites had zero issues with users being able to log on, install the agent and get authenticated. Even logon scripts are running properly thanks to a loop that pings a specific set of IPs. Those IPs are blocked by default and can only be reached once the user is dropped into their appropriate user role. Successful ping = logon script execution.

I’ve still got about 20 locations left. Right now, the NAC is only performing authentication and assessing whether or not Windows updates are installed. Once I have all of the locations up, I’m going to implement a few more checks (i.e. antivirus software running and updated). I tested the AV check on a couple of users and it worked properly so I don’t expect any big issues when I role the check out for all locations.

Consider the postage stamp: its usefulness consists in the ability to stick to one thing till it gets there. – Josh Billings

16.Oct.08 Networking, Security Comments (0)

Moving the WSUS database

We use WSUS 3.0 on our corporate network in order to maintain Microsoft patches for the 5,000+ computers.  For the last seven or eight months, everything had been running smoothly.  I received a call from the network engineer in charge of that server this past Tuesday.  He said that he couldn’t connect to the admin interface and he had a lot of SQL errors in the event log.

When I looked at the logs, I saw that the errors were for licensing.  A default installation of WSUS will install Microsoft SQL Server 2005 Embeded Edition. This version of SQL Server is limited version of Microsoft SQL Server 2005 Express Edition that only allows connections from a short list of Microsoft products (i.e. WSUS, Sharepoint, etc.).  SQL Server Express Edition has a file size limit of 4 GB (database files, not logs) and the WSUS database had grown over that limit.

I told the engineer that we should move the database to my SQL Server 2005 Enterprise cluster.  Not only would that allow us to have a larger database, it would be much faster than the embedded SQL Server he was running on the WSUS server.  I found the instructions for performing the move on the Microsoft Technet Windows Server Update Services site.  The article is called Migrating from Windows Internal Database to SQL Server 2005.

There were two things I had to do in order to make the WSUS server work with my SQL Server cluster. Under Migrating the WSUS database from a Windows Internal Database instance to a SQL Server 2005 instance on a remote server:

• Step 7
  • Also had to change the HKLM\SOFTWARE\Microsoft\UpdateServices\Server\Setup\SqlInstanceIsRemote key to 1
• Step 8
  • Instead of starting the IIS Admin Service, I started the World Wide Web Publishing service. The reason is that starting the IIS Admin Service does not start the WWW Publishing service. The WWW Publishing service must be running if you want to connect to the WSUS admin interface (Administrative Tools → Microsoft Windows Server Update Services 3.0).

Our WSUS server is running much better now. The engineer told me that the reports he pulls from the WSUS admin interface run a lot quicker thanks to the better SQL Server. Adding the WSUS database to the SQL Server caused a very small resource hit, even while running large reports that perform multiple queries. I was glad that something went right this week.

A positive attitude may not solve all your problems, but it will annoy enough people to make it worth the effort. – Herm Albright

21.Sep.08 Microsoft SQL, Networking, Software, Technology Comments (6)