Troubleshooting

SharePoint Throwing 503, Cause is Managed Account Failure

I knew that when I had a few dozen emails this morning regarding SharePoint being unavailable for some users that we had a real problem. Generally I can just chalk it up to users not understanding what they are doing, or locking themselves or others out due to configuration changes that they make on their own sites. But this morning seemed legit.

Our production farm has 2 web front ends. Both were “up” but I found that one of them (let’s say WEB01) was throwing lots of “503 Service Unavailable” errors in the logs. I looked at the server and noticed that the main application pool was stopped. So I started it up and went off to look at some information in the logs. I found this:

Application pool SharePoint – 80 has been disabled. Windows Process Activation Service (WAS) encountered a failure when it started a worker process to serve the application pool.

Of course this meant that the affected application pool would stop immediately as well. A bit of research revealed that this probably meant that the identity of the application pool was somehow compromised or not configured appropriately. I am using a SharePoint-managed account for the identity of this application pool and noticed that our password policy had changed the password this morning. Our policy has the passwords changing every month on the 7th between 2 and 3AM.

To make a long story short — somehow the process either did not complete or did not execute on WEB01, and it had invalid credentials for the application pool’s identity. I found that you can force a password reset for the managed accounts inside of SharePoint Central Administration. So I forced that reset, had SharePoint generate its own password, and everything is now fine.

Have others experienced the issue of the passwords not being configured appropriately during one of these changes? Is this a bug in SharePoint 2010?

Directory.GetFiles VS Directory.EnumerateFiles

Where I work, we have fairly large archives of files due to the large volume of messages received from various clients. Most of these messages are received through a VLTrader or ASP(M)X front-end. Eventually they are archived onto the file system according to some pre-determined process. The teams supporting these archives had grown concerned about the ever-increasing amount of storage required for these files. There are thousands of directories (nested many levels deep) and hundreds of thousands of files, potentially millions.

I was asked to help come up with a solution for this problem. The app needed to be configurable when run to specify the root directory and the number of days back to check the date on the file. I needed to allow them to specify that all files older than 90 days should be deleted, for example.

My initial reaction was to use the excellent (and very convenient) System.IO.Directory.GetFiles and System.IO.Directory.GetDirectories methods to simply get an array of the files and directories I would need to enumerate in order to accomplish the task. So I wrote a quick app, utilizing these methods, and saw the IOPS go crazy for a while, then do nothing, then go crazy again. All the while, not much was being accomplished. The issue, as anyone who has tried to “browse” the file system using Windows Explorer may tell you, is that getting the properties of the entire tree, including the number/size of directories and number/size of files, is quite an expensive process.

After doing a bit more research, I came upon the Directory.EnumerateFiles method, which (you guessed it) returns an enumerable collection of file names in a specified path, as opposed to Directory.GetFiles, which returns an array of file names in a specified path. The difference when checking a path with thousands of directories and hundreds of thousands if files is huge. In fact you don’t even have to have that many files/directories to see a dramatic difference. This is only available in .NET 4.0 and above. I have seen others suggest ways of doing something similar with the win32 API, but it was much easier for me to make sure I had .NET 4.0 available than it was to try and implement something using the win32 API.

Usage is simply:

foreach (string file in Directory.EnumerateFiles(rootDirectory, "*", SearchOption.AllDirectories))
                    ShouldDeleteFile(file);

When using these methods, be sure that proper permissions are available on the entire tree. See this post at Stack Overflow for more information. Otherwise you may get an exception. Speaking of permissions — part of my requirement was that I was supposed to delete all files more than 90 days old and all directories which were empty. To avoid any potential conflicts with permissions and/or file properties, the application will run as an administrator and

File.SetAttributes(filePath, FileAttributes.Normal);

is being set each time through. I’m not sure of the performance penalty this may result it. I’ll have to research and see what the hit would be.

Get host name from IP

My networking knowledge is fairly limited. I know more than the average person, I’m sure, but when I start talking to people who are true network engineers, I realize very quickly that my knowledge is pretty cursory.

I had a need recently to get the host name from a windows server. I only had the IP address available. I was told about nbtstat, which allows you to request NetBIOS information from the IP address. I’m sure it’s useful for much more than that, but this was my immediate need.

Its usage is as follows: nbtstat -a ipaddress

T510 Wireless Problems, Connection Drops

I got a new Lenovo T510 laptop at work. My old T60 was on its last leg and was needing some rest. It got to the point where it would literally take about 30 minutes for the thing to fully boot up and become usable. The T510 is nice. The screen is bright and wide, the keyboard is great. The trackpad is a little bit quirky, but that’s probably more a reflection of what I’m used to than anything.

The one main (big, huge) complaint that I have about this thing is that the wireless connection constantly drops, as in (literally) every 60 seconds. The connection will be acceptable, but will literally drop every 60 seconds. It is absolutely maddening.

I have a Linksys WRT160N router (WPA2, static channel, etc) and am running W7 (32-bit). I have 5 other devices connecting to this same router, without issue:

1. Dell XPS M1330 (W7 64-bit)
2. Lenovo T60 (mentioned above, running WXP)
3. Wii
4. Blackberry
5. iMac G5 (an old one — PPC!)

I did have problems with the Dell once upon a time, but a new wireless card solved that one. Perhaps that is what is needed here?

I un-installed the Thinkpad Access Connections software and am just allowing Windows to manage the connection. I also updated the drivers from the Real Tek website. They released a new version last week. As I’ve been typing this post, my connection has dropped and reconnected probably 5 times.

I’m not sure what to do next. Any suggestions?

Update: Well I turned off the wireless N and am just connecting with G. It works OK now.

Update 2: I’m still having the problem where the wireless connection will not return after the computer goes to sleep. It’s incredibly annoying. I end up having to reboot the machine to get the wireless connection back. I’ve tried all sorts of stuff with the power management settings, etc., but nothing really seems to work. Has anyone else experienced this?