I have come across this situation a few times now out in the field so I thought it might be a good time to describe the problem and some easy ways to mitigate or avoid the issue. The January 2010 OCS Server updates related to NTLM reminded me about this issue.
It is pretty common place to lock down, with a GPO or registry setting, the NTLM settings on member servers, domain controllers and client computers. Although uncommon, there are scenarios where you can effectively break any NTLM negotiation between domain controllers, member servers and clients.
Thankfully, these are well documented in KB
823659, “Client, service, and program incompatibilities that may occur when you modify security settings and user rights assignments”.
Consider this scenario:
OCS servers and internal network (local LAN) clients are Active Directory domain joined.
Non-domain joined clients connect through an OCS Edge.
If NTLMMINCLIENTSEC and/or NTLMMINSERVERSEC differ between clients and servers, for the internal clients and servers, Kerberos authentication will still be functional, assuming you haven't set the pool to only accept NTLM.
Clients that connect through the Edge will be rely entirely on NTLM because Kerberos is not available as an authentication method, as noted in the excellent OCS 2007 R2 Resource Kit.
If NTLM is “broken” inside the domain between domain controllers and OCS servers (front End/edge), the Office Communicator client will act as if the user entered an invalid username or password. The error message on the client computer is very misleading and everyone external will not be able to log in.
As noted in TechNet
here, OCS is very particular about the NTLMv2 settings.
These settings, for server and client, can be set in a group policy under
Network security: Minimum session security for NTLM SSP based (including secure RPC), or by use of a registry setting.
To directly quote Technet:
Sometimes the server will be configured to require encryption, and the client will not. In this case, the client NTLM request is not passed on by the front-end server. This situation primarily affects external users, because NTLM is the only authentication protocol that external clients can use to sign in. For example, if the server key is configured to have a value of 0x20080030, which specifies 128-bit encryption, and clients are not, clients will be unable to sign in. You should ensure that this key on the client is configured to match the server’s setting.
As operating systems have evolved, the default security settings for NTLMMINCLIENTSEC and NTLMMINSERVERSEC have been changed, which is a good thing.
By default, anything older than Windows 7 and Server 2008 R2, these registry settings will be configured to
not require 128-bit encryption and
not require NTLMv2 session security.
Windows 7 and Server 2008 R2 require 128-bit encryption by default,
only.
As Microsoft de-emphasizes NTLM in favor of Kerberos and other plug-in authentication methods, you most likely will want to raise the minimum for NTLM for everyone as legacy operating systems are retired from your environment.
You might even want to follow the Server 2008 Security Guide (http://www.microsoft.com/downloads/details.aspx?FamilyID=5534bee1-3cad-4bf0-b92b-a8e545573a3e) when setting up your policies, which specifies requiring 128-bit encryption and NTLMv2 session security.
In a misconfigured environment, the packets going back and forth between the OCS Edge and OCS Front End will look normal except that the NTLM negotiations will always fail.
The obvious fix is to make sure you have these settings consistent across your organization to begin with and you will never see this problem. It can be problematic if you configure all the servers without configuring all clients with identical settings because clients will be unable to connect to your OCS servers through the Edge without modifying their default operating system settings.
In particular, if you have an unmanaged client environment outside of the office (a very common scenario), you might want to provide the following registry file as a way to help secure your environment and enable unmanaged clients to connect:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0]
"NtlmMinClientSec"=dword:20080000
"NtlmMinServerSec"=dword:20080000
Those registry settings above are equivalent to configuring the following group policy with these options, for client and server:
On a somewhat related note, since we are touching upon NTLM, one of the best tutorials about NTLM gotchas appeared in the
August 2006 Security Watch article by Jesper Johansson.
In particular, pay attention to the section detailing the specifics of
LMCompatibilityLevel.
In group policy,
LMCompatibilityLevel, has a description that looks promising as a potential “most compatible secure” setting but does not behave like the description might lead you to believe.
If you set
LMCompatibilityLevel to
1 (Send LM and NTLM – use NTLMv2 session security if negotiated), LM, NTLM and NTLMv2 will be accepted but NTLMv2
will never be sent from the client.
Using
1 might look appealing in a debugging scenario but I would use
LMCompatibilityLevel set to
5 to eliminate use of the LM and NTLMv1 protocols.
The gory details are best explained in the above mentioned Security Watch article.
Overall, assuming all your software and operating systems on your network work properly with NTLMv2, I recommend using the recommendations from the Server 2008 Security Guide and setting
NTLMMINCLIENTSEC and
NTLMMINSERVERSEC to
0x20080000 for all servers and clients. You’ll avoid the OCS Edge headache and you will help avoid older l0phtcrack attacks.
Windows 7 and Server 2008 R2 add some handy NTLM auditing policies that can be used to restrict NTLM but also audit NTLM usage. You can configure the restrictions in audit only mode to see what servers and clients are using NTLM for authentication. It can be handy as a debugging tool and I used it originally when I initially ran into this issue. A good writeup about this can be found
here on the Microsoft site. Using these new tools, you might be find applications you never knew that were using NTLM and potentially verify that you can enable NTLMv2 only everywhere.