Unified Comm

The Exchange Pre-Deployment Analyzer performs an overall topology readiness scan of your environment and provides you with a list of decisions that need to be made before you deploy Exchange Server 2010.

The checks performed by ExPDA are similar to the pre-requisite checks implemented (via Exchange Best Practices Analyzer) in the Exchange 2010 Setup program; in fact ExPDA is based off the Exchange Best Practices Analyzer (ExBPA) engine. However, unlike Exchange 2010 setup, this tool focuses only on overall topology readiness and not the ability to run Exchange 2010 on the local computer. The scan also performs a deep analysis of each existing Exchange 2003/2007 server to verify that it has the necessary updates and configuration in-place to support Exchange 2010.

Another good thing to note that hasn't been documented very well, but I have seen happen a few times already:

If you plan on ever deploying Exchange 2007 portions with Exchange 2010, for instance if you are coming from an Exchange 2003 environment and need a few Exchange 2007 servers due to 3rd party applications not catching up to Exchange 2010 yet, make sure you run the schema update for Exchange 2007 before running the schema update for Exchange 2010.

If you run the Exchange 2010 schema update without ever having any Exchange 2007 schema in place, you can no longer install Exchange 2007. You can go back to AD backups and recover, and I have heard of some unsupported/daring changes to schema versions to force it to work, but for pre-deploying, make sure you take this into account.

A common scenario like this is with Cisco Unity or some archival products. Little by little, they are catching up to Exchange 2010 with support.

I know it is a work in progress but I believe that this hotfix will end up being a pre-req for installing on Server 2008 R2.

KB 975858 - An application or service that calls the InitializeSecurityContext function together with the ISC_REQ_EXTENDED_ERROR flag may encounter a TLS/SSL negotiation failure on a computer that is running Windows Server 2008 R2 or Windows 7

Specifically due to this section of the article:
For example, a Transport Layer Security (TLS) negotiation fails when Microsoft Office Communicator Server 2007 R2 is installed on a computer that is running Windows Server 2008 R2. This problem causes some functions not to work in Office Communicator Server 2007 R2.

A few months ago I detailed about a problem with OCS services not starting after a reboot, a general loss of network connectivity during boot and the inability to RDP to those servers once this scenario was triggered.

I bumped into a new article from a Paul Adams at MS that has a handy PowerShell script to add in the workaround for the deadlock problem. It is also noted, just from general banter on the Internet, forums and newsgroups, that the OCS servers involved are typically HP servers that used the SmartStart CD/DVD for installation. It could be that HP agents, drivers or services added to the base OS are causing this issue to flare up. Any OCS servers I have seen, so far, that had this problem were HP servers.

The PowerShell script can be downloaded from here, and it refers to an earlier post by the same author detailing the technical reasons as to why and how it can happen. Ideally, I'm hoping a more generic/official fix for it becomes available from Microsoft at some point for the OS. OCS unfortunately is just a possible trigger for this issue, and the issue comes from the underlying operating system.

Update: In a weird twist of fate, this issue/bug does not exist with Server 2008 R2, but OCS 2007 R2 isn't supported under Server 2008 R2 yet.

I now have complete feature parity with a 64-bit OS (Windows 7) running 64-bit Office 2010 with LiveMeeting and Office Communicator, that I had with 32-bit Office 2007 not too long ago.

You can read about the changes for LiveMeeting here and the changes for the Conferencing Add-in here.

You can download the updated LiveMeeting client here and the updated Conferencing Add-in here.

For the add-in, you will have to choose 32-bit or 64-bit which depends on which flavor (32-bit or 64-bit) of Office you are using. The only 64-bit version of Office to date is the Office 2010 Technical Preview/Beta. Any version before that is 32-bit.

Anyone running a 32-bit OS is going to use 32-bit Office and will want the 32-bit version.

You can certainly run 32-bit Office on a 64-bit OS, and in this case, you would want the 32-bit Conferencing Add-in. One reason I can think of is if you want to keep using Xobni until a new version of that application comes out, or you have other 32-bit Office plugins that you use.

I am using 64-bit Office primarily for a relatively strange reason. Sure, running native applications is a good thing, but I am mainly interested in applications not "mucking around" with my Office configuration by adding in things covertly.

The biggest offender I can rattle off the top of my head is the plugin that iTunes installs into Outlook without asking. It has a notorious history of causing problems with delaying Outlook to shut down. I'm not picking on Apple, I just wish it was an optional component of iTunes.

Plus, I'm just a fan of 64-bit applications in general.

Note: One limitation of using Live Meeting on a 64-bit OS still remains - uploading Office documents other than PowerPoint to LiveMeeting and having LiveMeeting dynamically convert/scale the documents to each client's workspace. I generally don't use this feature so it is a non-issue for me.

The technical reason - a 64-bit Microsoft Office Document Imaging (MODI) print driver does not exist.

In July 2007, I was originally awarded Microsoft MVP for Windows User/Shell. This area tended to confuse people because they thought it was related to batch files or PowerShell.

In the following months, I was nominated for Forefront MVP and OCS MVP. Both were very flattering because it meant I was getting attention from Microsoft from multiple sources. At the time, I had no idea you could have a 'primary' competency and a few secondaries. OCS and Forefront ended up as my secondary competencies.

2007 and 2008 were very busy years for this blog and for myself. Mostly positive, and I enjoyed what I was doing.

Thankfully, in July 2008, I was re-awarded the MVP and the category changed from Windows User/Shell to Windows Desktop Experience. This confused people much less and I was honored to be chosen again.

Midway through that MVP year, I was invited into the newly formed Windows Systems and Performance MVP group. It was a smaller group of MVPs and more in line with the topics I had been covering during that time. A great match, and I was able to keep my same MVP lead.

At the end of 2008, my work had a reorganization and I changed from being just a Microsoft Consultant to a Microsoft Unified Communications Consultant. This meant my day to day work life would revolve around Exchange 2010 and OCS 2007 R2 for the most part. Thankfully, I've been doing Exchange since version 5.5 and OCS since it was called LCS 2005, in beta.

Overall, this meant this blog would slightly shift focus as I worked specifically on UC projects.

In July 2009, I was re-awarded the MVP for Windows Systems and Performance.

After a lot of thought and consideration, I recently asked to be moved from the Windows Systems and Performance MVP group over to the OCS MVP group.

I'm happy to announce that they approved the move and now, I'm in my 3rd MVP category in 3 years, Office Communications Server.

In general, that doesn't mean I am going to leave the other aspects I've traditionally covered or gave speeches about, including Windows 2008 R2, Windows 7, Exchange 2010 and anything else that captures my attention.

Overall, I'm just excited to be in this MVP group as it aligns with my work environment.

At some point, I want to go for the OCS MCM (Microsoft Certified Master).

I have come across this situation a few times now out in the field so I thought it might be a good time to describe the problem and some easy ways to mitigate or avoid the issue. The January 2010 OCS Server updates related to NTLM reminded me about this issue.

It is pretty common place to lock down, with a GPO or registry setting, the NTLM settings on member servers, domain controllers and client computers. Although uncommon, there are scenarios where you can effectively break any NTLM negotiation between domain controllers, member servers and clients.

Thankfully, these are well documented in KB 823659, “Client, service, and program incompatibilities that may occur when you modify security settings and user rights assignments”.

Consider this scenario:

OCS servers and internal network (local LAN) clients are Active Directory domain joined.
Non-domain joined clients connect through an OCS Edge.

If NTLMMINCLIENTSEC and/or NTLMMINSERVERSEC differ between clients and servers, for the internal clients and servers, Kerberos authentication will still be functional, assuming you haven't set the pool to only accept NTLM.

Clients that connect through the Edge will be rely entirely on NTLM because Kerberos is not available as an authentication method, as noted in the excellent OCS 2007 R2 Resource Kit.

If NTLM is “broken” inside the domain between domain controllers and OCS servers (front End/edge), the Office Communicator client will act as if the user entered an invalid username or password. The error message on the client computer is very misleading and everyone external will not be able to log in.

As noted in TechNet here, OCS is very particular about the NTLMv2 settings.

These settings, for server and client, can be set in a group policy under Network security: Minimum session security for NTLM SSP based (including secure RPC), or by use of a registry setting.

To directly quote Technet:
Sometimes the server will be configured to require encryption, and the client will not. In this case, the client NTLM request is not passed on by the front-end server. This situation primarily affects external users, because NTLM is the only authentication protocol that external clients can use to sign in. For example, if the server key is configured to have a value of 0x20080030, which specifies 128-bit encryption, and clients are not, clients will be unable to sign in. You should ensure that this key on the client is configured to match the server’s setting.

As operating systems have evolved, the default security settings for NTLMMINCLIENTSEC and NTLMMINSERVERSEC have been changed, which is a good thing.

By default, anything older than Windows 7 and Server 2008 R2, these registry settings will be configured to not require 128-bit encryption and not require NTLMv2 session security.

Windows 7 and Server 2008 R2 require 128-bit encryption by default, only.

As Microsoft de-emphasizes NTLM in favor of Kerberos and other plug-in authentication methods, you most likely will want to raise the minimum for NTLM for everyone as legacy operating systems are retired from your environment.

You might even want to follow the Server 2008 Security Guide (http://www.microsoft.com/downloads/details.aspx?FamilyID=5534bee1-3cad-4bf0-b92b-a8e545573a3e) when setting up your policies, which specifies requiring 128-bit encryption and NTLMv2 session security.

In a misconfigured environment, the packets going back and forth between the OCS Edge and OCS Front End will look normal except that the NTLM negotiations will always fail.

The obvious fix is to make sure you have these settings consistent across your organization to begin with and you will never see this problem. It can be problematic if you configure all the servers without configuring all clients with identical settings because clients will be unable to connect to your OCS servers through the Edge without modifying their default operating system settings.

In particular, if you have an unmanaged client environment outside of the office (a very common scenario), you might want to provide the following registry file as a way to help secure your environment and enable unmanaged clients to connect:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0]
"NtlmMinClientSec"=dword:20080000
"NtlmMinServerSec"=dword:20080000

Those registry settings above are equivalent to configuring the following group policy with these options, for client and server:




On a somewhat related note, since we are touching upon NTLM, one of the best tutorials about NTLM gotchas appeared in the August 2006 Security Watch article by Jesper Johansson.

In particular, pay attention to the section detailing the specifics of LMCompatibilityLevel.

In group policy, LMCompatibilityLevel, has a description that looks promising as a potential “most compatible secure” setting but does not behave like the description might lead you to believe.

If you set LMCompatibilityLevel to 1 (Send LM and NTLM – use NTLMv2 session security if negotiated), LM, NTLM and NTLMv2 will be accepted but NTLMv2 will never be sent from the client.

Using 1 might look appealing in a debugging scenario but I would use LMCompatibilityLevel set to 5 to eliminate use of the LM and NTLMv1 protocols.

The gory details are best explained in the above mentioned Security Watch article.

Overall, assuming all your software and operating systems on your network work properly with NTLMv2, I recommend using the recommendations from the Server 2008 Security Guide and setting NTLMMINCLIENTSEC and NTLMMINSERVERSEC to 0x20080000 for all servers and clients. You’ll avoid the OCS Edge headache and you will help avoid older l0phtcrack attacks.

Windows 7 and Server 2008 R2 add some handy NTLM auditing policies that can be used to restrict NTLM but also audit NTLM usage. You can configure the restrictions in audit only mode to see what servers and clients are using NTLM for authentication. It can be handy as a debugging tool and I used it originally when I initially ran into this issue. A good writeup about this can be found here on the Microsoft site. Using these new tools, you might be find applications you never knew that were using NTLM and potentially verify that you can enable NTLMv2 only everywhere.

This update (KB 976135) contains the following fixes:

KB 978161 - Office Communicator 2007 R2 crashes when you receive an incoming instant message notification

KB 978162 - A hyperlink is converted to plain text format when you input the hyperlink into an Office Communicator 2007 R2 conversation window

KB 978163 - You cannot call a user by using the Actions button in Office Outlook 2007, but you can call the user through Office Communicator 2007 R2

You can download the update here.

I'm not sure if this client update addresses the compact delta GAL/stale GAL problem mentioned here yet or not.

Update: I had the version wrong in the title originally and a few new discoveries have surfaced. This version has the proper 64-bit MAPI/EWS/API support for 64-bit Office 2010, which means that Outlook integration (missed call notification, voice mail notification, "In a meeting" status) works for people brave enough (like me) to go completely 64-bit with Office 2010.

Also of note, which might be a side effect of the new 64-bit support, the client is now reporting "x64" instead of "x32" to the OCS Client Filter, as already noted numerous places. Read more about it here and here. For now, it is best just to duplicate your "x32" directory structure as "x64" until a "real" x64 client is available with patches or further guidance is provided from MS. I'm going to update my Client Filter post to reflect this shortly too.

Although the release notes are not available yet, this brings the Group Chat version to 3.5.6907.83, up from the last version 3.5.6907.41 in October 2009.

When published, the details will be at KB 977338.

You can download the update from here.

Although the release notes are not available yet, this brings the Tanjay/Phone Edition version to 3.5.6907.82, up from the last version 3.5.6907.55 in October 2009.

When published, the details will be at KB 978737.

You can download the update from here.

Not a lot of changes, server side for OCS 2007 R2 in these updates but a few that have been longstanding annoyances. On a plus note, these updates remove the need for applying the OCS ASN fix caused by KB 974571 (MS09-056).

For the overall list of server updates and update guidance, check out KB 968802.

If you look over the list at the KB article above, you'll notice that only certain components were updated for January 2010:

Administration Tools (AdminTools.msp) and Core Components (Ocscore.msp):

KB 978371 - An offline certificate request is formatted incorrectly when you create the certificate request in Office Communications Server 2007 R2 on a computer that is running Windows Server 2008

This fix is particularly welcome because the certificate request output varied depending on if the server used Server 2003 or Server 2008. Some certificate authorities require -----BEGIN CERTIFICATE REQUEST----- at the beginning of the certificate request and -----END CERTIFICATE REQUEST----- at the end.

Server (Server.msp):

KB 978363 - You experience performance issues in Office Communications Server 2007 R2 during NTLM authentication

KB 978364 - You experience performance issues in Office Communications Server 2007 R2 when you move many users between pools at the same time

KB 978366 - Instant messages are blocked in Office Communications Server 2007 R2 if a third-party application uses the "otherIpPhone" attribute

KB 978367 - You cannot start the Office Communications Server 2007 R2 Front-End service if the connection to the root domain controller is lost

KB 978369 - Instant messaging fails when the Office Communications Server 2007 R2 front-end server that you connect to changes

Database (OCS2007-DBUpgrade.msi):

Not installed by ServerUpdateInstaller.exe - must be installed manually with all the other updates if you want proper operation.

The link to the Download Center seems invalid at the moment but the Request Hotfix link works.

KB 978362 - You experience performance issues in Office Communications Server 2007 R2 when many legacy presence users are converted to enhanced presence users at the same time

Unified Communications Managed API 2.0 Core Redist 64-bit:

KB 978372 - CPU usage increases when you turn on tracing for the S4 component in Office Communications Server 2007 R2

As always, the easiest method of installing the updates is to use the ServerUpdateInstaller.exe, combined with the manual application of the database update.

I have seen sporadic reports of stale GAL (Global Address List/Book) reports from time to time from end users and on user forums.

9 times out of 10, a simple deletion of the galcontacts.db and galcontacts.db.idx located in %userprofile%\Appdata\Local\Microsoft\Communicator\\ would solve the problem.

It was noted that the RTM version of Communicator 2007 R2 client (6.5.6907.0) had less reports of the issue. I chalked it up as a bit of a fluke because there didn't seem to be a common pattern other than stale info and a possible bug/change in behavior in post-RTM updates.

Using my workstation as an example path for the galcontact[.db/.idx], under Vista, Windows 2008 R1/R2, and Windows 7:
C:\Users\atiensivu\AppData\Local\Microsoft\Communicator\sip_aaron@tiensivu.com\GalContacts.db
C:\Users\atiensivu\AppData\Local\Microsoft\Communicator\sip_aaron@tiensivu.com\GalContacts.db.idx
For earlier versions of Windows, the path would be:
C:\Documents and Settings\atiensivu\Local Settings\Application Data\Microsoft\Communicator\sip_aaron@tiensivu.com\GalContacts.db
C:\Documents and Settings\atiensivu\Local Settings\Application Data\Microsoft\Communicator\sip_aaron@tiensivu.com\GalContacts.db.idx

Now, however, we have an answer for the stale GAL problem with this OCS escalation scenario documented here. I really appreciated that they posted all the technical details.

Most notably:
The behavior of the Communicator client has been changed in .37+, whereby Communicator downloads Full Files ONLY if the local GalContacts.DB hasn’t been updated in last 7 days AND a newer delta file is not available. Thus if the server generates full files but no corresponding delta file for 7 consecutive days then ONLY the client will download a full file.

As a result the local GAL stops gets updated daily since the Delta Files exists, but the Hash stored in the Delta File (H2) does not match the Hash stored in local GAL (H1).

Resolution

#1 Stop running manual absserver -syncNow and absserver -regenUR all together as per http://technet.microsoft.com/en-us/library/ee323612(office.13).aspx. (Most standard installs do not do this to begin with - you would have to set up a scheduled task manually to do this)

#2 Allow a 24 hour period to elapse after performing Step 1.
#3 Delete the GalContacts.DB on every client machine for every user ONLY ONCE for the next logon.

I'm hoping a future Communicator update fixes up this logic change but in the meantime, it can be worked around.

Unfortunately, with the logic change in 3.5.6907.37+, the behavior is now dependent on various patch levels of the OCS client, which is something that was problematic with a few of the Communicator 2007 R1 updates. I'm hoping TechNet is either updated or the behavior is refined a bit.

However, I doubt most installations will have seen this overall.

I figured I'd add my on-site/consultant perspective and get the word out a bit more since the July 2009 updates and newer patches have been offered on Microsoft Update and my standard mode of operation for almost all installations is to use the latest MOC client available.

I have some time off this week from work so I am finishing up a few more things to post here. Mainly OCS, Exchange 2010 and Windows 7/2008 R2 related. Good stuff!

BTW: New Year's Resolution - 1920x1200x32bit - bad joke, I know.

Announced today at PDC and Twitter'd by @DrRez - a nice portal/landing page for OCS resources. Hopefully some of my articles will end up on there someday.

So I have a history of getting interesting projects, whether they start out relatively simple or are complex from the start. Sometimes, the seemingly standard/not-out-of-the-ordinary deployments end up with the most obscure problems. So far, I've only seen this happen once and was able to recreate it the issue as long as the same Active Directory information was used.

Background on the scenario:
Install any OCS role that depends heavily on WMI (practically all of them - the only roles not affected were the Archiving, Monitoring, and Communicator Web Access roles) on a Server 2008 SP2 server with .NET 3.5 SP1 and the usual post-3.5 SP1 updates (KB 958481 + 958483 + 958484).

Upon reboot, the networking stack would not come online and the system would stall during logon at the "Applying Internet Explorer Policy" or "Preparing Desktop". After waiting for the failed services to finally stall out (varies per server role), you will notice that most or all of the OCS related services are stuck at 'Starting'. RDP will be unavailable. The network icon will show as if the cable is disconnected.

If you set all the OCS services to 'Manual' and reboot, everything will start up normally. If you log in and start up the services manually, everything works fine. Obviously, this isn't an ideal setup for a production server.

During a sleepless weekend of research, I bumped into a few others that had experienced the same issue, most notably here. We ended up using the work around of placing all the OCS services dependent on wmiApSrv. In addition to that, we also changed the services startup from "Automatic" to "Automatic (delayed)".

Soon after, Ben Kohn (@axilar) found an interesting Server 2008 specific KB article that seemed to explain the situation completely, namely KB 2004121.

For future installs that have this problem, if I run into it again, I'll be doing the following, which comes from the above mentioned KB article:

To resolve this issue we need to modify the behavior of HTTP.SYS to depend on another service being started first.

1. Open Registry Editor
2. Navigate to HKLM\CurrentControlSet\Serivces\HTTP and create the following Multi-string value: DependOnService
3. Double click the new DependOnService value that you created
4. Enter CRYPTSVC in the Value Data field and click OK

The only aspect that I am still fuzzy on - why doesn't this show up in more OCS installs under Server 2008? Is it configuration specific? Obviously the CRYPTSVC isn't getting triggered on demand properly which in turn is causing the HTTP services to fail to start, which in turn causes this cascading OCS service/networking failure.

I'm glad it is fixable and you can work around it but I'm still puzzled.

As noted here, the OCS team has setup a page on TechNet that lists the latest updates and download links for:

Server side (as of 11-17-09):

Office Communications Server 2007 R2 (3.5.6907.56)
Cumulative Updates for Office Communications Server 2007 R2 - October 2009

Update for OCS 2007 R2 Database – July 2009

Standalone all-in-one updater for every server role (except July 2009 SQL database update mentioned above - must be installed otherwise Dial-in Audio Conferencing and other functionality will break - needs SQL 2005/2008 workstation client tools installed from SQL 2005/2008 installation media)

October 2009 ASN fix for OCS Front End and Edge roles due to KB 974571 (MS09-056)

Speech Server 2007 R2
Update for Speech Server 2007 R2 – April 2009

Group Chat Server 2007 R2 (3.5.6907.41)
Update for Group Chat Server 2007 R2 – July 2009
Update for Group Chat Server 2007 R2 Administration Tool - July 2009

Best Practices Analyzer (3.0.6362.80)
Office Communications Server 2007 Best Practices Analyzer

Client side (as of 11-17-09):

Office Communicator 2007 R2 (3.5.6907.56)
Update for Office Communicator 2007 R2 – October 2009
Mac Messenger 7.0.2

Office Live Meeting (8.0.6362.143)
Update for Live Meeting 2007 Client – September 2009 (latest version is always located here)
Update for Live Meeting 2007 Conferencing Add-in for Outlook – September 2009 (latest version is always located here)

Group Chat Client 2007 R2 (3.5.6907.41)
Update for Group Chat Client 2007 R2 – July 2009

Attendant 2007 R2 (3.5.6907.37)
Update for Office Communications Server 2007 R2 Attendant – July 2009

Office Communicator Phone Edition 2007 R2 - Tanjay (3.5.6907.55)
Update for Office Communicator Phone Edition 2007 R2 – October 2009

Office Communicator Mobile 2007 R2 (6906.28)
Update for Office Communicator Mobile 2007 R2 – July 2009 (latest version is always located here)

It is cool that they have done this because I've had a blog entry stuck in Draft mode ever since the July and October 2009 updates for OCS were released. It was going to be a two part post that I had planned on publishing shortly.

The first part would have been much like the diagram/links posted here and the links above are a mixture of that chart and my original post.

The second part would have been links to the individual KB articles and the download binaries and which download binaries matched up with the KB articles. If you downloaded each part individually, it was very easy to mismatch the .MSP files with the server roles/KB articles. I am going to make an abbreviated version of the original post after this one because there are some aspects that are still valid/noteworthy.

If you want a quick reference for customers or yourself, which seems to be updated as more products become supported under 2008 R2, go to the Microsoft website here.

The most interesting aspect, at least to me, is that there looks to be upcoming support for Server 2008 R2 for OCS 2007 R2 and later versions, and as mentioned on the Exchange blog, support for Exchange 2007 SP2+. The reversal of support on Exchange is a very welcome one in my eyes.