NSPI connection limits when using a Microsoft Windows Server 2008 - 2012 Global Catalog Server - NSPI Bind Limit exceeded

Article ID: KB17325

Type: Support Content

Last Modified: 06-02-2014

 

Product(s) Affected:

  • BlackBerry Enterprise Server for Microsoft Exchange
CollapseEnvironment
  • BlackBerry Enterprise Server for Microsoft Exchange
  • Microsoft Windows Server 2008, 2008 R2, or 2012
  • Microsoft Exchange Server 2003 to 2010
CollapseOverview

In an environment using exclusively, or a majority of, global catalog servers on Microsoft Windows Server 2008, it is possible that the BlackBerry Enterprise Server might encounter a range of issues relating to name resolution. In the BlackBerry Enterprise Server MAGT debug logs, the following events are displayed:

[40206] (12/22 08:58:11.625):{0x1A38} MailboxManager::SubsystemInitialize - Using MAPI profile 'BlackBerryServer'
[50000] (12/22 08:58:11.672):{0x394} Controller: This BES Agent is under control of BlackBerry Agent Controller
[20137] (12/22 08:58:11.719):{0x1A38} MailboxManager::SubsystemInitialize - g_pSession->OpenMsgStore (0x80040111)
[40206] (12/22 08:58:11.719):{0x1A38} MailboxManager::SubsystemInitialize - Using MAPI profile 'BlackBerryServer'
[20137] (12/22 08:58:11.797):{0x1A38} MailboxManager::SubsystemInitialize - g_pSession->OpenMsgStore (0x80040111)
[10277] (12/22 08:58:11.797):{0x1A38} BlackBerry Messaging Agent SERVER01 Agent 1 failed to start. Error code 5305
[50106] (12/22 08:58:11.797):{0x1A38} Stopping BlackBerry Mailbox Agent 1 for Server <BlackBerry Server Name>

or

[40000] (12/22 09:44:49.654):{0x4EC} CDO initializing failure in CDO helper 1046a930 (2)
[30001] (12/22 09:44:49.764):{0x4EC} CDOCalendar::Initialize - Code = 800406f9, WCode = 04f9, Code meaning = IDispatch error #1273,
[30002] (12/22 09:44:49.764):{0x4EC} Server = SERVER01, Mailbox = <Mailbox Distinguished Name> Description = The information store could not be opened. [MAPI 1.0 - [MAPI_E_LOGON_FAILED(80040111)]]
[30180] (12/22 09:44:49.764):{0x4EC} {saskit} CDOCalendar::Initialize - Error in call m_spCalendarFolder = m_spCDOSession->GetDefaultFolder
[40000] (12/22 09:44:49.764):{0x4EC} CDO initializing failure in CDO helper 1046a930 (4)
[30181] (12/22 15:04:32.463):{0x1A38} Performing system health check (BlackBerry Mailbox Agent 1 - BESX Version 4.1.6.11)
[30038] (12/22 15:04:32.463):{0x1A38} Worker Thread: *** No Response *** Thread Id=0x1350, Handle=0x7B0, WaitCount=6, WorkingTime=68 min, LastActivity=68 min, Event: NEW_MB_PCKT_RESCAN, User: user01@example.com, Server: exchserver01, Activity: MAPISendertoRIMSender - RIM_HrGWResolveProxy
[30038] (12/22 15:04:32.463):{0x1A38} Worker Thread: *** No Response *** Thread Id=0x1564, Handle=0x1664, WaitCount=6, WorkingTime=68 min, LastActivity=68 min, Event: NEW_MB_PCKT_RESCAN, User: user01@example.com, Server: exchserver01, Activity: MAPISendertoRIMSender - RIM_HrGWResolveProxy
[50020] (12/22 15:04:32.463):{0x1A38} Some worker threads have been blocked for 6 health checks

or

[20265] (09/08 10:09:54.784):{0x1378} {user@domain.com} MAPIMailbox::Send(ppMAPIMessage) - SubmitMessage (0x80004005) failed
[20472] (09/08 10:09:54.830):{0x1378} {user@domain.com} Send() failed: ERR_SUBMIT_MAIL, RefId=1053279042, Tag=2133553
[40277] (09/08 10:09:54.830):{0x1378} {user@domain.com} Sending message error to device for message 1053279042

Note: For this issue the BlackBerry smartphone will get a red x when sending an email to an external domain but will be able to send internally.

or

[30038] (03/05 12:21:28.390):{0x1C2C} Worker Thread: *** No Response *** Thread Id=0x1808, Handle=0xd84, WaitCount=6, WorkingTime=69 min, LastActivity=69 min, Event: NEW_MB_PCKT_RESCAN, User: user01@example.com, Server: SERVER01, Activity: Starting CDO helper

Note: If the majority of the *** No Response *** threads are Activity: Starting CDO helper then it is most likely this issue. The symptom will be mail delay if this is the log lines that are showing in the MAGT logs.

In addition, it is likely that if this article applies, the BlackBerry Manager will return an error message when opened regarding being unable to open the default message store.

Note: It is possible that any of these events can occur independently and not be related to this article. For this issue to apply, it is likely that two or more of the events will occur.

An alternative manifestation of this issue could be that, following a failover to an apparently healthy BES, the majority of users (90% or more on each agent)fail to start and the health scores indicate a disconnection from the messaging server, as in the following example from the DISP debug log:

[30505] (05/12 20:06:18.949):{0x1538} (001,11) Received REPORT_HEALTH_SCORE command, Health=0x000000000200150D, Mask=0x000000000200150F, Users=60, Servers=0 [30505] (05/12 20:06:18.965):{0x1538} (002,4) Received REPORT_HEALTH_SCORE command, Health=0x000000000200150D, Mask=0x000000000200150F, Users=59, Servers=0 [30505] (05/12 20:06:18.949):{0x1538} (003,7) Received REPORT_HEALTH_SCORE command, Health=0x000000000200150D, Mask=0x000000000200150F, Users=61, Servers=0

This may be accompanied by reports in the MAGT debug logs of network issues preventing connection to the Exchange server.

CollapseCause

Cause 1

NSPI bind limit on the Microsoft Windows 2008 global catalog server reached the limit of 50.

As of Microsoft Windows Server 2008, Microsoft has changed the default behavior of the Domain Controller with regards to Named Service Provider Interface (NSPI) connections. NSPI is the interface that allows Messaging Application Programming Interface (MAPI) to interact with the global catalog server to use the Microsoft Exchange address book and to perform name resolution tasks requiring the information stored in the global catalog. Prior to Microsoft Windows Server 2008, any individual MAPI client could make virtually unlimited numbers of NSPI connections to a global catalog without consequence. In order to more appropriately manage these connections from MAPI clients, Windows Server 2008 introduced a limit of 50 NSPI connections per user. For more details, visit the Microsoft Help and Support site and search for Error: Trying to connect to Microsoft Exchange Server results in MAPI_E_LOGON_FAILED.

This limit has little to no impact on a single user MAPI client; however, the BlackBerry Enterprise Server has to monitor the mailbox for each BlackBerry smartphone user that is added and requires more NSPI connections than an application such as Microsoft Outlook would.

Cause 2

Microsoft Exchange 2010 Service Pack 1 client throttling setting must be applied.

In Microsoft Exchange 2010, the global catalog server (NSPI) concurrent connection limit is increased by changing the <drive>:\Program Files\Microsoft\Exchange Server\V14\Bin\Microsoft.exchange.addressbook.service.exe.config file MaxSessionsPerUser key to 100000. This line is not present in the config file after service pack 1 of Exchange 2010 for those versions only apply the changes to the throttling policy.

However, in Microsoft Exchange 2010 Service Pack 1, the setting was moved to the Client Throttling Policy settings CPAMaxConcurrency, CPAPercentTimeInCAS, and CPAPercentTimeInMailboxRPC.


Cause 3

In Microsoft Exchange 2013 environments the NSPI limit does not affect BlackBerry Enterprise server but if the client throttling setting are not applied similar errors can be seen in the logs. Additional problems can also be encountered preventing sending emails to external domains.

[20265] (05/02 00:38:13.169):{0x18E0} {User@Domain.com} MAPIMailbox::Send(ppMAPIMessage) - SubmitMessage (0x80004005) failed
[20472] (05/02 00:38:13.372):{0x18E0} {User@Domain.com} Send() failed: ERR_SUBMIT_MAIL, RefId=441723089, Tag=46112

CollapseResolution

Cause 1

NSPI bind limit on the Microsoft Windows 2008 global catalog server reached the limit of 50.

Resolution 1

See the Additional Information section to confirm if the NSPI connection limit is the issue.

The number of NSPI connections required by a BlackBerry Enterprise Server can vary based on the number of BlackBerry smartphone users and the frequency of calendaring activity. It is recommended to configure a value based on the maximum possible concurrent connections a BlackBerry Enterprise Server could possibly make, rather than the typical average. For guidelines, see the Additional Information section.

This should be done on any Microsoft Windows Server 2008 global catalog server that the BlackBerry Enterprise Server may be required to connect to (for example, all global catalog servers in the same site as the service account mailbox):

  1. Click Start, click Run, type regedit, and then click OK.
  2. Locate and then click the following registry key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\NTDS
  3. On the Edit menu, point to New, and then click Key.
  4. Type Parameters, and then press ENTER.
  5. Click the Parameters key.
  6. On the Edit menu, point to New, and then click DWORD Value.
  7. Type NSPI max sessions per user , and then press ENTER.
  8. Double-click NSPI max sessions per user, type the maximum number of the NSPI connections, and then click OK.
  9. Exit the Registry Editor.
  10. Restart the computer or restart Active Directory Domain Services.

As long as the limit exceeds the number of required connections found above normal, functionality will be restored.

Cause 2

Microsoft Exchange 2010 Service Pack 1 client throttling setting.

Resolution 2

  1. Apply the following Microsoft Exchange 2010 Service Pack 1 setting to the BESPolicy client throttling policy.
    • Set-ThrottlingPolicy BESPolicy -CPAMaxConcurrency $NULL -CPAPercentTimeInCAS $NULL -CPAPercentTimeInMailboxRPC $NULL
  2. Stop/start the BlackBerry Controller service.

Important: Restarting the BlackBerry Enterprise Server or its services may delay message delivery to BlackBerry smartphones.

Cause 3

Similar errors can be seen in Microsoft Exchange 2013 environments if the client access server (CAS) throttling setting are not set to the recommended levels.

Resolution 3

  1. Apply the following Microsoft Exchange 2013 setting to the BESPolicy client throttling policy and make sure its assigned to the service account.
    • Set-ThrottlingPolicy BESPolicy -RCAMaxConcurrency Unlimited -EWSMaxConcurrency Unlimited
  2. Stop/start the BlackBerry Controller service.

Important: Restarting the BlackBerry Enterprise Server or its services may delay message delivery to BlackBerry smartphones.

Note: For more information, see KB20608.

CollapseWorkaround

In order to limit the impact of this issue, multiple BlackBerry Enterprise Server instances in the environment can be run under different service accounts.

CollapseAdditional Information

Cause 1

Confirming NSPI bind limit issue:

To confirm the cause of the issue, turn on additional logging on the Global Catalog Server by completing the following steps:

  1. On the domain controller that is targeted for the NspiBind connection, click Start, click Run, type regedit, and then click OK.
  2. Double-click the following registry entry:
    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\NTDS\Diagnostics\4 MAPI Interface Events
  3. In the Value data box, type 5, and then click OK.
  4. Restart the computer or restart Active Directory Domain Services.

Open Server Manager and navigate to Diagnostics, Event Viewer, Applications and Services Logs, Directory Services. Monitor the Directory Services event viewer for the following Event ID:

Event ID: 2820
NSPI max connection limit for the user has reached.
You need to do NSPI unbind on old connections before making new connections.
Additional Data
Max NSPI connections per user:
%1
User:
%2

Before making these changes and to obtain current information, refer to the Microsoft Help and Support site website and search for KB949469.

This information has been included under the Pre-requisites section of the BlackBerry Enterprise Server for Microsoft Exchange Installation and Configuration Guide for 5.0 SP2.

Guidelines for NSPI bind limit value:

It is suggested to account for 1000 NSPI Connections per 1000 BlackBerry smartphone users on a BlackBerry Enterprise Server, rounded up to the nearest thousand. For example, a BlackBerry Enterprise Server with 0 to 1000 BlackBerry smartphone users could use 1000 NSPI connections. A BlackBerry Enterprise Server with 1001 to 2000 users could use 2000 NSPI Connections.

It is important to note that the above method of determining the number of required NSPI connections is on a per server basis. The limit on NSPI connections is on a per user basis. If there are multiple BlackBerry Enterprise Server instances in the environment running with the same service account, the number of NSPI connections that all the BlackBerry Enterprise Server instances require must be added together. For example, four BlackBerry Enterprise Servers with 600 users each will use up to 2,400 concurrent NSPI connections and therefore, the NSPI limit should be set to 3000.

For example, in an environment with five BlackBerry Enterprise Server instances averaging 1500 BlackBerry smartphone users, each where all BlackBerry Enterprise Server instances run under the same service account, the maximum number of concurrent NSPI Connections required would be 10000. In an environment with the same five BlackBerry Enterprise Server instances where each server runs under a different BlackBerry service account, the maximum number of concurrent connections required would only be 2000.

Disclaimer

By downloading, accessing or otherwise using the Knowledge Base documents you agree:

   (a) that the terms of use for the documents found at www.blackberry.com/legal/knowledgebase apply to your use or reference to these documents; and

   (b) not to copy, distribute, disclose or reproduce, in full or in part any of the documents without the express written consent of RIM.


Visit the BlackBerry Technical Solution Center at www.blackberry.com/btsc.