Intermittent mail delivery issues are experienced on BlackBerry smartphones when Microsoft Exchange is configured behind a load balancing appliance

Article ID: KB28704

Type: Support Content

Last Modified: 10-31-2013

 

Product(s) Affected:

  • BlackBerry Enterprise Server for Microsoft Exchange
  • BlackBerry Enterprise Server Express for Microsoft Exchange
CollapseEnvironment
  • BlackBerry Enterprise Server 5.0 SP1 MR1 to 5.0 SP4 for Microsoft Exchange
  • BlackBerry Enterprise Server Express 5.0 SP1
  • Microsoft Exchange 2007 to Microsoft Exchange 2010
CollapseOverview

BlackBerry smartphone user receive a red X when sending new email or replying to emails from the BlackBerry smartphone in what appears to be a random pattern. If the affected user sends the message again, it is delivered.

In environments that incorporate hardware or software load balancing appliances, such as F5 Big-IP, Cisco ACE or Barracuda, issues can be encountered in which sporadic error messages regarding reaching BlackBerry smartphone users' mailboxes or performing basic communication functions appear in both Windows Event Viewer and the BlackBerry Messaging Agent (MAGT) logs. To enable debug logging agent refer to KB00765. These error messages are typically repetitive and occur approximately every 5 to 7 minutes once the initial issue is encountered.

The following is an example of an error message:

[40720] (06/11 16:47:37.522):{0x2308} MAPI call failed. Error 'Network problems are preventing connection to the Microsoft Exchange Server computer.', LowLevelError 6, Component 'Microsoft Exchange Server Information Store', Context 1300
[20301] (06/11 16:47:37.522):{0x2308} {user01@example.com} Unable to save configuration settings or statistics

[20280] (06/11 16:47:39.225):{0xEB8} {user01@example.com} DoICS() failed: ERR_FAIL
[20179] (06/11 16:47:39.225):{0xEB8} {user01@example.com} ConsistencyCheck() failed: ERR_FAIL

[20265] (06/11 16:47:38.381):{0x1AE4} {user01@example.com} MAPIMailbox::GetMessageStatusUpdates - GetContentsTable (0x80040115) failed
[20274] (06/11 16:47:38.381):{0x1AE4} {user01@example.com} GetMessageStatusUpdates() failed

[40202] (06/11 16:37:52.896):{0x20A4} ResolveProxyMAPI(N) - Properties not found
[30186] (06/11 16:37:52.896):{0x20A4} ResolveProxyMAPI(N) failed for 'LastName, FirstName', so using 'EX:/O=company/OU=organizational unit/CN=RECIPIENTS/CN=mailboxname' as the Recip address

[40681] (06/11 16:42:20.097):{0x1654} {user01@example.com} Failed to reach user's mailbox
[20709] (06/11 16:42:20.394):{0x2438} {user01@example.com} Failed to reach user's mailbox

Various versions of ERR_NETWORK_PROBLEM
[20176] (06/11 16:42:18.690):{0xF10} {user01@example.com} GetNewMessages() failed: ERR_NETWORK_PROBLEM
[20185] (06/11 12:26:21.735):{0x25E0} {user01@example.com} GetRIMMessage() failed: ERR_NETWORK_PROBLEM, EntryId=11874
[20190] (06/11 12:26:22.001):{0x2704} {user01@example.com} GetRIMCalendar() failed: ERR_NETWORK_PROBLEM, EntryId=376780
[20205] (06/11 12:26:28.798):{0x18DC} {user01@example.com} AddRefIDProp() failed: ERR_NETWORK_PROBLEM, EntryId=14203
[20216] (06/11 14:58:51.865):{0x1E68} {user01@example.com} Synchronize() failed: ERR_NETWORK_PROBLEM, Tag=276815
[20222] (06/11 16:47:37.522):{0x2308} {user01@example.com} TransactionComplete() failed: ERR_NETWORK_PROBLEM
[20223] (06/11 12:34:14.783):{0x210C} {user01@example.com} MeetingTransactionComplete() failed: ERR_NETWORK_PROBLEM
[20539] (06/11 16:47:38.225):{0x1EB0} {user01@example.com} ERR_NETWORK_PROBLEM(-3174737) failed:

Various versions of 0x80040115 errors
[20001] (06/11 13:23:07.146):{0x1BF4} {user01@example.com} CalSyncState::GetSyncStateTable - GetContentsTable(0x80040115) failed
[20048] (06/11 13:01:38.252):{0x19AC} CalICSAgent:: - LoadSyncInfo Open Root Folder(0x80040115) failed for MailboxName
[20265] (06/11 12:50:17.128):{0x19AC} {user01@example.com} MAPIMailbox::OpenMessage - OpenEntry (0x80040115) failed
[20265] (06/11 12:50:19.879):{0x19AC} {user01@example.com} MAPIMailbox::RefIDtoEntryID - GetContentsTable (0x80040115) failed
[20265] (06/11 12:50:20.645):{0xF10} {user01@example.com} MAPIMailbox::GetNewMessages - GetContentsTable (0x80040115) failed
[20265] (06/11 12:50:26.990):{0x1C18} {user01@example.com} MAPIMailbox::CheckLastChangeTime - GetProps (0x80040115) failed
[20421] (06/11 12:34:10.239):{0x1678} {user01@example.com} - GetAndCreateFolderInfo OpenFolder failed (0x80040115). FolderID=0
[20535] (06/11 13:01:34.767):{0x12FC} {user01@example.com} RescanPIMItems - Failed getting contents table (type=Memos) (0x80040115)
[20535] (06/11 13:01:37.236):{0x1EA8} {user01@example.com} RescanPIMItems - Failed getting contents table (type=Address Book) (0x80040115)
[40405] (06/11 13:23:00.489):{0x1D0C} {user01@example.com} MAPIMailbox::IsMAPIMessageSecureEmail - HrQueryAllRows (0x80040115) failed
[40405] (06/11 14:57:30.214):{0x1EA4} {user01@example.com} MAPIMailbox::SetMessageAsTransferred - SaveChanges (0x80040115) failed
[40406] (06/11 12:26:21.720):{0x1684} {user01@example.com} CDOCalendar::GetMeetingCorrelation - GetProps (0x80040115) failed
[40406] (06/11 12:26:22.267):{0xEF8} {user01@example.com} CDOCalendar::ReadRecurrenceBlob - pMAPIMessage->GetProps (0x80040115) failed
[40406] (06/11 12:55:35.281):{0x1C24} {user01@example.com} CDOCalendar::GetMeetingCorrelation - QueryRows (0x80040115) failed
[40810] (06/11 13:28:36.516):{0x1304} {user01@example.com} UpdatePIMItem - Could not open item for update. OpenEntry (0x80040115) failed

Note: Other error messages also appear, but not as frequently as these error messages appear such as receiving Unlisted Error Message when sending from the smartphone

{0x1378} {user01@example.com} Send() failed: ERR_SUBMIT_MAIL, RefId=1053279042, Tag=2133553

CollapseCause

Hardware and software load balancing appliances actively harden TCP traffic overheads in an effort to stop all potential high bandwidth traffic from causing network delays. All MAPI communication that occurs between the BlackBerry Enterprise Server and the Microsoft Exchange Server is TCP in the form of DCERPC. These connections between Microsoft Exchange and the BlackBerry Enterprise Server are usually open for extended amounts of time. When a load balancer is implemented with a Microsoft Exchange Server, TCP traffic is given a timeout value. This timeout value actively ceases all extended TCP sessions between the two communication points.

The 5 to 7 minute frequency in which error messages appear is caused by the closing of TCP sessions between the BlackBerry Enterprise Server and the Microsoft Exchange Server. When these connections are dropped, the BlackBerry Enterprise Server loses connectivity to the Microsoft Exchange Server and erratic behavior occurs. The load balancing appliance follows TCP Stack Hardening guidelines that force any open TCP traffic that is passing through it and is over the 5-minute threshold to be closed.

CollapseResolution

The use of a CAS Array in Microsoft Exchange 2010 requires the use of a hardware or software load balancer. BlackBerry Technical Support Services supports BlackBerry Enterprise Server connections to Exchange in this configuration, provided that the load balancer is configured properly however, load-balancing BlackBerry Enterprise Server components remains best effort support.

Please contact the vendor of the load balancer for optimizations of DCERPC traffic between the BlackBerry Enterprise Server and Microsoft Exchange components. No specific configuration information is available at this time. Instead, refer to KB26490 if this is not helpful in determining configuration options, the required settings can be described as source-IP based persistent routing.

CollapseWorkaround

Bypass the hardware load balancer with a hosts file entry so that the BlackBerry Enterprise Server may communicate directly with a single Client Access Server.

Important: Restarting the BlackBerry Enterprise Server or its services might delay message delivery to BlackBerry smartphones.

  1. Browse to c:\Windows\System32\drivers\etc and edit the hosts file.
  2. Make an entry with the IP address of one of the Client Access Server nodes and the network name of the hardware load balancer. The name of the hardware load balancer will match the Exchange 2010 CAS Array name. 
     Example: 10.0.0.25 (IP address of the CAS Node) CASArray.domain.com (Fully Qualified Domain Name of the CAS Array)
  3. Save the file.
  4. Stop the BlackBerry Controller and BlackBerry Dispatcher Services.
  5. Open a Command Prompt and run ipconfig /flushdns
  6. Start the BlackBerry Controller and BlackBerry Dispatcher Services.
Note: To also allow for NetBIOS name resolution change the Host file to IP Address TAB FQDN TAB NetBIOS name.
CollapseAdditional Information

To validate that TCP connections are being dropped on the network, use a network capture utility, such as Wireshark, that is installed on the BlackBerry Enterprise Server. The most efficient way to filter traffic is by source and destination IP address. More information regarding Wireshark filters can be found on the Wireshark wiki. The traffic will eventually come back with error messages that indicate that the server is experiencing network connectivity issues.

NOTE: It may be required that  the maximum permitted MAPI sessions per user value is increased on the Microsoft Exchange servers after this change. This will be needed on servers hosting the Mailbox and CAS roles. Article KB20066 has information on how to achieve this. 

This article contains information from previous article KB22812.

Note: A similar error can also been seen if using MAPI CDO 1.2.1 version 6.5.8165 and the BlackBerry Enterprise Server should be using a later version. Additional information available in Microsoft KB2277593

Disclaimer

By downloading, accessing or otherwise using the Knowledge Base documents you agree:

   (a) that the terms of use for the documents found at www.blackberry.com/legal/knowledgebase apply to your use or reference to these documents; and

   (b) not to copy, distribute, disclose or reproduce, in full or in part any of the documents without the express written consent of RIM.


Visit the BlackBerry Technical Solution Center at www.blackberry.com/btsc.