Issue

Initial complaint was that all users homed to a specific pool could not reach voicemail, neither could any callers leave messages.

Environment

Lync 2010 with an Enterprise Pool (co-located Mediation servers), multiple SBA’s and SBC’s as well as an Edge Pool. Voicemail in O365. To make things interesting, each pool has its own UM Policy.

Replicated the issue by dialing the subscriber access number in O365. Basically, the call is established, I hear silence for 10 seconds and the call is dropped.

Troubleshooting

Investigating I found that any users homed to the Enterprise Pool were experiencing the issue. Looking at the monitoring reports, diagnostic report shows the diagnostic ID 22 with a reason “Call failed to establish due to a media connectivity failure when both endpoints are internal”. Now that sounds familiar..

In the past I have seen these sorts of issues and found that they usually occur when there is a routing issue involving the Edge Pools and thus a breakdown in the candidate negotiation process. However, this is the first time I have seen this sort of issue when dialing a PSTN number.

Move the user to another pool and the issue disappears, right so pool specific.

Next stop, Edge server to see what its thoughts are on this call failure. Looking at the Event logs I find the error Event ID 14402 which corelates to the attempts to O365 Voicemail.

The non-internal servers that are named in the error is the on-prem UM server. “Add it to the list of internal servers on the Access Edge Server”, well I don’t think I have seen that list since the OCS days. There must be a list of sorts but no bells ringing just yet.

Running snooper trace I got the same Diagnostic ID and reason as we saw earlier in the diagnostic report. But wait, there is also another clue. The exception thrown is “Proxy side ICE connectivity check failed”, and the component is “mediation server”.

Looking at the Lync Workloads Protocal poster, this ICE traffic is on the internal interface. The Lync Workloads Protocal poster also confirms that the components affected are Mediation, UM and Enterprise Pool. So in summary, a specific Pool is unable to complete an ICE check with the Edge Pool. Testing from these servers I can confirm that this traffic is allowed (simply 443 TCP and 3478 UDP).

Wonder if anyone else can shed light on this, off to google. Not a single hit with the same diagnostic error or call scenario. Not entirely surprised, also I suspect that this is not the only symptom of the underlying issue. My google search did however find a possibility that the Mediation service and Edge service are not associated correctley (thanks Mattias)

And there it is..or in the case of the EdgeServer entry for the pool in question, there it is not.

Solution

Add the association between the Mediation Server and the Edge Server, of course. Test and sorted. Found the list 😉