Introduction

This document describes how to troubleshoot the Jabber Log in when it fails on an internal or corporate network.

Background Information

Jabber Log in comprises of two stages; Cisco Unified Communications Manager server (CUCM) Log in and IM and Presence server (IM&P) Log in.

How to Collect Logs

Clear the cache on the PC and collect the clean Jabber Problem Report (PRT).

Step 1. Sign out and exit the Jabber application.
Step 2. Delete all the logs located at

%AppData%\Local\Cisco\Unified Communications\Jabber\
%AppData%\Roaming\Cisco\Unified Communications\Jabber\

Step 3. Restart Jabber and recreate the problem.

Step 4. Collect the Problem report. (From the Jabber Help menu , select Report a problem option to launch the problem report tool. The instructions are found there)

Link to these resources:

Keywords to Search in Logs

IMPStackCap::Log in::OnLog inError 
ServiceDiscoveryHandlerResult
@CupSoapCli: log in cup succeeds - shows when the SOAP log in was successful.
[CTriTPConnectionClient::OnConnectSuccess] - @XmppSDK: - shows when the XMPP log in was successful.
LERR - shows the Log in Errors when the Jabber fails to log in to the IM&P Server.

Stages to Troubleshoot

Stage 1. CUCM Service Discovery

Sample Log Snippet

017-03-19 17:55:00,422 WARN  [0x000050ac] [src\dnsutils\win32\win32DnsUtils.cpp(52)] [csf.dns]   [csf::dns::mapFromWindowsDNSResult] - *-----* DNS query _cisco-uds._tcp.applab has failed: DNS name does not exist.  (9003).

2017-03-19 17:55:00,438 WARN  [0x000050ac] [src\dnsutils\win32\win32DnsUtils.cpp(52)] [csf.dns]   [csf::dns::mapFromWindowsDNSResult] - *-----* DNS query _cuplogin._tcp.applab has failed: DNS name does not exist. (9003).

Steps to Resolve

Step 1. Start the command prompt (on a Windows Client) and then enter nslookup .

Step 2. Set the query type to SRV
set type = SRV

Step 3. Insert the SRV record we need to check
_cisco-uds._tcp.example.com

Step 4. This returns the DNS A records which point to the CUCM servers.

This is an example of the Successful _cisco-uds SRV record.

If no records are returned, contact your DNS administrator to configure the SRV records

Steps to Resolve

Step 1. Validate that the CUCM nodes configured as TFTP Servers are up and running.

Step 2. Restart these services on all the CUCM nodes.

Stage 2. CUCM User Authentication

Sample Log Snippet

2017-01-09 08:59:10,652 INFO  [0x00001740] [vices\impl\DiscoveryHandlerImpl.cpp(460)] [service-discovery] [CSFUnified::DiscoveryHandlerImpl::evaluateServiceDiscoveryResult] - ServiceDiscoveryHandlerResult return code FAILED_UCM90_AUTHENTICATION

Steps to Resolve

Step 1. Ensure that the User is configured as an enduser in CUCM. Navigate to CUCM Administration > Enduser page.

Step 2. Verify that credentials are correct and the user is active. Log in to the CUCM Self-care Portal.

This image refers to the scenario where the LDAP is unable to authenticate the user either because the user is not a valid user or the password supplied is incorrect.

Step 3. If this issue is seen for all users, verify if the LDAP synchronisation and LDAP Authentication settings on CUCM Administration > System > LDAP is correct.

Tip : From the LDAP Server perspective, ensure that the account is not locked, that the passwords are not expired, and that all the users are synchronized with the CUCM server.

Sample Log Snippet

2017-08-28 12:04:00,282 INFO  [0x00004290] [vices\impl\DiscoveryHandlerImpl.cpp(452)] [service-discovery] [CSFUnified::DiscoveryHandlerImpl::evaluateServiceDiscoveryResult] - ServiceDiscoveryHandlerResult return code FAILED_UCM90_CONNECTION

Steps to Resolve

Step 1. Test if you are able to open this URL in the browser on the PC https://<CUCM IP/FQDN> :8443/cucm-uds/version

Unsuccessful

Step 2. If the response is unsuccessful, verify that the DNS is configured correctly to resolve them and also if no Network Elements like Firewall/ASA block the port 8443.

Step 3. This URL must be tested for all CUCM Servers in the cluster. For a list of Servers, navigate to CUCM Administration > System > Server.

Sample Log Snippet

2016-08-18 13:14:49,943 INFO  [0x000036e4] [vices\impl\DiscoveryHandlerImpl.cpp(367)] [service-discovery] [DiscoveryHandlerImpl::evaluateServiceDiscoveryResult] - ServiceDiscoveryHandlerResult return code FAILED_USER_LOOKUP

Steps to Resolve

Step 1. Verify that you are able to open this URL in the browser on the PC https://CUCM:8443/cucm-uds/clusterUser?username= <userid>

Step 2. Verify that the userid that is entered in Jabber matches the userid on CUCM End-user page.

Tip : Jabber has UPN discovery enabled by default and hence get the userid prepopulated from the LDAP UPN field. Check if UPN is the same as configured in CUCM, if you need to disable UPN discovery, set U PN_DISCOVERY_ENABLED =false during installation

Stage 3. SOAP Log in (IM and Presence Log in)

Sample Log Snippet

2017-01-14 15:55:09,615 INFO  [0x00000dc0] [ts\adapters\imp\components\Login.cpp(99)] [imp.service] [IMPStackCap::Login::OnLoginError] - ****************************************************************
2017-01-14 15:55:09,615 INFO  [0x00000dc0] [s\adapters\imp\components\Login.cpp(100)] [imp.service] [IMPStackCap::Login::OnLoginError] - OnLoginError: (data=0) LERR_CUP_AUTH <12>:
201-01-14 15:55:09,615 INFO  [0x00000dc0] [s\adapters\imp\components\Login.cpp(101)] [imp.service] [IMPStackCap::Login::OnLoginError] - ****************************************************************

Steps to Resolve

Step 1. Verify that the user is assigned to a Presence node and there are no duplicates for the user ( IM and presence Administration > Diagnostics > System troubleshooter ).

Step 2. Verify that the High Availability (HA) state is Normal and no Failover has occurred.

If you have tried to assign the user during an abnormal HA state, Users are not assigned to any IMP node and log in fails.

Recover the HA state first and re-assign the user.

Step 3. Verify that the credentials are valid.

Step 4. Check the server has high TOMCAT CPU consumption

Step 5. Set these services log to DEBUG and then recreate the Log in issue and collect the logs

Sample Log Snippet

[IMPServices] [CSFUnified::IMPStackCap::Log in::OnLog inError] - ****************************************************************
[IMPServices] [CSFUnified::IMPStackCap::Log in::OnLog inError] - OnLog inError: LERR_CUP_AUTH <10>: 
[IMPServices] [CSFUnified::IMPStackCap::Log in::OnLog inError] - ****************************************************************
[http-bio-443-exec-15] handlers.Log inHandlerAbstract - preLog in:PRELOGIN reasoncode=FAILURE. User either not CUP licensed or not found in database.

Tip : For this error, it is also recommended to retrieve the Cisco Tomcat logs from the CUCM and IM&P servers.

From the Cisco Tomcat logs

2019-10-27 18:33:40,373 DEBUG [http-bio-443-exec-5] impl.LDAPHostnameVerifier - check : inside check with X509 cert
2019-10-27 18:33:40,373 DEBUG [http-bio-443-exec-5] impl.Certificates - getCNs :
2019-10-27 18:33:40,373 DEBUG [http-bio-443-exec-5] impl.LDAPHostnameVerifier - check : cns = [ldap.ciscolab.com]
2019-10-27 18:33:40,373 DEBUG [http-bio-443-exec-5] impl.Certificates - getDNSSubjectAlts :
2019-10-27 18:33:40,374 DEBUG [http-bio-443-exec-5] impl.LDAPHostnameVerifier - check : subjectAlts = [ldap.ciscolab.com, ldap2.ciscolab.com]
2019-10-27 18:33:40,374 ERROR [http-bio-443-exec-5] impl.AuthenticationLDAP - verifyHostName:Exception.javax.net.ssl.SSLPeerUnverifiedException: hostname of the server 'ldapdc.ciscolab.com' does not match the hostname in the server's certificate.
2019-10-27 18:33:40,374 DEBUG [http-bio-443-exec-5] impl.AuthenticationLDAP - value of hostnameverifiedfalse
2019-10-27 18:33:40,374 INFO [http-bio-443-exec-5] impl.AuthenticationLDAP - verifyHostName: Closing LDAP socket

Steps to Resolve

Two situations are encountered here if the Cisco Tomcat logs do not show any certificate error, this requires to be validated.

Step 1. Validate that the user is associated with an IM&P server

Step 2. If the user is associated with an IM&P server, bounce the user from the Home Node cluster

In case the Cisco Tomcat logs show the error from the Snippet previously showed, perform these steps:

Step 1. Confirm if the Cisco Jabber is configured to use Secure LDAP

Step 2. If Secure LDAP is in use, confirm the certificates' information associated with them like the Fully Qualified Domain Name (FQDN), Hostname and Common Name (CN).

Step 3. Validate how the CUCM and IM&P are configured, if with IP address or FQDN, and compare that with the information contain within the certificate

Step 4. If the servers are configured with IP address and the LDAP certificates are configured with FQDN, the next command requires to be executed on all the CUCM and IM&P nodes

Sample Log Snippet

2017-11-08 16:03:20,051 DEBUG [0x00003a0c] [s\adapters\imp\components\Login.cpp(127)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - ************************************************
2017-11-08 16:03:20,051 INFO  [0x00003a0c] [s\adapters\imp\components\Login.cpp(128)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - OnLoginError: LERR_CUP_UNREACHABLE 
2017-11-08 16:03:20,051 DEBUG [0x00003a0c] [s\adapters\imp\components\Login.cpp(129)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - *************************************************

Steps to Resolve

Step 1. Verify that IMP FQDN/Hostnames are resolvable from the client PC.

Step 2. Verify that you can open this URL in browser https://<IMP SERVER FQDN/IP>:8443/EPASSoap/service/v105

Successful

Step 3. Verify that firewall/VPN does not block the connectivity to IMP server ( Port 8443,5222)

Step 4. Verify if this service runs in IMP server: Cisco Client profile Agent

Step 5. Set these services log to DEBUG, recreate the Log in issue and then collect the logs if the previous steps do not resolve the problem.

2017-11-08 16:03:20,051 DEBUG [0x00003a0c] [s\adapters\imp\components\Login.cpp(127)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - ************************************************
2017-11-08 16:03:20,051 INFO  [0x00003a0c] [s\adapters\imp\components\Login.cpp(128)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - OnLoginError: LERR_CUP_INTERNAL_ERROR 
2017-11-08 16:03:20,051 DEBUG [0x00003a0c] [s\adapters\imp\components\Login.cpp(129)] [IMPServices] [CSFUnified::IMPStackCap::Login::OnLoginError] - *************************************************

Step 3. Check if this Field notice is applicable

Field Notice: FN - 64267 - Cisco Unified Communications Manager IM & Presence causes Cisco Jabber log in failures - Software Upgrade Recommended

Step 4. Set these services log to DEBUG, recreate the Log in issue and then collect the logs if the previous steps do not resolve the problem.

Step 5. Reboot the cluster to recover the situation.

Stage 4. XMPP Log in (IM and Presence Log in)

Sample Log Snippet

2019-05-03 15:19:32,225 DEBUG [0x0000000109732f80] [s/adapters/imp/components/Log in.cpp(128)] [IMPServices] [OnLog inError] - ****************************************************************
2019-05-03 15:19:32,225 INFO  [0x0000000109732f80] [s/adapters/imp/components/Log in.cpp(129)] [IMPServices] [OnLog inError] - OnLog inError: LERR_JABBER_AUTH <14>: Authentication error with server, resource bind, TLS, create session or SASL error
2019-05-03 15:19:32,225 DEBUG [0x0000000109732f80] [s/adapters/imp/components/Log in.cpp(130)] [IMPServices] [OnLog inError] - ****************************************************************

Steps to Resolve

Step 1. Verify that port 5222 is opened between the IM&P servers and the Expressways.

Step 2. Verify that these services are running in the IM&P server and restart them once.

Step 3. Disable the High Availability from the CUCM Presence Redundancy Groups.

Step 4. Restart the Cisco XCP Router service on all the IM&P nodes, first with the IM&P Publisher and then in the Subscribers.

Step 5. Reenable the High Availability from the CUCM Presence Redundancy Groups.

Sample Log Snippet

2017-10-27 10:56:47,396 DEBUG [0x00007fff8b3d7340] [s/adapters/imp/components/Login.cpp(127)] [IMPServices] [OnLoginError] - ****************************************************************
2017-10-27 10:56:47,396 INFO  [0x00007fff8b3d7340] [s/adapters/imp/components/Login.cpp(128)] [IMPServices] [OnLoginError] - OnLoginError: LERR_JABBER_AUTH <17>: Authentication error with server, for example, resource bind, TLS, create session or SASL error
2017-10-27 10:56:47,396 DEBUG [0x00007fff8b3d7340] [s/adapters/imp/components/Login.cpp(129)] [IMPServices] [OnLoginError] - ****************************************************************

Steps to Resolve

Step 1. Verify that the cup-xmpp Certificates are valid.

Step 2. Verify that Port 5222 is open.

Step 3. Set these services log to DEBUG and then recreate the Log in issue and collect the logs before step 4.

If Root cause to be identified as Reboot of the server is the only fix known.

Step 4. Reboot the server to resolve the issue.

Sample Log Snippet

2014-12-15 12:07:31,600 INFO  [0x00001670] [ts\adapters\imp\components\Login.cpp(96)] [imp.service] [IMPStackCap::Login::OnLoginError] - ****************************************************************
2014-12-15 12:07:31,600 INFO  [0x00001670] [ts\adapters\imp\components\Login.cpp(97)] [imp.service] [IMPStackCap::Login::OnLoginError] - OnLoginError: (data=0) LERR_JABBER_UNREACHABLE <16>:
2014-12-15 12:07:31,600 INFO  [0x00001670] [ts\adapters\imp\components\Login.cpp(98)] [imp.service] [IMPStackCap::Login::OnLoginError] - ****************************************************************

Steps to Resolve

Step 1. Check if IMP FQDN/Hostnames are resolvable.

Step 2. Verify that the firewall/VPN does not block the connectivity to the IM&P server ( Port 8443,5222).

Step 3. Verify if these services are running in the IM&P server and restart them once.

Step 4. Perform Mandatory checks .

Step 5. Set these services log to DEBUG, recreate the Log in issue and then collect the logs if the previous steps do not resolve the problem.

Step 6. In case of all users experience the same error, a server Reboot can be done for quick recovery.

Sample Log Snippet

2020-03-12 19:55:01,283 DEBUG [0x000000010b71d800][apters/imp/components/Log inUtils.cpp(96)][IMPServices][Log inErrortoErrorCode] - Log inErrortoErrorCode: 27 mapped to: UnknownLog inError
2020-03-12 19:55:01,283 DEBUG [0x000000010b71d800][isteners/Log inEventListenerImpl.cpp(148)][IMPServices][OnLog inError] - errCode: UnknownLog inError
2020-03-12 19:55:01,283 INFO [0x000000016b61f000][ers/imp/lifecycle/Log inExecutor.cpp(314)][IMPServices][signOn] - logged in using User ID: 35309769, failed
2020-03-12 19:55:01,478 INFO [0x000000010b71d800][pp/tahiti/ui/log in/YLCLog inBaseVC.m(500)][UI.Action.System] [-[YLCLog inBaseVC getPresenceErrorMessgaWithCode:]] - Jabber log in failed and show errorcode:200 string: Cannot Sign in your account. Contact your administrator.

Steps to Resolve

Step 1. Validate that the user is assigned to the IM&P.

Step 2. Validate that the certificates are correctly exchanged between the nodes and the Jabber.

Step 3. Validate the OAuth Signing and Encryption keys are correctly configured on all the nodes. See this document under Verify section.

Step 4. Perform Mandatory checks .

Step 5. Set these services log to DEBUG, recreate the Log in issue and then collect the logs if the previous steps do not resolve the problem.

Mandatory Checks

Step 1. Verify that the user is assigned to a Presence node (navigate to IM and Presence Administration > System > Topology) and there are no duplicates for the user (navigate to IM and Presence Administration > Diagnostics > System troubleshooter)

Step 2. If High Availability is enabled, navigate to CUCM Administration > Server > Presence Redundancy Group and check if they are in Normal state . This is the image of how Normal state looks. For more information about High Availability can be found here .

Abnormal State

Note : These services are used by the Jabber to log in: Cisco Tomcat, Cisco Tomcat Security, Cisco Client Profile Agent, Cisco XCP Connection Manager, Cisco XCP Router and Cisco XCP Authentication.

Normal State

Step 3. Check High Availability Replication status.

a.utils dbreplication runtimestate

If you encounter issues on the database replication, navigate to this link.

b.run pe sql ttlog in select count(*) from typesysreplication

or utils imdb_replication status ( 10.5.2 SU2a and later)

The three datastores need to show PASSED, and the command needs to be run on all the IM&P nodes, as sometimes on one node all the datastores' replication can show Passed , but on another node, it can show Failed.

The implications if the IMDB (In-memory Database) replication is not correct can imply that some or all the users are unable to log in or that their presence status cannot be shown correctly.

The steps to resolve the IMDB replication issues are:

Step 1. Disable the High Availability (HA) for the IM&P Subcluster that is affected.

Step 2. Stop the Cisco Presence Engine on all nodes

utils service stop Cisco Presence Engine

Step 3. Verify all the Datastore Services are running: Cisco Log in Datastore, Cisco Route Datastore, Cisco Presence Datastore, Cisco SIP Registration Datastore.

utils service list

Step 4. Restart Cisco Config Agent on each node one at a time.

utils service restart Cisco Config Agent

Step 5. Start Cisco Presence Engine.

utils service start Cisco Presence Engine

Step 6. Enable HA for the Sub-cluster.

How to Set Logs to DEBUG

Step 1       Choose Navigation > Unified serviceability > Trace > Configuration .
Step 2       From the Server drop-down list, choose the server ( i.e IMP node) that runs the service to configure trace and click Go .
Step 3       From the Service Group drop-down list box, choose the service group for the service to configure trace; then click Go .
Step 4       From the Service drop-down list box, choose the service for which you want to configure trace; then click Go .
Step 5       Check box ' Apply to All Nodes ' and select the trace level to ' DEBUG '
Step 6       To save your trace parameters configuration, click the Save button

For more information on how to set trace levels refer to the Cisco Unified Serviceability Administration Guide.

Helpful videos: