Search

Well, I am back to Client certificate again, guess the reason being a lot of support calls that we getting off late are related to any of the following four errors, especially the first two.

403.7

403.13

403.16

403.17 ( I will cover .16 and .17 very briefly since they are very self-explanatory and easy to troubleshoot)

Earlier I had discussed the setup of the client certificate with IIS and AD for authentication mappi... .

Here I will discuss the troubleshooting strategies on client certificate related errors that are listed above.

To understand how Client certificate is used while accessing a resource on the server, you may prefer to look at this brief but quite explanatory KB by David Dietz from IIS support.

http://support.microsoft.com/kb/907274/en-us

So here we go...

1) 403.7

We see that 403.7 can be thrown by IIS when Client certificate is required and the browser is not sending the client certificate details to the web server (IIS). Either the client did not send the certificate for some reason or else the client did not have a certificate issued by a CA that was also trusted by IIS server. If the client sends a certificate which is not mutually trusted by both client and the server you may see this error.

You may get a meaningful error like this in the browser:

HTTP Error 403 403.7 Forbidden: Client certificate required
This error occurs when the resource you are attempting to access requires your browser to have a client Secure Sockets Layer (SSL) certificate that the server recognizes. This is used for authenticating you as a valid user of the resource.
Please contact the Web server's administrator to obtain a valid client certificate.

To start with, follow this KB http://support.microsoft.com/kb/332077/en-us

You need to make sure that the client certificate is issued by a CA which is in the trusted root CA store on both the server and the client machine. Confirm whether the trusted root CA is part of CTL. The reason being that if your certificate's CA is not in the CTL; although present in the trusted root CA store in the server machine, you may still see the error.

A CTL is a list of trusted certification authorities (CAs) that can be used for client authentication for a particular Web site . You can use CTLs to configure your Web server to accept certificates from a specific list of CAs, and automatically verify client certificates against this list. Only users with a client certificate that is issued by a CA in the CTL can gain access to the server.
Each Web site on your server can be configured to accept certificates from a different CTL. You may want to do this if you need a different list of trusted CAs for each Web site.

If CTL is present, this is the list which is actually used to check for CA's which can issue client certificate to a user. If it is disabled then root CA store will be used for the above. Also make sure that the certificate is a valid client certificate. Make sure it is intended for user authentication.

Check the certificate for "Ensures the identity of a remote computer" and Enhanced Key usage says Client Authentication.

Also Using >Certutil -verify -urlfetch should show:

Verified Application Policies:
1.3.6.1.5.5.7.3.2 Client Authentication

You may also see 403.7 due to an update to the trusted Root CA list. This creates a list that is too large based on the size limit we enforce, the result being truncation of the list when this is sent to the client during the client certificate handshake. The limit is based on data size not CA count so there is no way to say this happens at a certain count of trusted CA’s.
To resolve this we need to delete some of the expired and unused/unknown trusted root certificates from the Trusted Root Certification Authorities list until it is working again.

The problem can also be identified when the following entry is logged on the Web server. It is quite explanatory in itself.

Event Type: Warning
Event Source: Schannel
Event Category: None
Event ID: 36885
Date: 2/9/2007
Time: 9:32:44 AM
User: N/A
Computer: USMASVGDOIM259
Description:
When asking for client authentication, this server sends a list of trusted
certificate authorities to the client. The client uses this list to choose a client
certificate that is trusted by the server. Currently, this server trusts so many
certificate authorities that the list has grown too long. This list has thus been
truncated. The administrator of this machine should review the certificate
authorities trusted for client authentication and remove those that do not really
need to be trusted.

Related articles:

Trusted root certificates that are required by Windows Server 2003, by Windows XP,
and by Windows 2000
http://support.microsoft.com/kb/293781
931125 Microsoft root certificate program members (January 2007)
http://support.microsoft.com/default.aspx?scid=kb;EN-US;931125

2) 403.13

The error you may see in the browser will be as shown below:

HTTP 403.13 Forbidden: Client certificate revoked
The page requires a valid client certificate

For an understanding of this error message check KB 248058.

This error message means that the client sent a certificate, but either the certificate shows up as revoked in the issuing authority's Certificate Revocation List or the server could not retrieve a CRL from the issuing authority.

You need to crosscheck whether the client certificate is revoked or not with the respective CA.

If CA confirms the certificate as valid and not revoked then the issue could be with IIS being unable to retrieve CRL from the CA.

IIS , by default retrieves a CRL whenever it receives a client cert to make sure that cert is not revoked as long as local cache is expired. For this it contacts the CA to get the CRL which is a list of revoked certificates and compares the list with the presented client cert. If for any reason it cannot retrieve the CRL, it will go ahead and throw error message as 403.13 even if cert is valid and not revoked.  This can happen in cases where some Proxy/firewall may block access to CDP to get the CRLs.

If a CDP extension is present in a certificate that is part of the certification path, IIS must be able to download at least one of the CRLs. If IIS is unable to resolve the CRL, it returns the HTTP 403.13 error. In this case, we cannot access the above CDP so we fail. Prior to MS04-011 Win2k did not limit validation based on this. However, we now require that the CDP be reachable when validating a certificate chain.
To work around this we must either use a reachable CDP in the client certificate or disable CertCheckMode on the IIS server, thus preventing it from doing any revocation checking.

So, if we are getting Client certificate revoked errors, then check to see if the server can get to the CRL distribution point specified in the client certificate and if it can and is still giving this error, then download the Root and Subordinate CA CRLs and install them on the IIS server so that it can get to it locally.

Also there is a metabase key in IIS called certcheckmode, which if disabled will stop IIS from trying to retrieve CRL checking. In such a case client cert will be accepted even if the cert is revoked. Disabling CRL checking is a quick way to test the cause.

The CertCheckMode property enables or disables Certificate Revocation List (CRL) checking. When CertCheckMode is set to a value greater than 0 (CertCheckMode>0), the CRL does not search for certificates that have been revoked. When CertCheckMode is equal to 0 (CertCheckMode=0), the CRL searches for certificates that have been revoked.

With CertCheckMode disabled, IIS will no longer try to verify revocation of incoming client certificate requests. The client certificates will still need to be within their valid dates and still must be trusted by the IIS server (the IIS server must trust the issuing CA).

We disable the Certcheckmode key by setting it to 1.

> C:\Inetpub\Adminscript\cscript.exe adsutil.vbs Set W3SVC /<Website identifier>/CertCheckMode 1

I had seen an interesting case where 2 of the websites were accepting the same client cert whereas another one was not accepting it on the same web server.

Checking the metabase.xml for the server showed this:

Non-Working site:

=================

<IIsWebServer Location ="/LM/W3SVC/690402"

AuthFlags="0"

LogPluginClsid="{FF160663-DE82-11CF-BC0A-00AA006111E0}"

SSLCertHash="8bcfc28e346bb9ec49374d87479021354349cf85"

SSLStoreName="MY"

SecureBindings="XX.XX.XX.X:443:"

ServerAutoStart="TRUE"

ServerBindings="XX.XX.XX.X:80:"

ServerComment="CDB"

</IIsWebServer>

Working Site:

=============

<IIsWebServer Location ="/LM/W3SVC/90326589"

AuthFlags="0"

CertCheckMode="1"

LogPluginClsid="{FF160663-DE82-11CF-BC0A-00AA006111E0}"

SSLCertHash="a640634e38ff20ebd8c29c32aae635e5575e57f6"

SSLStoreName="MY"

SecureBindings="XX.XX.XX.Y:443:"

ServerAutoStart="TRUE"

ServerBindings="XX.XX.XX.Y:80:wcdb"

ServerComment="WCDB"

</IIsWebServer>

Look at the difference between them. You see CertCheckMode is absent in the Non-working site, and absence of this key is equivalent to it being enabled. So once we put the CertCheckMode set to "1" for non-working site we should be able to resolve the issue. But this means that CRL chekcing is disabled. You may downlaod the CRL on to the server or else open up the relevant ports in order to allow CRL to be retrieved.

Check the KB 294305.

You may also check KB 841632 if IIS 5.0 is in picture.

There was an interesting case, where users were getting 403.13 even when client cert was not revoked and we were able to access the get the CRL from the CDP for the client cert by accessing it through a browser. Yet after a lot of tracing and monitoring we found that there was a 4-level hierarchy in the certificate chain, with let's say Root CA1 ->Subordinate Root CA2->Subordinate Root CA3 -> Client certificate and one of the subordinate root CA's crl was not accessible. There are tools like certutil or SSLspy that can come handy. We ran certutil.exe -verify -urlfetch <location of the client cert.cer> on the IIS server and found that CRL retrieval for Subordinate Root CA2 was failing, and hence the issue.

So remember that we need to make sure that the CDPs for all the subordinate CAs certifcates in the chain should also be reachable.

Let's say for my client certificate, the Certification path shows:

Microsoft Corporate Root CA

|--> Microsoft Corp Enterprise CA 2

|--> Saurabh Singh

Here is the information for certificate " Saurabh singh "

CRL Distribution Points (Under Details->Field) shows:

[1]CRL Distribution Point
Distribution Point Name:
Full Name:
URL=ldap:///CN=Microsoft%20Corp%20Enterprise%20CA%202(4),CN=CRL,CN=CDP,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=corp,DC=microsoft,DC=com?certificateRevocationList?base?objectClass=cRLDistributionPoint
URL= http://corppki/crl/Microsoft%20Corp%20Enterprise%20CA%202(4).crl
URL= http://crl.microsoft.com/pki/mscorp/crl/Microsoft%20Corp%20Enterprise%20CA%202(4).crl

Authority Information Access shows:

[1]Authority Info Access
Access Method=Certification Authority Issuer (1.3.6.1.5.5.7.48.2)
Alternative Name:
URL= http://corppki/aia/Microsoft%20Corp%20Enterprise%20CA%202(4).crt
[2]Authority Info Access
Access Method=Certification Authority Issuer (1.3.6.1.5.5.7.48.2)
Alternative Name:
URL= http://www.microsoft.com/pki/mscorp/Microsoft%20Corp%20Enterprise%20CA%202(4).crt

Here is the information for certificate  " Microsoft Corp Enterprise CA 2 ":

CRL Distribution Points (Under Details->Field) shows:

[1]CRL Distribution Point
Distribution Point Name:
Full Name:
URL= http://corppki/crl/mscrca.crl
URL= http://crl.microsoft.com/pki/mscorp/crl/mscrca.crl

Authority Information Access shows:

[1]Authority Info Access
Access Method=Certification Authority Issuer (1.3.6.1.5.5.7.48.2)
Alternative Name:
URL= http://corppki/aia/mscrca.crt
[2]Authority Info Access
Access Method=Certification Authority Issuer (1.3.6.1.5.5.7.48.2)
Alternative Name:
URL= http://www.microsoft.com/pki/mscorp/mscrca.crt

Now runnning the Certutil.exe as shown below:

cmd prompt> certutil.exe -verify -urlfetch cert.cer <-------the client certificate

Here is the output:

Issuer:
CN=Microsoft Corp Enterprise CA 2
Subject:
CN=Saurabh Singh
Cert Serial Number: 258a555c0004008b1c42

dwFlags = CA_VERIFY_FLAGS_DUMP_CHAIN (0x40000000)
ChainFlags = CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT (0x40000000)
HCCE_LOCAL_MACHINE
CERT_CHAIN_POLICY_BASE
-------- CERT_CHAIN_CONTEXT --------
ChainContext.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
ChainContext.dwRevocationFreshnessTime: 176 Days, 6 Hours, 5 Minutes, 17 Seconds

SimpleChain.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
SimpleChain.dwRevocationFreshnessTime: 176 Days, 6 Hours, 5 Minutes, 17 Seconds

CertContext[0][0]: dwInfoStatus=102 dwErrorStatus=0
Issuer: CN=Microsoft Corp Enterprise CA 2
Subject: CN=Saurabh Singh
Serial: 258a555c0004008b1c42
Template: AutoEnrolled Client Auth
48 b7 48 da 00 51 21 77 b3 e1 3a ce 98 7d 35 2f b7 e8 0c 1c
Element.dwInfoStatus = CERT_TRUST_HAS_KEY_MATCH_ISSUER (0x2)
Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
---------------- Certificate AIA ----------------
Verified "Certificate (0)" Time: 1
[0.0] http://corppki/aia/Microsoft%20Corp%20Enterprise%20CA%202(4).crt

Verified "Certificate (0)" Time: 1
[1.0] http://www.microsoft.com/pki/mscorp/Microsoft%20Corp%20Enterprise%20CA%202(4).crt

---------------- Certificate CDP ----------------
Verified "Base CRL (821)" Time: 0
[0.0] ldap:///CN=Microsoft%20Corp%20Enterprise%20CA%202(4),CN=CRL,CN=CDP,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=corp,DC=microsoft,DC=com?certificateRevocationList?base?objectClass=cRLDistributionPoint

Verified "Base CRL (821)" Time: 0
[1.0] http://corppki/crl/Microsoft%20Corp%20Enterprise%20CA%202(4).crl

Verified "Base CRL (821)" Time: 1
[2.0] http://crl.microsoft.com/pki/mscorp/crl/Microsoft%20Corp%20Enterprise%20CA%202(4).crl

---------------- Base CRL CDP ----------------
No URLs "None" Time: 0
--------------------------------
CRL 821:
Issuer: CN=Microsoft Corp Enterprise CA 2
fd 19 3c 2f 0c 24 ea 1c 4a 5d df c4 26 2a b0 1b 98 48 ef 99
Application[0] = 1.3.6.1.5.5.7.3.2 Client Authentication

CertContext[0][1]: dwInfoStatus=102 dwErrorStatus=0
Issuer: CN=Microsoft Corporate Root CA, O=Microsoft Corporation
Subject: CN=Microsoft Corp Enterprise CA 2
Serial: 610d1de0000000000019
Template: SubCA
17 0a 7b 9d 52 85 07 7e 74 1a f5 a0 6b db 05 78 9e bc f1 8d
Element.dwInfoStatus = CERT_TRUST_HAS_KEY_MATCH_ISSUER (0x2)
Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
---------------- Certificate AIA ----------------
No CRL "Certificate (0)" Time: 0
[0.0] http://corppki/aia/mscrca.crt

No CRL "Certificate (0)" Time: 1
[1.0] http://www.microsoft.com/pki/mscorp/mscrca.crt

---------------- Certificate CDP ----------------
Verified "Base CRL (18)" Time: 0
[0.0] http://corppki/crl/mscrca.crl

Verified "Base CRL (18)" Time: 0
[1.0] http://crl.microsoft.com/pki/mscorp/crl/mscrca.crl

---------------- Base CRL CDP ----------------
No URLs "None" Time: 0
--------------------------------
CRL 18:
Issuer: CN=Microsoft Corporate Root CA, O=Microsoft Corporation
0e 70 65 69 a7 4c f9 7d 9f 50 7b db 9c e1 b8 27 9e 53 ba f4

CertContext[0][2]: dwInfoStatus=10c dwErrorStatus=0
Issuer: CN=Microsoft Corporate Root CA, O=Microsoft Corporation
Subject: CN=Microsoft Corporate Root CA, O=Microsoft Corporation
Serial: 443c2a54b59cd69d4c09b18a9b02eb55
d2 d3 8e ba 60 ca a1 c1 20 55 a2 e1 c8 3b 15 ad 45 01 10 c2
Element.dwInfoStatus = CERT_TRUST_HAS_NAME_MATCH_ISSUER (0x4)
Element.dwInfoStatus = CERT_TRUST_IS_SELF_SIGNED (0x8)
Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
---------------- Certificate AIA ----------------
No URLs "None" Time: 0
---------------- Certificate CDP ----------------
No URLs "None" Time: 0
--------------------------------

Exclude leaf cert:
8a cf e9 23 e2 d7 cd d1 f0 bb 05 6e 63 b5 31 95 6e 46 0d ad
Full chain:
5b fa 04 32 34 21 49 11 92 56 b3 ee 41 94 b4 b8 f3 f6 44 f2
------------------------------------
Verified Issuance Policies: None
Verified Application Policies:
1.3.6.1.5.5.7.3.2 Client Authentication
Leaf certificate revocation check passed
CertUtil: -verify command completed successfully.

If you notice the Certutil.exe tries to check the CRL accessibility by accessing the CRL Distribution points. The above command ouptput should give you an idea regarding the cause. You may see an error in accessing the CRL in the output above in cases where you get the above errors.

Here is something similar when you get an error:

----------------  Certificate CDP  ----------------

Failed "CDP" Time: 0

Error retrieving URL: The specified network resource or device is no longer available. 0x80070037 (WIN32: 55)

ldap:///CN=CRL1, CN=xxxx, OU=xxxx, OU=xxxx. by ref. (limits liab.), O=xxxx, C=US?certificateRevocationList;binary,authorityRevocationList;binary,deltaRevocationList;binary

Failed "CDP" Time: 0

Error retrieving URL: The server name or address could not be resolved 0x80072ee7 (WIN32: 12007)

http://www.some_company.net/CRL/net1.crl

--------------------------------

Application[0] = 1.3.6.1.5.5.7.3.4 Secure Email

Application[1] = 1.3.6.1.5.5.7.3.1 Server Authentication

Application[2] = 1.3.6.1.5.5.7.3.2 Client Authentication

Application[3] = 1.3.6.1.5.5.7.3.3 Code Signing

Exclude leaf cert:

14 4c 46 42 11 66 a4 a9 42 70 ad b6 e0 1e 23 ca d4 9b 24 0e

Full chain:

fe 37 4a cf 76 3e 01 14 21 a6 c7 25 35 14 97 e5 91 87 e3 b7

Issuer: CN=A company, OU=PKI, DC=company, DC=com

Subject: OID.0.9.2342.19200300.100.1.1=ZALDI001, OU=People, OU=SAP, DC=company, DC=com

Serial: 42c550de

6e 33 5f 13 e1 67 ad 41 71 02 96 17 c7 57 c9 91 ea cb 1d 24

The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613)

------------------------------------

Revocation check skipped -- server offline

Cert is an End Entity certificate

ERROR: Verifying leaf certificate revocation status returned The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613)

CertUtil: The revocation function was unable to check revocation because the revocation server was offline.

Ensure that the necessary firewall/network configuration changes to allow the IIS server to access ALL of the external CDP’s listed in the client cert’s revocation chain, or download the CRL(s) to the IIS server manually and set CertCheckMode to MD_CERT_CACHE_RETRIEVAL_ONLY (see this link http://technet2.microsoft.com/windowsserver/en/library/173427fd-eb90-44ef-8a9c-d7bb4ff41ab81033.mspx... ).  That will tell IIS to look at the CRL in its local store and not try to attempt to access the CRL via the CDP entries specified in the client cert.

One more confusing point that should be clarified here:

If you have a certificate chain, let's say: Root CA -> Intermediate CA1 -> Intermediate CA2 ->..... -><Your Client ceritficate>, then CRL checking will be done for all the Certificates in the hierarchy except the Root CA.

If you are really interested to dig further as to how Certificate Revocation etc. works at a lower level, here is a real exhaustive link to check it out.

http://technet.microsoft.com/en-us/library/af1e419e-ede5-8c4b-bf6e-1fb17658a99d.aspx

Another issue that pops up from time to time is:

"Choose a digital certificate" popup window in Internet Explorer is blank when attempting to use client certificates to authenticate against IIS.

This can happen in situations as explained earlier too in cases where:

The total size of the certificates in the Trusted Root Certification Authorities store on the IIS server was too large to send to the client. The list was truncated as a result.

The following event was written to the System log:
Event Type: Warning
Event Source: Schannel
Event ID: 36885
Description: When asking for client authentication, this server sends a list of trusted certificate authorities to the client. The client uses this list to choose a client certificate that is trusted by the server. Currently, this server trusts so many certificate authorities that the list has grown too long. This list has thus been truncated. The administrator of this machine should review the certificate authorities trusted for client authentication and remove those that do not really need to be trusted.

Resolution would be to Remove unused certificates from the Trusted Root Certification Authorities store on the IIS server, reducing the number of certificates.

Also another cause may be when the any of the Subordinate CAs->certificate->Details->Edit Properties button has Client Authentication disabled in the intended purposes.

Also this should be of help at times http://support.microsoft.com/kb/285069/

Also if we have certificate trust list(CTL) enabled, CTL that IIS sends is what the client uses to know if it has a client cert it can use.

Also there is a <12kb> limit on this and if the customer has applied the Trusted Root CA update, then we may not send the full list of trusted CA’s. Make sure that CA is in CTL as well as the size limit. You may want to revisit the article
http://support.microsoft.com/kb/933430 . Either install the hotfix if it
is applicable or try deleting/moving to other store some of the unused/junk CAs from
the Trusted Root Certificate Authority Store on the IIS server. That
could do the trick for you!

Refer to one of our finest Escalation engineer (Andreas Klein)'s blog which talks about limiting the list of CA's allowed for Client authentication, without deleting the CAs from the store.

http://blogs.msdn.com/andrekl/archive/2005/04/19/409682.aspx

Also while you may have the certificate in your personal store (using the mmc snap-in it shows up properly), you may not see it in the IE browser. If you go through Internet Options->Content and click Certificates, it doesn’t show up at all. Open the certificate MMC and check whether the cert has a Private key or not.

If the General tab on the cert properties does not say at the bottom that you have a Private Key corresponding to this cert then you don’t, and this may lead to the above problem.

403.16 - Client certificate is untrusted or invalid.

This error message is primarily generated when the certificate that the client provided is improperly formed. It can also be generated if there are intermediate certification authorities in the certificate chain that are not trusted by the Web server.

403.17 - Client certificate has expired or is not yet valid

This error message is fairly self-explanatory. It means that the current date on the server is not within the valid date ranges that are presented in the client certificate. You may also want to ensure that the client certificate and its issuing CAs (including Intermediate CAs) are not expired or invalid.

Author: Saurabh Singh