We were having some issues with one of our VCSA’s and creating or subscribing to Content Libraries. So here is our resolution.
Symptoms:
- When creating a local Content Library, when clicking finish, it errors with: This Method Requires Authentication
- When subscribing to another Content Library that has authentication disabled, after copying the json URL into the field and clicking next, it halts the view and states: This Method Requires Authentication
- When attempting to download Support Bundles from the VAMI at https://<vCenter FQDN>:5480, Downloads timeout and fail
SSH into the VCSA and check the following log files:
/storage/log/vmware/vdcs/cls.log
/storage/log/vmware/vdcs/ovf.log
/storage/log/vmware/vdcs/ts.log
In cls.log, you will be looking for something like this:
cls.log
=========
2016-01-20T14:18:01.773Z | DEBUG | unset-opId | tomcat-http--39 | SsoOverRestVerifierUtil | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.800Z | ERROR | unset-opId | tomcat-http--39 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
at com.vmware.identity.token.impl.SamlTokenImpl.validate(SamlTokenImpl.java:535)
at com.vmware.vim.sso.client.DefaultTokenFactory.parseToken(DefaultTokenFactory.java:46)
at com.vmware.vim.sso.http.impl.AuthVerifierImpl.validateSamlToken(AuthVerifierImpl.java:77)
at com.vmware.vim.sso.http.impl.AuthVerifierImpl.verifyToken(AuthVerifierImpl.java:66)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:183)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
.
.
.
.
2016-01-20T14:18:01.801Z | ERROR | unset-opId | tomcat-http--39 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.801Z | ERROR | unset-opId | tomcat-http--39 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
In ovf.log, you are looking for:
ovf.log
-------
2016-01-20T14:18:01.792Z | DEBUG | unset-opId | tomcat-http--23 | SsoOverRestVerifierUtil | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.804Z | ERROR | unset-opId | tomcat-http--23 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--23 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--23 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at com.vmware.vcde.common.services.cm.servlet.DispatcherServlet.service(DispatcherServlet.java:53)
In ts.log, you are looking for:
Ts.log
---------
2016-01-20T14:18:01.792Z | DEBUG | unset-opId | tomcat-http--14 | SsoAuthenticatedFileStreamServlet | doGet: Entering (/ts/resourcebundle)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--14 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--14 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ts/resourcebundle
2016-01-20T14:18:01.806Z | ERROR | unset-opId | tomcat-http--14 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
Cause:
According to VMware support, these log entries show no security context for the user. Without that Security content the user cannot perform actions on the content library.
Resolution:
We found the signing cert and its root CA used by SSO from vmware-identity-sts.log and took out the ssoserverSign and the root certificate and added them to the CA to TRUSTED_ROOTS using the below mentioned vets command.
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo51 --cert 51root.crt
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo52 --cert 52root.crt
Then restart all services. Run the following commands. (No there is not a –restart or –reset, use both commands).
service-control --stop --all
service-control --start --all
Thats all for today folks. Hope this helped!
