Clouds, Compliance and other Tech Stuff
I’ve been meaning to do a write up on designing my homelab. In my last job, I had access to hardware and some essential networking bits, but now that I’m a Field SE, i’m in a different situation. I have access to internal tools and nested deployments (otherwise called PODs), as well as some Hands-On-Labs deployments. These are great for doing quick demo’s but for continued education purposes, the consensus among SE’s is that “Nothing beats a homelab”.
We were having some issues with one of our VCSA’s and creating or subscribing to Content Libraries. So here is our resolution.
SSH into the VCSA and check the following log files:
/storage/log/vmware/vdcs/cls.log
/storage/log/vmware/vdcs/ovf.log
/storage/log/vmware/vdcs/ts.log
cls.log
=========
2016-01-20T14:18:01.773Z | DEBUG | unset-opId | tomcat-http--39 | SsoOverRestVerifierUtil | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.800Z | ERROR | unset-opId | tomcat-http--39 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
at com.vmware.identity.token.impl.SamlTokenImpl.validate(SamlTokenImpl.java:535)
at com.vmware.vim.sso.client.DefaultTokenFactory.parseToken(DefaultTokenFactory.java:46)
at com.vmware.vim.sso.http.impl.AuthVerifierImpl.validateSamlToken(AuthVerifierImpl.java:77)
at com.vmware.vim.sso.http.impl.AuthVerifierImpl.verifyToken(AuthVerifierImpl.java:66)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:183)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
.
.
.
.
2016-01-20T14:18:01.801Z | ERROR | unset-opId | tomcat-http--39 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.801Z | ERROR | unset-opId | tomcat-http--39 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
ovf.log
-------
2016-01-20T14:18:01.792Z | DEBUG | unset-opId | tomcat-http--23 | SsoOverRestVerifierUtil | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.804Z | ERROR | unset-opId | tomcat-http--23 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--23 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--23 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at com.vmware.vcde.common.services.cm.servlet.DispatcherServlet.service(DispatcherServlet.java:53)
Ts.log
---------
2016-01-20T14:18:01.792Z | DEBUG | unset-opId | tomcat-http--14 | SsoAuthenticatedFileStreamServlet | doGet: Entering (/ts/resourcebundle)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--14 | SamlTokenImpl | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
2016-01-20T14:18:01.805Z | ERROR | unset-opId | tomcat-http--14 | SsoOverRestVerifierUtil | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ts/resourcebundle
2016-01-20T14:18:01.806Z | ERROR | unset-opId | tomcat-http--14 | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
According to VMware support, these log entries show no security context for the user. Without that Security content the user cannot perform actions on the content library.
We found the signing cert and its root CA used by SSO from vmware-identity-sts.log and took out the ssoserverSign and the root certificate and added them to the CA to TRUSTED_ROOTS using the below mentioned vets command.
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo51 --cert 51root.crt
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo52 --cert 52root.crt
Then restart all services. Run the following commands. (No there is not a –restart or –reset, use both commands).
service-control --stop --all
service-control --start --all
Thats all for today folks. Hope this helped!
A little bit ago, we upgraded to VCSA 6.0 Update 1a on all sites. One of the features that I looked forward to most was the Content Library. But, we ran into an issue with one of our VCSA’s… Continue reading “This Method Requires Authentication – Short Version”
I’ve attempted to blog 3 times in my life, first on tumblr with no topic, second as a blog on general tech as I was working in retail and a third time when I wanted to blog about tech I was learning and getting involved with. Each time amounted to about 2 posts in a year or so. I just couldn’t continue writing because I felt it wasn’t good enough or I was too busy trying to learn the tech to sit down and write about it.
The vDM30in30 challenge came up and I thought I would give it a try. Even as I wrote the 7 or so posts, it was hard to find the time in the beginning, let alone the 15 or so drafts that are still awaiting to be published. The challenge is hard, but just as @kylog pointed out in his post “Writing is Hard, Redux”, this is still more than I’ve written in a single year, and I did this in a month! This isn’t the end, especially as I try to focus on writing about virtualization for first-timers.
I spoke about this topic with @cxi during VMworld and feel like its how I really want to approach this blog. Unfortunately I did not take that approach during this month, its how I want to move forward with this. Ideally, presenting each new tech or feature as if using it for the first time. In the spirit of the category, I am truly #New2 blogging. Watch for those 15 drafts as I clean them up, and maybe as I rewrite a couple. I think I can keep this up, I really do, because I enjoyed it so I am looking forward to 2016!
First off, I won’t claim to speak on behalf of the entire industry… this is going to have a VMware bias. With that in mind, lets move forward.
So… in my last post (Virtualization, the way the school teaches it…) I talked about desktop virtualization apps. This is really where it all started. In the beginning there was CP-40 (1964)… no wait, too far back. Lets just start with the desktop virtualization on x86 server beginnings. IN THE EARLY DAYS (before I got into this), there was VMware who released a software called Vmware Server or Vmware GSX Server. This was the first x86 virtualization platform that ran on Windows Server. So, much like the last post, you have the Host, which is the physical hardware, the OS which is Windows Server, the Application which is VMware Server and the VM that runs inside of VMware server. (The VM has an OS and virtual hardware which are really just shared resources that the application pulls from the OS and the OS pulls from the physical server).
Ok, history lesson and recap over, lets move on. Unlike the desktop applications that most schools use during computer science or IT classes, working in an IT role, you’ll experience something called a Hypervisor. Fancy name! So what does a hypervisor do, that a desktop application can’t?
First lets consider the following in a desktop method, if the resources of the physical computer was cake and we had to feed those resources to each thing that needed it, you you would first have to feed the OS on the host. In this instance lets say that that is Windows. So the computer gives CPU, RAM, Network and Video resources to Windows. Well, Windows is running another application that is also running an Operating System, lets say Linux. So Windows needs to pass some cake to the App so it can pass it to Linux. BUT, we need to make sure there is enough cake for Windows too! If you don’t leave enough cake for Windows… its won’t be able to run the app, which won’t be able to run Linux. Starting to feel a bit like office space in here:
So now lets look at the hypervisor! In this case we will look at VMware’s current offering called ESXi. Instead of running Windows on the host physical hardware, lets remove it from the equation. Now thats one less thing to give cake to. ESXi is a very slim operating system, its sole purpose in life is to run VM’s. The biggest difference you will find, is that instead of plugging in a monitor into the computer, opening an app and SEEING the VM running and looking at its video output, you only see this:
Not much to look at huh? So here is the deal, as I stated before, the sole purpose of the hypervisor is to run VM’s. Running a VM and outputting the display from the VM to the VGA port isn’t a big concern because all Operating Systems have a method of remote access, and even if they don’t, VMware provides a tool to manage this server remotely, and that tool… can see the VM. GASP! Yes, you can still see the video output, but you have to use the tool. At this time, the tool is called the vSphere Client. The client is not only used to manage the VM’s but the configuration of host resources as well.
The hypervisor doesn’t just virtualize a server, it virtualizes the main components of the physical host. Unlike desktop virtualization where you are sharing the resources that the main OS is using, in a hypervisor, you setup a Virtual Switch for managing networking between hosts and VMs, Datastores for storing the virtual hard drives of VMs and CPU cores to allow multiple VMs to share the CPU clock scheduling.
So again, why the hypervisor? Short answer is, save the resources. Without Windows, you don’t have to provide resources to the bare-metal OS (Windows). Hypervisors are slim in comparison, and provide a LOT more functionality, which we will talk about later. So we save those resources, which means we can run MORE VM’s per server. Cool… more consolidation means more savings on hardware which makes your boss happy, the accountants happy and trust me… from a management perspective, you will be happier too.
So let me just add this here, ESXi is built by VMware and is provided free of charge. Yes, you read that correctly, FREE. The number of features available for free are slim in comparison to the paid version of vSphere (we will talk about the vNames later, there are a lot of them). But, for a small company, that may not matter. Free is free is free is free… too much? Nah. So for free, you have the ability to install a hypervisor (ESXi) on a physical server, configure it (later), and connect to it via a tool and build, manage and use Virtual Machines.
So lets recap:
Hypervisors will be briefly talked about in classes, but for the most part, not typically taught unless you are interested in taking a class towards a certification. If you are interested in trying out a hypervisor, seeing what it is capable of, then I suggest downloading a copy of ESXi from my.vmware.com (you have to register), and get started. If you are working towards a career in IT, I recommend that you start learning it sooner rather than later.
In schools, when a student that is working towards a degree in IT, MIS, IST or anything of the like… typically the virtualization method of choice by the teachers is to use Virtualbox or some other desktop virtualization software. These tools are great for building up an operating system and testing it, without actually installing it on hardware. If you have never done something like this before, its pretty neat.
Essentially, you build what is called a Virtual Machine or VM for short (remember that). A VM consists of a few files typically. First you have the configuration file. The config file tells the VM how many CPU processors or cores it has, how much RAM is allocated for the VM, what type of network card setup to use, how large the virtual hard drive is as well as where that disk is located. There are other items in there as well, but lets stick to the basics. It covers all the needs of a basic computer, CPU, RAM, Network, oh and of course Video card.
Things to remember:
Host – the physical computer
VM – the Virtual machine that is running inside of the App
The App (Virtualbox) – the Application that runs on the host, that provides resources from the host to the VM (CPU, RAM, Network)
So where is this whole thing running? Well, good question! Your actual computer is the physical machine, it is running its operating system, that would be either Windows, Mac OS X or some flavor of Linux (not as likely, but sure). Now inside of that Operating System you have applications. Virtualbox is just another application, but what it does is takes ANOTHER Operating System and pass the CPU, RAM, Network and Video capabilities of the physical machine, onto the Virtual Machine. Getting it yet? You are running an Operating System, inside of an application. You tricked the OS into thinking that it is on REAL hardware. Thats pretty nifty in an of itself.
Because of this, you share all of the same resources the “bare-metal” operating system uses. I.E. – don’t give the VM 100% of the CPU, RAM and Network… or else there isn’t enough for the actual computer to use that is running the VM. Now, because you share the network, the cool thing is you can connect to the VM just like its connected to your physical network. Inside the application that runs the VM, it is acting like a network switch (virtual networking, remember that). There are a couple different settings for this, but just know that you can either make it available to the host (your physical machine) only, or shared the network input and let the VM talk to the other devices on your network. To repeat, host-only means the VM can only talk to the physical machine running the app that runs the VM. Shared means that other devices on the network can also see the VM that is running inside the host.
Now, there is NOTHING magical about a VM on Virtualbox, it doesn’t have extra powers or run faster. You still have to install the OS by putting in a “CD”, or in this case you get an ISO (image file of a disc, what you normally burn to a CD) and you take that ISO and pass it to the VM, and install from the CD image. The same way you physically put a CD that was made by burning an ISO to a disc, and pop that sucker into a physical machine when you are installing an Operating System for the first… second or hundredth time.
After you install the OS, you’ve got a Virtual Machine, you can use it to test software out, and if it gets screwed up, you just delete it and start again. Its awesome because doing that on a physical machine is a pain in the butt. At least with this method, you aren’t wiping the machine, just a virtual hard drive. Let me just say that there is nothing wrong with this method, but once you step outside of this into the world of Hypervisors… it gets SOOO much more interesting.
Next Up: Virtualization, the way the industry sees it… with a little (a lot of) VMware bias.