Posted on

A short discussion on VMware’s Army of Service Providers and MSP’s

I’ve had an off and on relationship with the VMware Service Provider offerings over the last few years. I’ve worked for a partner, then on to VMware Perpetual Sales and now with VMware Service Provider division. I came back because I absolutely love the offering.

When I worked for a Service Provider Partner, this program was called the VMware Service Provider Program. Simple enough. Then the name was changed to vCloud Air Network. This was roughly around the time of vCloud Air becoming available. I’m not going to lie, it was pretty confusing for customers. With that confusion and some opinions on vCloud Air as a product, I’m curious if customer automatically assumed vCAN had the same reputation.

Now, vCloud Air has been sold to OVH. Who, funny enough is a VERY large vCloud Air Network Partner. It was a huge win-win in my book and really gave OVH more leverage in the US market. You see, OVH is killing it in Europe. So, when presented with the opportunity to onboard large set of new customers and operationally take on new data centers in one shot, it provided a unique opportunity.

So, this week, VMware announced that the vCloud Air Network program will now be called the VMware Cloud Provider Program. A smart move in my book to distance from the vCloud Air name as well as really driving their purpose in this Hybrid Cloud future VMware is painting. VMware wants to empower these partners and our customers with the tools to provide a great experience, whether its Infrastructure as a Service or Managed Services for those partners that want to go above and beyond. This Service Provider Program is over 4500 strong and growing each quarter.

Now, with last years announcement of VMware Cloud on AWS and it becoming generally available this week you wouldn’t be wrong to think this is another situation where VMware is competing against the service provider community. BUT! I challenge you to take another look with me.

VMware Cloud on AWS is a great offering with the ability to stand up new environments on the fly. It will help companies take advantage of AWS services using an infrastructure model they are intimitaly familier with. The idea as I see it is that companies with an IT workforce, have the engineering know-how to utilize such a service, but companies with minimal IT or who are already running on a service providers offering via MSP model may not know where to even begin.

This is where the MSP model in the future will step in. VMware is working to empower our MSP partners to resell the VMConAWS service, so they can continue to support their customers but offer the same flexibility the larger customers will now begin to enjoy. It’s not a competition when we still want our MSP partners to grow in this model with us.

I’m looking forward to watching these offerings grow, especially in how we offer them to our partners. The VCPP and MSP future is bright.

Posted on

Ramping up at VMware

I’m on week 3 at VMware, working my way through training and tasks designed to get me “Ramped Up” in the role. Its been incredible so far and for so many more reasons than I expected. First, let me say that I’m truly sorry I haven’t done the next Professional Growth post, I’m actually going to combine question 4 into post 3, and do a repost so look out for that. I’ve been a little busy but I promise to get back on track with those.

First, lets get this question out of the way.

“You went to VMware amidst the Dell Acquisition? Why?”

Yes, this was a major concern even before getting a call to do the first interview. There are so many articles and blog posts centered around VMware and Dell’s acquisition of EMC. Yes, there was a recent round of layoffs at VMware that was pretty significant. Ultimately, I felt comfortable with the stance VMware is showing and reached out to multiple contacts both inside and outside of VMware to get opinions.

I walked away from those conversations thinking that VMware, though part of the Federation, is very strong in its own right. This company still has new areas to grow and I’d like to be a part of that.

Alright, now that that is behind us, moving on.

Going to the “Dark Side”

I’ve heard this numerous times from my new coworkers and a few others. I understand that becoming a Pre-Sales System Engineer means that I am convincing companies that Product X is right for them, but its so much more than that. At the end of the first week, I questioned why that phrase is even used at all. By the end of the second, lets just say I don’t see it. This is an excellent opportunity and allows me to see and help so many people in so many different environments. I’m here to help validate, demo, show the value of these products and help solve problems. I don’t see a dark side to it at all.

First Impressions on my team and role

I wasn’t exactly sure what to expect going in. This is a vendor role, remember. Up until now, I’ve been a customer. One of the first things that struck me was all of the members willingness to help. People in different areas of the business have gone out of their way to get me slide decks and 1-on-1 meetings to discuss products, helping me get up and running.

My account reps took a good chunk of time to discuss and exchange knowledge and ideas, during a very busy part of their quarter, when they should be focusing on closing deals I obviously am not a part of. Above that, just being available far more than I expect them to be.

But, the best part of this actually happened around going to a customer’s business to do a demo. Now, as I mentioned earlier, I’ve been told “welcome to the Dark Side” plenty of times. This is part of sales, I support sales by doing the technical portion. It’s an understanding that I will convince companies that they need some software to solve their problem or reach their goal. What I wasn’t completely expecting was hearing the account rep and current systems engineer push on the idea of “we don’t sell shelf-ware”. The idea is that we don’t push products you don’t need to solve your current problem, or isn’t part of a soon to be project/goal.

That in itself makes this move all the sweeter. Why? Becuase it goes back to a core competency at my last job, a type of company culture if you will. At Voxeo/Aspect, they called it customer obsession. Doing what is right for the customer and seeing it through to the end. I’m reminded of that by this team and its amazing to see that idea being pushed, even in Pre-Sales.

Drinking from the firehose

Yeah, I have a lot of products to catch up on, but I’ve never been one to NOT want to learn a new enterprise tech. BRING. IT. ON.

Work | Life Balance

Coming from a customer background, I was presently surprised when my phone died the other night and I didn’t have to worry about an on-call rotation. But what really surprises me, is when my manager ends a friday call with “Alright, have a good weekend and remember, Family first… Work Second”. Work/Life balance is incredibly important here and as I agreed to this role, my one concern was how it might affect mine. I had spent a considerable amount of time working on this in my last job to improve it, to the best of my ability. While at VMware, its practically pushed in my favor by management.

Summary:

All in all, I already considered having VMware on my resume as a great career booster, but the perks keep rolling in. I don’t mean that in a way that suggests good discounts or benefits, which coincidentally are also great. The culture is good, the team is great and the role  will definitely cater professional growth beyond what I initially considered. I look forward the future that is at VMware. Now if I could only come up with a paper for VMworld… seriously, what should I talk about?!?!

Posted on

This Method Requires Authentication – Full Version

We were having some issues with one of our VCSA’s and creating or subscribing to Content Libraries. So here is our resolution.

Symptoms:

  • When creating a local Content Library, when clicking finish, it errors with: This Method Requires Authentication
  • When subscribing to another Content Library that has authentication disabled, after copying the json URL into the field and clicking next, it halts the view and states: This Method Requires Authentication
  • When attempting to download Support Bundles from the VAMI at https://<vCenter FQDN>:5480, Downloads timeout and fail

SSH into the VCSA and check the following log files:

/storage/log/vmware/vdcs/cls.log
/storage/log/vmware/vdcs/ovf.log
/storage/log/vmware/vdcs/ts.log

In cls.log, you will be looking for something like this:

cls.log
=========
2016-01-20T14:18:01.773Z | DEBUG    | unset-opId       | tomcat-http--39           | SsoOverRestVerifierUtil        | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.800Z | ERROR    | unset-opId       | tomcat-http--39           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
        at com.vmware.identity.token.impl.SamlTokenImpl.validate(SamlTokenImpl.java:535)
        at com.vmware.vim.sso.client.DefaultTokenFactory.parseToken(DefaultTokenFactory.java:46)
        at com.vmware.vim.sso.http.impl.AuthVerifierImpl.validateSamlToken(AuthVerifierImpl.java:77)
        at com.vmware.vim.sso.http.impl.AuthVerifierImpl.verifyToken(AuthVerifierImpl.java:66)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:183)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
.
.
.
.
2016-01-20T14:18:01.801Z | ERROR    | unset-opId       | tomcat-http--39           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.801Z | ERROR    | unset-opId       | tomcat-http--39           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)

In ovf.log, you are looking for:

ovf.log
-------
2016-01-20T14:18:01.792Z | DEBUG    | unset-opId       | tomcat-http--23           | SsoOverRestVerifierUtil        | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.804Z | ERROR    | unset-opId       | tomcat-http--23           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
        
        
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--23           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--23           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at com.vmware.vcde.common.services.cm.servlet.DispatcherServlet.service(DispatcherServlet.java:53)

In ts.log, you are looking for:

Ts.log
---------
2016-01-20T14:18:01.792Z | DEBUG    | unset-opId       | tomcat-http--14           | SsoAuthenticatedFileStreamServlet | doGet: Entering (/ts/resourcebundle)
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--14           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--14           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ts/resourcebundle
2016-01-20T14:18:01.806Z | ERROR    | unset-opId       | tomcat-http--14           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)

Cause:

According to VMware support, these log entries show no security context for the user. Without that Security content the user cannot perform actions on the content library.

Resolution:

We found the signing cert and its root CA used by SSO from vmware-identity-sts.log and took out the ssoserverSign and the root certificate and added them to the CA to TRUSTED_ROOTS using the below mentioned vets command.

/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo51 --cert 51root.crt
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo52 --cert 52root.crt

Then restart all services. Run the following commands. (No there is not a –restart or –reset, use both commands).

service-control --stop --all
service-control --start --all

Thats all for today folks. Hope this helped!

Posted on

I think I can, I think I can…

I’ve attempted to blog 3 times in my life, first on tumblr with no topic, second as a blog on general tech as I was working in retail and a third time when I wanted to blog about tech I was learning and getting involved with. Each time amounted to about 2 posts in a year or so. I just couldn’t continue writing because I felt it wasn’t good enough or I was too busy trying to learn the tech to sit down and write about it.

The vDM30in30 challenge came up and I thought I would give it a try. Even as I wrote the 7 or so posts, it was hard to find the time in the beginning, let alone the 15 or so drafts that are still awaiting to be published. The challenge is hard, but just as @kylog pointed out in his post “Writing is Hard, Redux”, this is still more than I’ve written in a single year, and I did this in a month! This isn’t the end, especially as I try to focus on writing about virtualization for first-timers.

I spoke about this topic with @cxi during VMworld and feel like its how I really want to approach this blog. Unfortunately I did not take that approach during this month, its how I want to move forward with this. Ideally, presenting each new tech or feature as if using it for the first time. In the spirit of the category, I am truly #New2 blogging. Watch for those 15 drafts as I clean them up, and maybe as I rewrite a couple. I think I can keep this up, I really do, because I enjoyed it so I am looking forward to 2016!

 

Posted on

This one goes out to all the students

This is just my opinion based on my experiences and having watched others go through the same program(s) that I did. Take it as you  like, but I hope it helps some of the students out there getting ready to hit that “requires X experience wall” in the job market.

A week or so ago I attended a Program Advisory Committee meeting at the local ITT Tech school. They handed out some folders with the programs that relating to my field of work. In this case thats Information Technology. There were others there representing IT ranging from Government Security to IT Consulting. Overall the experience was pleasant, but showed that there are areas still for improvement. Which of course, was why each of us were there.

They asked a few questions that stood out among the rest. I’ll list the questions below as well as some questions and discussions that I wish we had time for. Continue reading “This one goes out to all the students”

Posted on

Active Directory, Aliases and Hostnames, OH MY!

Before I get into this, I need to set it up.

NOTE: Be prepared to reboot the VCSA a couple times… it takes forever for the web-client to initialize and seriously, that needs to stop VMware. Sub 20 second Web-Client Initialization… NEEDS to be in the next release. I think I speak for everyone when I say that the Web-Client is initializing page is old and needs to die. Anywho. back to the fixing.

We have our active directory domains and our DNS domains. They aren’t the same. We are using alias’s, as support loved to keep saying (I didn’t set it up but sure!) Continue reading “Active Directory, Aliases and Hostnames, OH MY!”

Posted on

Adding vCenters with the similar hostnames to the same Root AD Domain

If you add a Windows Server to a Domain, you have the option of preventing the Domain Controller from changing the servers hostname. The example is if I have server1.acme.com, and I want to add it to my ad.acme.com domain, during the process of registering the server, it will change the servers hostname to server1.ad.acme.com. Not always ideal.

Whats worse is that since the VCSA is based on SUSE Linux, there is no checkbox the uncheck to prevent this during the PSC’s Join Domain functionality. Continue reading “Adding vCenters with the similar hostnames to the same Root AD Domain”

Posted on

Client is not authenticated to VMware Inventory Service after VCSA Upgrade to 6

Had this issue today after upgrading from VCSA 5.5 U3 to 6.6 U1A.

How was the VCSA setup before?

  • DNS servers configured to Prod DNS
  • Time configured to AD servers (automatic when you join the domain from the VAMI)
  • AD configured

Symptoms:

  • Logging into vCenter Server completes successfully with Administrator@vsphere.local
  • AD Users that used to authenticate with vCenter (even Admins) see the following error pop up: “Client is not authenticated to VMware Inventory Service – http://localhost:10080/invsvc”

After looking around, I didn’t see anything that pointed to a specific answer. Also to note, nothing related to vSphere or VCSA 6. A lot of items we found discussed that it was something that was fixed in 5.5 U2. Hmmm.. ok.

Resolution:

  • Since we were using AD for time server because 5.5 U3 VCSA made the AD server the time server, we had no NTP setup in the VAMI. Added the time server and moved on. Oddly for one of our servers, the DNS was missing as well, its odd, but even though it SHOWED the correct DNS servers, when I went to edit the network settings, it was set to automatic. Well that won’t do at all. Set them manually and moved on.

NOTE: Add the vCenter to the root domain if possible, or at least that is what I had to do. If you are adding more than one VCSA to the same domain, with very similar names, read this post as well.

  • Next, I removed any AD Identity sources and then left the domain.
    • If you are unfamiliar with how, first remove the Identity source from Administration > SSO> Configuration > Identity Sources.
    • Then leave the domain from Administration > Deployment > System Configuration > Nodes > {vCenter FQDN} > Manage > Active Directory.
    • Reboot the server after leaving the domain.
  • Once the VCSA comes back up, rejoin the domain:
    • Go back to Administration > Deployment > System Configuration > Nodes > {vCenter FQDN} > Manage > Active Directory, join the domain again.
    • REBOOT! (I know this takes forever, I’m sorry.)
    • Go back to Administration > SSO> Configuration > Identity Sources and add your AD identity source once again.
  • After this is complete, check your permissions and attempt logging in as an AD user.

This fixed it for us, but if you are having worse luck, let me know and I’ll try and work it out with you.

Posted on

Virtualization, the way the industry sees it. Part 1

First off, I won’t claim to speak on behalf of the entire industry… this is going to have a VMware bias. With that in mind, lets move forward.

So… in my last post (Virtualization, the way the school teaches it…) I talked about desktop virtualization apps. This is really where it all started. In the beginning there was CP-40 (1964)… no wait, too far back. Lets just start with the desktop virtualization on x86 server beginnings. IN THE EARLY DAYS (before I got into this), there was VMware who released a software called Vmware Server or Vmware GSX Server. This was the first x86 virtualization platform that ran on Windows Server. So, much like the last post, you have the Host, which is the physical hardware, the OS which is Windows Server, the Application which is VMware Server and the VM that runs inside of VMware server. (The VM has an OS and virtual hardware which are really just shared resources that the application pulls from the OS and the OS pulls from the physical server).

Ok, history lesson and recap over, lets move on. Unlike the desktop applications that most schools use during computer science or IT classes, working in an IT role, you’ll experience something called a Hypervisor. Fancy name! So what does a hypervisor do, that a desktop application can’t?

First lets consider the following in a desktop method, if the resources of the physical computer was cake and we had to feed those resources to each thing that needed it, you you would first have to feed the OS on the host. In this instance lets say that that is Windows. So the computer gives CPU, RAM, Network and Video resources to Windows. Well, Windows is running another application that is also running an Operating System, lets say Linux. So Windows needs to pass some cake to the App so it can pass it to Linux. BUT, we need to make sure there is enough cake for Windows too! If you don’t leave enough cake for Windows… its won’t be able to run the app, which won’t be able to run Linux. Starting to feel a bit like office space in here:

giphy

So now lets look at the hypervisor! In this case we will look at VMware’s current offering called ESXi. Instead of running Windows on the host physical hardware, lets remove it from the equation. Now thats one less thing to give cake to. ESXi is a very slim operating system, its sole purpose in life is to run VM’s. The biggest difference you will find, is that instead of plugging in a monitor into the computer, opening an app and SEEING the VM running and looking at its video output, you only see this:

  

Not much to look at huh? So here is the deal, as I stated before, the sole purpose of the hypervisor is to run VM’s. Running a VM and outputting the display from the VM to the VGA port isn’t a big concern because all Operating Systems have a method of remote access, and even if they don’t, VMware provides a tool to manage this server remotely, and that tool… can see the VM. GASP! Yes, you can still see the video output, but you have to use the tool. At this time, the tool is called the vSphere Client. The client is not only used to manage the VM’s but the configuration of host resources as well.

The hypervisor doesn’t just virtualize a server, it virtualizes the main components of the physical host. Unlike desktop virtualization where you are sharing the resources that the main OS is using, in a hypervisor, you setup a Virtual Switch for managing networking between hosts and VMs, Datastores for storing the virtual hard drives of VMs and CPU cores to allow multiple VMs to share the CPU clock scheduling.

So again, why the hypervisor? Short answer is, save the resources. Without Windows, you don’t have to provide resources to the bare-metal OS (Windows). Hypervisors are slim in comparison, and provide a LOT more functionality, which we will talk about later. So we save those resources, which means we can run MORE VM’s per server. Cool… more consolidation means more savings on hardware which makes your boss happy, the accountants happy and trust me… from a management perspective, you will be happier too.

So let me just add this here, ESXi is built by VMware and is provided free of charge. Yes, you read that correctly, FREE. The number of features available for free are slim in comparison to the paid version of vSphere (we will talk about the vNames later, there are a lot of them). But, for a small company, that may not matter. Free is free is free is free… too much? Nah. So for free, you have the ability to install a hypervisor (ESXi) on a physical server, configure it (later), and connect to it via a tool and build, manage and use Virtual Machines.

So lets recap:

  • Desktop Virtualization requires more resources to run the underlying OS
  • Hypervisor Virtualization does not output VM video to the host video output port
  • Hypervisors allow for greater density of servers per host by using a slimmed down OS built for virtualization
  • Hypervisors provide greater functionality, management and customization.
  • Hypervisor tools such as vSphere Client provide remote console access to VM’s while also managing host/VM resources and configuration.

Hypervisors will be briefly talked about in classes, but for the most part, not typically taught unless you are interested in taking a class towards a certification. If you are interested in trying out a hypervisor, seeing what it is capable of, then I suggest downloading a copy of ESXi from my.vmware.com (you have to register), and get started. If you are working towards a career in IT, I recommend that you start learning it sooner rather than later.