vCOPs - On Data Pruning and Kahadb

A couple of issues I ran into this week with vCOPs that are worth mentioning. One has to do with the infamous grey question mark. This replaces a badge when data collection is broke. The other has to do with pruning data collection in an environment where you are licensed for less per VM than what is being managed by a single vCenter instance. Yes a bit long winded, but let me explain…

First off the most noticeable tip off to vCOPs not collecting data is the grey question mark. In times of normalcy, each badge should be colored with a number indicating, visually and numerically, that it’s reading and making sense of the data. If it’s not there are a couple things that could contribute to this. First, disk space on the analytics VM, lack of..breaks ActiveMQ. VMware uses conservative guidelines as follows in regards to disk sizing, but keep in mind you can add disk space to the Analytics VM, albeit with a shutdown, at anytime.

A bit of background, ActiveMQ is an open source message broker that writes to a database called kahadb. Here it stores and forwards to other services. If the service can’t accept the messages, it writes it to this database. If the disk underflowth ActiveMQ is unable to write a complete file, there in potentially corrupting the database. To resolve this you can follow the steps in VMware KB 2013266. The process is pretty straight forward which haves you rename the kahadb and create a new one. However, it doesn’t state which default account to use to do so. I mistakenly used root (as opposed to admin) to make the changes which prevented the ActiveMQ service (which uses admin) from writing to the kahadb. Simply changing the permissions ( chown -R admin:admin /data/activemq/kahadb/)on the database resolved the issue.

Secondarily, unexpected shutdowns. As with any vApp make sure to shutdown, or restart the VM’s within the vApp at the vApp level. This will save you a lot of trouble down the line and keep that kahadb in working order.

Finally, data collection pruning is something I have heard numerous times from customers. How do I only report on 100 VM’s (because that is all I am licensed for) even though my vCenter instance see’s 300? Very simple. vCOPs collects data from vCenter using the vCenter enterprise adapter. Using the vCOPs admin UI, you establish this pull communication by registering vCOPs with your vCenter instance(s). Within this setup it asks for the IP address, Display name and the registration user for the vCenter plug-in. An optional setting is the collection user. This is the account it uses, based on its visibility within vCenter, to pull metrics. By default it will use the registration user. If you want to only collect on one datacenter or one cluster, for example, setup your collection user with just visibility to that datacenter or that cluster within vCenter. Perhaps in future versions this will be a little more stream lined, doable within the vSphere UI. But for now it’s the recommended way of pruning collections. Visit KB 1036195 for more information.