DDC Service Status Splunk Query

Here are the Splunk queries that I use to to find the status of each Citrix XenDesktop service that runs on each DDC server. It is really useful to find all services but with these Splunk queries are break out each XenDesktop service into its own query so you can slice and dice depending on what you are looking for. It speeds things up in Splunk if you can narrow the amount of data you pull in the first place.

DDC Service Status Splunk Query

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixAdIdentityService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixAnalytics ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixBrokerService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixConfigurationLogging ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixConfigurationService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixDelegatedAdmin ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixEnvTest ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixHostService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixMachineCreationService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixMonitor ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixPrivilegedService ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixStorefront ” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Splunk Queries for Citrix XenDesktop Services

I developed the following Splunk queries to count how many of each XenDesktop service is running in the Citrix Environment. Each query is based upon the Windows Service name and looks for the Start Mode to be AUTO. It will return a count of the service if the service is running or not. You only want to know the count if the service is running that is an easy change to the ‘State’ field value.

The Citrix EUEM service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”Citrix EUEM” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Encryption Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”Citrix Encryption Service” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Licensing service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”Citrix Licensing” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Peer Resolution Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”Citrix Peer Resolution Service” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix AD Identity Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixAdIdentityService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Analytics service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixAnalytics” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Broker Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixBrokerService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Configuration Logging service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixConfigurationLogging” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Configuration Replication service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixConfigurationReplication” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Configuration Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixConfigurationService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Credential Wallet service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixCredentialWallet” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix CSE Engine service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixCseEngine” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Default Domain Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixDefaultDomainService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Delegated Admin service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixDelegatedAdmin” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Environment Test service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixEnvTest” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Host Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixHostService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Machine Creation Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixMachineCreationService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Monitor service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixMonitor” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Privileged Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixPrivilegedService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Redirector service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixRedirector” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Service Monitor service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixServiceMonitor” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Storefront service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixStorefront” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Subscriptions Store service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixSubscriptionsStore” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Telemetry Service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixTelemetryService” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

The Citrix Web Services for Licensing service:

index=”windows” source=”service” eventtype=hostmon_windows Type=”Service” host=”*” Name=”CitrixWebServicesforLicensing” StartMode=”Auto” State=”*”
| dedup host
| chart count(Name) as y

StoreFront Service Status by Splunk Query

Use Splunk  to Query StoreFront Service Status

Here are the Splunk Queries I use to find StoreFront Services in the XenDesktop environment. They are specific to each StoreFront service. The last SPL query will pull on the StoreFront services from hosts that you specify. Please let me know what you think and if they help in your XenDesktop environment.

Citrix Configuration Replication Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixConfigurationReplication” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Credential Wallet Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixCredentialWallet” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Default Domain Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixDefaultDomainService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Peer Resolution Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix Peer Resolution Service” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Service Monitor:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixServiceMonitor” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Privileged Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixPrivilegedService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Subscriptions Store Service:
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixSubscriptionsStore” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Or if you want a single query to return the XenDesktop services status from a list hosts use this SPL Query:

index=”windows” eventtype=hostmon_windows Type=Service host=”StoreFrontServer01″ OR host=”StoreFrontServer02″ OR host=”StoreFrontServer03″ OR host=”StoreFrontServer04″ OR host=”StoreFrontServer05″ OR host=”StoreFrontServer06″ Name=”Citrix*” StartMode=”Auto” State=”*” | sort host, Name, State | table host, Name, State

XenDesktop Error Visualization in Splunk

Citrix XenDesktop Errors Visualized with Splunk

Here are the Splunk Queries I use when I need to create Splunk timechart dashboards to visualize Citrix XenDesktop errors. Try them out and let me know how they work in your environment.

This SPL Query grabs all Citrix related errors:

  • All Citrix Errors
    index=”wineventlog” sourcetype=”WinEventLog:Application” SourceName=”Citrix*” error | timechart count(EventCode) by SourceName

This query displays database errors for the last 7 days:

  • All Citrix Database Errors (7 Days)
    index=”wineventlog” sourcetype=”WinEventLog:Application” SourceName=”Citrix*” database* | timechart count(EventCode) by SourceName

Here is a query to display all events flagged as “failed” in the event log:

  • All Citrix Failures
    index=”wineventlog” sourcetype=”WinEventLog:Application” SourceName=”Citrix*” failed| timechart count(EventCode) by SourceName

It is really not good to have timeouts anywhere in your Citrix XenDesktop environment so here is a timechart query to display when they are happening:

  • All Citrix Timeouts
    index=”wineventlog” sourcetype=”WinEventLog:Application” SourceName=”Citrix*” timeout | timechart count(EventCode) by SourceName

Virtual machine events are easily displayed using this timechart query:

  • All Citrix Virtual Machine Events
    index=”wineventlog” sourcetype=”WinEventLog:Application” SourceName=”Citrix*” virtual | timechart count(EventCode) by SourceName

Splunk StoreFront Dashboard

Need to create a StoreFront Dashboard in Splunk?Splunk-Citrix-Event-Visualzation

Here is a quick SPL query you can run to gather some basic information on that state of the Citrix XenDesktop services that are running in your environment. If you want to narrow it down to your StoreFront hosts just replace the asterisk in the “host=”*”” with the names of your StoreFront servers or the beginning pattern of the host names for you StoreFront servers.

 

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix*” StartMode=”Auto” State=”*” | sort Name, State | table host, Name, StartMode, State

Want to know how many of each Citrix service is running across those hosts?

Just add  | stats count by Name

Want to know how many Citrix services are running on each host?

Just add | stats count by host

 

 

Splunk Query for Xen Desktop Services

If you have Citrix Xen Desktop in your environment and want to check on which server they are installed and their state use these Splunk queries. It is helpful so that you don’t have to keep an active list of the Xen Desktop servers. You could keep the list of Xen Desktop servers in a lookup but these queries are dynamic which saves you the headache of keeping the lookup current. These Splunk queries are also great if you are spinning up Xen servers on demand.

There is a Splunk query for each Citrix service that runs in a Xen Desktop environment so you can create a display for each one individually if you want.

XAV Server
Citrix Audio Redirection Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CtxAudioSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Desktop Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BrokerAgent” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Diagnostic Facility COM Server
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CdfSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Encryption Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix Encryption Service” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix End User Experiencing Monitoring
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix EUEM” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Group Policy Engine
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixCseEngine” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix HDX MediaStream for Flash Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CtxFlashSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Location and Sensor Virtual Channel Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CtxSensVcSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Mobile Receiver Virtual Channel Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”MRVCSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Print Manager Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”cpsvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Profile Management
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”ctxProfile” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Pvs for VMs agent
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”PvsVmAgent” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Services Manager
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”ServicesManager” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Smart Card Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CtxSmartCardSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Stack Control Service
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”StackControlService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

CitrixTelemetryService
index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CitrixTelemetryService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Licensing Services Splunk Query

Find the Status of Citrix Licensing Services with Splunk

Citrix Licensing

During this exercise to get Splunk to ingest data from Citrix and make since of it we came to the Licensing server. We have ours running on a dedicated server; however,  the Splunk queries still search all hosts to keep the queries standard. Also if you ever planned on expanding your Citrix environment later you are covered. There are only 2 services that run on the license server concerned with Citrix: Citrix Licensing and CitrixWebServicesforLicensing.

These Splunk queries will give you a table with with hosts and the state of those 2 services. The dedup statement will prevent the same server from showing up more than once depending on the time frame your Splunk Universal Forwarder gathers data and the time frame your run your query for the dashboard.

index="windows" eventtype=hostmon_windows Type=Service host="*" Name="Citrix Licensing" 
StartMode="Auto" State="*" | dedup host | sort host, State | table host, State
index="windows" eventtype=hostmon_windows Type=Service host="*" Name="CitrixWebServicesforLicensing" 
StartMode="Auto" State="*" | dedup host | sort host, State | table host, State

As always please let me know what you think.

Thank you,
Logan Bingham

Citrix PVS Boot Time Splunk Query

Ever wonder how to get the boot time that Citrix XenDesktop PVS puts in the Windows Application Event Log in a format that you can calculate in Splunk?

I have my Windows Event Logs dumped into index=wineventlog with a sourcetype=WinEventLog:Application. When looking for the PVS bootime you need search for SourceName=StreamProcess. That will give you all the events for Stream Process and all of those events use EventCode=10 EventType=4 Type=Information. Unfortunately the way PVS records the boot time for each VM is in the Message field in the following format: Message=Device VMNAME boot time: X minutes Y seconds.

Here is my attempt that uses REGEX in the query to get the numbers out of the message field and into a table:

index="wineventlog" sourcetype=WinEventLog:Application  SourceName=StreamProcess Message="Device * boot time: * minutes * seconds." | rex field=_raw "(?ms)^\\d+/\\d+/\\d+\\s+\\d+:\\d+:\\d+\\s+\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+\\d+\\w+\\d+\\.\\w+\\.\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+(?P<PVSDesktopName>\\s+\\w+\\s+)[^:\\n]*:\\s(?P<PVSBootTimeMin>[^\\s]+)\\s+\\w+\\s(?P<PVSBootTimeSec>[^\\s]+)" offset_field=_extracted_fields_bounds | table PVSDesktopName, PVSBootTimeMin, PVSBootTimeSec | sort -PVSBootTimeMin

Now you can work with the numbers to do some math.

index="wineventlog" sourcetype=WinEventLog:Application  SourceName=StreamProcess Message="Device * boot time: * minutes * seconds." | rex field=_raw "(?ms)^\\d+/\\d+/\\d+\\s+\\d+:\\d+:\\d+\\s+\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+\\d+\\w+\\d+\\.\\w+\\.\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+(?P\\s+\\w+\\s+)[^:\\n]*:\\s(?P[^\\s]+)\\s+\\w+\\s(?P[^\\s]+)" offset_field=_extracted_fields_bounds | eval BootTimeSec=((PVSBootTimeMin*60)+(PVSBootTimeSec)) | table PVSDesktopName, BootTimeSec

Splunk PVS Boot Time In Seconds

 

 

 

 

 

Splunk Query Finds All Citrix XenDesktop Services

Splunk query to find status of Citrix XenDesktop Services

If you ever need to find all the Citrix services that run for XenDesktop in Splunk you can use this Splunk query. It will also give you the State of each service so you query if the service is running or not. I used this as the basis for many dashboards to report which Xen services are running on which servers and what is the current state.

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix*” OR Name=”ComTradeMPPVSAgent” OR Name=”CdfSvc” OR Name=”CtxLSPortSvc” OR Name=”CdfSvc” OR Name=”BNBOOTP” OR Name=”BNPXE” OR Name=”BNAbs” OR Name=”soapserver” OR Name=”StreamService” OR Name=”BNTFTP” OR Name=”PVSTSB” OR Name=”CtxAudioSvc” OR Name=”CtxFlashSvc” OR Name=”CtxSensVcSvc” OR Name=”MRVCSvc” OR Name=”cpsvc” OR Name=”ctxProfile” OR Name=”PvsVmAgent” OR Name=”ServicesManager” OR Name=”CtxSmartCardSvc” OR Name=”StackControlService” OR Name=”CitrixTelemetryService” StartMode=”Auto” State=”*” | sort Name, State | table host, Name, StartMode, State

How are you going to use this query?

If you modified please share the changes you made.

Splunk Query for Citrix PVS Services

Splunk Plus Citrix

If you have Citrix Xen Desktop in your environment you might be using PVS Server. Citrix PVS Server is a great way to decrease operational maintenance and VM density in your virtual environment. In case you want to use Splunk to analyze the data about your PVS environment here are the queries that I use. It is not defined by host so it will dynamically build the list for you based on the PVS services that are running on each host in your environment.

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CdfSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”ComTradeMPPVSAgent” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNBOOTP” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNPXE” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNAbs” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”soapserver” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”StreamService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State
index=”windows”

eventtype=hostmon_windows Type=Service host=”*” Name=”BNTFTP” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”PVSTSB” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State