PowerShell Commands to Install Hyper-V

Install Hyper-V using Windows PowerShellPowerShell Icon

install-windowsfeature –name hyper-v -includemanagementtools –restart

Install Hyper-V using DISM.exe, run the following command from an elevated command prompt:
dism /online /enable-feature /featurename:microsoft-hyper-v

 

 

Install the management tools with Windows PowerShell, you use the Install-Windows-Feature cmdlet, as follows:
install-windowsfeature -name rsat-hyper-v-tools

Install just the Hyper-V Manager or just the Hyper-V PowerShell module, you use one of the following commands:
install-windowsfeature -name hyper-v-tools
install-windowsfeature -name hyper-v-powershell

NetScaler Traffic Management Features

Here are the traffic management features of the Citrix NetScaler:

Cache Redirection

NetScaler Traffic Managaement analyzes incoming requests and forwards the requests for already cached data to cache servers. Dynamic HTTP requests and non-cacheable requests are forwarded to the origin servers.

Content Switching

Analyzes client requests and redirects the requests to specific servers on the basis of geographical area, authorization credentials, and device from which the request was initiated.

DataStream

Ensures optimal distribution of traffic from the application and web servers to the database servers. Enables you to segment traffic according to information in the SQL query and on the basis of database names, user names, character sets, and packet size.

Domain Name System

Provides authoritative domain name server (ADNS server) functionality for a domain. The NetScaler appliance functions as a DNS end resolver and forwarder, and also helps in name resolution when fully qualified domain names are not configured.

Firewall Load Balancing

Distributes the traffic across multiple firewalls, providing fault tolerance, increased throughput, and high availability.

Global Server Load Balancing

Enables disaster recovery and ensures continuous availability of applications by protecting against points of failure in a wide area network (WAN).

Link Load Balancing

Load balances outbound traffic across multiple Internet connections to transmit packets seamlessly over the best possible link.

Load Balancing

Distributes user requests for web pages and other protected applications across multiple servers to prevent server overloading and failure. Load balancing also provides fault tolerance.

SSL Offload and Acceleration

Offloads SSL processing from a server to the NetScaler appliance to accelerate SSL transactions.

Splunk Antivirus Exemptions

Splunk Antivirus Exemptions

splunk anti virus exemptionsThere are 2 main parts of your Splunk environment that will need exemptions from Antivirus software: the hosts that run Splunk and the hosts that run Universal Forwarder. The file locations are different for Windows and Linux. The file paths assume you have installed Splunk to the default location. If your environment required you to install to another directory/location just apply those changes to the paths below. You will also need to check and see if any of the scripts that are run as part of the apps or add-ons that you installed need to be exempted as well. This is the case in my environment and I have to get any file changes exempted before they will run in the production environment. This can get tricky when you are pushing changes to Universal Forwarders and the folders/files are transferred successfully but do not run to return results.

So as of the current release when this was written those are:

Splunk Enterprise on Windows Server

\Program Files\Splunk and all sub-directories
\Program Files\Splunk\var\lib\splunk and all sub-directories

Splunk Universal Forwarder on Windows OS

\Program Files\SplunkUniversalForwarder and all subdirectories

Splunk Enterprise on Linux

/opt/splunk and all sub-directories
/opt/splunk/var/lib/splunk and all sub-directories

Splunk Universal Forwarder on Linux OS

/opt/splunkforwarder and all subdirectories

Files to exclude on Windows

splunk-admon.exe
splunk-compresstool.exe
splunk-MonitorNoHandle.exe
splunk-netmon.exe
splunk-optimize-lex.exe
splunk-optimize.exe
splunk-perfmon.exe
splunk-regmon.exe
splunk-winevtlog.exe
splunk-winhostinfo.exe
splunk-winprintmon.exe
splunk-wmi.exe
splunk.exe
splunkd.exe
splunkweb.exe

Files to exclude on Linux

bloom
btool
btprobe
bzip2
cherryd
classify
exporttool
locktest
locktool
node
python*
splunk
splunkd
splunkmon
tsidxprobe
tsidxprobe_plo
walklex

 

Citrix Licensing Services Splunk Query

Find the Status of Citrix Licensing Services with Splunk

Citrix Licensing

During this exercise to get Splunk to ingest data from Citrix and make since of it we came to the Licensing server. We have ours running on a dedicated server; however,  the Splunk queries still search all hosts to keep the queries standard. Also if you ever planned on expanding your Citrix environment later you are covered. There are only 2 services that run on the license server concerned with Citrix: Citrix Licensing and CitrixWebServicesforLicensing.

These Splunk queries will give you a table with with hosts and the state of those 2 services. The dedup statement will prevent the same server from showing up more than once depending on the time frame your Splunk Universal Forwarder gathers data and the time frame your run your query for the dashboard.

index="windows" eventtype=hostmon_windows Type=Service host="*" Name="Citrix Licensing" 
StartMode="Auto" State="*" | dedup host | sort host, State | table host, State
index="windows" eventtype=hostmon_windows Type=Service host="*" Name="CitrixWebServicesforLicensing" 
StartMode="Auto" State="*" | dedup host | sort host, State | table host, State

As always please let me know what you think.

Thank you,
Logan Bingham

Citrix PVS Boot Time Splunk Query

Ever wonder how to get the boot time that Citrix XenDesktop PVS puts in the Windows Application Event Log in a format that you can calculate in Splunk?

I have my Windows Event Logs dumped into index=wineventlog with a sourcetype=WinEventLog:Application. When looking for the PVS bootime you need search for SourceName=StreamProcess. That will give you all the events for Stream Process and all of those events use EventCode=10 EventType=4 Type=Information. Unfortunately the way PVS records the boot time for each VM is in the Message field in the following format: Message=Device VMNAME boot time: X minutes Y seconds.

Here is my attempt that uses REGEX in the query to get the numbers out of the message field and into a table:

index="wineventlog" sourcetype=WinEventLog:Application  SourceName=StreamProcess Message="Device * boot time: * minutes * seconds." | rex field=_raw "(?ms)^\\d+/\\d+/\\d+\\s+\\d+:\\d+:\\d+\\s+\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+\\d+\\w+\\d+\\.\\w+\\.\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+(?P<PVSDesktopName>\\s+\\w+\\s+)[^:\\n]*:\\s(?P<PVSBootTimeMin>[^\\s]+)\\s+\\w+\\s(?P<PVSBootTimeSec>[^\\s]+)" offset_field=_extracted_fields_bounds | table PVSDesktopName, PVSBootTimeMin, PVSBootTimeSec | sort -PVSBootTimeMin

Now you can work with the numbers to do some math.

index="wineventlog" sourcetype=WinEventLog:Application  SourceName=StreamProcess Message="Device * boot time: * minutes * seconds." | rex field=_raw "(?ms)^\\d+/\\d+/\\d+\\s+\\d+:\\d+:\\d+\\s+\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+\\d+\\w+\\d+\\.\\w+\\.\\w+\\s+\\w+=\\w+\\s+\\w+=\\w+\\s+\\w+=\\d+\\s+\\w+=\\w+\\s+\\w+=\\w+(?P\\s+\\w+\\s+)[^:\\n]*:\\s(?P[^\\s]+)\\s+\\w+\\s(?P[^\\s]+)" offset_field=_extracted_fields_bounds | eval BootTimeSec=((PVSBootTimeMin*60)+(PVSBootTimeSec)) | table PVSDesktopName, BootTimeSec

Splunk PVS Boot Time In Seconds

 

 

 

 

 

Splunk Query Determine NetScaler Load Balanced VIP Status

Determine the status of an individual Load Balanced Service with this Splunk query

When you have your Netscaler load-balancing traffic for a service Splunk is a great way to monitor and report on the status of the load balanced VIP. Now we are talking about the the load balanced VIP resource that is per VIP, not per service or service group.
Netscaler Load Balancing

 index="netscaler"
 source="stat:lbvserver"
 sourcetype="citrix:netscaler:nitro"
 actsvcs="*"
 avl_status="1"
 avl_status_string="UP"
 citrix_netscaler_avl_status="1"
 citrix_netscaler_state="UP"
 name="*"

The index is the index you are using for netscaler data.

The source is stat:lbserver and the sourcetype is citrix:netscaler:nitro.

The name of the load balanced VIP is the name to look for using this query.

There are a few fields that you can call depending if you want a binary or text result returned. Which one is up to you and can determine that type of visualization you can use.

For binary status of the load balanced VIP use avl:status or citrix_netscaler_avl_status they give you “0” or “1”

For text status of the load balanced VIP use avl_status_string and citrix_netscaler_state they give you “UP” or “DOWN”

I hope this helps to create reports that not generated by Citrix products but by Splunk. If you make a cool report or chart in Splunk let me know in the comments.

The data that the query looks for is captured using the Splunk Add-on for Citrix NetScaler which is a Splunk supported Add-on so you can get support for it if you have a valid support contract. I know that having the vendor provide support sometimes makes a difference when it comes to choosing the add-ons you use in your Splunk environment. Here is the link to the Splunk Add-on for Citrix NetScaler docs if you want to have a look.

Thank you,

Logan Bingham

Splunk Query Finds All Citrix XenDesktop Services

Splunk query to find status of Citrix XenDesktop Services

If you ever need to find all the Citrix services that run for XenDesktop in Splunk you can use this Splunk query. It will also give you the State of each service so you query if the service is running or not. I used this as the basis for many dashboards to report which Xen services are running on which servers and what is the current state.

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”Citrix*” OR Name=”ComTradeMPPVSAgent” OR Name=”CdfSvc” OR Name=”CtxLSPortSvc” OR Name=”CdfSvc” OR Name=”BNBOOTP” OR Name=”BNPXE” OR Name=”BNAbs” OR Name=”soapserver” OR Name=”StreamService” OR Name=”BNTFTP” OR Name=”PVSTSB” OR Name=”CtxAudioSvc” OR Name=”CtxFlashSvc” OR Name=”CtxSensVcSvc” OR Name=”MRVCSvc” OR Name=”cpsvc” OR Name=”ctxProfile” OR Name=”PvsVmAgent” OR Name=”ServicesManager” OR Name=”CtxSmartCardSvc” OR Name=”StackControlService” OR Name=”CitrixTelemetryService” StartMode=”Auto” State=”*” | sort Name, State | table host, Name, StartMode, State

How are you going to use this query?

If you modified please share the changes you made.

Splunk Query for Citrix PVS Services

Splunk Plus Citrix

If you have Citrix Xen Desktop in your environment you might be using PVS Server. Citrix PVS Server is a great way to decrease operational maintenance and VM density in your virtual environment. In case you want to use Splunk to analyze the data about your PVS environment here are the queries that I use. It is not defined by host so it will dynamically build the list for you based on the PVS services that are running on each host in your environment.

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”CdfSvc” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”ComTradeMPPVSAgent” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNBOOTP” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNPXE” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”BNAbs” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”soapserver” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”StreamService” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State
index=”windows”

eventtype=hostmon_windows Type=Service host=”*” Name=”BNTFTP” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

index=”windows” eventtype=hostmon_windows Type=Service host=”*” Name=”PVSTSB” StartMode=”Auto” State=”*” | dedup host | sort host, State | table host, State

Citrix Event IDs 2000 to 2102 to Monitor with Splunk

Citrix Event ID 2000 to Citrix Event ID 2102

Citrix Event ID
Splunking Citrix? If you are or just need to figure out what the Event IDs are that Citrix is filling up your Windows Event Logs with here are Citrix Event IDs 2000 to 2102. These are more detailed than the earlier Event IDs and mainly have to do with the Citrix Broker Services. So if you are tracking down issues with XML services like the Citrix Broker Service this is range of Citrix Event IDs to look into.

Looking for Citrix Event IDs 1 to 509

Looking for Citrix Event IDs 1000 to 1201

Event ID to Monitor
Event Message Text
Event ID 2000
The Citrix Profile management Group Policy Extension has started. Cause: The Citrix Profile management Group Policy Extension has started to process policies for user ‘Domain\user’. Action: This message is informational and no action is required.
Event ID 2001
The Citrix Broker Service failed to initialize XML services. The services will attempt to initialize again in approximately 1 minute(s).Exception ‘Input string was not in a correct format.’ of type ‘System.FormatException’.
Event ID 2003
The Citrix Broker Service successfully started XML services.
Event ID 2004
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: WerFault.exe (15428) consumed 6796857344 bytes, BrokerService.exe (9432) consumed 722894848 bytes, and BrokerService.exe (9528) consumed 637378560 bytes.
Event ID 2007
The Citrix Broker Service is stopping XML HTTP services.
Event ID 2008
The Citrix Broker Service successfully stopped XML HTTP services.
Event ID 2012
The Citrix Broker Service was unable to send a response to the XML client.
Event ID 2012
The Citrix Broker Service was unable to send a response to the XML client. Details: Request URL: ‘/SCRIPTS/WPNBR.DLL’ Error Code: ‘1229’ Error Message: ‘An operation was attempted on a nonexistent network connection’ Exception Type: ‘System.Net.HttpListenerException’ Exception Call Stack: ‘ at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 size) at Citrix.Cds.Xms.Multiplexer.XmlMultiplexer.HandleRequest(HttpListenerContext context, DateTime startTime)’
An unexpected exception occurred while the Citrix Broker Service processed an HTTP request. Error details: Request URL: ‘https://ddc.Domain.com/scripts/wpnbr.dll’ Exception Type: ‘Citrix.Cds.Broker.DAL.DALPowerActionIgnoredException’ Exception Call Stack: ‘ at Citrix.Xms.XmlSupport.XmlPerf.TimeoutAction(Int32 timeoutMs, Action action) at Citrix.Cds.Xms.Wpnbr.WpnbrServer.HandleRequest(HttpListenerRequest request, WindowsIdentity identity, DateTime startTime)’
Event ID 2100
The Citrix Broker Service failed to validate a user’s credentials on an XML service. Verify the trust relationships between your domains. Error details: User: ‘Domain\timothy.frazier.wa’ Error: ‘InvalidCredentials’ Message: ‘Failed Windows logon, error code 50’ Stack Trace: ”
Event ID 2101
The Citrix Broker Service failed to validate a user’s credentials on an XML service again. A previously detected problem still exists. To avoid excessive event logging, the service is suppressing related messages (event ID 2100) until the problem is resolved. Initialization attempts will continue.
Event ID 2102
The Citrix Broker Service successfully validated user credentials. It is no longer suppressing the related messages (event ID 2100).

Splunk with btool for Combined Inputs.conf

Inputs not showing up in Splunk?

The other day I pointed some additional data at my Splunk Indexer but it was not showing up. I wanted to make sure I did not fat finger something or that I did not have overlapping inputs from other apps inputs. I found the following BTOOL command to run when you want to find out how Splunk has combined all the inputs.conf files that is it processing.

./splunk btool inputs list

Splunk BtoolYou run it from $SPLUNK_HOME/bin folder and it does not matter if you are on a Search Head, Indexer, DCN, or Universal Forwarder.
It outputs all the Stanzas in all the INPUTS.conf files in the final order they are processed in. You get to see which Stanzas won due to the folder and file precedence that Splunk uses when processing .conf files.

By the way if you want to get the same information about any .conf file other than inputs.conf you just replace place the name of .conf file after btool and before list.

If you want to save the results in a file so you can slice and dice the results in text editor or sort it with Excel use the following btool command.

./splunk btool inputs list > nameOfFile.txt

That list command will give you just the names of the Stanzas but what if you want to know the entire path to each file that contains those Stanzas?

Use this btool command that includes the list and –debug switches.

./splunk btool inputs list --debug > nameOfFile.txt

The output gives you the full path to each .conf file of each processed Stanza.

It is a lot of data about those little old Stanza that mean so much to Splunk and the data you sent your way.

Try btool for props.conf and transforms.conf it will help provide even more insight to your data and the is what it is all about.

Here is the link to the Splunk>docs btool page.

Here is the link to a great Splunk Blog page about btool