On XenApp 7.5 servers (and possibly 7.1 and 7.0), you may notice the registration state of the machine is stuck on “Initializing” in Citrix Studio and no one will be able to launch any apps. I’m still investigating if the 7.6 VDA also has this behavior. This is how it looks in Studio:
You can also run the following powershell command:
Add-PSSnapin Citrix.*.Admin.V*
followed by:
Get-BrokerMachine | select MachineName, SummaryState, MachineInternalState, RegistrationState
which results in:
You will notice the impacted server in question has a MachineInternalState set to “Unavailable” and the RegistrationState is stuck on “Initializing”. We’ve noticed this happening on XenApp servers that have been up around the 24 or 25 day mark. They suddenly stop being registered.
The work around is to restart the “Citrix Desktop Service” on the impacted server. The service is not in a Stopped state so it’s hard to setup monitoring using a 3rd party monitoring tool. I just wrote this PowerShell script that queries the delivery controller for the actual state of registration on every XenApp server to see if it’s stuck Initializing, restart the Citrix Desktop Service on the impacted servers, log a .csv file with the names of the impacted servers for historical purposes, and send an email notification out to the Citrix admins and NOC. I have this running as a scheduled task (stacked a few min apart) every 5 minutes on all my delivery controllers:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
##Written by Jason Samuel - jasonsamuel.com ##http://www.jasonsamuel.com/2014/10/16/citrix-xenapp-7-x-vda-registration-state-stuck-in-initializing-and-a-self-healing-powershell-script-to-fix-it Add-PSSnapin Citrix.*.Admin.V* $Results = @() $Date = (Get-Date -DisplayHint Date) $save_date = $Date.ToString("MM-dd-yyyy-hh-mm-ss-tt") $Results += Get-BrokerMachine -RegistrationState "Initializing" | select DNSname, AgentVersion If (!$Results) { exit } Else { ##Restart the Citrix Desktop Service on stuck servers foreach ($vda in $Results) { Restart-Service -InputObject $(get-service -ComputerName $vda.DNSName -Name "Citrix Desktop Service") -Verbose $EmailBodyList += $vda.DNSName + "`t" + $vda.AgentVersion + "`r`n" } } ##Writes the result to a CSV file on your delivery controller for historical purposes $file_output = ('D:\Citrix_VDA_Restart\XA_VDA_' + $save_date + '.csv') $Results | Export-CSV -Path $file_output -NoTypeInformation ##Sends an email with the list of servers in the body as well as the CSV attachment $filename = $file_output $smtpServer = “YOURSMTPSERVER” $msg = new-object Net.Mail.MailMessage $att = new-object Net.Mail.Attachment($filename) $smtp = new-object Net.Mail.SmtpClient($smtpServer) $msg.From = “citrix@YOURDOMAIN.com” $msg.To.Add(“citrixadmins@YOURDOMAIN.com”) $msg.Subject = “XenApp 7.6 servers stuck Initializing” $msg.Body = “The below XenApp 7.6 servers are stuck Initializing on the Delivery Controllers. This script is now attempting to restart the Citrix Desktop Service on the impacted servers and force the VDA to register.” + "`r`n" + "`r`n" + $EmailBodyList $msg.Attachments.Add($att) $smtp.Send($msg) Start-Sleep -s 5 exit |
If the code runs off the page above you can highlight all of it and paste to Notepad or download it here in text format (make sure you right click – save as). Just change the extension to .ps1 and set your path for the CSV files, SMTP server, and email addresses you want notifications to go to:
Anyhow, you will get a nice notification email like this when servers get stuck. In this example I had 2 servers that were stuck Initializing and by the time the email was received the VDA had already been restarted on both and were Registered again. 🙂
I’m hoping this is fixed in the the 7.6 VDA and am in the process of testing. I really don’t like to have to setup scripts to monitor service health like this but in a pinch, running a self-healing monitoring script is your best bet to prevent an application outage until the issue is resolved. I’m currently monitoring all servers with the 7.6 VDA and will update this post if I see them exhibit the same behavior as the 7.5 VDA.
UPDATE:
Just updated the script above to include the agent version. Here’s how your notification email and .csv file will look:
Jason Samuel is a visionary product leader and trusted advisor with a proven track record of shaping strategy and driving technology innovation. With extensive expertise in enterprise end-user computing, security, cloud, automation, and virtualization technologies, Jason has become a globally recognized authority in the IT industry. His career spans consulting for hundreds of Fortune 500 enterprises across diverse business sectors worldwide, delivering cutting-edge digital solutions from Citrix, Microsoft, VMware, Amazon, Google, and NVIDIA that seamlessly balance security with exceptional user experiences.
Jason’s leadership is amplified by his dedication to knowledge-sharing as an author, speaker, podcaster, and mentor within the global IT and technology community. Recognized with numerous prestigious awards, Jason’s contributions underscore his commitment to advancing technology and empowering organizations to achieve transformative results. Follow him on LinkedIn.
Lal Mohan
February 12, 2015 at 8:00 PM
Wouldn’t restarting the Desktop service on the failing server kill all the existing user sessions?
Gizmo
September 13, 2016 at 9:39 AM
Furthermore, restarting the service shows in the event log that it successfully connected to the DDC yet the DDC still does not show the targets registered 🙁
Arun
March 10, 2017 at 1:29 AM
Hi Jason,
Recently we migrated XenApp 6.5 to XD 7.9 and we get the same issue reported
Opened a case with Citrix and will keep posted with the updates on it.
Magnus
July 6, 2017 at 8:40 AM
Restarting the desktop service doesn’t kill existing sessions (at least from my testing).
When seeing the agent in an “initializing” state on a Win7 VDA, typically its an issue with either the video driver, or general connectivity type issue with the DDC or DNS entries not being correct. Restarting the service, in my experience, fixes about 20% of the machines that this issue occurs to.
lschums
July 18, 2023 at 10:01 AM
We see this issue with pretty recent 2203 LTSR CU2 VDAs.
It is nice that there is an easy workaround, but has the underlaying cause ever be found ?
In our case the initializing state is also not reported in director as an error so you only notice, that there are a less sessions than usual.
lschums
July 19, 2023 at 1:19 AM
We see this behaviour with 2203 CU2 VDAs and while being happy, that there is at leasy an workaround without having to reboot workers is is strange to see something like that many years later.
In our case we observed multiple Kerne-General 1 Messages in System-Log about time adjustment in fractions of seconds (talking about ~0,05s each) and this immediately stops after restarting Citrix Desktop Service and all former sessions are visible again in Director and Studio.