Citrix 15. Citrix Fixes – XenServer

Citrix Fixes – XenServer

 

A list containing the majority of Citrix XenServer support articles collated to make this page a one stop place for you to search for and find information regarding any issues you have with the product and its related dependencies.

 

The page is updated daily with new support articles and information. Articles will change from time and if information here is outdated or incorrect please let me know using the comments. Links may also expire or change so if you find broken links, please again let me know. For each issue, known product versions affected are recorded however that does not mean product versions that aren’t listed are not affected.

There is a search box that you can use if looking for a specific fault. For example if you have an error code or error message, use that to perform a search. You can also use your browsers search feature which will perform a search against the whole page based on the words you enter.

 

 

Search: 

Brief Description of IssueBrief Description of FixApplicable Product Versions Affected (if known)Link to supplemental Support Article(s)A XenServer 6.2 to 7.1 upgrade fails with "[Errno 28] No space left on device:'/tmp/backup-rORnXu/.xen-backup-partition'".Find the folders with high inode usage, check which folders have a high number of files e.g. temporary files, and then delete the files from each folder. https://support.citrix.com/article/CTX227290When attempting to install XenTools you receive error "Your Tools ISO is on a broken SR, XenServer will attempt to fix it before proceeding. Do you wish to continue?". Within "xensource.log" is error "Storage_interface.Sr_not_attached".Search for stale/duplicate XenServer Tools Storage Repository by running command "xe sr-list name-label="XenServer Tools". If there are multiple entries, find each one with state "currently-attached (RO): false" and forget them. https://support.citrix.com/article/CTX220219XenServer time is incorrect after changing the BIOS time.Make sure all XenServer hosts in the pool have identical NTP servrs specified. You may have to manually restart the "ntpd" service and then run a manual sync as explained in the CTX article. https://support.citrix.com/article/CTX226572Virtual Machine cannot start with error "VDI is being used by another operation".Look for instances where the VBD is attached to other VMs as described in the CTX article. https://support.citrix.com/article/CTX214407On some high resolution monitors, the XenServer console does not display.Use an alternative monitor or connect remotely via SSH for example and edit the grub config file to edit the VGA section, as outlined in the CTX article. https://support.citrix.com/article/CTX226191You receive error "Internal error: Connection failed: no host resolved" when adding a new server to the pool.Reconnect XenCenter with the IP address of the pool master and then try again. https://support.citrix.com/article/CTX224728Applying a patch through XenCenter results in error "invalid_file". Trying to patch via CLI hangs.This is caused by no SR being elected as the default SR. Right-click on an SR and set it as default.Citrix XenServer 7.1.https://support.citrix.com/article/CTX224142After upgrading an HPSA driver XenServer can no longer boot. You see error "Kernel panic" on the XenServer console.Before upgrading the HPSA driver, add options "hpsa hpsa_allow_any=1" to "/etc/modprob.conf". https://support.citrix.com/article/CTX223518When importing a VHD into XenCenter you receive error "No local storage and no default storage: cannot import Transfer VM" on the XenServer console.Set a default SR for the pool in XenServer and then reinstall the Transfer VM using the steps provided in CTX article. https://support.citrix.com/article/CTX215142The Management Interface shows blank in the Nteworking tab on XenCenter and the pool Management Network bond shows as "unknown" after adding a host to the pool.Find and forget all NICs that are disconnected on both the master host and the host you just added.Citrix XenServer 7.1.https://support.citrix.com/article/CTX223314When selecting a Master Image during Machine Catalog creation you receive error "You must select a disk image with at least one network card",This can happen if you have a VM imported from a previous version of XenServer into a higher version. Create a diskless VM but do not start it. Detach the disk associated to the problematic VM and attach it to the newly created diskless VM. Create a Machine Catalog using this VM as the template. https://support.citrix.com/article/CTX222490XenCenter shows dom0 memory as 4GB after changing it to 32GB, even after a reboot.During reboot the XenServer was booting to "Trusted Boot" mode rather than "XenServer" mode. The Trusted Mode memory setting was set to 4GB. You configure the default XenServer boot mode to boot into "XenServer" mode as explained in the CTX article.Citrix XenServer 7.1.https://support.citrix.com/article/CTX221790On a host that has been removed from the pool, a red X displays beside the host name in XenCenter. When you issue a "xe host-forget" against the removed host you receive error "This host cannot be forgotten because there are some user VMs still running host".You need to determne which VMs are still running on the host and shut them down. This is done via CLI, following the CTX article steps. https://support.citrix.com/article/CTX126382When starting a virtual machine you receive error "Internal error: xenopsd internal error: Device.Ioemu_failed('vgpu exited unexpectedly')".If the ECC feature is enabled on a GRID K2 card (disabled by default), virtual GPU fails to start. vGPU is currently not supported with ECC active. Use "nvidia-smi" to list the status of all GPUs and then issue another nvidia-smi command to disable ECC on the culprit GPU as explained in the CTX article.NVIDIA GRID.https://support.citrix.com/article/CTX219835When using older NVIDIA drivers, XenServer experiences hanging or crashing.Update drivers to latest version from NVIDIA.Citrix XenServer 7.0.https://support.citrix.com/article/CTX223092When pointing XenServer to your license server you receive error "A license for XenServer could not be checked-out because the license server could not be reached at the given address and port. Please check the connection details and verify that the license server is running".Restart the XenServer toolstack, verify the time and date on affected XenServer host, make sure DNS settings are configured correctly so that the license server is resolvable, make sure the firewall is allowing connections. https://support.citrix.com/article/CTX200015Virtual Machines fail to boot with error "No bootable device found".Check the position of the OS disk. The disk should be at position 0. This could also be an issue with the RAM on the physical XenServer host. The "mcelog" will contain any RAM related errors. https://support.citrix.com/article/CTX216911When starting multiple vGPU enabled Virtual Machines, the XenServer host crashes.This is caused by a feature (PML) introduced with Broadwell CPUs that has been adopted in Xen 4.6 and XenServer 7. As a workaround, disable PML via CLI on each host in the pool.Citrix XenServer 7.0.https://support.citrix.com/article/CTX220674When upgrading to XenServer Tools 7.0 you receive error "Windows Management Agent failed to install".Remove "XENBUS" and "XENVIF" keys from the registry using "psexec". Reoot and install the tools again. https://support.citrix.com/article/CTX215427Window Server 2016 VM template is missing from XenServer 7.1.Create the template manually by running command "/usr/bin/create-guest-templates".Citrix XenServer 7.1.https://support.citrix.com/article/CTX221332After upgrading to XenServer 7.1, creating new or reattaching existing NFS storage repositories can fail.Install Hotfix "XS71E003".Citrix XenServer 7.1.https://support.citrix.com/article/CTX223285You may experience a system leak in "systemd" when using SSH.This is a known issue, due to a defect in systemd. Citrix have published a hotfix for XenServer 7.0 and the fix is part of 7.1 CU1.Citrix XenServer 7.0 and 7.1.https://support.citrix.com/article/CTX228129After an upgrade to XenServer 7.1, the hosts do not reconnect to storage.The PDB(s) are likely damaged and need recreated, by following steps from the CTX article. https://support.citrix.com/article/CTX229625When you try and remove a host from the pool you receive error "the SR.shared flag cannot be set to false while the SR remains connected to multple servers".The CTX article explains how destroying a PBD (Physical Block Device) will get around this issue. https://support.citrix.com/article/CTX230692The uploaded update package is invalid.This issue happens when the date on XenServer is older than the date of the "gpg" key. Synchronise the time by running "ntpdate -U 'NTP server IP'" as described in the CTX article.Citrix XenServer 7.1.https://support.citrix.com/article/CTX231197When adding Hardware HBA Storage through XenCenter you receive error "The SCSlid parameter is missing or incorrect".Upgrade XenCenter from 7.1.1.4743 to 7.12.5459.Citrix XenServer 7.1 LTSR CU1.https://support.citrix.com/article/CTX231200XenServer 6.5 upgrade to 7.1 becomes stuck at 72%.The upgrade isn't actually stuck, it is trying to restore all your previous performance related information. Manually archive the files under "/var/xapi/blobs" and then proceed with the upgrade.Citrix XenServer 6.5.https://support.citrix.com/article/CTX224419XenServer unexpectedly reboots due to multipath failovers on a Storage Repository.Download and apply patch "XS71ECU1007".Citrix XenServer 7.0, 7.1, 7.1 CU1.https://supp

Citrix 14. Citrix Receiver launch.ica file does not open – nothing happens

Citrix Receiver launch.ica file does not open – nothing happens

Just reporting on a small issue I had with a machine and brand new Citrix Receiver 4.4 install. Clicking on a published application or desktop resulted in nothing happening. Seemed like the launch.ica file just wasn’t calling Citrix Receiver properly to start the connecting process.

IE wasn’t asking me to save or open launch.ica, this was differrent.

After a Citrix Receiver cleanup and reinstall the same problem remained. I checked the .ica file extension to see what program was associated with it but everything looked OK.

Clutching at straws I decided to manually associate .ica files with the Citrix Connection Manager.

This file by default is located in C:\Program Files (x86)\Citrix\ICA Client\wfcrun32.exe.

Once set, ICA files mysteriously started to launch. Something to look out for!

To set the association, you can choose default applications by file type within Windows or download the ICA file by following the steps located at

Citrix 13. Citrix Receiver for Windows Logging

Citrix Receiver for Windows Logging

 

List of the different types of data you can log with Receiver for Windows.

 

General Receiver Logging

Add the following values to HKLM\Software\Citrix\ (32-bit) or HKLM\Software\WOW6432Node\Citrix\ (64-bit):

DWORD ReceiverVerboseTracingEnabled = 1

When you stop and start Receiver a Receiver_.log file will be created in %USERPROFILE%\AppData\Local\Citrix\Receiver\.

Single Sign-On Logging

You can enable SSON logging which may help in identifying an issue with single sign-on.

Add the following values to HKLM\Software\Citrix\Install\SSON (32-bit) or HKLM\Software\WOW6432Node\Citrix\Install\SSON (64-bit):

REG_SZ DebugEnabled = true

REG_SZ LogPath = Path location

When you log off and on again log files will be created relating to SSON in the path you specified.

The trace-pnsson.log file shows information such as the credentials captured and packaged by SSON.

Authentication Manager Logging

Add the following values to HKLM\Software\Citrix\AuthManager (32-bit) or HKLM\Software\WOW6432Node\Citrix\AuthManager (64-bit):

REG_SZ LoggingMode = Verbose

REG_SZ TracingEnabled = True

REG_SZ SDKTracingEnabled = True

When you stop and start Receiver log files will be created within the sub-folders that are present under %USERPROFILE%\AppData\Local\Citrix\AuthManager\.

Self-Service Plugin Logging

Add the following values to HKLM\Software\Citrix\Dazzle\ (32-bit) or HKLM\Software\WOW6432Node\Citrix\Dazzle\ (64-bit):

REG_SZ Tracing = True

REG_SZ AuxTracing = True

REG_SZ DefaultTracingConfiguration = global all -detail

When you stop and start Receiver log files will be created within the sub-folders that are present under %USERPROFILE%\AppData\Local\Citrix\SelfService\.

Citrix 12. Reduce Citrix logon times by up to 75%

Reduce Citrix logon times by up to 75%

 

Citrix Director is great at recording logon times per session and logon averages over periods of time. We can even produce logon reports and show them off to managers or other teams within the organisation to show them how good (or bad) the virtual workspace performs! Though without any effort you’ll likely be wowing everyone for the wrong reasons until you put in the background work to get logon times down to a low number. Citrix unfortunately doesn’t magically make logons quicker than any other desktop.

Many of the logon friendly optimisations and best practices out there today are straight forward and common sense and help to get you started:

Keep GPOs at a minimum (don’t be GPO happy).

Keep close control of Group Policy incuding monitoring – http://www.jgspiers.com/audit-group-policy-changes/

Don’t map tonnes of drives, especially to users who do not need them.

Don’t map tonnes of printers. Joe who prints to two printers doesn’t need 13 printers mapped to his machine.

Avoid using logon scripts, these are only going to add time to the logon.

Move Group Policy settings to Citrix WEM.

There are more, and I’ll cover off some additional ones in this post to really reduce logon times. If you’re interested in some more tips, see http://www.jgspiers.com/citrix-tips-tricks-tweaks-suggestions/

Also, if you’re interested in finding out more about the logon process see http://www.jgspiers.com/digging-in-to-citrix-logon-process/

So you’ve performed all of the above and more, you’re timings tells you that logon times are no longer than 20 seconds. You look at Director and the logon times are double. Why? There is still one recorded metric that:

Not everyone knows what exactly it is and or struggles to understand it.

Takes a bit of work getting the metric value to reduce, although once you know what it does it’s easier to shave the seconds off.

That metric is: Interactive Session Time.

What is Interactive Session Time?

From https://www.citrix.com/blogs/2016/08/19/interactive-session-of-logon-duration-in-citrix-director-explained/

It is the time taken to handoff keyboard and mouse control to the user after the profile of the user is loaded for a session.

Event ID 2 is initially logged on the VDA shortly after the desktop/application icon is clicked within Receiver client or Receiver for Web. This event triggers the Interactive Session timer which ends once Event 1000 is logged to indicate that the session is ready for use. Event ID 1000 is logged by the Citrix Profile Management service.

So whilst Director records logon times, it is important to understand that this is the time taken from clicking to launch a resource until the machine is actually usable even though the actual logon may have completed some time before that. This produces innacurate results in Director for true logon times so let me show you how you can almost eliminate Interactive Session times, get overall logon times reduced and get Director logging much more accurate data.

My testing environment

For the following logon tests, I used XenDesktop 7.13, running PVS 7.13 and a 7.13 VDA configured with 2GB RAM and 2vCPU. The VDA runs Windows Server 2016 with no optimisations to start however as you will see later it does become optimised and improves logon times. The Target Device Write-Cache is configured as RAM w/overflow to HDD (which is on SSD storage).

Note: The following configurations can be applied to both Server and Desktop OS in persistent and non-persistent environments. You don’t have to implement all of them, consider each one individually. Logon times will also fluctuate based on factors such as load (busy periods), VDA performance and underlying hardware used.

Non-optimised image vs optimised image – logon time results

I built a brand new Windows Server 2016 VDA streaming from PVS. Nothing else was performed on the image. Logged on three times. The average logon time is 68 seconds. That time has advantages such as being able to go and make coffee or produce a logon time report from Director to show your boss, which probably won’t go down all that well. Add applications, larger profiles, Group Policies in to the mix and more seconds get added.

So I’ve gone and optimised my image using the Server 2016 optimisation script. You know optimising an image brings a great sense of satisfaction, not to mention the average time is now down to 38 seconds! That is 20 seconds shaven off just by optimising the image.  Notice also that the Interactive Session time has been greatly reduced, that is because the image is a lot more leaner and can get a session ready more quickly.

Serialize/StartupDelayInMSec – logon time results

It was blogged about here https://xenappblog.com/2016/optimize-logon-times/. Windows Server 2012 and Windows 8 introduced a startup delay for applications which has a negative effect on Interactive Session times. By disabling this delay we can start the applications immediately, not an issue if you have only a few. On the write-mode PVS/MCS gold image or Citrix App Layering OS Layer, launch RegEdit -> HKEY_USERS -> File -> Load Hive.

Load the default user hive by navigating to C:\Users\Default and double-clicking NTUSER.DAT.

Give the hive a name and click OK.

Within the hive navigate to SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer and create a new key called Serialize.

Create a new DWORD 32-bit value within the Serialize key.

Give the value a name of StartupDelayInMSec and a data value of 0x0.

Click File -> Unload Hive.

Click Yes. Finalise the image.

Note: You could have done this through Group Policy, but since it applies to all users we want to reduce the need for Group Policy processing and extra logon processing.

The results of this optimisation show logon time averages down to 27 seconds. An 11 second drop. Remember that each logon here is on a non-persistent machine. The machine is restarted between each logon so as to mimic a first-time session logon (post restart) to VDA where no profile is cached. These current logon times look a lot better and are good for a first-time logon after VDA restart.

Autologon account/the second logon is quicker – logon time results

When a VDA restarts as part of scheduled reboots for example or when non-persistent desktops reboot to reset, the first logon is generally always the longest. So I thought of the idea to use an auto-logon account to be the guinea pig and be the one who first logs on when a VDA restarts. This works well particularly in server OS since the autologon account when it logs off doesn’t trigger any sort of restart to the VDA.

You can use Autologon as pointed out by Chris in the comments. This tool easily configures an autologon account and encrypts the password. https://technet.microsoft.com/en-us/sysinternals/autologon.aspx

When launching autologon, enter the credentials to your autologon account and click Enable.

Click OK. The Winlogon Registry Key will be configured with DefaultUsername/DefaultDomainName string values and so on however the DefaultPassword will not be present and is encrypted.

Restart the VDA. If autologon does not work the first time run through the Autologon tool again and it should work on second attempt.

You can now configure the logoff procedure as described below.

For an alternative method of autologon (this section includes the autologoff procedure):

Open the gold PVS/MCS image again or the OS Layer (Citrix App Layering). On the C:\ drive, create a batch file and call it something like AutoLogon.bat.Within the batch file, enter the following:

 

1

2

call reg delete"HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v"DefaultPassword" /f

call logoff

Now open RegEdit and navigate to HKLM\SOFTWARE\Microsoft\Windows NT\CurrrentVersion\Winlogon.

Set the AutoAdminLogon value to 1.

Set DefaultDomainName to your domain name as below.

Set DefaultUserName to a user account (service account) which has rights to log on to each VDA. This user account should be secured with a strong password and be a Domain User only. If this DefaultUserName REG_SZ string does not exist, create it.

Set DefaultPassword to the password of the autologon account. Click OK.

Right-click the Winlogon key and select Permissions.

Click Add. Search and add the autologon account.

Give Full Control permissions. This allows the autologon account to delete the DefaultPassword string after each logon. Finalise the image.

Now using Group Policy, create a GPO which is filtered to the autologon account as below. Edit the Group Policy obejct.

Expand User Configuration -> Preferences -> Control Panel Settings -> Scheduled Tasks -> New -> Scheduled Task (At least Windows 7).

Under General specify a name. Specify Run only when user is logged on and run the task under the autologon account. For Configure for choose Windows 7 or the highest possible OS.

On the Triggers tab click New.

For Begin the task choose At log on. Check Specific user or group and select the autologon account. Click OK.

On the Actions tab click New.

For Action choose Start a program. Under Program/script enter the path of your batch file which resides on the gold image. Click OK.

Your scheduled task is now created.

Now when the VDA boots up, an autologon occurs. The Scheduled Task runs a batch file which deletes the DefaultPassword string immediately for security and then logs off. The machine is then ready for real user logons.

As a result, the average logon time has dropped to 20 seconds. A 7 second drop. Interactive Session times are a lot lower than when we started these optimisations, over a 40 reduction!

UPMEvent – logon time results – Saving the best to last

Note: This really only applies now to VDA 7.13 and below. Citrix made a change in 7.14.1 that allows upmEvent.exe to run quickly out of the box. To read more visit http://www.jgspiers.com/reduce-citrix-director-interactive-session-time/

For VDA 7.13 and below:

If you had implemented what I am about to show you first, you probably could have cut Interactive Session time by more than 60% immediately.

The Interactive Session time is calculated once Event ID 1000 is logged on the VDA. The faster UPMEvent.exe runs the quicker Event 1000 is logged and the calculation is complete.

So ideally we want the UPMEvent.exe to run once we see that desktop wallpaper screen as that is when the logon is complete. By default, it instead runs some time after the profile has loaded.

The StartupDelayInMSec key added earlier simply speeds up when run keys (startup applications) are started. Hence why the Interactive Session time is decreased because UPMEvent.exe is started quicker since we removed the startup delay.

So what is faster than startup applications specified within run? Removing the upmEvent.exe run key and moving it to the Userinit string as described in http://www.jgspiers.com/reduce-citrix-director-interactive-session-time/

What else is faster than startup applications specified within run? A log on Scheduled Task.

Open your gold image or Citrix App Layering Platform Layer (the Platform Layer should contain your VDA). Launch RegEdit and navigate to HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run. Delete the Citrix UPM UserMsg string. Finalise the image.

Now using Group Policy, create a new GPO which applies to all users logging on to the VDA.

Within the GPO expand User Configuration -> Preferences -> Control Panel Settings -> Scheduled Tasks -> New -> Scheduled Task (At least Windows 7).

On the General tab specify a name. Keep the task running under %LogonDomain%\%LogonUser%. Set Configure for to Windows 7 or the highest available OS.

On the Triggers tab click New.

For Begin the task choose At log on and for Any user. Click OK.

On the Actions tab click New.

Under Action select Start a program. Under Program/script enter “C:\Program Files\Citrix\Virtual Desktop Agent\upmEvent.exe” and beside Add arguments (optional) enter wait. Click OK.

Click OK to finish creating the Scheduled Task. Now UPMEvent.exe will be run by the Scheduled Task immediately when the desktop shell has loaded.

With UPMEvent.exe being ran now by the Scheduled Task the average logon time has dropped to 13seconds. A further 7 second drop. Notice the Interactive Session times are all at 3 seconds, more than 50 seconds lower than when we first started. Director is logging true logon times and our future reports will be much more accurate.

Note: In VDA versions before 7.7, upmEvent was called upmUserMsg.