July 26, 2019

OpenCL ICD and Multiple vendors

Some of this applies to VULKAN too Separate post coming soon

I’ve been battling with OpenCL implementations for a while now. Has this ever happened to you:

No Opencl Platforms or devices visible at runtime?
Reinstalling drivers doesn’t fix
Having multiple Vendors installed causing issues?
Not clear how or what is even installed?

So I had enough of this and wanted to dive deeper. Also I need to get CL working for my PhD work. This is going through Windows specific stuff, some logic will apply to Linux issues.

ICD what now?

When you install a graphics driver or similar, this will install that vendors OpenCl implementation into your system. What happens when you have multiple Drivers installed? Things get Hairy.

Khronos came up with a solution to having multiple different versions and vendors of OpenCL on one system, they published a spec and code for a generic installable Client Driver.

Spec: khronos.org/registry/OpenCL/specs/2.2/html/OpenCL_ICD_Installation.html

Code: github.com/KhronosGroup/OpenCL-ICD-Loader

This should be the first call from your CL app. It has code within it so search around the system and report all the different implementations available to your app.

For this to work:

The ICD needs to be actually installed
The different CL implementations need to be installed in a manor in which the ICD spec dictates so it can find it.

If 1 and 2 aren’t true, which is pretty common, you won’t see any or all of your devices at runtime. Here’s how to diagnose this:

How to get OpenCL

You can get OpenCl dependencies from multiple places.

Windows Update may install a default Khronos OpenCL.dll in system32
Installing graphics drivers should bring along a vendor specific OpenCL.dll and should also being along the Kronos ICD and put it in system32.
- but some are nasty and put their own specifc dll in system32
Installing the NVidia Cuda SDK will get you the build dependencies (.h/.lib)
~~Installing the AMD APP SDK will get you the build dependencies (.h/.lib)~~
- APP SDK seesm to have fallen off the internet, I’m using this instead: github.com/GPUOpen-Tools/common-lib-amd-APPSDK-3.0

To build an opencl program you need the headders:

(for C) cl/cl.h, or (C++) cl/cl.hpp or cl/cl2.hpp
These should be identical from all sources, they should have come from Kronos after being generated directly from the spec

You also need the .Lib to link against

These DO differ between vendors, however this shouldn’t matter
Linking against AMD’s opencl .lib, and running your app on a system with only NVidias opencl runtime installed should work. (but might not)
Almost always called OpenCL.lib

And then the runtime .dll

When you Launch an application, it needs to find the openCll.dll. This is where things get hairy.

CMAKE

If you use cmake to locate OpenCL for building your app: You can use the FindCL module , although I find it always ends at the CUDA install, which atm is rather outdated and only has headders for CL 1.2. You may want to find these things manually.

Which DLL

So if you are having trouble, here’s what you can do.

Step one, determine what .dll is being loaded.

Visual studio Output window can show you which .dlls are being loaded:

You should see:

'MAIN.exe' (Win32): Loaded 'C:\Windows\System32\OpenCL.dll'.

Nice, let’s go check that dll out

Ok. If you don’t see this and instead see, AMD/Nvidia/ something else. You may want to go compile the ICD yourself and replace this. Skip to ‘Roll your own ICD’ section.

Back to Visual studio, if everything is working. In my system I have both an Nvidia and AMD card, and I should see the following at some point:

'MAIN.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\u0342855.inf_amd64_b40622b25c29110f\B342717\amdocl12cl64.dll'. 
'MAIN.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_b49751b9038af669\nvopencl64.dll'.

If this is the case, fantastic. it means the ICD loaded first, then went and found and loaded the two other vendor’s CL implementation. My application reports the following:

2 OpenCL Platforms
OpenCL Platform: AMD Accelerated Parallel Processing by: Advanced Micro Devices, Inc. Version:OpenCL 2.1 AMD-APP (2841.5)
1 Platform devices
        Device: gfx804 - Advanced Micro Devices, Inc. - TYPE_GPU  - 8

OpenCL Platform: NVIDIA CUDA by: NVIDIA Corporation Version:OpenCL 1.2 CUDA 10.2.120
1 Platform devices
        Device: GeForce RTX 2080 Ti - NVIDIA Corporation - TYPE_GPU  - 68

Fixing this if it isn’t working

So Above shows an ideal working situation, if you aren’t in this place, here’s some of the things that could have gone wrong, and an idiot-check list

Do you actually have a graphics driver installed
Did this driver install an opencl runtime?
- Do a search for cl64.dll, cl.dll, opencl.dll, nvopencl, andocl. in C:/windows
- If you get nada, try Driver reinstall
Is there an Opencl.dll somewhere your application can find. I.e system32
- If not, you can make your own, see below, or try driver re-install.
Is this Opencl.dll the ICD, or a vendor specific .dll
- If it’s vendor specific, it probably will only show that Vendors devices.
- very common issue in my experience, in this case, roll your own ICD
ICD is there, vendor Dll also there, but can’t see any/all devices at runtime
- ICD can’t find the vendor dll. Try rolling your own and turn on debugging
Does everything look right but your application crashes after any CL call?
- I’ve been here before, sorry no advice, nuke it form orbit and start again.

Roll your own ICD

So the ICD is open source, so you can compile it yourself, thankfully it’s straightforward.

why would you want to do this?

A. you don’t have the ICD installed at all
B. So you can debug it

Here’s what saved my bacon. Get the code, build it, and modify loader/a/icd.h as

 // internal tracing macros
#if 0

for

// internal tracing macros
#if 1

Now grab that new .dll and plonk it in the same folder as your app (or system32 if you want).

Run it and have a look at the console:

[TrcEvent] CL Init
KHR ICD trace at \loader\windows\icd_windows_dxgk.c:116: D3DKMT_QUERYADAPTERINFO status != SUCCESS
KHR ICD trace at \loader\windows\icd_windows.c:47: Failed to load via DXGK interface on RS4, continuing
KHR ICD trace at \loader\windows\icd_windows_hkr.c:277: Device ID: PCI\VEN_1002&DEV_699F&SUBSYS_05111043&REV_C7\4&10ee519c&0&001A
KHR ICD trace at \loader\windows\icd_windows_hkr.c:282:     devinst: 1
KHR ICD trace at \loader\windows\icd_windows_hkr.c:295:     Trying to look for the key in the display adapter HKR...
KHR ICD trace at \loader\windows\icd_windows_hkr.c:118: RegQueryValueExA, hkey:0x23c, lpValueName:OpenCLDriverName
KHR ICD trace at \loader\windows\icd_windows_hkr.c:132: lpType: 7, lpData: C:\Windows\System32\DriverStore\FileRepository\u0342855.inf_amd64_b40622b25c29110f\B342717\amdocl64.dll
KHR ICD trace at \loader\windows\icd_windows_hkr.c:140:     Path: C:\Windows\System32\DriverStore\FileRepository\u0342855.inf_amd64_b40622b25c29110f\B342717\amdocl64.dll
KHR ICD trace at \loader\icd.c:50: attempting to add vendor C:\Windows\System32\DriverStore\FileRepository\u0342855.inf_amd64_b40622b25c29110f\B342717\amdocl64.dll...
KHR ICD trace at \loader\icd.c:176: successfully added vendor C:\Windows\System32\DriverStore\FileRepository\u0342855.inf_amd64_b40622b25c29110f\B342717\amdocl64.dll with suffix AMD
KHR ICD trace at \loader\windows\icd_windows_hkr.c:277: Device ID: PCI\VEN_10DE&DEV_1E04&SUBSYS_86751043&REV_A1\4&1da95f35&0&0019
KHR ICD trace at \loader\windows\icd_windows_hkr.c:282:     devinst: 2
KHR ICD trace at \loader\windows\icd_windows_hkr.c:295:     Trying to look for the key in the display adapter HKR...
KHR ICD trace at \loader\windows\icd_windows_hkr.c:118: RegQueryValueExA, hkey:0x23c, lpValueName:OpenCLDriverName
KHR ICD trace at \loader\windows\icd_windows_hkr.c:132: lpType: 1, lpData: C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_b49751b9038af669\nvopencl64.dll
KHR ICD trace at \loader\windows\icd_windows_hkr.c:140:     Path: C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_b49751b9038af669\nvopencl64.dll
KHR ICD trace at \loader\icd.c:50: attempting to add vendor C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_b49751b9038af669\nvopencl64.dll...
KHR ICD trace at \loader\icd.c:176: successfully added vendor C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_b49751b9038af669\nvopencl64.dll with suffix NV
KHR ICD trace at \loader\windows\icd_windows.c:54: Opening key HKLM\SOFTWARE\Khronos\OpenCL\Vendors...
KHR ICD trace at \loader\windows\icd_windows.c:77: Reading value 0...
KHR ICD trace at \loader\windows\icd_windows.c:90: Failed to read value 0, done reading key.

Beautiful, now you can get a good idea what is going on.

How is this useful

Well in a few cases now, I’ve not had AMD’s platform appear. Turns out this was because they weren’t following the ICD Spec and as such the ICD wouldn’t load it

See the issue here.

Using this output I determined this and fixed it myself, and reported the issue to AMD who have now fixed that issue.

I now keep this dll in system32 for all my CL app to use, so I can keep an eye on things and spot if a driver update has borked things.

Other solutions

Go read the ICD spec, specifically how it finds the vendor platforms, then follow the steps yourself to see if anything is amiss. 9/10 times it’s a registry value that’s been (suspiciously) wiped