Backend

Backend Manager

The Backend Manager is a singleton class that manages backend selection and initialization. It is designed to allow only one backend to be active at a time. Users can select backends at runtime, during the initialization phase. By default, the library will try to initialize the OpenCL backend. If initialization fails, it will attempt to initialize the CUDA backend. If both fail, the library will throw an exception.

Warning

Switching between backends during runtime is not supported and may lead to undefined behavior.

class BackendManager

BackendManager class This class is a singleton that holds the current backend and provides a way to change it.

Public Functions

void setBackend(const std::string &backend = "opencl")

Set the backend.

Parameters:

backend – The backend to be set, default is “opencl”

const Backend &getBackend() const

Get the backend.

Returns:

const Backend&

~BackendManager() = default

Destroy the BackendManager object.

BackendManager(const BackendManager&) = delete

Copy constructor (delete)

BackendManager &=delete operator= (const BackendManager &)

Copy assignment operator (delete)

Public Static Functions

static BackendManager &getInstance()

Get the singleton instance of the BackendManager.

static std::vector<std::string> getBackendsList()

Get the list of available backends.

static bool cudaEnabled()

Check if CUDA is enabled.

Returns:

bool True if CUDA is enabled, False otherwise

static bool openCLEnabled()

Check if OpenCL is enabled.

Returns:

bool True if OpenCL is enabled, False otherwise

Friends

inline friend std::ostream &operator<<(std::ostream &out, const BackendManager &backend_manager)

Operator << to print the backend manager.

Backend Class

The Backend class is an abstract class defining the interface for the different hardware backends supported by CLIc. Inherited classes implement the necessary low-level functions to operate the hardware. This abstraction enforces a design pattern allowing switching between different hardware backends without changing the high-level code. Currently, the library supports OpenCL and CUDA backends, with the possibility of adding more in the future.

Note

This class operates closest to the hardware, and casual developers should not need to interact with it.

Warning

The CUDABackend class is operational but not yet fully released.

class Backend

Backend class.

This class holds low-level device operations. It is used to allocate memory, copy data, execute kernels, etc. It is standardized to operate with different hardware API (CUDA, OpenCL, etc).

Subclassed by cle::CUDABackend, cle::OpenCLBackend

Public Functions

Backend() = default

Construct a new Backend object.

Backend(const Backend&) = default

Construct by copy a new Backend object.

Backend(Backend&&) = default

Construct by move a new Backend object.

virtual ~Backend() = default

Destroy the Backend object.

Backend &=default operator= (const Backend &)

Copy assignment operator.

Backend &=default operator= (Backend &&)

Move assignment operator.

virtual Backend::Type=0 getType () const

Get the type of the backend.

Returns:

Backend::Type

virtual std::vector< std::string >=0 getDevicesList (const std::string &type) const

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual std::vector< Device::Pointer >=0 getDevices (const std::string &type) const

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual Device::Pointer=0 getDevice (const std::string &name, const std::string &type) const

Get a device by name and type.

Parameters:

• name –

• type –

Returns:

Device::Pointer

virtual Device::Pointer=0 getDeviceFromIndex (size_t index, const std::string &type) const

Get a device by index and type.

Parameters:

• index –

• type –

Returns:

Device::Pointer

virtual std::string=0 getPreamble () const

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string

virtual void=0 allocateMemory (const Device::Pointer &device, const std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void **data_ptr) const

Allocate a memory space in the device.

Parameters:

• device –

• size –

• data_ptr –

virtual void=0 freeMemory (const Device::Pointer &device, const mType &mtype, void **data_ptr) const

Free a memory space in the device.

Parameters:

• device –

• mtype –

• data_ptr –

virtual void=0 writeMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const void *host_ptr) const

Write data from host to device buffer.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void=0 readMemory (const Device::Pointer &device, const void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void *host_ptr) const

Read data from device buffer to host.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void=0 copyMemoryBufferToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void=0 copyMemoryImageToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void=0 copyMemoryBufferToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void=0 copyMemoryImageToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void=0 setMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const float &value) const

Set a memory space to a specific value.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• dtype –

• mtype –

• value –

virtual void=0 buildKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, void *kernel) const

Build a kernel from source code.

Parameters:

• device –

• kernel_source –

• kernel_name –

• kernel –

virtual void=0 executeKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array< size_t, 3 > &global_size, const std::vector< void * > &args, const std::vector< size_t > &sizes) const

Execute a kernel.

Parameters:

• device –

• kernel_source –

• kernel_name –

• global_size –

• args –

• sizes –

Friends

inline friend std::ostream &operator<<(std::ostream &out, const Backend::Type &backend_type)

Operator << to print the backend type.

inline friend std::ostream &operator<<(std::ostream &out, const Backend &backend)

Operator << to print the backend.

OpenCL Backend

class OpenCLBackend : public cle::Backend

OpenCL backend class This class holds low-level device operations for OpenCL devices.

Public Functions

virtual std::vector< Device::Pointer > override getDevices (const std::string &type) const

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual Device::Pointer override getDevice (const std::string &name, const std::string &type) const

Get a device by name and type.

Parameters:

• name –

• type –

Returns:

Device::Pointer

virtual Device::Pointer override getDeviceFromIndex (size_t index, const std::string &type) const

Get a device by index and type.

Parameters:

• index –

• type –

Returns:

Device::Pointer

virtual std::vector< std::string > override getDevicesList (const std::string &type) const

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual Backend::Type override getType () const

Get the type of the backend.

Returns:

Backend::Type

virtual void override allocateMemory (const Device::Pointer &device, const std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void **data_ptr) const

Allocate a memory space in the device.

Parameters:

• device –

• size –

• data_ptr –

virtual void override freeMemory (const Device::Pointer &device, const mType &mtype, void **data_ptr) const

Free a memory space in the device.

Parameters:

• device –

• mtype –

• data_ptr –

virtual void override writeMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const void *host_ptr) const

Write data from host to device buffer.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void override readMemory (const Device::Pointer &device, const void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void *host_ptr) const

Read data from device buffer to host.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void override copyMemoryBufferToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryImageToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryBufferToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryImageToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override setMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const float &value) const

Set a memory space to a specific value.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• dtype –

• mtype –

• value –

virtual void override buildKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, void *kernel) const

Build a kernel from source code.

Parameters:

• device –

• kernel_source –

• kernel_name –

• kernel –

virtual void override executeKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array< size_t, 3 > &global_size, const std::vector< void * > &args, const std::vector< size_t > &sizes) const

Execute a kernel.

Parameters:

• device –

• kernel_source –

• kernel_name –

• global_size –

• args –

• sizes –

virtual std::string override getPreamble () const

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string

CUDA Backend

class CUDABackend : public cle::Backend

CUDA backend class This class holds low-level device operations for CUDA devices.

Public Functions

virtual std::vector< Device::Pointer > override getDevices (const std::string &type) const

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual Device::Pointer override getDevice (const std::string &name, const std::string &type) const

Get a device by name and type.

Parameters:

• name –

• type –

Returns:

Device::Pointer

virtual Device::Pointer override getDeviceFromIndex (size_t index, const std::string &type) const

Get a device by index and type.

Parameters:

• index –

• type –

Returns:

Device::Pointer

virtual std::vector< std::string > override getDevicesList (const std::string &type) const

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual Backend::Type override getType () const

Get the type of the backend.

Returns:

Backend::Type

virtual void override allocateMemory (const Device::Pointer &device, const std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void **data_ptr) const

Allocate a memory space in the device.

Parameters:

• device –

• size –

• data_ptr –

virtual void override freeMemory (const Device::Pointer &device, const mType &mtype, void **data_ptr) const

Free a memory space in the device.

Parameters:

• device –

• mtype –

• data_ptr –

virtual void override writeMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const void *host_ptr) const

Write data from host to device buffer.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void override readMemory (const Device::Pointer &device, const void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, void *host_ptr) const

Read data from device buffer to host.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• host_ptr –

virtual void override copyMemoryBufferToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryImageToBuffer (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device buffer.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryBufferToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device buffer to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override copyMemoryImageToImage (const Device::Pointer &device, const void **src_ptr, std::array< size_t, 3 > &src_origin, std::array< size_t, 3 > &src_shape, void **dst_ptr, std::array< size_t, 3 > &dst_origin, std::array< size_t, 3 > &dst_shape, std::array< size_t, 3 > &region, const size_t &bytes) const

Copy data from device image to device image.

Parameters:

• device –

• src_ptr –

• src_origin –

• src_shape –

• dst_ptr –

• dst_origin –

• dst_shape –

• region –

• bytes –

virtual void override setMemory (const Device::Pointer &device, void **buffer_ptr, std::array< size_t, 3 > &buffer_shape, std::array< size_t, 3 > &buffer_origin, std::array< size_t, 3 > &region, const dType &dtype, const mType &mtype, const float &value) const

Set a memory space to a specific value.

Parameters:

• device –

• buffer_ptr –

• buffer_shape –

• buffer_origin –

• region –

• dtype –

• mtype –

• value –

virtual void override buildKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, void *kernel) const

Build a kernel from source code.

Parameters:

• device –

• kernel_source –

• kernel_name –

• kernel –

virtual void override executeKernel (const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array< size_t, 3 > &global_size, const std::vector< void * > &args, const std::vector< size_t > &sizes) const

Execute a kernel.

Parameters:

• device –

• kernel_source –

• kernel_name –

• global_size –

• args –

• sizes –

virtual std::string override getPreamble () const

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string