API reference

onnx2akida.convert(model, input_shape=None, input_dtype='uint8', samples=None, num_samples=1, device=None, enable_hwpr=False, sram_size=None, minimal_memory=False)[source]

Check ONNX model compatibility with Akida and convert the model into a HybridModel.

Parameters:

model (onnx.ModelProto) – The ONNX model.
input_shape (Iterable, optional) – An iterable specifying the new model input shape excluding batch dimension. Defaults to None.
input_dtype (np.dtype or str, optional) – expected model input format. If given as a string, should follow numpy string type requirements. Defaults to ‘uint8’.
samples (list of numpy arrays, optional) – List of input samples to use for calibration. If not provided, random samples will be generated. Defaults to None.
num_samples (int, optional) – Number of samples to use for calibration. Defaults to 1.
device (akida.Device, optional) – the Akida device to map the Akida sub models. Defaults to None.
enable_hwpr (bool, optional) – if True, the device is computed assuming partial reconfiguration. Used when device is None. Defaults to False.
sram_size (akida.NP.SramSize, optional) – Size of shared SRAM available inside the mesh. Ignored when minimal_memory is True. Used when device is None. Defaults to None.
minimal_memory (bool, optional) – if True, computes and sets the minimal required inputs and weights memory for the device. Used when device is None. Defaults to False.

Returns:

converted model and object containing information about model compatibility.

Return type:

HybridModel, ModelCompatibilityInfo

onnx2akida.print_report(hybrid_model, compatibility_info)[source]

Prints a report of the model compatibility with Akida.

Parameters:

hybrid_model (HybridModel) – The converted hybrid model.
compatibility_info (ModelCompatibilityInfo) – The compatibility information.

Compatibility

class onnx2akida.compatibility_info.ModelCompatibilityInfo(model)[source]

Tracks Akida compatibility of an ONNX model.

Parameters:: model (onnx.ModelProto) – The ONNX model to analyze.

Attributes:

`compatibility_percentage`	Returns the model compatibility percentage with the Akida accelerator.
`incompatibilities`	Returns a list of incompatibilities with reasons and other information.
`incompatible_nodes`	Returns a list of all incompatible nodes.
`incompatible_op_types`	Returns a list of unique op types of incompatible nodes.

Methods:

save_tagged_model(save_path)

Saves a model with node op_types tagged as 'AK<>' for compatible nodes and 'CPU<>' for incompatible nodes.

property compatibility_percentage

Returns the model compatibility percentage with the Akida accelerator.

Returns:: percentage of compatible nodes in the model.
Return type:: float

property incompatibilities

Returns a list of incompatibilities with reasons and other information.

Example

[
    {
        "node_sequence": [
            {"name": "cos_node", "op_type": "Cos"},
            ...
        ],
        "stage": "Quantization",
        "faulty_node": "cos_node",
        "reason": "Unsupported op."
    },
    ...
]

Returns:: list of incompatibilities with reasons and other information.
Return type:: list

property incompatible_nodes

Returns a list of all incompatible nodes.

Returns:: list of nodes from all incompatible sequences.
Return type:: list

property incompatible_op_types

Returns a list of unique op types of incompatible nodes.

Returns:: list of unique op types of incompatible nodes.
Return type:: list

save_tagged_model(save_path)[source]

Saves a model with node op_types tagged as ‘AK<>’ for compatible nodes and ‘CPU<>’ for incompatible nodes.

Parameters:: save_path (str) – File path to save the tagged model.

HybridModel

class onnx2akida.hybrid_model.HybridModel(model, name='HybridModel')[source]

Tensor-driven container for mixing ONNX and Akida execution.

HybridModel wraps a single ONNX graph and allows sections of the graph to be replaced by Akida models. Integration is performed by identifying an input tensor (the entry point of the Akida model) and an output tensor (where execution returns to the ONNX graph). Akida segments are tracked internally so they can be mapped to hardware devices.

Parameters:

model (onnx.ModelProto or ir.Model) – the base graph to augment.
name (str, optional) – name of the hybrid model. Defaults to “HybridModel”.

Attributes:

`akida_models`	Returns a list of Akida models within the hybrid model.
`input_shape`	Returns the hybrid model input shape.
`output_shape`	Returns the hybrid model output shape.

Methods:

`compute_data_movement`()	Computes the data movement between CPU and Akida for models in the HybridModel.
`generate_inference_model`([dirpath])	Generates a unified ONNX inference model from all sub-models in the HybridModel.
`map`(device[, mode])	Map (if possible) all akida models to a given device.
`summary`()	Prints a string summary of the hybrid model.

property akida_models

Returns a list of Akida models within the hybrid model.

Returns:: the akida models within the hybrid model.
Return type:: tuple

compute_data_movement()[source]

Computes the data movement between CPU and Akida for models in the HybridModel.

For each Akida model sequence, this method calculates: - The amount of data transferred from CPU to Akida. - The amount of data transferred from Akida to CPU.

The size is computed as the product of the input or output dimensions.

Returns:

a list of dictionaries, each containing :

”layer”: the Akida layer involved in the data transfer.
”type”: a string indicating the direction (“CPU -> Akida” or “Akida -> CPU”).
”size”: The size in bytes of the data movement.

Return type:

list of dict

generate_inference_model(dirpath='.')[source]

Generates a unified ONNX inference model from all sub-models in the HybridModel.

This method inject all Akida sub-models added to the HybridModel inside of the ONNX model suitable for inference. It handles the conversion of Akida models to ONNX and connects sub-models according to their inbounds.

>>> inference_model = model.generate_inference_model()
>>> sess = AkidaInferenceSession(inference_model.SerializeToString())
>>> outputs = sess.run(None, feeds)

Parameters:: dirpath (str, optional) – directory path where Akida programs will be saved. Defaults to the current directory.
Returns:: the combined ONNX model ready for inference.
Return type:: onnx.ModelProto

property input_shape

Returns the hybrid model input shape.

Returns:: the model input shape.
Return type:: list

map(device, mode=MapMode.AllNps)[source]

Map (if possible) all akida models to a given device.

Parameters:

device (akida.Device) – An Akida device.
mode (akida.MapMode, optional) – The mapping mode. Defaults to AllNps.

property output_shape

Returns the hybrid model output shape.

Returns:: the model output shape.
Return type:: list

summary()[source]

Prints a string summary of the hybrid model.

This method prints a summary with details for every layer: - Layer name and type - Backend (ONNX or Akida) - Output shape - Inbounds (list of inbound layer names) - Data movement (in bytes, only for layers involved in data transfer)

Inference

class onnx2akida.inference.AkidaInferenceSession(path_or_bytes, sess_options=None, providers=None, provider_options=None, **kwargs)[source]

A wrapper around onnxruntime.InferenceSession that ensures the Akida custom ops library is registered.

Parameters:

path_or_bytes (str or bytes) – path to the ONNX model file or the model in bytes.
sess_options (ort.SessionOptions, optional) – session options for the inference session. Defaults to None.
providers (list, optional) – list of execution providers. Defaults to None.
provider_options (list, optional) – list of provider options. Defaults to None.
**kwargs – additional keyword arguments for the InferenceSession.