liveBook · Manning

MLComputePlan

As explained in Quantization and Fast Inference

MLComputePlan is a Core ML API for inspecting the compute plan and operation-level execution details of a model.

MLComputePlan inspects a compiled Core ML model’s preferred compute device for each operation. Use it to check whether operations are assigned to the ANE, GPU, or CPU; compute_units=ALL permits those units but does not force their use. Operation counts show placement by count, not the share of computation, so they do not alone establish which device did most of the work. section 12.2

Listing 12.5 Reading op placement from the compiled model
import coremltools as ct
from collections import Counter
 
_DEVICE_TO_UNIT = {                                     #A
    "MLNeuralEngineComputeDevice": "ane",
    "MLGPUComputeDevice":          "gpu",
    "MLCPUComputeDevice":          "cpu",
}
 
mlmodel = ct.models.MLModel(model_path, compute_units=ct.ComputeUnit.ALL)
compiled_path = mlmodel.get_compiled_model_path()       #B
plan = ct.models.compute_plan.MLComputePlan.load_from_path(
    compiled_path, compute_units=ct.ComputeUnit.ALL,
)
 
main_fn = plan.model_structure.program.functions["main"]
counts = Counter()
for op in main_fn.block.operations:
    usage = plan.get_compute_device_usage_for_mlprogram_operation(op)   #C
    pref = getattr(usage, "preferred_compute_device", None) if usage else None
    counts[_DEVICE_TO_UNIT.get(type(pref).__name__, "cpu")] += 1        #D
 
total = sum(counts.values())
frac = {u: round(counts.get(u, 0) / total, 4) for u in ("ane", "gpu", "cpu")}
# {'ane': 0.19, 'gpu': 0.0, 'cpu': 0.81}

Inspect the compiled model and count operations by preferred compute device. section 12.2

How Quantization and Fast Inference covers it

How deep it goes
Reports preferred-device assignments and operation counts, but does not measure each device’s share of floating-point work.
Examples
Counts a model’s operations as 19% ANE, 81% CPU, and none GPU, while cautioning that counts can misrepresent arithmetic share.
What it assumes
Requires a compiled Core ML model and access to its operation-level program structure.
What you can do after
Evaluate preferred operation placement in a compiled Core ML model without treating operation counts as a measure of compute share.
How it connects
Links Core ML compute-unit permissions with preferred-device assignments across the ANE, GPU, and CPU.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.