pub enum KernelPattern {
Show 17 variants
MatMul {
inputs: [TensorBinding; 2],
output: TensorBinding,
shape: MatMulShape,
},
ElementWise {
op: ElementWiseOp,
inputs: [TensorBinding; 2],
output: TensorBinding,
dim_name: String,
},
ElementWiseChain {
base: TensorBinding,
cast: Option<i32>,
steps: Vec<ChainStep>,
output: TensorBinding,
dim_name: String,
},
Conv2D {
input: TensorBinding,
weight: TensorBinding,
output: TensorBinding,
bias: Option<TensorBinding>,
shape: Conv2DShape,
activation: Option<ActivationOp>,
},
Pool {
kind: PoolKind,
input: TensorBinding,
output: TensorBinding,
shape: PoolShape,
},
Activation {
op: ActivationOp,
input: TensorBinding,
output: TensorBinding,
dim_name: String,
},
Reduce {
op: ReduceOp,
input: TensorBinding,
output: TensorBinding,
axis: i64,
},
Transpose {
input: TensorBinding,
output: TensorBinding,
perm: Vec<i64>,
},
Reshape {
input: TensorBinding,
output: TensorBinding,
},
Normalization {
input: TensorBinding,
scale: TensorBinding,
bias: TensorBinding,
output: TensorBinding,
epsilon: f32,
norm_type: NormType,
},
Concat {
inputs: Vec<TensorBinding>,
output: TensorBinding,
axis: i64,
},
Split {
input: TensorBinding,
outputs: Vec<TensorBinding>,
axis: i64,
},
Attention {
query: TensorBinding,
key: TensorBinding,
value: TensorBinding,
output: TensorBinding,
d_k: String,
seq_len: String,
num_heads: u32,
num_kv_heads: u32,
causal: bool,
},
Gather {
data: TensorBinding,
indices: TensorBinding,
output: TensorBinding,
axis: i64,
},
Scatter {
data: TensorBinding,
indices: TensorBinding,
updates: TensorBinding,
output: TensorBinding,
axis: i64,
},
QuantizedMatMul {
input: TensorBinding,
weight: TensorBinding,
scale: TensorBinding,
bias: Option<TensorBinding>,
output: TensorBinding,
shape: MatMulShape,
},
Unknown {
reason: String,
},
}Expand description
A classified kernel pattern that maps to ONNX operators.
Variants§
MatMul
Loop + accumulation + 2 read arrays + 1 write array → ONNX MatMul.
ElementWise
No loop + binary op on arrays → ONNX Add/Sub/Mul/Div.
ElementWiseChain
A per-element expression one level deeper than a binary op: one tensor, an optional conversion, and a short chain of element-wise steps whose operands are whole tensors or dispatch-time scalars.
Two shapes in the vendored corpus need this and nothing narrower:
axpy,output[i] = y[i] + a * x[i]withaa uniform scalar —x, scaled, then added toy.dequantize,output[i] = f32(input[i]) * s1 * s2with two uniform scalars — a conversion, then two scales.
Neither is one operation over two tensors, and Self::ElementWise
holds exactly two operands and one op, so both were refused. Neither is
a fused activation either: nothing here is a unary function of one
element, so widening Activation would not have held them.
The chain is deliberately linear. Every node has exactly one operand
that is not a leaf, which is what makes it a sequence of graph nodes
rather than a tree; snake and alibi, whose multiplies have two
non-leaf operands, do not match and stay refused.
Fields
base: TensorBindingThe tensor the chain starts from.
cast: Option<i32>The element type base is converted to before the first step, when
the kernel converts it. dequantize reads array<i32> and writes
array<f32>, and dropping the conversion would reinterpret bits.
output: TensorBindingConv2D
2D convolution: nested loops + kernel window + accumulation.
Fields
input: TensorBindingweight: TensorBindingoutput: TensorBindingbias: Option<TensorBinding>Per-output-channel bias, when the kernel has one.
Optional because a convolution need not have a bias, not because dropping it is acceptable: a backend that cannot emit one must refuse rather than silently compute a convolution without it.
shape: Conv2DShapeactivation: Option<ActivationOp>The activation the kernel applies to what it stores.
output[i] = max(sum, 0.0) is a convolution and a ReLU, and this
was silently dropped: the emitted model held a CONV_2D and nothing
else, so it loaded, was accelerated, and returned unclipped values.
The TFLite backend folds it into Conv2DOptions; a backend that
cannot express it must say so rather than emit the convolution
alone.
Pool
Pooling: nested loops + reduction over spatial window.
Activation
Activation function: no loop, single input, unary math op.
Reduce
Reduction over an axis: loop + accumulation, single input.
Transpose
Transpose: permute tensor axes.
Reshape
Reshape: change tensor shape without data copy.
Normalization
Normalization: mean + variance + scale + bias.
Concat
Concatenation of multiple inputs along an axis.
Split
Split a single input into multiple outputs along an axis.
Attention
Scaled dot-product attention.
Gather
Gather: index into data tensor using indices.
Scatter
Scatter: write updates into output tensor at given indices.
QuantizedMatMul
A matrix multiplication whose right operand arrives as integer codes
with one scale per output channel: output = input @ (weight * scale)ᵀ.
This is not Self::MatMul with an odd element type. A dense matmul
has two operands; this has three, and the third is not a second matrix.
scale carries one factor per row of the weight, so it multiplies the
contraction’s result rather than participating in the contraction.
Lowering it as a MatMul over weight and dropping scale would produce
a graph whose every output is wrong by a per-channel factor, which is
the failure a refusal is preferable to.
The weight’s rows are the output channels, so the contraction runs along the second axis of both operands and the lowerings transpose it. That is the layout every quantized kernel in the vendored corpus uses, and the one per-channel quantization is defined against: a scale per row is a scale per output feature only if a row is an output feature.
weight is reported with an element type of INT8 although the buffer
the kernel binds is array<u32>. Four two’s-complement codes are packed
per word, least-significant byte first, which is the byte layout of a
contiguous i8 row. The packing is how the GPU kernel buys four columns
per memory transaction; it is not part of what the kernel computes, and
the graph names the codes rather than the words they arrived in.
Fields
input: TensorBindingThe f32 activations, [m, k].
weight: TensorBindingThe integer codes, [n, k] — one row per output channel.
scale: TensorBindingf32, [n] — one scale per weight row, applied after the contraction.
bias: Option<TensorBinding>A per-output-channel addend the kernel fuses onto the result.
matvec/q8_residual adds a residual after the row scale. Optional
because a quantized matmul need not have one, not because dropping
it is acceptable — the same rule Self::Conv2D’s bias carries.
output: TensorBindingshape: MatMulShapeUnknown
Unrecognized pattern — classification could not determine a known op.
Trait Implementations§
Source§impl Clone for KernelPattern
impl Clone for KernelPattern
Source§fn clone(&self) -> KernelPattern
fn clone(&self) -> KernelPattern
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more