Nonzero values remaining after the mask.
Pruning creates zeros, but zeros only become speed when storage formats and kernels can exploit their pattern. Compare unstructured, block, and channel pruning while tracking error and hardware usefulness.
Open sparse matrixMagnitude pruning removes small weights under the assumption that they contribute less. Unstructured masks preserve flexibility but scatter zeros. Structured masks remove blocks or channels that hardware can skip efficiently, often at a higher accuracy cost.
Nonzero values remaining after the mask.
Normalized energy removed after recovery.
Dense count divided by retained count, excluding indices.
Modeled fraction of sparsity that available kernels exploit.
Magnitude is the simplest proxy. Calibration-aware methods also consider activations, asking whether a weight participates strongly on representative inputs.
score(wᵢⱼ) = |wᵢⱼ| · activation_importanceⱼUnstructured pruning chooses individual values. N:M, block, and channel masks constrain removals into predictable layouts that reduce indexing and improve kernel efficiency.
Wpruned = W ⊙ M, M ∈ {0,1}Fine-tuning lets remaining weights compensate for removed connections. Recovery cannot restore capacity that an overly aggressive structured mask eliminated.
minimize L(W ⊙ M) while M stays fixedBest flexibility at a given sparsity, but index overhead and irregular memory access can erase wall-clock gains.
Zeros are not automatically speed.Entire tiles disappear, matching matrix kernels more naturally while retaining finer control than channels.
Align blocks to hardware.Removing complete rows or columns changes layer dimensions and yields direct dense-kernel savings.
Highest structure, highest capacity risk.