Why Value-Aware Stratification Cuts Fault-Injection Budgets: Reading TreeFI Against Exhaustive Campaigns

The budget problem: why exhaustive and value-uniform fault injection cannot scale

Software-implemented fault injection is attractive because it is hardware-agnostic and applicable early in development, but exhaustive campaigns over all possible faults in modern DNNs are intractable [1]. Statistical fault injection reduces cost by injecting only a selected subset and using classical sampling theory to estimate the full fault population under a desired confidence level and error margin [1]. State-of-the-art methods like the data-aware one-shot method (SFI) and the later iterative method (IFI) organize sampling at the layer and bit level, but they do not explicitly organize the campaign by value range [1]. SFI was originally proposed for weight fault injection and is less favorable on activation faults, where the value distribution is more heterogeneous and input-dependent; IFI can spend many injections in early iterations on bits with little or no impact, and its iterative nature makes total runtime difficult to predict [1].

The cost problem is not unique to DNNs. In laser fault injection campaigns on secure microcontrollers, exhaustive search over spatial positions was infeasible, and researchers used Optical Beam Induced Current measurements to identify flip-flop locations, reducing the search space by factors of 6.6 to 375 [6]. That work established that profiling the device to find sensitive locations can dramatically cut campaign cost, but it targeted hardware security rather than DNN reliability and relied on physical measurements rather than statistical sampling theory [6]. TreeFI brings an analogous insight to DNN fault injection: profile the value distribution to find sensitive ranges, then allocate injections accordingly [1].

Value-dependent bit-flip effects: the structure existing SFI methods overlooked

The key empirical observation motivating TreeFI is that for a fixed bit position, the effect of a bit flip on model accuracy depends strongly on the original activation value [1]. Figure 2 in the paper shows this on a representative ResNet8 activation layer using 65,536 single-bit fault injections per bit position: for bit 30 (exponent MSB) and bit 27, some activation ranges show little or no accuracy drop when the bit is flipped, while other ranges show strong drops [1]. The same value-dependent behavior is observed for weight faults, though the figure focuses on activations [1]. This means two fault injections targeting the same layer and bit position can produce very different outcomes because the corrupted values belong to different value ranges [1].

This value dependence is consistent with broader evidence on bit-level fault sensitivity. In quantized neural networks for on-chip inference, the most significant bits produce the largest errors, and the variability of error magnitude across bits of the same significance can vary greatly—many dense-layer MSB weights have high error impact while many have low impact [5]. That work also notes that monotonicity of bit significance has been used previously to guide fault injection campaigns [5]. TreeFI extends this by learning data-driven intervals from the value distribution itself rather than assuming a fixed bit-significance ordering, and by using those intervals to stratify sampling [1].

From regression-tree intervals to stratified allocation: how TreeFI works

TreeFI operates in two phases. First, an offline characterization collects the value distribution of each target layer using weighted histograms, then partitions the value axis into intervals with similar local bit-flip effects using one-dimensional regression trees trained per layer–bit pair [1]. The trees place split points where the local effect of flipping the selected bit changes significantly, using histogram counts so that learned splits are supported by enough observed values; in experiments, trees contain at most about 15 leaves per layer–bit pair [1]. Each interval receives a risk score derived from the average log-distance between original and bit-flipped values, mapped through a sigmoid function [1]. Second, the fault injection campaign allocates injections across intervals proportionally to interval frequency and to the uncertainty of the risk estimate, following standard stratified sampling principles [1]. The final failure-rate estimate combines interval-level measured failure rates weighted by how often each interval appears during fault-free execution [1].

This design differs from prior statistical FI methods in a specific way: TreeFI uses the value axis itself to guide sampling, whereas existing approaches allocate injections across layers or bit positions without explicitly organizing by value range [1]. The weighted combination of interval results is also a key difference—without it, all intervals would contribute equally even though some occur much more often than others [1]. The method preserves the advantages of one-shot, upfront-budgeted campaigns while targeting a desired confidence level and error margin [1].

Validation on ResNet8 and beyond: accuracy against exhaustive injection and budget reductions

On ResNet8, where exhaustive activation fault injection is feasible, TreeFI provides more accurate estimates than state-of-the-art statistical FI baselines under the same campaign setting [1]. This is the strongest validation in the paper because it compares against a ground-truth exhaustive reference rather than only against other sampling methods [1]. Across the evaluated CNN and Transformer models using CIFAR-10 and ImageNet, TreeFI reduces the required injection budget by up to 72.1x, with average reductions of 44.9x for activation faults and 11.2x for the executed weight campaigns [1]. The results also show that TreeFI preserves the main per-layer and per-bit failure-rate trends in larger activation-fault experiments, making it useful even when exhaustive validation is impractical [1].

The weight-fault validation is narrower: it covers CNN models and a 6-layer DeiT-Tiny subset [1]. The paper does not claim that the same budget reductions generalize to all architectures or all weight-fault scenarios [1]. The comparison with prior statistical methods is explicit: SFI and IFI are the state-of-the-art baselines, and TreeFI outperforms them on ResNet8 activation faults under the same campaign setting [1]. The paper does not report head-to-head comparisons on all models for all fault types, so the scope of superiority claims is limited to the tested configurations [1].

Where the evidence stops: fault models, precision formats, and unvalidated extensions

The experimental study focuses on FP32 single-bit faults in activations and weights [1]. The paper explicitly states that the method itself is more general—it only requires a fault model defining how a bit flip modifies a value and a value distribution that can be collected and partitioned—and that the same interval-learning and stratified-allocation principle could extend to FP16, bfloat16, or quantized integer representations [1]. However, validating this experimentally is left as future work [1]. The conclusions therefore do not cover other precision formats, other fault models such as multi-bit or stuck-at faults, or hardware architectures beyond the software fault injection setting [1].

The broader fault-injection literature includes approaches that are complementary rather than competing. For example, hybrid deep learning methods for fault detection and classification in hardware-in-the-loop testing achieve high accuracy (98.8% average classification accuracy) but address fault diagnosis rather than reliability estimation under injected faults [2]. Targeted bit-flip adversarial weight attacks show that flipping as few as 27 out of 88 million weight bits in ResNet-18 can misclassify all images from one class to another with 100% attack success rate, demonstrating extreme value and location dependence in weight faults [3]. These works reinforce that fault effects are highly structured, but they do not provide statistical estimation frameworks comparable to TreeFI [2][3]. Competing approaches to fault-injection acceleration, such as heterogeneous spatio-temporal graph learning for digital integrated circuits, target gate-level injection rather than DNN value distributions [4]. The open question is whether TreeFI's interval-learning principle transfers to these adjacent settings and whether the budget reductions hold when the fault model or numerical format changes [1].

About These Sources

This research page is built on 6 studies (5 peer-reviewed, 1 preprint) — published from 2015 to 2026, 3 from 2024 or later — selected as the most relevant from 7 studies that passed quality screening, drawn from 24 papers retrieved from a database of over 500 million.

Sources used in this answer