While qualifying language models (tinystories, Mamba, SmolLM2) across Apple devices (multiple generations of iPhone, Apple Watch, Apple TV, and iPad), I found a test-oracle boundary that might be worth clarifying: WPT’s ULP helper considers maximum finite and same-sign infinity one ULP apart.
With testharness.js and webnn/resources/utils.js loaded, all four assertions currently pass:
assert_array_approx_equals_ulp([0x7bff], [Infinity], 1, 'float16');
assert_array_approx_equals_ulp([0x7c00], [65504], 1, 'float16');
assert_array_approx_equals_ulp([3.4028234663852886e38], [Infinity], 1, 'float32');
assert_array_approx_equals_ulp([Infinity], [3.4028234663852886e38], 1, 'float32');
Could we require matching finite/infinite/NaN classification before applying numerical tolerances, after converting the reference to the output dtype? If approximate overflow permits exceptions, those should be explicit in the operation’s error model, right?
Our RustNN strict comparator already tests this as an additional safeguard, and when I searched wider to see if other toolchains/runtimes had run into something similar, I found ONNX Runtime #31761. ONNX fixed a related finite-versus-infinite comparator problem. I'm not saying it always makes sense to pull in ONNX problems/solutions, but it seemed close to me in this case.
If the intended contract is agreed, I can contribute WPT helper regressions covering both signs, both directions and adjacent finite controls. Nearly all of which came up in my multi-model work in RustNN when pinning down accuacy/precision problems that compounded in recurrent graphs.
While qualifying language models (tinystories, Mamba, SmolLM2) across Apple devices (multiple generations of iPhone, Apple Watch, Apple TV, and iPad), I found a test-oracle boundary that might be worth clarifying: WPT’s ULP helper considers maximum finite and same-sign infinity one ULP apart.
With testharness.js and webnn/resources/utils.js loaded, all four assertions currently pass:
Could we require matching finite/infinite/NaN classification before applying numerical tolerances, after converting the reference to the output dtype? If approximate overflow permits exceptions, those should be explicit in the operation’s error model, right?
Our RustNN strict comparator already tests this as an additional safeguard, and when I searched wider to see if other toolchains/runtimes had run into something similar, I found ONNX Runtime #31761. ONNX fixed a related finite-versus-infinite comparator problem. I'm not saying it always makes sense to pull in ONNX problems/solutions, but it seemed close to me in this case.
If the intended contract is agreed, I can contribute WPT helper regressions covering both signs, both directions and adjacent finite controls. Nearly all of which came up in my multi-model work in RustNN when pinning down accuacy/precision problems that compounded in recurrent graphs.