Web Neural Network API

Draft Community Group Report, 28 November

This version:
https://webmachinelearning.github.io/webnn/
Issue Tracking:
GitHub
Editors:
Ningxin Hu ( Intel Corporation )
Chai Chaoweeraprasit ( Microsoft Corporation )
Explainer:
explainer.md
Polyfill:
webnn-polyfill / webnn-samples
Not Ready For Implementation

This spec is not yet ready for implementation. It exists in this repository to record the ideas and promote discussion.

Before attempting to implement this spec, please contact the editors.

Copyright © 2020 the Contributors to the Web Neural Network API Specification, published by the Machine Learning for the Web Community Group under the W3C Community Contributor License Agreement (CLA) . A human-readable summary is available.

Abstract

This document describes a dedicated low-level API for neural network inference hardware acceleration.

Status of this document

This specification was published by the Machine Learning for the Web Community Group . It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups .

1. Introduction

We’re working on this section. Meanwhile, please take a look at the explainer .

2. Use cases

2.1. Application Use Cases

This section illustrates application-level use cases for neural network inference hardware acceleration. All applications in those use cases can be built on top of pre-trained deep neural network (DNN) [models] .

2.1.1. Person Detection

A user opens a web-based video conferencing application, but she temporarily leaves from her room. The application is watching whether she is in front of her PC by using object detection (for example, using object detection approaches such as [SSD] or [YOLO] that use a single DNN) to detect regions in a camera input frame that include persons.

When she comes back, the application automatically detects her and notifies other online users that she is active now.

2.1.2. Semantic Segmentation

A user joins a teleconference via a web-based video conferencing application at her desk since no meeting room in her office is available. During the teleconference, she does not wish that her room and people in the background are visible. To protect the privacy of the other people and the surroundings, the application runs a machine learning model such as [DeepLabv3+] or [MaskR-CNN] to semantically split an image into segments and replaces segments that represent other people and background with another picture.

2.1.3. Skeleton Detection

A web-based video conferencing application tracks a pose of user’s skeleton by running a machine learning model, which allows for real-time human pose estimation, such as [PoseNet] to recognize her gesture and body language. When she raises her hand, her microphone is automatically unmuted and she can start speaking on the teleconference.

2.1.4. Face Recognition

There are multiple people in the conference room and they join an online meeting using a web-based video conferencing application. The application detects faces of participants by using object detection (for example, using object detection approaches such as [SSD] ) and checks whether each face was present at the previous meeting or not by running a machine learning model such as [FaceNet] , which verifies whether two faces would be identical or not.

2.1.5. Facial Landmark Detection

A user wants to find new glasses that beautifully fits her on an online glasses store. The online store offers web-based try-on simulator that runs a machine learning model such as Face Alignment Network [FAN] to detect facial landmarks like eyes, nose, mouth, etc. When she chooses a pair of glasses, the simulator properly renders the selected glasses on the detected position of eyes on her facial image.

2.1.6. Style Transfer

A user is looking for cosmetics on an online store and wondering which color may fit her face. The online store shows sample facial makeup images of cosmetics, and offers makeup simulator that runs a machine learning model like [ContextualLoss] or [PairedCycleGAN] to transfer the makeup style of the sample makeup image to her facial image. She can check how the selected makeup looks like on her face by the simulator.

2.1.7. Super Resolution

A web-based video conferencing is receiving a video stream from its peer, but the resolution of the video becomes lower due to network congestion. To prevent degradation of the perceived video quality, the application runs a machine learning model for super-resolution such as [SRGAN] to generate higher-resolution video frames.

2.1.8. Image Captioning

For better accessibility, a web-based presentation application provides automatic image captioning by running a machine learning model such as [im2txt] which predicts explanatory words of the presentation slides.

2.1.9. Machine Translation

Multiple people from various countries are talking via a web-based real-time text chat application. The application translates their conversation by using a machine learning model such as [GNMT] or [OpenNMT] , which translates every text into different language.

2.1.10. Emotion Analysis

A user is talking to her friend via a web-based real-time text chat application, and she is wondering how the friend feels because she cannot see the friend’s face. The application analyses the friend’s emotion by using a machine learning model such as [DeepMoji] , which infers emotion from input texts, and displays an emoji that represents the estimated emotion.

2.1.11. Video Summarization

A web-based video conferencing application records received video streams, and it needs to reduce recorded video data to be stored. The application generates the short version of the recorded video by using a machine learning model for video summarization such as [Video-Summarization-with-LSTM] .

2.1.12. Noise Suppression

A web-based video conferencing application records received audio streams, but usually the background noise is everywhere. The application leverages real-time noise suppression using Recurrent Neural Network such as [RNNoise] for suppressing background dynamic noise like baby cry or dog barking to improve audio experiences in video conferences.

2.2. Framework Use Cases

This section collects framework-level use cases for a dedicated low-level API for neural network inference hardware acceleration. It is expected that Machine Learning frameworks will be key consumers of the Web Neural Network API (WebNN API) and the low-level details exposed through the WebNN API are abstracted out from typical web developers. However, it is also expected that web developers with specific interest and competence in Machine Learning will want to interface with the WebNN API directly instead of a higher-level ML framework.

2.2.1. Custom Layer

A web application developer wants to run a DNN model on the WebNN API. However, she has found that some of activation functions like [LeakyReLU] , [ELU] , etc. are not included in the WebNN API. To address this issue, she constructs custom layers of the additional activation functions on top of the WebNN API. Note that the scope of custom layers may include convolution, normalization, etc. as well as activation.

2.2.2. Network Concatenation

A web application uses a DNN model, and its model data of upper convolutional layers and lower fully-connected layers are stored in separate files, since model data of the fully-connected layers are periodically updated due to fine tuning at the server side.

Therefore, the application downloads both partial model files at first and concatenates them into a single model. When the model is updated, the application downloads fine-tuned part of the model and replace only the fully-connected layers with it.

2.2.3. Performance Adaptation

A web application developer has a concern about performance of her DNN model on mobile devices. She has confirmed that it may run too slow on mobile devices which do not have GPU acceleration. To address this issue, her web application refers to the WebNN API to confirm whether acceleration is available or not, so that the application can display the warning for devices without acceleration.

After several weeks, she has developed a tiny DNN model that can even run on CPU. In order to accommodate CPU execution, she modifies the application so that the application loads the tiny model in the case of CPU-only devices.

3. API

3.1. Navigator 

partial interface Navigator {
  [SecureContext] readonly attribute ML ml;
};

3.2. ML 

[SecureContext, Exposed=Window]
interface ML {
  NeuralNetworkContext getNeuralNetworkContext();
};

3.3. OperandDescriptor 

enum OperandLayout {
  "nchw",
  "nhwc"
};
enum OperandType {
  "float32",
  "float16",
  "int32",
  "uint32",
  "int8",
  "uint8"
};
dictionary OperandDescriptor {
  // The operand type.
  required OperandType type;
  // The dimensions field is only required for tensor operands.
  // The negative value means an unknown dimension.
  sequence<long> dimensions;
};

3.4. Operand 

[SecureContext, Exposed=Window]
interface Operand {};

3.5. NeuralNetworkContext

The NeuralNetworkContext interface represents a global state of neural network compute workload and execution processes. 

[SecureContext, Exposed=Window]
interface NeuralNetworkContext {
  ModelBuilder createModelBuilder();
};

3.6. ModelBuilder

The ModelBuilder interface defines a set of operations as identified by the § 2 Use cases that can be composed into a computational graph. It also represents the intermediate state of a graph building session. 

typedef record<DOMString, Operand> NamedOperands;
[SecureContext, Exposed=Window]
interface ModelBuilder {
  // Create an operand that represents a model input.
  Operand input(DOMString name, OperandDescriptor desc);
  // Create an operand that represents a model constant.
  Operand constant(OperandDescriptor desc, ArrayBufferView value);
  // Create a single-value operand from the specified number of the specified type.
  Operand constant(double value, optional OperandType type = "float32");
  // Create a model that encapsulates the composition of operands by identifying the output operands.
  Model createModel(NamedOperands outputs);
};

3.6.1. batchNormalization

Normalize the tensor values of input features across dimensions in a the batch through a specialized [BatchNorm] transform dimension using [Batch-Normalization] . The For each input feature, the mean and variance tensors values of that feature are previously calculated computed across the batch dimension during model training pass. training. 
dictionary BatchNormalizationOptions {
  Operand scale;
  Operand bias;
  long axis = 1;
  float epsilon = 1e-5;
};
partial interface ModelBuilder {
  Operand batchNormalization(Operand input, Operand mean, Operand variance,
                             optional BatchNormalizationOptions options = {});
};
Arguments:

Returns: an Operand . The batch normalized batch-normalized N-D tensor of the same shape as the input tensor.

When input is a 4-D tensor of the "nchw" or "nhwc" layout, axis options.axis should be set to 1 or 3 respectively, to designate respectively. The axis value designates the feature or channel count dimension of the input tensor.

The behavior of this operation when the input tensor is 4-D of the "nchw" layout can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
const shape = [1,-1,1,1];
return builder.add(
    builder.mul(
      builder.reshape(options.scale, shape),
      builder.div(
        builder.sub(input, builder.reshape(mean, shape)),
        builder

        builder.pow(
          builder.add(builder.reshape(variance, shape), builder.constant(options.epsilon)),
          builder.constant(0.5))

        )
      ),
    builder.reshape(options.bias, shape)
  );

3.6.2. clamp

Clamp the input tensor element-wise within a range specified by the minimum and maximum values. 
dictionary ClampOptions {
  Operand minValue;
  Operand maxValue;
};
partial interface ModelBuilder {
  Operand clamp(Operand x, optional ClampOptions options = {});
};
Arguments:

Returns: an Operand . The output tensor of the same shape as x .

Clamp the input tensor element-wise within a range specified by minValue and maxValue . The calculation follows the expression min(max(x, minValue), maxValue). When minValue is not specified, the clamping is not performed on the lower limit. When maxValue is not specified, the clamping is not performed on the upper limit.

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
if (options.minValue === undefined) {
  if (options.maxValue === undefined) {
    return x;
  } else {
    return builder.min(x, options.maxValue);
  }
} else {
  if (options.maxValue === undefined) {
    return builder.max(x, options.minValue);
  } else {
    return builder.min(builder.max(x, options.minValue), options.maxValue);
  }
}

3.6.3. concat

Concatenates the input tensors along a given axis. 
partial interface ModelBuilder {
  Operand concat(sequence<Operand> inputs, long axis);
};
Arguments:

Returns: an Operand . The concatenated tensor of all the inputs along the axis . The output tensor has the same shape except on the dimension that all the inputs concatenated along. The size of that dimension is computed as the sum of all the input sizes of the same dimension.

3.6.4. conv2d

Compute a 2-D convolution given 4-D input and filter tensors { 
enum AutoPad {
  "explicit",
  "same-upper",
  "same-lower"
};
dictionary Conv2dOptions {
  sequence<long> padding;
  sequence<long> strides;
  sequence<long> dilations;
   = 1;

  sequence<long> outputPadding;
  sequence<long> outputSizes;
  AutoPad autoPad = "explicit";
  boolean transpose = false;
  long groups = 1;
  OperandLayout layout = "nchw";
};
partial interface ModelBuilder {
  Operand conv2d(Operand input, Operand filter, optional Conv2dOptions options = {});
};
Arguments:

Returns: an Operand . The output 4-D tensor that contains the result of the convolution. convolution result. The logical output shape is interpreted according to the value options.layout value. More specifically the sizes of the last two dimensions of the output tensor, the spatial dimensions, for the convolution operation can be calculated as follow:

layout . output size = 1 + (input size - filter size + beginning padding + ending padding) / stride

Whereas for the transposed convolution case with options.transpose set to true , unless the options.outputSizes values are explicitly specified, the options.outputPadding may be needed to compute the spatial dimension values of the output tensor as follow:

output size = (input size - 1) * stride + filter size - beginning padding - ending padding + output padding

A depthwise conv2d operation is a variant of grouped conv2d, convolution, used in models like the MobileNet, where the options.groups = input_channels = output_channels and the shape of filter tensor is [options.groups, 1, height, width] for "nchw" layout or [height, width, 1, options.groups] for "nhwc" layout.

3.6.5. element-wise binary operations

Compute the element-wise binary addition, subtraction, multiplication, division, maximum and minimum of the two input tensors. 
partial interface ModelBuilder {
  Operand add(Operand a, Operand b);
  Operand sub(Operand a, Operand b);
  Operand mul(Operand a, Operand b);
  Operand div(Operand a, Operand b);
  Operand max(Operand a, Operand b);
  Operand min(Operand a, Operand b);
  Operand pow(Operand a, Operand b);
};
Arguments:

Returns: an Operand . The output tensor that contains the result of element-wise binary operation of the two input tensors.

The element-wise binary operation will be broadcasted according to [numpy-broadcasting-rule] . The rank of the output tensor is the maximum rank of the input tensors. For each dimension of the output tensor, its size is the maximum size along that dimension of the input tensors.

Operation types:

3.6.6. element-wise unary operations

Compute the element-wise unary operation for input tensor. 
partial interface ModelBuilder {
  );
  );
  );
  );
  );
  );
  );
  );
  );
  );
  );
  );

  Operand abs(Operand x);
  Operand ceil(Operand x);
  Operand cos(Operand x);
  Operand exp(Operand x);
  Operand floor(Operand x);
  Operand log(Operand x);
  Operand neg(Operand x);
  Operand sigmoid(Operand x);
  Operand sin(Operand x);
  Operand tan(Operand x);
  Operand tanh(Operand x);
};
Arguments:

Returns: an Operand . The output tensor that contains the result of element-wise unary operation of the input tensor. The shape of the output tensor is the same as the shape of input tensor.

Operation types:

3.6.7. gemm

Calculate the general matrix multiplication of the Basic Linear Algebra Subprograms . The calculation follows the expression alpha * A * B + beta * C , where A , B , and C are matrices, and A and B may optionally be transposed prior to the calculation. 
dictionary GemmOptions {
  ;

  Operand c;
  float alpha = 1.0;
  float beta = 1.0;
  ;
  ;

  boolean aTranspose = false;
  boolean bTranspose = false;
};
partial interface ModelBuilder {
   = {});

  Operand gemm(Operand a, Operand b, optional GemmOptions options = {});
};
Arguments:

Returns: an Operand . The output 2-D tensor that contains the calculated product of all the inputs.

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
if (options.aTranspose)
  a = builder.transpose(a);
if (options.bTranspose)
  b = builder.transpose(b);
let ab = builder.matmul(builder.mul(builder.constant(options.alpha), a), b);
return (c ? builder.add(ab, builder.mul(builder.constant(options.beta), c)) : ab);

3.6.8. gru

Gated Recurrent Unit [GRU] recurrent network using an update gate and a reset gate to compute the hidden state that rolls into the output across the temporal sequence of the Network 
enum RecurrentNetworkWeightLayout {
  "zrn",  // update-reset-new gate ordering
  "rzn"   // reset-update-new gate ordering
};
enum RecurrentNetworkActivation {
  "relu",
  "sigmoid",
  "tanh"
};
enum RecurrentNetworkDirection {
  "forward",
  "backward",
  "both"
};
dictionary GruOptions {
  ;
  ;
  ;
  ;
  ;

  Operand bias;
  Operand recurrentBias;
  Operand initialHiddenState;
  boolean resetAfter = true;
  boolean returnSequence = false;
  RecurrentNetworkDirection direction = "forward";
  RecurrentNetworkWeightLayout layout = "zrn";
  ;

  sequence<RecurrentNetworkActivation> activations;
};
partial interface ModelBuilder {
  , 
                         = {});

  sequence<Operand> gru(Operand input, Operand weight, Operand recurrentWeight, 
                        long steps, long hiddenSize, optional GruOptions options = {});
};
Arguments:

Returns: a sequence of Operand . The first element of the sequence is a 3-D tensor of shape [num_directions, batch_size, hidden_size], the cell output from the last time step of the network. Additionally, if returnSequence is set to true, the second element is the 4-D output tensor of shape [steps, num_directions, batch_size, hidden_size] containing every cell outputs from each time step in the temporal sequence.

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
const numDirections = (options.direction == "both" ? 2 : 1);
let hiddenState = options.initialHiddenState;
if (!hiddenState) {
  const desc = { type: 'float32', dimensions: [numDirections, 1, hiddenSize] };
  const totalSize = numDirections * hiddenSize;
  hiddenState = builder.constant(desc, new Float32Array(totalSize).fill(0));
}
let sequence = null;
let cellWeight = [];
let cellRecurrentWeight = [];
let cellBias = [];
let cellRecurrentBias = [];
for (let slot = 0; slot < numDirections; ++slot) {
  cellWeight.push(builder.squeeze(builder.slice(weight, [slot, 0, 0], [1, -1, -1]), { axes: [0] }));
  cellRecurrentWeight.push(builder.squeeze(builder.slice(recurrentWeight, [slot, 0, 0], [1, -1, -1]), { axes: [0] }));
  cellBias.push(options.bias ? (builder.squeeze(builder.slice(options.bias, [slot, 0], [1, -1]), { axes: [0] })) : null);
  cellRecurrentBias.push(options.recurrentBias ? 
    (builder.squeeze(builder.slice(options.recurrentBias, [slot, 0], [1, -1]), { axes: [0] })) : null);
}
for (let step = 0; step < steps; ++step) {
  let cellHidden = [];
  let cellOutput = null;
  for (let slot = 0; slot < numDirections; ++slot) {
    cellHidden.push(builder.squeeze(builder.slice(hiddenState, [slot, 0, 0], [1, -1, -1]), { axes: [0] }));
  }
  for (let slot = 0; slot < numDirections; ++slot) {
    let slice = (slot == 1 || options.direction == "backward" ? steps - step - 1 : step);
    let cellInput = builder.squeeze(builder.slice(input, [slice, 0, 0], [1, -1, -1]), { axes: [0] });
    let result = builder.reshape(
      builder.gruCell(
        cellInput, cellWeight[slot], cellRecurrentWeight[slot],
        cellHidden[slot], hiddenSize, { bias: cellBias[slot],
        recurrentBias: cellRecurrentBias[slot], resetAfter: options.resetAfter,
        layout: options.layout, activations: options.activations }),
      [1, -1, hiddenSize]);
    cellOutput = (cellOutput ? builder.concat([cellOutput, result], 0) : result);
  }
  hiddenState = cellOutput;
  if (options.returnSequence) {
    cellOutput = builder.reshape(cellOutput, [1, numDirections, -1, hiddenSize]);
    sequence = (sequence ? builder.concat([sequence, cellOutput], 0) : cellOutput);
  }
}
return (sequence ? [hiddenState, sequence] : [hiddenState]);

3.6.9. gruCell

A single time step of the Gated Recurrent Unit [GRU] recurrent network using an update gate and a reset gate to compute the hidden state that rolls into the output across the temporal sequence of a recurrent network. 
dictionary GruCellOptions {
  ;
  ;
  ;

  Operand bias;
  Operand recurrentBias;
  boolean resetAfter = true;
  RecurrentNetworkWeightLayout layout = "zrn";
  ;

  sequence<RecurrentNetworkActivation> activations;
};
partial interface ModelBuilder {
  , 
                   = {});

  Operand gruCell(Operand input, Operand weight, Operand recurrentWeight, 
                  Operand hiddenState, long hiddenSize, optional GruCellOptions options = {});
};
Arguments:

Returns: an Operand . The 2-D tensor of shape [batch_size, hidden_size], the cell output hidden state of a single time step of the recurrent network.

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
const one = builder.constant(1);
const zero = builder.constant(0);
// update gate
let z = builder.sigmoid(
  builder.add(
    builder.add(
      (options.bias ? builder.slice(options.bias, [0], [hiddenSize]) : zero), 
      (options.recurrentBias ? builder.slice(options.recurrentBias, [0], [hiddenSize]) : zero)
      ),
    builder.add(
      builder.matmul(
        input, 
        builder.transpose(builder.slice(weight, [0, 0], [hiddenSize, -1]))
        ),
      builder.matmul(
        hiddenState,
        builder.transpose(builder.slice(recurrentWeight, [0, 0], [hiddenSize, -1]))
        )
      )
    )
  );
// reset gate
let r = builder.sigmoid(
  builder.add(
    builder.add(
      (options.bias ? builder.slice(options.bias, [hiddenSize], [hiddenSize]) : zero),
      (options.recurrentBias ? builder.slice(options.recurrentBias, [hiddenSize], [hiddenSize]) : zero)
      ),
    builder.add(
      builder.matmul(
        input, 
        builder.transpose(builder.slice(weight, [hiddenSize, 0], [hiddenSize, -1]))
        ),
      builder.matmul(
        hiddenState, 
        builder.transpose(builder.slice(recurrentWeight, [hiddenSize, 0], [hiddenSize, -1]))
        )
      )
    )
  );
// new gate
let n;
if (resetAfter) {
  n = builder.tanh(
    builder.add(
      (options.bias ? builder.slice(options.bias, [2 * hiddenSize], [hiddenSize]) : zero),
      builder.add(
        builder.matmul(
          input, 
          builder.transpose(builder.slice(weight, [2 * hiddenSize, 0], [hiddenSize, -1]))
          ),
        builder.mul(
          r,
          builder.add(
            (options.recurrentBias ? builder.slice(options.recurrentBias, [2 * hiddenSize], [hiddenSize]) : zero),
            builder.matmul(
              hiddenState, 
              builder.transpose(builder.slice(recurrentWeight, [2 * hiddenSize, 0], [hiddenSize, -1]))
              )
            )
          )
        )
      )
    );
}
else {
  n = builder.tanh(
    builder.add(
      builder.add(
        (options.bias ? builder.slice(options.bias, [2 * hiddenSize], [hiddenSize]) : zero),
        (options.recurrentBias ? builder.slice(options.recurrentBias, [2 * hiddenSize], [hiddenSize]) : zero)
        ),
      builder.add(
        builder.matmul(
          input, 
          builder.transpose(builder.slice(weight, [2 * hiddenSize, 0], [hiddenSize, -1]))
          ),
        builder.matmul(
          builder.mul(r, hiddenState),
          builder.transpose(builder.slice(recurrentWeight, [2 * hiddenSize, 0], [hiddenSize, -1]))
          )
        )
      )
    );
}
// compute the new hidden state
return builder.add(builder.mul(z, hiddenState), builder.mul(n, builder.sub(one, z)));

3.6.10. instanceNormalization

Normalize the input features using [Instance-Normalization] . Unlike § 3.6.1 batchNormalization where the mean and variance values are computed across the batch dimension during training, the mean and variance values of instance normalization are computed on the fly per input feature. 
dictionary InstanceNormalizationOptions {  Operand scale;  Operand bias;  float epsilon = 1e-5;  OperandLayout layout = "nchw";
};
partial interface ModelBuilder {  Operand instanceNormalization(Operand input,                                 optional InstanceNormalizationOptions options = {});
};
Arguments:

Returns: an Operand . The instance-normalized 4-D tensor of the same shape as the input tensor.

The behavior of this operation when the input tensor is 4-D of the "nchw" layout can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
// The mean reductions happen over the spatial dimensions of the input// e.g. axis 2 and 3 of the input tensor.const reduceOptions = { axes: [2,3], keepDimensions: true };const mean = builder.reduceMean(input, reduceOptions);const variance = builder.reduceMean(
  builder.pow(
    builder.sub(input, mean), 
    buider.constant(2)),
  reduceOptions
  );// The scale and bias values are applied per input feature// e.g. axis 1 of the input tensor.const shape = [1,-1,1,1];return builder.add(
  builder.mul(
    builder.reshape(options.scale, shape),
    builder.div(
      builder.sub(input, mean),
      buidler.pow(
        builder.add(variance, options.epsilon), 
        builder.constant(0.5))
      )    ),
  builder.reshape(options.bias, shape)
  );

3.6.11. leakyRelu 

dictionary LeakyReluOptions {
   = 0.01;

  float alpha = 0.01;
};
 {
   = {});

partial interface ModelBuilder {
  Operand leakyRelu(Operand x, optional LeakyReluOptions options = {});
};
Arguments:

Returns: an Operand . The output tensor of the same shape as x .

Calculate the leaky version of rectified linear function on the input tensor element-wise. The calculation follows the expression max(0, x) + alpha ∗ min(0, x) .

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
return builder.add(builder.max(builder.constant(0), x),
          builder.mul(builder.constant(options.alpha), builder.min(builder.constant(0), x)));

3.6.11. 3.6.12. matmul

Compute the matrix product of two input tensors. { ); 
partial interface ModelBuilder {
  Operand matmul(Operand a, Operand b);
};
Arguments:

Returns: an Operand . The output N-D tensor that contains the matrix product of two input tensors.

Compute the matrix product of two input tensors. It behaves as following:

3.6.12. 3.6.13. pad

Inflate the tensor with constant or mirrored values on the edges. 
enum PaddingMode {  "constant",  "edge",  "reflection",  "symmetric"
};
dictionary PadOptions {  PaddingMode mode = "constant";  float value = 0;
};
partial interface ModelBuilder {  Operand pad(Operand input, Operand padding, optional PadOptions options = {});
};
Arguments:

Returns: an Operand . The padded output tensor. 

// input: [[1,2,3], [4,5,6]]const input = builder.constant(  { type: 'float32', dimensions: [2,3] }, new Float32Array([1,2,3,4,5,6]));// padding: [[1,1], [2,2]]const padding = builder.constant(  { type: 'float32', dimensions: [2,2] }, new Float32Array([1,1,2,2]));// "constant" padded://    [[0,0,0,0,0,0,0],//     [0,0,1,2,3,0,0],//     [0,0,4,5,6,0,0],//     [0,0,0,0,0,0,0]]
builder.pad(input, padding);
// "edge" padded://    [[1,1,1,2,3,3,3],//     [1,1,1,2,3,3,3],//     [4,4,4,5,6,6,6],//     [4,4,4,5,6,6,6]]
builder.pad(input, padding, { mode: "edge" });
// "reflection" padded://    [[6,5,4,5,6,5,4],//     [3,2,1,2,3,2,1],//     [6,5,4,5,6,5,4],//     [3,2,1,2,3,2,1]]
builder.pad(input, padding, { mode: "reflection" });
// "symmetric" padded://    [[2,1,1,2,3,3,2],//     [2,1,1,2,3,3,2],//     [5,4,4,5,6,6,5],//     [5,4,4,5,6,6,5]]
builder.pad(input, padding, { mode: "symmetric" });

3.6.14. pooling operations

Compute a mean , L2 norm , or max reduction operation across all the elements within the moving window over the input tensor. See the description of each type of reduction in § 3.6.13 § 3.6.15 reduction operations . 
dictionary Pool2dOptions {
  ;
  ;
  ;
  ;
   = "nchw";

  sequence<long> windowDimensions;
  sequence<long> padding;
  sequence<long> strides;
  sequence<long> dilations;
  AutoPad autoPad = "explicit";
  OperandLayout layout = "nchw";
};
 {
   = {});
   = {});
   = {});

partial interface ModelBuilder {
  Operand averagePool2d(Operand input, optional Pool2dOptions options = {});
  Operand l2Pool2d(Operand input, optional Pool2dOptions options = {});
  Operand maxPool2d(Operand input, optional Pool2dOptions options = {});
};
Arguments:

Returns: an Operand . The output 4-D tensor that contains the result of the reduction. The logical shape is interpreted according to the value of layout .

A global pooling operation such as one for the max pooling operation is a variant of pooling where the window dimensions is the spatial dimensions (last two dimensions) of the input shape, as follow. 
// 'global' max pooling
builder.maxPool2d(input);

3.6.13. 3.6.15. reduction operations

Reduce the input along the dimensions given in axes . 
dictionary ReduceOptions {
  ;
  ;

  sequence<long> axes = null;
  boolean keepDimensions = false;
};
 {
   = {});
   = {});
   = {});
   = {});
   = {});
   = {});
   = {});
   = {});
   = {});
   = {});

partial interface ModelBuilder {
  Operand reduceL1(Operand input, optional ReduceOptions options = {});
  Operand reduceL2(Operand input, optional ReduceOptions options = {});
  Operand reduceLogSum(Operand input, optional ReduceOptions options = {});
  Operand reduceLogSumExp(Operand input, optional ReduceOptions options = {});
  Operand reduceMax(Operand input, optional ReduceOptions options = {});
  Operand reduceMean(Operand input, optional ReduceOptions options = {});
  Operand reduceMin(Operand input, optional ReduceOptions options = {});
  Operand reduceProduct(Operand input, optional ReduceOptions options = {});
  Operand reduceSum(Operand input, optional ReduceOptions options = {});
  Operand reduceSumSquare(Operand input, optional ReduceOptions options = {});
};
Arguments:

Returns: an Operand . The reduced output tensor.

Reduction types:

3.6.14. 3.6.16. relu

Calculate the rectified linear function on the input tensor element-wise. The calculation follows the expression max(0, x) . { ); 
partial interface ModelBuilder {
  Operand relu(Operand x);
};
Arguments:

Returns: an Operand . The output tensor of the same shape as x .

Calculate the rectified linear function on the input tensor element-wise. The calculation follows the expression max(0, x) .

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
return builder.max(builder.constant(0), x);

3.6.15. 3.6.17. resample

Resample the tensor values from the source to the destination dimensions according to the scaling factors. 
enum InterpolationMode {  "nearest-neighbor",  "linear"
};
dictionary ResampleOptions {  InterpolationMode mode = "nearest-neighbor";  sequence<float> scales;  sequence<long> sizes;
};
partial interface ModelBuilder {  Operand resample(Operand input, optional ResampleOptions options = {});
};
Arguments:

Returns: an Operand . The output 4-D tensor.

3.6.18. reshape

Reshapes a tensor to a given new shape. { ); 
partial interface ModelBuilder {
  Operand reshape(Operand input, sequence<long> newShape);
};
Arguments:

Returns: an Operand . The output tensor. The values of the output tensor are the same as values of the input tensor. The shape of the output tensor is specified by the newShape argument.

3.6.16. 3.6.19. slice

Produce a slice of the input tensor. 
dictionary SliceOptions {
  ;

  sequence<long> axes;
};
 {
  ,

partial interface ModelBuilder {
  Operand slice(Operand input, sequence<long> starts, sequence<long> sizes,
                optional SliceOptions options = {});
};
Arguments:

Returns: an Operand . The output tensor of the same rank as the input tensor with tensor values stripped to the specified starting and ending indices in each dimension.

3.6.17. 3.6.20. softmax

Compute the softmax values of the 2-D input tensor along axis 1. { ); 
partial interface ModelBuilder {
  Operand softmax(Operand x);
};
Arguments:

Returns: an Operand . The output 2-D tensor that contains the softmax results, of the same shape as the input tensor.

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
// This sample deploys a well-known implementation trick [1] to compute the
// exponentials of the distances to the max value, instead of the exponentials
// of the input values itself, in order to increase the numerical stability of
// the result.
// [1]: https://cs231n.github.io/linear-classify/#softmax
const max_x = builder.reduceMax(x, { axes: [1], keepDimensions: true });
const exp_x = builder.exp(builder.sub(x, max));
return builder.div(exp_x, builder.reduceSum(exp_x, { axes: [1], keepDimensions: true }));

3.6.18. 3.6.21. split

Split the input tensor into a number of sub tensors along the given axis. 
dictionary SplitOptions {
   = 0;

  long axis = 0;
};
 {
  ,
                          (,

partial interface ModelBuilder {
  sequence<Operand> split(Operand input,
                          (unsigned long or sequence<unsigned long>) splits,

                          optional SplitOptions options = {});
};
Arguments:

Returns: a sequence of Operand . The splitted output tensors. If splits is an unsigned long , the length of the output sequence equals to splits . The shape of each output tensor is the same as input except the dimension size of axis equals to the quotient of dividing the dimension size of input along axis by splits . If splits is a sequence of unsigned long , the length of the output sequence equals to the length of splits . The shape of the i-th output tensor is the same as as input except along axis where the dimension size is splits[i] .

The behavior of this operation can be generically emulated from the usage of other operations as follow. However, user agents typically have a more efficient implementation for it, therefore its usage is encouraged from the performance standpoint. 
// This sample shows the case that the splits parameter is an array.
const outputs = [];
let start = 0;
for (const size of splits) {
  outputs.push(builder.slice(input, [start], [size], { axis: [options.axis] }));
  start += size;
}
return outputs;

3.6.19. 3.6.22. squeeze

Reduce the rank of a tensor without affecting its values by eliminating dimensions with size 1 of the tensor shape. 
dictionary SqueezeOptions {
  ;

  sequence<long> axes;
};
 {
   = {});

partial interface ModelBuilder {
  Operand squeeze(Operand input, optional SqueezeOptions options = {});
};
Arguments:

Returns: an Operand . The output tensor of the same or reduced rank with the shape dimensions of size 1 eliminated.

3.6.20. 3.6.23. transpose

Permute the dimensions of the input tensor according to the permutation argument. 
dictionary TransposeOptions {
  ;

  sequence<long> permutation;
};
 {
   = {});

partial interface ModelBuilder {
  Operand transpose(Operand input, optional TransposeOptions options = {});
};
Arguments:

Returns: an Operand . The permuted or transposed N-D tensor.

3.7. Model

The Model interface represents an immutable computational graph constructed by the ModelBuilder interface. The operands to the starting operations of the graph are assumed to be the model’s input operands. 
enum PowerPreference {
  // Let the user agent decide the most suitable behavior. This is the default value.
  "default",
  // Prioritizes execution speed over other considerations e.g. power consumption
  "high-performance",
  // Prioritizes power consumption over other considerations e.g. execution speed
  "low-power"
};
dictionary CompilationOptions {
  // Compilation preference as related to power consumption level
  PowerPreference powerPreference = "default";
};
[SecureContext, Exposed=Window]
interface Model {
  Promise<Compilation> compile(optional CompilationOptions options = {});
};

3.8. Compilation

The Compilation interface represents a compiled version of the computational graph of a Model according to the compilation options. A compilation of a graph once constructed is immutable and cannot be subsequently changed. 
dictionary Input {
  required ArrayBufferView buffer;
  ;

  sequence<long> dimensions;
};
dictionary Output {
  ArrayBufferView buffer;
  ;

  sequence<long> dimensions;
};
typedef record<DOMString, Input> NamedInputs;
typedef record<DOMString, Output> NamedOutputs;
[SecureContext, Exposed=Window]
interface Compilation {
  Promise<NamedOutputs> compute(NamedInputs inputs, optional NamedOutputs outputs = {});
};

4. Examples

The following code gets the NeuralNetworkContext object. 
const nn = navigator.ml.getNeuralNetworkContext();
The following code builds a graph as: 
constant1 ---+
             +--- Add ---> intermediateOutput1 ---+
input1    ---+                                    |
                                                  +--- Mul---> output
constant2 ---+                                    |
             +--- Add ---> intermediateOutput2 ---+
input2    ---+

// Use tensors in 4 dimensions.
const TENSOR_DIMS = [1, 2, 2, 2];
const TENSOR_SIZE = 8;
const builder = nn.createModelBuilder();
// Create OperandDescriptor object.
const desc = {type: 'float32', dimensions: TENSOR_DIMS};
// constant1 is a constant operand with the value 0.5.
const constantBuffer1 = new Float32Array(TENSOR_SIZE).fill(0.5);
const constant1 = builder.constant(desc, constantBuffer1);
// input1 is one of the input operands. Its value will be set before execution.
const input1 = builder.input('input1', desc);
// constant2 is another constant operand with the value 0.5.
const constantBuffer2 = new Float32Array(TENSOR_SIZE).fill(0.5);
const constant2 = builder.constant(desc, constantBuffer2);
// input2 is another input operand. Its value will be set before execution.
const input2 = builder.input('input2', desc);
// intermediateOutput1 is the output of the first Add operation.
const intermediateOutput1 = builder.add(constant1, input1);
// intermediateOutput2 is the output of the second Add operation.
const intermediateOutput2 = builder.add(constant2, input2);
// output is the output operand of the Mul operation.
const output = builder.mul(intermediateOutput1, intermediateOutput2);
// Create the model by identifying the outputs.
const model = builder.createModel({'output': output});
The following code compiles the model by prioritizing lower level of power consumption over time. This option could be particularly useful for long-running models. 
// Compile the constructed model.
const compilation = await model.compile({powerPreference: 'low-power'});
The following code executes the compiled model. 
// Setup the input buffers with value 1.
const inputBuffer1 = new Float32Array(TENSOR_SIZE).fill(1);
const inputBuffer2 = new Float32Array(TENSOR_SIZE).fill(1);
// Asynchronously execute the compiled model with the specified inputs.
const inputs = {
  'input1': {buffer: inputBuffer1},
  'input2': {buffer: inputBuffer2},
};
const outputs = await compilation.compute(inputs);
// Log the shape and computed result of the output operand.
console.log('Output shape: ' + outputs.output.dimensions);
// Output shape: 1,2,2,2
console.log('Output value: ' + outputs.output.buffer);
// Output value: 2.25,2.25,2.25,2.25,2.25,2.25,2.25,2.25

5. Acknowledgements

This specification follows the concepts of the Android Neural Networks API C API.

Thanks to Tomoyuki Shimizu, Ningxin Hu, Zhiqiang Yu and Belem Zhang for the use cases.

Thanks to Nikhil Thorat, Daniel Smilkov, Ganesan Ramalingam, Rafael Cintron and Benjamin Poulain for their contributions to the API specification.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example" , like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note" , like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[HTML]
Anne van Kesteren; et al. HTML Standard . Living Standard. URL: https://html.spec.whatwg.org/multipage/
[NUMPY-BROADCASTING-RULE]
The SciPy community. General Broadcasting Rules of NumPy . July 2019. URL: https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html#general-broadcasting-rules
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels . March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[WebIDL]
Boris Zbarsky. Web IDL . 15 December 2016. ED. URL: https://heycam.github.io/webidl/

Informative References

[BatchNorm]
[Batch-Normalization]
Sergey Ioffe; Christian Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift . March 2015. URL: https://arxiv.org/abs/1502.03167
[ContextualLoss]
Roey Mechrez; Itamar Talmi; Lihi Zelnik-Manor. The Contextual Loss for Image Transformation with Non-Aligned Data . July 2018. URL: https://arxiv.org/abs/1803.02077
[DeepLabv3+]
Liang-Chieh Chen; et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation . August 2018. URL: https://arxiv.org/abs/1802.02611
[DeepMoji]
Bjarke Felbo; et al. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm . October 2017. URL: https://arxiv.org/abs/1708.00524
[ELU]
Djork-Arné Clevert; Thomas Unterthiner; Sepp Hochreiter. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) . February 2016. URL: https://arxiv.org/abs/1511.07289
[FaceNet]
Florian Schroff; Dmitry Kalenichenko; James Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering . June 2015. URL: https://arxiv.org/abs/1503.03832
[FAN]
Adrian Bulat; Georgios Tzimiropoulos. How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks) . September 2017. URL: https://arxiv.org/abs/1703.07332
[GNMT]
Minh-Thang Luong; Eugene Brevdo; Rui Zhao. Neural Machine Translation (seq2seq) Tutorial . May 2017. URL: https://github.com/tensorflow/nmt
[GRU]
Kyunghyun Cho; et al. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation . September 2014. URL: https://arxiv.org/pdf/1406.1078.pdf
[IM2TXT]
Oriol Vinyals; et al. Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge . September 2016. URL: https://arxiv.org/abs/1609.06647
[Instance-Normalization]
Dmitry Ulyanov; Andrea Vedaldi; Victor Lempitsky. Instance Normalization: The Missing Ingredient for Fast Stylization . July 2016. URL: https://arxiv.org/abs/1607.08022
[LeakyReLU]
Andrew L. Maas; Awni Y. Hannun; Andrew Y. Ng. Rectifier Nonlinearities Improve Neural Network Acoustic Models . June 2013. URL: https://pdfs.semanticscholar.org/367f/2c63a6f6a10b3b64b8729d601e69337ee3cc.pdf
[MaskR-CNN]
Kaiming He; et al. Mask R-CNN . January 2018. URL: https://arxiv.org/abs/1703.06870
[MODELS]
Machine Learning for the Web Community Group. The first-wave models . 2020. URL: https://github.com/webmachinelearning/webnn/blob/master/op_compatibility/first_wave_models.md
[OpenNMT]
Guillaume Klein; et al. OpenNMT: Open-Source Toolkit for Neural Machine Translation . March 2017. URL: https://arxiv.org/abs/1701.02810
[PairedCycleGAN]
Huiwen Chang; et al. PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup . June 2018. URL: http://openaccess.thecvf.com/content_cvpr_2018/html/Chang_PairedCycleGAN_Asymmetric_Style_CVPR_2018_paper.html
[PoseNet]
Dan Oved. Real-time Human Pose Estimation in the Browser with TensorFlow.js . May 2018. URL: https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
[RNNoise]
Jean-Marc Valin. Recurrent neural network for audio noise reduction . September 2017. URL: https://github.com/xiph/rnnoise
[SRGAN]
Christian Ledig; et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network . May 2017. URL: https://arxiv.org/abs/1609.04802
[SSD]
Wei Liu; et al. SSD: Single Shot MultiBox Detector . December 2016. URL: https://arxiv.org/abs/1512.02325
[Video-Summarization-with-LSTM]
Ke Zhang; et al. Video summarization with long short-term memory . October 2016. URL: http://www-scf.usc.edu/~zhan355/ke_eccv2016.pdf
[YOLO]
Joseph Redmon; et al. You Only Look Once: Unified, Real-Time Object Detection . May 2016. URL: https://arxiv.org/abs/1506.02640

IDL Index 

partial interface Navigator {
  [SecureContext] readonly attribute ML ml;
};
[SecureContext, Exposed=Window]
interface ML {
  NeuralNetworkContext getNeuralNetworkContext();
};
enum OperandLayout {
  "nchw",
  "nhwc"
};
enum OperandType {
  "float32",
  "float16",
  "int32",
  "uint32",
  "int8",
  "uint8"
};
dictionary OperandDescriptor {
  // The operand type.
  required OperandType type;
  // The dimensions field is only required for tensor operands.
  // The negative value means an unknown dimension.
  sequence<long> dimensions;
};
[SecureContext, Exposed=Window]
interface Operand {};
[SecureContext, Exposed=Window]
interface NeuralNetworkContext {
  ModelBuilder createModelBuilder();
};
typedef record<DOMString, Operand> NamedOperands;
[SecureContext, Exposed=Window]
interface ModelBuilder {
  // Create an operand that represents a model input.
  Operand input(DOMString name, OperandDescriptor desc);
  // Create an operand that represents a model constant.
  Operand constant(OperandDescriptor desc, ArrayBufferView value);
  // Create a single-value operand from the specified number of the specified type.
  Operand constant(double value, optional OperandType type = "float32");
  // Create a model that encapsulates the composition of operands by identifying the output operands.
  Model createModel(NamedOperands outputs);
};
dictionary BatchNormalizationOptions {
  Operand scale;
  Operand bias;
  long axis = 1;
  float epsilon = 1e-5;
};
partial interface ModelBuilder {
  Operand batchNormalization(Operand input, Operand mean, Operand variance,
                             optional BatchNormalizationOptions options = {});
};
dictionary ClampOptions {
  Operand minValue;
  Operand maxValue;
};
partial interface ModelBuilder {
  Operand clamp(Operand x, optional ClampOptions options = {});
};
partial interface ModelBuilder {
  Operand concat(sequence<Operand> inputs, long axis);
};
enum AutoPad {
  "explicit",
  "same-upper",
  "same-lower"
};

dictionary Conv2dOptions {
  sequence<long> padding;
  sequence<long> strides;
  sequence<long> dilations;
  sequence<long> outputPadding;
  sequence<long> outputSizes;
  AutoPad autoPad = "explicit";
  boolean transpose = false;
  long groups = 1;
  OperandLayout layout = "nchw";
};
partial interface ModelBuilder {
  Operand conv2d(Operand input, Operand filter, optional Conv2dOptions options = {});
};
partial interface ModelBuilder {
  Operand add(Operand a, Operand b);
  Operand sub(Operand a, Operand b);
  Operand mul(Operand a, Operand b);
  Operand div(Operand a, Operand b);
  Operand max(Operand a, Operand b);
  Operand min(Operand a, Operand b);
  Operand pow(Operand a, Operand b);
};
partial interface ModelBuilder {
  Operand abs(Operand x);
  Operand ceil(Operand x);
  Operand cos(Operand x);
  Operand exp(Operand x);
  Operand floor(Operand x);
  Operand log(Operand x);
  Operand neg(Operand x);
  Operand sigmoid(Operand x);
  Operand sin(Operand x);
  );

  Operand tan(Operand x);
  Operand tanh(Operand x);
};
dictionary GemmOptions {
  Operand c;
  float alpha = 1.0;
  float beta = 1.0;
  boolean aTranspose = false;
  boolean bTranspose = false;
};
partial interface ModelBuilder {
  Operand gemm(Operand a, Operand b, optional GemmOptions options = {});
};
enum RecurrentNetworkWeightLayout {
  "zrn",  // update-reset-new gate ordering
  "rzn"   // reset-update-new gate ordering
};
enum RecurrentNetworkActivation {
  "relu",
  "sigmoid",
  "tanh"
};
enum RecurrentNetworkDirection {
  "forward",
  "backward",
  "both"
};
dictionary GruOptions {
  Operand bias;
  Operand recurrentBias;
  Operand initialHiddenState;
  boolean resetAfter = true;
  boolean returnSequence = false;
  RecurrentNetworkDirection direction = "forward";
  RecurrentNetworkWeightLayout layout = "zrn";
  sequence<RecurrentNetworkActivation> activations;
};
partial interface ModelBuilder {
  sequence<Operand> gru(Operand input, Operand weight, Operand recurrentWeight, 
                        long steps, long hiddenSize, optional GruOptions options = {});
};
dictionary GruCellOptions {
  Operand bias;
  Operand recurrentBias;
  boolean resetAfter = true;
  RecurrentNetworkWeightLayout layout = "zrn";
  sequence<RecurrentNetworkActivation> activations;
};
partial interface ModelBuilder {
  Operand gruCell(Operand input, Operand weight, Operand recurrentWeight, 
                  Operand hiddenState, long hiddenSize, optional GruCellOptions options = {});
};
dictionary InstanceNormalizationOptions {
  Operand scale;
  Operand bias;
  float epsilon = 1e-5;
  OperandLayout layout = "nchw";
};
partial interface ModelBuilder {  Operand instanceNormalization(Operand input,                                 optional InstanceNormalizationOptions options = {});
};

dictionary LeakyReluOptions {
  float alpha = 0.01;
};
partial interface ModelBuilder {
  Operand leakyRelu(Operand x, optional LeakyReluOptions options = {});
};
partial interface ModelBuilder {
  Operand matmul(Operand a, Operand b);
};
enum PaddingMode {
  "constant",
  "edge",
  "reflection",
  "symmetric"
};
dictionary PadOptions {  PaddingMode mode = "constant";  float value = 0;
};
partial interface ModelBuilder {  Operand pad(Operand input, Operand padding, optional PadOptions options = {});
};

dictionary Pool2dOptions {
  sequence<long> windowDimensions;
  sequence<long> padding;
  sequence<long> strides;
  sequence<long> dilations;
  AutoPad autoPad = "explicit";
  OperandLayout layout = "nchw";
};
partial interface ModelBuilder {
  Operand averagePool2d(Operand input, optional Pool2dOptions options = {});
  Operand l2Pool2d(Operand input, optional Pool2dOptions options = {});
  Operand maxPool2d(Operand input, optional Pool2dOptions options = {});
};
dictionary ReduceOptions {
  sequence<long> axes = null;
  boolean keepDimensions = false;
};
partial interface ModelBuilder {
  Operand reduceL1(Operand input, optional ReduceOptions options = {});
  Operand reduceL2(Operand input, optional ReduceOptions options = {});
  Operand reduceLogSum(Operand input, optional ReduceOptions options = {});
  Operand reduceLogSumExp(Operand input, optional ReduceOptions options = {});
  Operand reduceMax(Operand input, optional ReduceOptions options = {});
  Operand reduceMean(Operand input, optional ReduceOptions options = {});
  Operand reduceMin(Operand input, optional ReduceOptions options = {});
  Operand reduceProduct(Operand input, optional ReduceOptions options = {});
  Operand reduceSum(Operand input, optional ReduceOptions options = {});
  Operand reduceSumSquare(Operand input, optional ReduceOptions options = {});
};
partial interface ModelBuilder {
  Operand relu(Operand x);
};
enum InterpolationMode {
  "nearest-neighbor",
  "linear"
};
dictionary ResampleOptions {  InterpolationMode mode = "nearest-neighbor";  sequence<float> scales;  sequence<long> sizes;
};
partial interface ModelBuilder {  Operand resample(Operand input, optional ResampleOptions options = {});
};

partial interface ModelBuilder {
  Operand reshape(Operand input, sequence<long> newShape);
};
dictionary SliceOptions {
  sequence<long> axes;
};
partial interface ModelBuilder {
  Operand slice(Operand input, sequence<long> starts, sequence<long> sizes,
                optional SliceOptions options = {});
};
partial interface ModelBuilder {
  Operand softmax(Operand x);
};
dictionary SplitOptions {
  long axis = 0;
};
partial interface ModelBuilder {
  sequence<Operand> split(Operand input,
                          (unsigned long or sequence<unsigned long>) splits,
                          optional SplitOptions options = {});
};
dictionary SqueezeOptions {
  sequence<long> axes;
};
partial interface ModelBuilder {
  Operand squeeze(Operand input, optional SqueezeOptions options = {});
};
dictionary TransposeOptions {
  sequence<long> permutation;
};
partial interface ModelBuilder {
  Operand transpose(Operand input, optional TransposeOptions options = {});
};
enum PowerPreference {
  // Let the user agent decide the most suitable behavior. This is the default value.
  "default",
  // Prioritizes execution speed over other considerations e.g. power consumption
  "high-performance",
  // Prioritizes power consumption over other considerations e.g. execution speed
  "low-power"
};
dictionary CompilationOptions {
  // Compilation preference as related to power consumption level
  PowerPreference powerPreference = "default";
};
[SecureContext, Exposed=Window]
interface Model {
  Promise<Compilation> compile(optional CompilationOptions options = {});
};
dictionary Input {
  required ArrayBufferView buffer;
  sequence<long> dimensions;
};
dictionary Output {
  ArrayBufferView buffer;
  sequence<long> dimensions;
};
typedef record<DOMString, Input> NamedInputs;
typedef record<DOMString, Output> NamedOutputs;
[SecureContext, Exposed=Window]
interface Compilation {
  Promise<NamedOutputs> compute(NamedInputs inputs, optional NamedOutputs outputs = {});
};