You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since the <a href="https://www.w3.org/TR/2023/CR-webnn-20230330/">initial Candidate Recommendation Snapshot</a> the Working Group has gathered further <a href="https://webmachinelearning.github.io/webnn-status/">implementation experience</a> and added new operations and data types needed for well-known <a href="https://github.com/webmachinelearning/webnn/issues/375">transformers to support generative AI use cases</a>. In addition, informed by this implementation experience, the group removed <code>MLCommandEncoder</code>, support for synchronous execution, and higher-level operations that can be expressed in terms of lower-level primitives in a performant manner. The group has also updated the specification to use modern authoring conventions to improve interoperability and precision of normative definitions.
26
27
The group is developing a new feature, a <a href="https://github.com/webmachinelearning/webnn/issues/482">backend-agnostic storage type</a>, to improve performance and interoperability between the WebNN, WebGPU APIs and purpose-built hardware for ML and expects to republish this document as a Candidate Recommendation Snapshot when ready for implementation.
@@ -401,7 +402,7 @@ This section illustrates application-level use cases for neural network
401
402
inference hardware acceleration. All applications in those use cases can be
402
403
built on top of pre-trained deep neural network (DNN) [[models]].
403
404
404
-
Note: Please be aware that some of the use cases described here, are by their very nature, privacy-invasive. Developers who are planning to use the API for such use cases should ensure that the API is being used to benefit users, for purposes that users understand, and approve. They should apply the Ethical Principles for Web Machine Learning [[webmachinelearning-ethics]] and implement appropriate privacy risk mitigations such as transparency, data minimisation, and users controls.
405
+
Note: Please be aware that some of the use cases described here, are by their very nature, privacy-invasive. Developers who are planning to use the API for such use cases <span class=allow-2119>should</span> ensure that the API is being used to benefit users, for purposes that users understand, and approve. They <span class=allow-2119>should</span> apply the Ethical Principles for Web Machine Learning [[webmachinelearning-ethics]] and implement appropriate privacy risk mitigations such as transparency, data minimisation, and users controls.
405
406
406
407
### Person Detection ### {#usecase-person-detection}
407
408
@@ -630,22 +631,28 @@ Purpose-built Web APIs for measuring high-resolution time mitigate against timin
630
631
631
632
## Guidelines for new operations ## {#security-new-ops}
632
633
634
+
*This section is non-normative.*
635
+
636
+
<div class=informative>
637
+
633
638
To ensure operations defined in this specification are shaped in a way they can be implemented securely, this section includes guidelines on how operations are expected to be defined to reduce potential for implementation problems. These guidelines are expected to evolve over time to align with industry best practices:
634
639
635
640
- Prefer simplicity of arguments
636
641
- Don't use parsers for complex data formats
637
642
- If an operation can be decomposed to low level primitives:
638
643
- Add an informative emulation path
639
644
- Prefer primitives over new high level operations but consider performance consequences
640
-
- Operations should follow a consistent style for inputs and attributes
641
-
- Operation families such as pooling and reduction should share API shape and options
645
+
- Follow a consistent style for operation inputs and attributes
646
+
- Share API shape and options for operation families such as pooling and reduction
642
647
- Formalize failure cases into test cases whenever possible
643
-
- When in doubt, leave it out: API surface should be as small as possible required to satisfy the use cases, but no smaller
648
+
- When in doubt, leave it out: keep the API surface as small as possible to satisfy the use cases, but no smaller
644
649
- Try to keep the API free of implementation details that might inhibit future evolution, do not overspecify
645
650
- Fail fast: the sooner the web developer is informed of an issue, the better
646
651
647
652
In general, always consider the security and privacy implications as documented in [[security-privacy-questionnaire]] by the Technical Architecture Group and the Privacy Interest Group when adding new features.
648
653
654
+
</div>
655
+
649
656
Privacy Considerations {#privacy}
650
657
===================================
651
658
@@ -665,7 +672,7 @@ The WebNN API defines two developer-settable preferences to help inform [[#progr
665
672
666
673
Issue(623): {{MLContextOptions}} is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community.
667
674
668
-
If a future version of this specification introduces support for a new {{MLDeviceType}} that can only support a subset of {{MLOperandDataType}}s, that may introduce a new fingerprint.
675
+
If a future version of this specification introduces support for a new {{MLDeviceType}} that can only support a subset of {{MLOperandDataType}}s, that could introduce a new fingerprint.
669
676
670
677
In general, implementers of this API are expected to apply <a href="https://gpuweb.github.io/gpuweb/#privacy-considerations">WebGPU Privacy Considerations</a> to their implementations where applicable.
671
678
@@ -951,7 +958,7 @@ Schedules the computational workload of a compiled {{MLGraph}} on the {{MLContex
951
958
**Returns:** {{undefined}}.
952
959
</div>
953
960
954
-
Note: `dispatch()` itself provides no signal that graph execution has completed. Rather, callers should await the results of reading back the output tensors. See [[#api-mlcontext-dispatch-examples]] below.
961
+
Note: `dispatch()` itself provides no signal that graph execution has completed. Rather, callers can `await` the results of reading back the output tensors. See [[#api-mlcontext-dispatch-examples]] below.
955
962
956
963
<details open algorithm>
957
964
<summary>
@@ -1108,7 +1115,7 @@ Bring-your-own-buffer variant of {{MLContext/readTensor(tensor)}}. Reads back th
1108
1115
1. Otherwise, [=queue an ML task=] with |global| and the following steps:
1109
1116
1. If |outputData| is [=BufferSource/detached=], [=reject=] |promise| with a {{TypeError}}, and abort these steps.
1110
1117
1111
-
Note: [=Validating buffer with descriptor=] above will fail if |outputData| is detached, but it's possible |outputData| may detach between then and now.
1118
+
Note: [=Validating buffer with descriptor=] above will fail if |outputData| is detached, but it is possible that |outputData| could detach between then and now.
1112
1119
1113
1120
1. [=ArrayBuffer/Write=] |bytes| to |outputData|.
1114
1121
1. [=Resolve=] |promise| with {{undefined}}.
@@ -1145,7 +1152,7 @@ Writes data to the {{MLTensor/[[data]]}} of an {{MLTensor}} on the {{MLContext}}
1145
1152
1. Return {{undefined}}.
1146
1153
</details>
1147
1154
1148
-
Note: Similar to `dispatch()`, `writeTensor()` itself provides no signal that the write has completed. To inspect the contents of a tensor, callers should await the results of reading back the tensor.
1155
+
Note: Similar to `dispatch()`, `writeTensor()` itself provides no signal that the write has completed. To inspect the contents of a tensor, callers can `await` the results of reading back the tensor.
The {{MLContext/opSupportLimits()}} exposes level of support that differs across implementations at operator level. Consumers of the WebNN API are encouraged to probe feature support level by using {{MLContext/opSupportLimits()}} to determine the optimal model architecture to be deployed for each target platform.
@@ -1325,7 +1332,7 @@ Issue(391): Should 0-size dimensions be supported?
1325
1332
1326
1333
An {{MLOperand}} represents an intermediary graph being constructed as a result of compositing parts of an operation into a fully composed operation.
1327
1334
1328
-
For instance, an {{MLOperand}}may represent a constant feeding to an operation or the result from combining multiple constants together into an operation. See also [[#programming-model]].
1335
+
For instance, an {{MLOperand}}can represent a constant feeding to an operation or the result from combining multiple constants together into an operation. See also [[#programming-model]].
The {{MLTensor}} interface represents a tensor which may be used as an input or output to an {{MLGraph}}. The memory backing an {{MLTensor}} should be allocated in an [=implementation-defined=] fashion according to the requirements of the {{MLContext}} and the {{MLTensorDescriptor}} used to create it. Operations involving the {{MLTensor/[[data]]}} of an {{MLTensor}} occur on the {{MLContext/[[timeline]]}} of its associated {{MLContext}}.
1448
1455
1449
-
Note: The [=implementation-defined=] requirements of how an {{MLTensor}} is allocated may include constraints such as that the memory is allocated with a particular byte alignment or in a particular memory pool.
1456
+
The [=implementation-defined=] requirements of how an {{MLTensor}} is allocated may include constraints such as that the memory is allocated with a particular byte alignment or in a particular memory pool.
@@ -1614,7 +1621,7 @@ Create a named {{MLOperand}} based on a descriptor, that can be used as an input
1614
1621
</details>
1615
1622
1616
1623
<div class="note">
1617
-
The {{MLGraphBuilder}} API allows creating an {{MLGraph}} without input operands. If the underlying platform doesn't support that, implementations may add a stub input, or pass constants as inputs to the graph.
1624
+
The {{MLGraphBuilder}} API allows creating an {{MLGraph}} without input operands. If the underlying platform doesn't support that, implementations can add a stub input, or pass constants as inputs to the graph.
interpreted according to the value of *options*.{{MLConvTranspose2dOptions/filterLayout}} and {{MLConvTranspose2dOptions/groups}}.
2677
2684
- <dfn>options</dfn>: an optional {{MLConvTranspose2dOptions}}.
2678
2685
2679
-
**Returns:** an {{MLOperand}}. The output 4-D tensor that contains the transposed convolution result. The output shape is interpreted according to the *options*.{{MLConvTranspose2dOptions/inputLayout}} value. More specifically, unless the *options*.{{MLConvTranspose2dOptions/outputSizes}} values are explicitly specified, the *options*.{{MLConvTranspose2dOptions/outputPadding}}may be needed to compute the spatial dimension values of the output tensor as follows:
2686
+
**Returns:** an {{MLOperand}}. The output 4-D tensor that contains the transposed convolution result. The output shape is interpreted according to the *options*.{{MLConvTranspose2dOptions/inputLayout}} value. More specifically, unless the *options*.{{MLConvTranspose2dOptions/outputSizes}} values are explicitly specified, the *options*.{{MLConvTranspose2dOptions/outputPadding}}is be needed to compute the spatial dimension values of the output tensor as follows:
The {{MLGraphBuilder/gather(input, indices, options)/indices}} parameter to {{MLGraphBuilder/gather()}} can not be clamped to the allowed range when the graph is built because the inputs are not known until execution. Implementations can introduce {{MLGraphBuilder/clamp()}} in the compiled graph if the required clamping behavior is not provided by the underlying platform. Similarly, if the underlying platform does not support negative indices, the implementation can introduce operations in the compiled graph to transform a negative index from the end of the dimension into a positive index.
3597
+
The {{MLGraphBuilder/gather(input, indices, options)/indices}} parameter to {{MLGraphBuilder/gather()}} can not be clamped to the allowed range when the graph is built because the inputs are not known until execution. Implementations can introduce {{MLGraphBuilder/clamp()}} in the compiled graph if the specified clamping behavior is not provided by the underlying platform. Similarly, if the underlying platform does not support negative indices, the implementation can introduce operations in the compiled graph to transform a negative index from the end of the dimension into a positive index.
Calculate the [general matrix multiplication of the Basic Linear Algebra Subprograms](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3). The calculation follows the expression `alpha * A * B + beta * C`, where `A` is a 2-D tensor with shape *[M, K]* or *[K, M]*, `B` is a 2-D tensor with shape *[K, N]* or *[N, K]*, and `C` is [=unidirectionally broadcastable=] to the shape *[M, N]*. `A` and `B` may optionally be transposed prior to the calculation.
3821
+
Calculate the [general matrix multiplication of the Basic Linear Algebra Subprograms](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3). The calculation follows the expression `alpha * A * B + beta * C`, where `A` is a 2-D tensor with shape *[M, K]* or *[K, M]*, `B` is a 2-D tensor with shape *[K, N]* or *[N, K]*, and `C` is [=unidirectionally broadcastable=] to the shape *[M, N]*. `A` and `B` can optionally be transposed prior to the calculation.
1. If |hiddenSize| * 6 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
4170
4177
<details class=note>
4171
4178
<summary>Why |hiddenSize| * 6 ?</summary>
4172
-
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruOptions/bias}} and {{MLGruOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| must also be a [=valid dimension=].
4179
+
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruOptions/bias}} and {{MLGruOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| also needs to be a [=valid dimension=].
4173
4180
</details>
4174
4181
1. If |options|.{{MLGruOptions/bias}}[=map/exists=]:
4175
4182
1. If its [=MLOperand/dataType=] is not one of its [=/allowed data types=] (according to [this table](#constraints-gru)), then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 6 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
4463
4470
<details class=note>
4464
4471
<summary>Why |hiddenSize| * 6 ?</summary>
4465
-
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruCellOptions/bias}} and {{MLGruCellOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| must also be a [=valid dimension=].
4472
+
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruCellOptions/bias}} and {{MLGruCellOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| also needs to be a [=valid dimension=].
4466
4473
</details>
4467
4474
1. If |options|.{{MLGruCellOptions/bias}}[=map/exists=]:
4468
4475
1. If its [=MLOperand/dataType=] is not one of its [=/allowed data types=] (according to [this table](#constraints-gruCell)), then [=exception/throw=] a {{TypeError}}.
The 3-D initial hidden state tensor of shape *[numDirections, batchSize, hiddenSize]*. When not specified, implementations SHOULD use a tensor filled with zero.
5353
+
The 3-D initial hidden state tensor of shape *[numDirections, batchSize, hiddenSize]*. When not specified, implementations must use a tensor filled with zero.
5347
5354
5348
5355
: <dfn>initialCellState</dfn>
5349
5356
::
5350
-
The 3-D initial hidden state tensor of shape *[numDirections, batchSize, hiddenSize]*. When not specified, implementations SHOULD use a tensor filled with zero.
5357
+
The 3-D initial hidden state tensor of shape *[numDirections, batchSize, hiddenSize]*. When not specified, implementations must use a tensor filled with zero.
1. If |hiddenSize| * 8 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
5490
5497
<details class=note>
5491
5498
<summary>Why |hiddenSize| * 8 ?</summary>
5492
-
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmOptions/bias}} and {{MLLstmOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| must also be a [=valid dimension=].
5499
+
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmOptions/bias}} and {{MLLstmOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| also needs to be a [=valid dimension=].
5493
5500
</details>
5494
5501
1. If |options|.{{MLLstmOptions/bias}}[=map/exists=]:
5495
5502
1. If its [=MLOperand/dataType=] is not one of its [=/allowed data types=] (according to [this table](#constraints-lstm)), then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 8 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
5845
5852
<details class=note>
5846
5853
<summary>Why |hiddenSize| * 8 ?</summary>
5847
-
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmCellOptions/bias}} and {{MLLstmCellOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| must also be a [=valid dimension=].
5854
+
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmCellOptions/bias}} and {{MLLstmCellOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| also needs to be a [=valid dimension=].
5848
5855
</details>
5849
5856
1. If |options|.{{MLLstmCellOptions/bias}}[=map/exists=]:
5850
5857
1. If its [=MLOperand/dataType=] is not one of its [=/allowed data types=] (according to [this table](#constraints-lstmCell)), then [=exception/throw=] a {{TypeError}}.
0 commit comments