Skip to content

Commit 4404fce

Browse files
huningxinhuningxin
committed
Drop the support of synchronous execution
Remove the definition and algorithm steps for - ML.createContextSync() - MLGraphBuilder.buildSync() - MLContext.computeSync() Fix #531
1 parent 479ce17 commit 4404fce

File tree

1 file changed

+25
-165
lines changed

1 file changed

+25
-165
lines changed

‎index.bs

+25-165
Original file line numberDiff line numberDiff line change
@@ -725,24 +725,9 @@ The implementation may use views, as above, for intermediate values.
725725

726726
Before the execution, the computation graph that is used to compute one or more specified outputs needs to be compiled and optimized. The key purpose of the compilation step is to enable optimizations that span two or more operations, such as operation or loop fusion.
727727

728-
There are multiple ways by which the graph may be compiled. The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method compiles the graph in the background without blocking the calling thread, and returns a {{Promise}} that resolves to an {{MLGraph}}. The {{MLGraphBuilder}}.{{MLGraphBuilder/buildSync()}} method compiles the graph immediately on the calling thread, which must be a worker thread running on CPU or GPU device, and returns an {{MLGraph}}. Both compilation methods produce an {{MLGraph}} that represents a compiled graph for optimal execution.
728+
The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method compiles the graph in the background without blocking the calling thread, and returns a {{Promise}} that resolves to an {{MLGraph}}. The compilation step produces an {{MLGraph}} that represents a compiled graph for optimal execution.
729729

730-
Once the {{MLGraph}} is constructed, there are multiple ways by which the graph may be executed. The
731-
{{MLContext}}.{{MLContext/computeSync()}} method represents a way the execution of the graph is carried out immediately
732-
on the calling thread, which must also be a worker thread, either on a CPU or GPU device. The execution
733-
produces the results of the computation from all the inputs bound to the graph.
734-
735-
The {{MLContext}}.{{MLContext/compute()}} method represents a way the execution of the graph is performed asynchronously
736-
either on a parallel timeline in a separate worker thread for the CPU execution or on a GPU timeline in a GPU
737-
command queue. This method returns immediately without blocking the calling thread while the actual execution is
738-
offloaded to a different timeline. This type of execution is appropriate when the responsiveness of the calling
739-
thread is critical to good user experience. The computation results will be placed at the bound outputs at the
740-
time the operation is successfully completed on the offloaded timeline at which time the calling thread is
741-
signaled. This type of execution supports both the CPU and GPU device.
742-
743-
In both the {{MLContext}}.{{MLContext/compute()}} and {{MLContext}}.{{MLContext/computeSync()}} execution methods, the caller supplies
744-
the input values using {{MLNamedArrayBufferViews}}, binding the input {{MLOperand}}s to their values. The caller
745-
then supplies pre-allocated buffers for output {{MLOperand}}s using {{MLNamedArrayBufferViews}}.
730+
Once the {{MLGraph}} is constructed, the {{MLContext}}.{{MLContext/compute()}} method performs the execution of the graph asynchronously either on a parallel timeline in a separate worker thread for the CPU execution or on a GPU timeline in a GPU command queue. This method returns immediately without blocking the calling thread while the actual execution is offloaded to a different timeline. The caller supplies the input values using {{MLNamedArrayBufferViews}}, binding the input {{MLOperand}}s to their values. The caller then supplies pre-allocated buffers for output {{MLOperand}}s using {{MLNamedArrayBufferViews}}. The execution produces the results of the computation from all the inputs bound to the graph. The computation results will be placed at the bound outputs at the time the operation is successfully completed on the offloaded timeline at which time the calling thread is signaled. This type of execution supports both the CPU and GPU device.
746731

747732
The {{MLCommandEncoder}} interface created by the {{MLContext}}.{{MLContext/createCommandEncoder()}} method supports
748733
a graph execution method that provides the maximum flexibility to callers that also utilize WebGPU in their
@@ -807,11 +792,6 @@ dictionary MLContextOptions {
807792
interface ML {
808793
Promise<MLContext> createContext(optional MLContextOptions options = {});
809794
Promise<MLContext> createContext(GPUDevice gpuDevice);
810-
811-
[Exposed=(DedicatedWorker)]
812-
MLContext createContextSync(optional MLContextOptions options = {});
813-
[Exposed=(DedicatedWorker)]
814-
MLContext createContextSync(GPUDevice gpuDevice);
815795
};
816796
</script>
817797

@@ -868,30 +848,6 @@ Its <a>default allowlist</a> is <code>'self'</code>.
868848
</div>
869849
</details>
870850

871-
### {{ML/createContextSync}} ### {#api-ml-createcontextsync}
872-
873-
<details open algorithm>
874-
<summary>
875-
The <dfn method for=ML>createContextSync(|options|)</dfn> method steps are:
876-
</summary>
877-
<div class=algorithm-steps>
878-
1. If [=this=]'s [=relevant global object=]'s [=associated Document=] is not [=allowed to use=] the [=webnn-feature|webnn=] feature, then [=exception/throw=] a "{{SecurityError}}" {{DOMException}}.
879-
1. Let |context| be the result of [=creating a context=] with |options|. If that returns failure, then [=exception/throw=] a "{{NotSupportedError}}" {{DOMException}}.
880-
1. Return |context|.
881-
</div>
882-
</details>
883-
884-
<details open algorithm>
885-
<summary>
886-
The <dfn method for=ML>createContextSync(|gpuDevice|)</dfn> method steps are:
887-
</summary>
888-
<div class=algorithm-steps>
889-
1. If [=this=]'s [=relevant global object=]'s [=associated Document=] is not [=allowed to use=] the [=webnn-feature|webnn=] feature, then [=exception/throw=] a "{{SecurityError}}" {{DOMException}}.
890-
1. Let |context| be the result of [=creating a context=] with |gpuDevice|. If that returns failure, then [=exception/throw=] a "{{NotSupportedError}}" {{DOMException}}.
891-
1. Return |context|.
892-
</div>
893-
</details>
894-
895851
## {{MLActivation}} interface ## {#api-mlactivation}
896852

897853
Objects implementing the {{MLActivation}} interface represent activation function types.
@@ -1132,40 +1088,6 @@ interface MLContext {};
11321088
When the {{[[contextType]]}} is set to [=context type/default=] with the {{MLContextOptions}}.{{deviceType}} set to [=device type/gpu=], the user agent is responsible for creating an internal GPU device that operates within the context and is capable of ML workload submission on behalf of the calling application. In this setting however, only {{ArrayBufferView}} inputs and outputs are allowed in and out of the graph execution since the application has no way to know what type of internal GPU device is being created on their behalf. In this case, the user agent is responsible for automatic uploads and downloads of the inputs and outputs to and from the GPU memory using this said internal device.
11331089
</div>
11341090

1135-
### Synchronous Execution ### {#api-mlcontext-sync-execution}
1136-
Synchronously carries out the computational workload of a compiled graph {{MLGraph}} on the calling thread, which must be a worker thread, to produce results as defined by the operations in the graph. This method of execution requires an {{MLContext}} created with {{MLContextOptions}}. Otherwise, it [=exception/throws=] an "{{OperationError}}" {{DOMException}}.
1137-
1138-
<script type=idl>
1139-
partial interface MLContext {
1140-
[Exposed=(DedicatedWorker)]
1141-
undefined computeSync(
1142-
MLGraph graph, MLNamedArrayBufferViews inputs, MLNamedArrayBufferViews outputs);
1143-
};
1144-
</script>
1145-
1146-
<div>
1147-
**Arguments:**
1148-
- *graph*: an {{MLGraph}}. The compiled graph to be executed.
1149-
- *inputs*: an {{MLNamedArrayBufferViews}}. The resources of inputs.
1150-
- *outputs*: an {{MLNamedArrayBufferViews}}. The pre-allocated resources of required outputs.
1151-
1152-
**Returns:** {{undefined}}.
1153-
</div>
1154-
1155-
<details open algorithm>
1156-
<summary>
1157-
The <dfn method for=MLContext>computeSync(|graph|, |inputs|, |outputs|)</dfn> method steps are:
1158-
</summary>
1159-
<div class=algorithm-steps>
1160-
1. If |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[contextType]]}} is not "[=context type/default=]", [=exception/throw=] an "{{OperationError}}" {{DOMException}}.
1161-
1. If [=validating graph resources=] given |inputs| and |graph|.{{MLGraph/[[inputDescriptors]]}} returns false, then [=exception/throw=] a "{{DataError}}" {{DOMException}}.
1162-
1. If [=validating graph resources=] given |outputs| and |graph|.{{MLGraph/[[outputDescriptors]]}} returns false, then [=exception/throw=] a "{{DataError}}" {{DOMException}}.
1163-
1. Invoke [=execute graph=] given |graph|, |inputs| and |outputs|.
1164-
1. If that [=exception/throws=] an error, re-[=exception/throw=] the error.
1165-
1. Return {{undefined}}.
1166-
</div>
1167-
</details>
1168-
11691091
<details open algorithm>
11701092
<summary>
11711093
To <dfn>validate graph resources</dfn>, given |resources| and |descriptors|, run the following steps:
@@ -1213,46 +1135,6 @@ partial interface MLContext {
12131135
</div>
12141136
</details>
12151137

1216-
#### Examples #### {#api-mlcontext-sync-execution-examples}
1217-
1218-
<div class="example">
1219-
<details open>
1220-
<summary>
1221-
The following code showcases the synchronous computation with optional outputs in a worker.
1222-
</summary>
1223-
<pre highlight="js">
1224-
const context = navigator.ml.createContextSync();
1225-
1226-
// Build a graph with two outputs.
1227-
const builder = new MLGraphBuilder(context);
1228-
const descA = {dataType: 'float32', dimensions: [3, 4]};
1229-
const a = builder.input('a', descA);
1230-
const descB = {dataType: 'float32', dimensions: [4, 3]};
1231-
const bufferB = new Float32Array(sizeOfShape(descB.dimensions)).fill(0.5);
1232-
const b = builder.constant(descB, bufferB);
1233-
const descC = {dataType: 'float32', dimensions: [3, 3]};
1234-
const bufferC = new Float32Array(sizeOfShape(descC.dimensions)).fill(1);
1235-
const c = builder.constant(descC, bufferC);
1236-
const d = builder.matmul(a, b);
1237-
const e = builder.add(d, c);
1238-
const graph = builder.buildSync({'d': d, 'e': e});
1239-
1240-
const bufferA = new Float32Array(sizeOfShape(descA.dimensions)).fill(0.5);
1241-
const inputs = {'a': bufferA};
1242-
1243-
// Compute d.
1244-
const bufferD = new Float32Array(sizeOfShape([3, 3]));
1245-
context.computeSync(graph, inputs, {'d': bufferD});
1246-
console.log(&#96;values: ${bufferD}&#96;);
1247-
1248-
// Compute e.
1249-
const bufferE = new Float32Array(sizeOfShape([3, 3]));
1250-
context.computeSync(graph, inputs, {'e': bufferE});
1251-
console.log(&#96;values: ${bufferE}&#96;);
1252-
</pre>
1253-
</details>
1254-
</div>
1255-
12561138
### {{MLNamedArrayBufferViews}} transfer algorithm ### {#mlnamedarraybufferviews-transfer-alg}
12571139

12581140
<details open algorithm>
@@ -1426,15 +1308,11 @@ interface MLGraphBuilder {
14261308

14271309
// Compile the graph up to the specified output operands asynchronously.
14281310
Promise<MLGraph> build(MLNamedOperands outputs);
1429-
1430-
// Compile the graph up to the specified output operands synchronously.
1431-
[Exposed=(DedicatedWorker)]
1432-
MLGraph buildSync(MLNamedOperands outputs);
14331311
};
14341312
</script>
14351313

14361314
<div class="note">
1437-
Both {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} and {{MLGraphBuilder}}.{{MLGraphBuilder/buildSync()}} methods compile the graph builder state up to the specified output operands into a compiled graph according to the type of {{MLContext}} that creates it. Since this operation can be costly in some machine configurations, the calling thread of the {{MLGraphBuilder}}.{{MLGraphBuilder/buildSync()}} method must only be a worker thread to avoid potential disruption of the user experience. When the {{[[contextType]]}} of the {{MLContext}} is set to "[=context type/default=]", the compiled graph is initialized right before the {{MLGraph}} is returned. This graph initialization stage is important for optimal performance of the subsequent graph executions. See [[#api-mlcommandencoder-graph-initialization]] for more detail.
1315+
The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method compiles the graph builder state up to the specified output operands into a compiled graph according to the type of {{MLContext}} that creates it. When the {{[[contextType]]}} of the {{MLContext}} is set to "[=context type/default=]", the compiled graph is initialized right before the {{MLGraph}} is returned. This graph initialization stage is important for optimal performance of the subsequent graph executions. See [[#api-mlcommandencoder-graph-initialization]] for more detail.
14381316
</div>
14391317

14401318
{{MLBufferResourceView}} has the following members:
@@ -1653,7 +1531,7 @@ partial interface MLGraphBuilder {
16531531
</div>
16541532

16551533
### build ### {#api-mlgraphbuilder-build}
1656-
Build a composed graph up to a given output operand into a computational graph, asynchronously or synchronously.
1534+
Build a composed graph up to a given output operand into a computational graph asynchronously.
16571535

16581536
#### {{MLGraphBuilder/build(outputs)}} #### {#api-mlgraphbuilder-build-outputs}
16591537

@@ -1662,46 +1540,28 @@ Build a composed graph up to a given output operand into a computational graph,
16621540
The <dfn method for=MLGraphBuilder>build(|outputs|)</dfn> method steps are:
16631541
</summary>
16641542
<div class=algorithm-steps>
1665-
<div class="note">
1666-
The permissions and context validity have been checked by [[#api-mlgraphbuilder-constructor]] steps.
1667-
</div>
16681543
1. Let |promise| be [=a new promise=].
1669-
1. Return |promise| and run the following steps [=in parallel=].
1670-
1. Return the result of invoking {{MLGraphBuilder/buildSync(outputs)}} given |outputs|.
1671-
1. If that [=exception/throws=], re-[=exception/throw=] the error.
1672-
</div>
1673-
</details>
1674-
1675-
#### {{MLGraphBuilder/buildSync(outputs)}} #### {#api-mlgraphbuilder-buildsync-outputs}
1676-
1677-
<details open algorithm>
1678-
<summary>
1679-
The <dfn method for=MLGraphBuilder>buildSync(|outputs|)</dfn> method steps are:
1680-
</summary>
1681-
<div class=algorithm-steps>
1682-
<div class="note">
1683-
The permissions and context validity have been checked by [[#api-mlgraphbuilder-constructor]] steps.
1684-
</div>
1685-
1. If |outputs| is empty, then [=exception/throw=] a {{TypeError}}.
1686-
1. [=map/For each=] |name| &rarr; |operand| of |outputs|:
1687-
1. If |name| is empty, then [=exception/throw=] a {{TypeError}}.
1688-
1. If any of the following sub-steps fail, [=exception/throw=] an "{{OperationError}}" {{DOMException}}.
1689-
1. Let |graph| be a new {{MLGraph}}:
1690-
1. Set |graph|.{{MLGraph/[[context]]}} to [=this=].{{MLGraphBuilder/[[context]]}}.
1691-
1. Make a request to the underlying platform to:
1692-
1. Connect |graph| to a new [=implementation-defined=] graph implementation |graphImpl| given |graph|.
1693-
1. Set |graph|.{{MLGraph/[[implementation]]}} to |graphImpl|.
1694-
1. Make a request to the underlying platform to initialize the graph:
1695-
1. [=map/For each=] |name| &rarr; |operand| of |outputs|:
1696-
1. If [=validating MLOperand=] given |operand| and [=this=] returns false, then [=exception/throw=] a {{TypeError}}.
1697-
1. If |operand| was created as an input by the underlying platform:
1698-
1. If |operand|.{{MLOperand/[[name]]}}] is not unique for |graphImpl|, then [=exception/throw=] a {{TypeError}}.
1699-
1. Add |operand|.{{MLOperand/[[descriptor]]}} to |graph|.{{MLGraph/[[inputDescriptors]]}}[|operand|.{{MLOperand/[[name]]}}].
1700-
1. If |operand| was created as a constant by the underlying platform:
1701-
1. Implementations MAY preprocess and optimize the tensor data of |operand| for the underlying platform.
1702-
1. Register |operand|.{{MLOperand/[[operand]]}} in |graphImpl| as graph output.
1703-
1. Register |operand|.{{MLOperand/[[operator]]}} to |graphImpl|.
1704-
1. Return |graph|.
1544+
1. Return |promise| and run the following steps [=in parallel=]:
1545+
1. If |outputs| is empty, then [=reject=] |promise| with a "{{TypeError}}" {{DOMException}}.
1546+
1. [=map/For each=] |name| &rarr; |operand| of |outputs|:
1547+
1. If |name| is empty, then [=reject=] |promise| with a "{{TypeError}}" {{DOMException}}.
1548+
1. If any of the following sub-steps fail, then [=reject=] |promise| with an "{{OperationError}}" {{DOMException}}.
1549+
1. Let |graph| be a new {{MLGraph}}:
1550+
1. Set |graph|.{{MLGraph/[[context]]}} to [=this=].{{MLGraphBuilder/[[context]]}}.
1551+
1. Make a request to the underlying platform to:
1552+
1. Connect |graph| to a new [=implementation-defined=] graph implementation |graphImpl| given |graph|.
1553+
1. Set |graph|.{{MLGraph/[[implementation]]}} to |graphImpl|.
1554+
1. Make a request to the underlying platform to initialize the graph:
1555+
1. [=map/For each=] |name| &rarr; |operand| of |outputs|:
1556+
1. If [=validating MLOperand=] given |operand| and [=this=] returns false, then [=reject=] |promise| with a "{{TypeError}}" {{DOMException}}.
1557+
1. If |operand| was created as an input by the underlying platform:
1558+
1. If |operand|.{{MLOperand/[[name]]}} is not unique for |graphImpl|, then [=reject=] |promise| with a "{{TypeError}}" {{DOMException}}.
1559+
1. Add |operand|.{{MLOperand/[[descriptor]]}} to |graph|.{{MLGraph/[[inputDescriptors]]}}[|operand|.{{MLOperand/[[name]]}}].
1560+
1. If |operand| was created as a constant by the underlying platform:
1561+
1. Implementations MAY preprocess and optimize the tensor data of |operand| for the underlying platform.
1562+
1. Register |operand|.{{MLOperand/[[operand]]}} in |graphImpl| as graph output.
1563+
1. Register |operand|.{{MLOperand/[[operator]]}} to |graphImpl|.
1564+
1. [=Resolve=] |promise| with |graph|.
17051565
</div>
17061566
</details>
17071567

0 commit comments

Comments
 (0)