Skip to content

Commit 3ba10e9

Browse files
TedThemistokleousPrathik Rao
authored and
Prathik Rao
committed
Fixes to get stable diffusion benchmark running (#15755)
### Description Added changes to MIGraphX EP to suppoert stable diffusion 1. Added parameterized input dimensions to not trigger a precompile to set input parameters in the EP 2. Removed input checking for Resize operator in EP as MIGraphX already performs these checks 3. Add support to benchmark script to use the MIGraphX execution provider 4. Add support for an odd valued batch size (3) that was seen on other benchmarks we were performing comparison on. ### Motivation and Context These changes are required to get stable diffusion mdoels to run on MIGraphX through the EP. Without these changes we see the following incorrect behavior. 1. Resize operators are pushed onto the CPU EP instead of MIGraphX, causing a significant slowdown during runs 2. Precompile operations incorrectly parse input_ids parameter for our text model, with a 1, which breaks during MIGraphX Compile of onnx. This in turn throws an error and stops any setup before inference. 3. Selecting the correct EP in the benchmark script which was previously missing the MIGraphX option 5. Suppressed an error we keep seeing with pthread_set_affinity - this is a quality of life change when using the MIGraphX EP This was testing with the benchmark.py script using stable diffusion v2 located in onnxruntime/onnxruntime/python/tools/transformers/models/stable_diffusion/ --------- Co-authored-by: Ted Themistokleous <[email protected]>
1 parent fe1ab4f commit 3ba10e9

File tree

3 files changed

+14
-13
lines changed

3 files changed

+14
-13
lines changed

onnxruntime/core/platform/posix/env.cc

+2
Original file line numberDiff line numberDiff line change
@@ -248,11 +248,13 @@ class PosixThread : public EnvThread {
248248
<< ", mask: " << *p->affinity;
249249
} else {
250250
auto [err_no, err_msg] = GetSystemError(ret);
251+
#if !defined(USE_MIGRAPHX)
251252
LOGS_DEFAULT(ERROR) << "pthread_setaffinity_np failed for thread: " << syscall(SYS_gettid)
252253
<< ", index: " << p->index
253254
<< ", mask: " << *p->affinity
254255
<< ", error code: " << err_no << " error msg: " << err_msg
255256
<< ". Specify the number of threads explicitly so the affinity is not set.";
257+
#endif
256258
}
257259
}
258260
#endif

onnxruntime/core/providers/migraphx/migraphx_execution_provider.cc

+10-12
Original file line numberDiff line numberDiff line change
@@ -451,15 +451,6 @@ static bool IsUnsupportedOpMode(const onnxruntime::GraphViewer& graph_viewer, co
451451
}
452452
}
453453

454-
const auto& args = node->InputDefs();
455-
if (args.size() > 1) {
456-
std::vector<std::size_t> indices(args.size() - 1);
457-
std::iota(indices.begin(), indices.end(), 1);
458-
if (canEvalNodeArgument(graph_viewer, node, indices, input_nodes)) {
459-
return false;
460-
}
461-
return true;
462-
}
463454
} else if (optype == "ReduceSum") {
464455
const auto& args = node->InputDefs();
465456
if (args.size() == 2) {
@@ -952,8 +943,15 @@ bool get_input_output_names(const GraphViewer& graph,
952943
if (sptr == nullptr)
953944
return true;
954945

955-
auto dim_size = sptr->dim_size();
956-
return (dim_size == 0);
946+
if (sptr->dim_size() == 0)
947+
return true;
948+
949+
for (int i = 0; i < sptr->dim_size(); i++) {
950+
if (sptr->dim(i).has_dim_param())
951+
return true;
952+
}
953+
954+
return false;
957955
});
958956

959957
const auto& out_args = graph.GetOutputs();
@@ -1002,7 +1000,7 @@ Status MIGraphXExecutionProvider::Compile(const std::vector<FusedNodeAndGraph>&
10021000
}
10031001

10041002
std::vector<std::string> input_names, output_names;
1005-
no_input_shape = no_input_shape or get_input_output_names(graph_body_viewer, input_names, output_names);
1003+
no_input_shape = get_input_output_names(graph_body_viewer, input_names, output_names);
10061004

10071005
// by parsing the model_proto, create a program corresponding to
10081006
// the input fused_node

onnxruntime/python/tools/transformers/models/stable_diffusion/benchmark.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
PROVIDERS = {
2020
"cuda": "CUDAExecutionProvider",
2121
"rocm": "ROCMExecutionProvider",
22+
"migraphx": "MIGraphXExecutionProvider",
2223
}
2324

2425

@@ -570,7 +571,7 @@ def parse_arguments():
570571
"--batch_size",
571572
type=int,
572573
default=1,
573-
choices=[1, 2, 4, 8, 10, 16, 32],
574+
choices=[1, 2, 3, 4, 8, 10, 16, 32],
574575
help="Number of images per batch. Default is 1.",
575576
)
576577

0 commit comments

Comments
 (0)