Disabling threading completely for prediction (in C++)

[mike-jn]

Hi,

My application is single-threaded and pinned to a core and I would like to make sure that the library does not create any threads and runs in the context of the caller. Other than compiling with OMP support disabled, is there anything else I need to do so that LGBM_BoosterPredictForMatSingleRowFast() runs in the context of the caller and does not dispatch work to any other threads?

Thanks!

[jameslamb]

Thanks for using LightGBM.

The most reliable way to do what you've asked is build without OpenMP enabled, as you mentioned. That'll have the nice benefit of also creating a library that is smaller and more portable.

cmake -B build -S . -DUSE_OPENMP=OFF

# or '--target lightgbm' for the CLI
cmake --build build --target _lightgbm

I don't think there are other sources of thread creation in LGBM_BoosterPredictForMatSingleRowFast() besides those from OpenMP (e.g. no direct calls to pthread_create() or creation of std::thread).

---

If using a build of LightGBM with OpenMP support, you can pass num_threads=1 through parameter when you call LGBM_BoosterPredictForMatSingleRowFastInit().

LightGBM/src/c_api.cpp

Lines 2546 to 2553 in 768f642

	int LGBM_BoosterPredictForMatSingleRowFastInit(BoosterHandle handle,
	const int predict_type,
	const int start_iteration,
	const int num_iteration,
	const int data_type,
	const int32_t ncol,
	const char* parameter,
	FastConfigHandle *out_fastConfig) {

Docs on that parameter: https://lightgbm.readthedocs.io/en/latest/Parameters.html#num_threads

However, I'm not 100% sure if that setting num_threads=1 will mean "the library does not create any threads"... that might be implementation-dependent, based on which OpenMP implementation is being linked into your program.

---

If there is an underlying performance problem or other issue that's leading to this question, we'd appreciate a minimal, reproducible example demonstrating that problem. That might help us to provide a more useful answer.

[mike-jn]

Thank you for such a detailed response! ❤

I will go with building without OMP then.

[jameslamb]

Sure, happy to help! We'll close this for now, comment in the future if you have other questions related to this topic (or open new issues if you have new questions.

Thanks again for using LightGBM!