TAC topic request -- thread pools
|
Larry Gritz
If we have time, can we add to the agenda some discussion about thread pools? In some side conversations with SESI, we're realizing that there is a proliferation of thread pools in our libraries (somewhat mirroring the proliferation of vector libraries) that can cause problems. It wouldn't be hard for an app using OIIO, OpenEXR (either directly or transitively through OIIO), OpenVDB, plus the app uses its own thread pool -- now you have at least four thread pools active in the app, each unaware of the others. That's a ton of threads and no coordination or smart resource allocation among them. Subtopics: (a) Is this enough of a problem to call for some coordinated action? (b) With TBB named in VFX Platform, is it nowadays a common enough dependency that VFX open source packages should all be encouraged to adopt it rather than rolling their own (as a build-time option at least)? (c) Should the projects coordinate on a simple API to make it possible to communicate at runtime a pointer to the app's single shared thread pool to each library to use, and only fall back on its internal and/or homegrown pool if none is supplied? (d) Should ASWF and ASWF-adjacent projects be strongly encouraged (if not required) to do b or c? It used to be that there were only a few open source projects, and a given app was unlikely to use more than one or two simultaneously, so there was little penalty from reinvention of the wheel and little benefit to coordinating much across multiple projects. But now, it's hard to imagine any application of substance that doesn't pull in a whole bunch of these projects, so this kind of incompatible replication (and also my other favourite topic, vector libraries) is getting to be a big pain. ASWF is in a good position to help coordinate some of these ecosystem-wide issues that span multiple projects. Larry Gritz lg@... |
|
|