Skip to content

Mutex-free TGeo material budget in Propagator for multithreaded users - #16146

Open
shahor02 wants to merge 1 commit into
AliceO2Group:devfrom
shahor02:pr_propMT
Open

shahor02 wants to merge 1 commit into
AliceO2Group:devfrom
shahor02:pr_propMT

Conversation

@shahor02

Copy link
Copy Markdown
Collaborator

Propagator::getMatBudget (TGeo branch) used the shared TGeo navigator under GeometryManager::sTGMutex. Add opt-in per-thread navigators:

  • GeometryManager::setMaxThreads(n): wraps TGeoManager::SetMaxThreads and raises the flag enabling per-thread navigators (one-way, like TGeo's).
  • GeometryManager::getThreadNavigator(): lazily books a thread_local navigator; returns nullptr if setMaxThreads() was never called or if more than n threads ask for one, in which case meanMaterialBudget(..., nullptr) falls back to the old mutexed shared-navigator path.
  • Propagator::getMatBudget passes getThreadNavigator() to GeometryManager::meanMaterialBudget.
  • MatLayerCylSet LUT build uses GeometryManager::setMaxThreads.

How to use in code calling Propagator methods from several threads:

  1. Load the geometry (GeometryManager::loadGeometry or from CCDB).
  2. Serially, BEFORE starting the worker threads, call o2::base::GeometryManager::setMaxThreads(nThreads); with nThreads >= the maximum number of threads that will call the Propagator (it walks all volumes, so do it once, not per call).
  3. Call Propagator methods (propagateToDCA, propagateTo with MatCorrType::USEMatCorrTGeo, or with the TGeo fall-back when no MatLUT) from the worker threads as usual; each thread gets its own navigator on first use and releases it on thread exit. No other change needed.

Without step 2 behaviour is unchanged (mutexed, thread-safe but serialized). Threads beyond nThreads log a warning and use the mutexed path. Using a MatLayerCylSet LUT (MatCorrType::USEMatCorrLUT) never touches TGeo.

Co-Authored-By: Claude Sonnet 5.5

@shahor02
shahor02 requested a review from sawenzel as a code owner November 17, 2026 13:57
GeometryManager::meanMaterialBudget(Ext) used the shared TGeo navigator under
sTGMutex, serializing all callers (Propagator TGeo branch, LUT building).
Add opt-in per-thread navigators:

 * GeometryManager::setMaxThreads(n): wraps TGeoManager::SetMaxThreads and
   enables per-thread navigation; returns the effective capacity.
 * GeometryManager::getThreadNavigator(): thread_local navigator, created on
   first use, removed at thread exit. nullptr if not enabled. Throws
   std::runtime_error if more threads than declared ask for one.
 * meanMaterialBudget / meanMaterialBudgetExt (and thus Propagator, unchanged)
   take the calling thread's navigator when enabled, otherwise the shared
   one under the mutex, as before. meanMaterialBudgetExt now navigates via a
   TGeoNavigator instead of the gGeoManager global API.
 * getOriginalMatrix / getOriginalMatrixFromPath navigate the same way (they
   also no longer re-lock sTGMutex when getOriginalMatrix(symname) falls back
   to getOriginalMatrixFromPath, which self-deadlocked).
 * rebuildVoxels(): the voxels, shared by all navigators and rebuilt lazily
   after TGeoPhysicalNode::Align, are rebuilt serially (also in setMaxThreads).
 * MatLayerCylSet LUT build uses the same mechanism and limits its
   parallelism to the enabled capacity.
 * Unit test testGeometryManagerMT.

How to use in code calling Propagator (or meanMaterialBudget) from threads:

  1. Load the geometry (and apply alignment).
  2. Serially, BEFORE starting the worker threads, call once
       o2::base::GeometryManager::setMaxThreads(nThreads);
     with nThreads >= the maximum number of threads which will ever use TGeo
     in the process (workers other than the calling thread).
  3. Call Propagator methods (USEMatCorrTGeo or TGeo fall-back without LUT)
     from the threads as usual: no further change needed.
  Without step 2 behaviour is unchanged (thread-safe but mutexed).
  MatCorrType::USEMatCorrLUT never touches TGeo.

Corner cases to avoid (TGeoManager is not designed for them):
  * setMaxThreads can be called effectively once per geometry: TGeo would
    reset the thread slots of threads already navigating. Repeated calls with
    n <= capacity are no-ops, a larger n is refused with an error.
  * TGeo thread slots (TGeoManager::ThreadId) are never released: every
    distinct thread which navigated keeps its slot after exit. With thread
    churn (short-lived threads) use a thread pool, otherwise slots run out and
    getThreadNavigator throws. Never silently fall back to the shared
    navigator in multithread mode: it would index the per-volume thread data
    with the calling thread's slot, possibly out of range.
  * Direct use of the gGeoManager navigation API (cd, InitTrack, GetCurrentNode,
    ...) from a worker thread is safe only after that thread has called
    getThreadNavigator() (or called setMaxThreads): GetCurrentNavigator
    returns nullptr for threads without a navigator. Prefer calling the
    methods of the navigator returned by getThreadNavigator(). Any TGeo
    navigation from a new thread consumes a slot.
  * TGeoPhysicalNode::Align after setMaxThreads: call rebuildVoxels() while no
    thread navigates, otherwise the lazy rebuild races between the threads.
  * Replacing gGeoManager disables multithreading (detected via
    IsMultiThread(), not only the pointer) and the stale navigators are
    dropped. The slot map is static in TGeo and not reset: threads which
    navigated before must not be reused and enabling again may find slots
    already consumed (warning issued).
  * setMaxThreads, rebuildVoxels and geometry replacement need all other
    threads to be idle.

Co-Authored-By: Claude Sonnet 5.5

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant