diff --git a/docs/ARCHITECTURE_DETAILS.md b/docs/ARCHITECTURE_DETAILS.md new file mode 100644 index 0000000000..aa8889078b --- /dev/null +++ b/docs/ARCHITECTURE_DETAILS.md @@ -0,0 +1,456 @@ +--- +doc_id: "THR-ARCH-003b" +doc_title: "Thread System - Architecture Details" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "ARCH" +--- + +# Thread System - Architecture Details + +> **SSOT**: This document is the single source of truth for **Thread System - Architecture Details** (component deep-dives). + +> **See also**: [Architecture Overview](ARCHITECTURE_OVERVIEW.md) for high-level system architecture diagrams. + +## 4. Queue Capabilities Architecture + +This diagram shows the interface hierarchy for queue implementations after Phase 1-5 enhancements. + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Queue Architecture (v2.0) │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐│ +│ │ scheduler_interface (UNCHANGED) ││ +│ │ + schedule(job) -> result_void ││ +│ │ + get_next_job() -> result> ││ +│ │ + has_pending() -> bool ││ +│ └────────────────────────────┬────────────────────────────────┘│ +│ │ │ +│ ┌────────────────────────────┴────────────────────────────────┐│ +│ │ queue_capabilities_interface (NEW - MIXIN) ││ +│ │ + get_capabilities() -> queue_capabilities ││ +│ │ + has_exact_size() / has_atomic_empty() / is_lock_free() ││ +│ │ + supports_batch() / supports_blocking_wait() ││ +│ └────────────────────────────┬────────────────────────────────┘│ +│ │ │ +│ ┌────────────────────┼────────────────────┐ │ +│ │ │ │ │ +│ ▼ ▼ ▼ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ job_queue │ │ lockfree_ │ │ adaptive_ │ │ +│ │ (UNCHANGED) │ │ job_queue │ │ job_queue │ │ +│ │ │ │ (EXTENDED) │ │ (NEW) │ │ +│ ├─────────────┤ ├─────────────┤ ├─────────────┤ │ +│ │ exact_size │ │ lock_free │ │ auto-tune │ │ +│ │ ~300K ops/s │ │ ~1.2M ops/s │ │ best-of-both│ │ +│ │ batch: yes │ │ batch: no │ │ policy-based│ │ +│ │ blocking:yes│ │ blocking:no │ │ RAII guard │ │ +│ └─────────────┘ └─────────────┘ └─────────────┘ │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐│ +│ │ queue_factory (NEW - OPTIONAL) ││ +│ │ + create_standard_queue() -> shared_ptr ││ +│ │ + create_lockfree_queue() -> unique_ptr││ +│ │ + create_adaptive_queue() -> unique_ptr││ +│ │ + create_for_requirements(reqs) -> unique_ptr ││ +│ │ + create_optimal() -> unique_ptr ││ +│ └─────────────────────────────────────────────────────────────┘│ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐│ +│ │ Compile-Time Type Selection (NEW) ││ +│ │ ││ +│ │ queue_t → job_queue (exact size) ││ +│ │ queue_t → lockfree_job_queue (fast) ││ +│ │ queue_t → adaptive_job_queue (balanced) ││ +│ │ ││ +│ │ Type Aliases: ││ +│ │ - accurate_queue_t = queue_t ││ +│ │ - fast_queue_t = queue_t ││ +│ │ - balanced_queue_t = queue_t ││ +│ └─────────────────────────────────────────────────────────────┘│ +│ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Key Points:** +- All existing queue classes remain backward compatible +- `queue_capabilities_interface` is a mixin (additive inheritance) +- `queue_factory` is optional utility for convenient queue creation +- Compile-time selection provides zero runtime overhead + +## 5. Queue Implementation Strategy Comparison + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ QUEUE IMPLEMENTATIONS │ +├──────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ job_queue (Mutex-Based FIFO) │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ ENQUEUE DEQUEUE │ │ +│ │ ┌─────────────────┐ ┌──────────────────┐ │ │ +│ │ │ lock_guard │ │ lock_guard │ │ │ +│ │ │ queue.push(job) │ │ if queue.empty │ │ │ +│ │ │ notify_one() │ │ wait(cond_var) │ │ │ +│ │ │ unlock │ │ job = queue.pop()│ │ │ +│ │ └─────────────────┘ │ unlock │ │ │ +│ │ │ return job │ │ │ +│ │ ✅ Simple, reliable └──────────────────┘ │ │ +│ │ ✅ Baseline performance │ │ +│ │ ⚠️ Contention under high load │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ lockfree_job_queue (Michael-Scott + Hazard Pointers) │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ ALGORITHM: Michael-Scott Queue (1996) │ │ +│ │ MEMORY: Hazard Pointers (Michael, 2004) │ │ +│ │ │ │ +│ │ ENQUEUE (wait-free) DEQUEUE (lock-free) │ │ +│ │ ┌──────────────────┐ ┌──────────────────┐ │ │ +│ │ │ create node │ │ get hazard ptr │ │ │ +│ │ │ CAS tail.next │ │ read head │ │ │ +│ │ │ CAS tail pointer │ │ CAS head pointer │ │ │ +│ │ │ no locks! │ │ retire old node │ │ │ +│ │ └──────────────────┘ │ return job │ │ │ +│ │ └──────────────────┘ │ │ +│ │ MEMORY RECLAMATION: │ │ +│ │ • Per-thread hazard list (MAX 4) │ │ +│ │ • Global hazard registry │ │ +│ │ • Scan all threads before deletion │ │ +│ │ │ │ +│ │ ✅ 4x faster (71 μs vs 291 μs per op) │ │ +│ │ ✅ Production-safe (no TLS bugs) │ │ +│ │ ✅ Zero contention overhead │ │ +│ │ ⚠️ ~256 bytes per thread │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ adaptive_job_queue (Auto-Switching Strategy) │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ MONITOR: SELECT STRATEGY: │ │ +│ │ • Contention ratio ──┐ │ │ +│ │ • Operation latency ├──▶ Low contention: │ │ +│ │ • Operation count │ Use Mutex (simpler) │ │ +│ │ │ │ │ +│ │ └──▶ High contention: │ │ +│ │ Use Lock-Free (faster) │ │ +│ │ │ │ +│ │ ✅ Up to 7.7x improvement under contention │ │ +│ │ ✅ Automatic optimization │ │ +│ │ ✅ Zero configuration │ │ +│ │ ✅ Best of both worlds │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ bounded_job_queue (Ring Buffer with Size Limit) │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ • Fixed-size ring buffer │ │ +│ │ • Memory-bounded (predictable) │ │ +│ │ • Wraparound on full │ │ +│ │ │ │ +│ │ ✅ Memory-predictable │ │ +│ │ ⚠️ Bounded capacity (enqueue may fail) │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +## 6. Type-Based Thread Pool Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ TYPED THREAD POOL (Type-Routed Scheduling) │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ APPLICATION LAYER │ +│ ┌──────────────────────┐ ┌──────────────────────┐ │ +│ │ enqueue(RealTime job)│ │ enqueue(Batch job) │ │ +│ └──────────┬───────────┘ └──────────┬───────────┘ │ +│ │ │ │ +│ ┌────────▼─────────────────────────▼────────┐ │ +│ │ TYPE-BASED JOB DISPATCH │ │ +│ │ (Enqueue-time decision) │ │ +│ └────┬─────────────┬───────────┬─────────┘ │ +│ │ │ │ │ +│ ┌────▼────┐ ┌────▼────┐ ┌──▼────┐ │ +│ │RealTime │ │ Normal │ │Background │ +│ │Queue │ │ Queue │ │Queue │ │ +│ │(FIFO) │ │ (FIFO) │ │(FIFO) │ │ +│ └────┬────┘ └────┬────┘ └──┬────┘ │ +│ │ │ │ │ +│ ┌────▼─────────────▼──────────▼────┐ │ +│ │ WORKER ASSIGNMENT │ │ +│ │ (Type-Aware Pool Partitioning) │ │ +│ └────┬─────────────┬────────────────┘ │ +│ │ │ │ │ +│ ┌────▼────┐ ┌────▼────┐ ┌──▼────────┐ │ +│ │Workers │ │Workers │ │Workers │ │ +│ │Set A │ │Set B │ │Set C │ │ +│ │(R.Time) │ │(Normal) │ │(Background │ +│ │ │ │ │ │) │ │ +│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ +│ │ │W₁ │ │ │ │W₄ │ │ │ │W₇ │ │ │ +│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ +│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ +│ │ │W₂ │ │ │ │W₅ │ │ │ │W₈ │ │ │ +│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ +│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ +│ │ │W₃ │ │ │ │W₆ │ │ │ │W₉ │ │ │ +│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ +│ │ │ │ │ │ │ │ +│ └─────────┘ └─────────┘ └───────────┘ │ +│ │ +│ BENEFITS: │ +│ ✅ Type-accuracy >99% under all conditions │ +│ ✅ Per-type FIFO ordering preserved │ +│ ✅ Optimal resource allocation │ +│ ✅ Priority-aware scheduling │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## 7. Hazard Pointer Memory Reclamation Flow + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ HAZARD POINTER MEMORY RECLAMATION MECHANISM │ +├──────────────────────────────────────────────────────────────────┤ +│ │ +│ THREAD 1 THREAD 2 THREAD 3 │ +│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ +│ │Hazard List│ │Hazard List│ │Hazard List│ │ +│ │ │ │ │ │ │ │ +│ │ [ptr→node]│ │ [NULL] │ │ [NULL] │ │ +│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ +│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ +│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ +│ │ │ │ │ │ │ │ +│ │ active=T │ │ active=T │ │ active=T │ │ +│ └───────┬───┘ └────┬──────┘ └────┬──────┘ │ +│ │ │ │ │ +│ └────────────────┼──────────────────┘ │ +│ │ │ +│ ┌─────────▼─────────┐ │ +│ │Global Hazard Ptr │ │ +│ │ Registry │ │ +│ │ │ │ +│ │ Head ──▶ [T1] ──▶ │ │ +│ │ ↓ │ │ +│ │ [T2] ──▶ │ │ +│ │ ↓ │ │ +│ │ [T3] │ │ +│ │ ↓ │ │ +│ │ NULL │ │ +│ └────────┬──────────┘ │ +│ │ │ +│ ┌───────────────────────┴────────────────────────┐ │ +│ │ DELETION PROCESS │ │ +│ ├─────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ 1. Mark node for retirement: │ │ +│ │ retire_list.push(node_ptr) │ │ +│ │ │ │ +│ │ 2. Scan all thread hazard lists: │ │ +│ │ protected = scan_hazards() │ │ +│ │ (Check: is node in any thread's list?) │ │ +│ │ │ │ +│ │ 3. Reclaim if safe: │ │ +│ │ for (auto node : retire_list) { │ │ +│ │ if (node not in protected) { │ │ +│ │ delete node; // SAFE TO DELETE │ │ +│ │ } else { │ │ +│ │ defer_delete(node); // Try later │ │ +│ │ } │ │ +│ │ } │ │ +│ │ │ │ +│ │ 4. Thread cleanup: │ │ +│ │ When thread exits: │ │ +│ │ mark_inactive() // Sets active=false │ │ +│ │ (TLS destructor no longer blocks delete) │ │ +│ │ │ │ +│ └────────────────────────────────────────────────┘ │ +│ │ +│ KEY BENEFITS: │ +│ ✅ True ABA prevention │ +│ ✅ No use-after-free │ +│ ✅ No memory leaks │ +│ ✅ Safe TLS destruction (no ordering dependency) │ +│ ✅ Production-safe │ +│ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +## 8. Cancellation Token Hierarchy + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ CANCELLATION TOKEN HIERARCHY & PROPAGATION │ +├──────────────────────────────────────────────────────────────────┤ +│ │ +│ SCOPE 1: System-level Shutdown │ +│ ┌──────────────────────────┐ │ +│ │ system_shutdown_token │ │ +│ │ │ │ +│ │ is_cancelled() = false │ │ +│ │ [callbacks = {}] │ │ +│ └──────────┬───────────────┘ │ +│ │ │ +│ │ (create_linked) │ +│ │ │ +│ ┌──────┴──────┬─────────────┬─────────────┐ │ +│ │ │ │ │ │ +│ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐ │ +│ │Token 1 │ │Token 2 │ │Token 3 │ │Token 4 │ │ +│ │(DB Op) │ │(Network│ │(Compute) │(Cache) │ │ +│ │ │ │ Op) │ │ │ │ │ │ +│ │callback│ │callback│ │callback │callback │ │ +│ │cleanup │ │cleanup │ │cleanup │ │cleanup │ │ +│ │database│ │conns │ │threads │ │entries │ │ +│ └────────┘ └────────┘ └────────┘ └────────┘ │ +│ │ +│ PROPAGATION FLOW: │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ User: system_shutdown_token.cancel() │ │ +│ │ │ │ │ +│ │ └─▶ set atomic flag │ │ +│ │ │ │ │ +│ │ └─▶ notify() on all registered callbacks │ │ +│ │ │ │ │ +│ │ ├─▶ Token 1: cleanup() │ │ +│ │ │ └─▶ close DB connections │ │ +│ │ │ │ │ +│ │ ├─▶ Token 2: cleanup() │ │ +│ │ │ └─▶ close network sockets │ │ +│ │ │ │ │ +│ │ ├─▶ Token 3: cleanup() │ │ +│ │ │ └─▶ terminate worker threads │ │ +│ │ │ │ │ +│ │ └─▶ Token 4: cleanup() │ │ +│ │ └─▶ flush cache entries │ │ +│ │ │ │ +│ │ All linked tokens observe is_cancelled() = true │ │ +│ │ │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ │ +│ WEAK POINTER SAFETY: │ +│ • Token stores shared_ptr │ +│ • Callbacks store weak_ptr │ +│ ✅ Prevents circular reference cycles │ +│ ✅ Safe automatic cleanup │ +│ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +## 9. Error Handling Flow + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ RESULT ERROR HANDLING PATTERN │ +├──────────────────────────────────────────────────────────────────┤ +│ │ +│ OPERATION WITH ERROR HANDLING: │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ result compute_with_error() { │ │ +│ │ // Try operation │ │ +│ │ if (invalid_input) { │ │ +│ │ return error{ │ │ +│ │ error_code::invalid_argument, │ │ +│ │ "Input value out of range" │ │ +│ │ }; │ │ +│ │ } │ │ +│ │ │ │ +│ │ // Success │ │ +│ │ return result(42); │ │ +│ │ } │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ │ +│ CONSUMING THE RESULT: │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ auto result = compute_with_error(); │ │ +│ │ │ │ +│ │ if (result.is_success()) { │ │ +│ │ int value = result.value(); // Use value │ │ +│ │ } else { │ │ +│ │ auto err = result.error_value(); │ │ +│ │ handle_error(err); // Handle error │ │ +│ │ } │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ │ +│ MONADIC OPERATIONS (Chaining): │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ // map: Transform value if success │ │ +│ │ auto squared = result │ │ +│ │ .map([](int x) { return x * x; }) // 42*42=1764 │ │ +│ │ .map([](int x) { return x + 1; }); // 1765 │ │ +│ │ │ │ +│ │ // and_then: Chain dependent operations │ │ +│ │ auto chained = parse_input() │ │ +│ │ .and_then([](auto val) { │ │ +│ │ return validate(val); // result │ │ +│ │ }) │ │ +│ │ .and_then([](auto val) { │ │ +│ │ return process(val); // result │ │ +│ │ }); │ │ +│ │ │ │ +│ │ // unwrap: Extract value or throw on error │ │ +│ │ int value = result.unwrap(); // Throws if error │ │ +│ │ │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ │ +│ ERROR CODE HIERARCHY: │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ enum class error_code { │ │ +│ │ success = 0, │ │ +│ │ unknown_error, │ │ +│ │ operation_canceled, │ │ +│ │ operation_timeout, │ │ +│ │ │ │ +│ │ thread_already_running = 100, // Thread (100+) │ │ +│ │ thread_not_running, │ │ +│ │ thread_start_failure, │ │ +│ │ thread_join_failure, │ │ +│ │ │ │ +│ │ queue_full = 200, // Queue (200+) │ │ +│ │ queue_empty, │ │ +│ │ queue_stopped, │ │ +│ │ │ │ +│ │ job_creation_failed = 300, // Job (300+) │ │ +│ │ job_execution_failed, │ │ +│ │ job_invalid, │ │ +│ │ │ │ +│ │ resource_allocation_failed = 400, // Resource (400+) │ │ +│ │ resource_limit_reached, │ │ +│ │ │ │ +│ │ mutex_error = 500, // Sync (500+) │ │ +│ │ deadlock_detected, │ │ +│ │ condition_variable_error, │ │ +│ │ │ │ +│ │ io_error = 600, // IO (600+) │ │ +│ │ ... │ │ +│ │ }; │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ │ +│ BENEFITS: │ +│ ✅ No exceptions in critical paths │ +│ ✅ Type-safe error information │ +│ ✅ Explicit error handling │ +│ ✅ Composable & chainable operations │ +│ ✅ Clear error propagation paths │ +│ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +--- + +*Last Updated: 2025-12-04* +*Visual diagrams for Thread System architecture and key mechanisms* diff --git a/docs/ARCHITECTURE_DIAGRAM.md b/docs/ARCHITECTURE_DIAGRAM.md index cab8ec1b86..6025779904 100644 --- a/docs/ARCHITECTURE_DIAGRAM.md +++ b/docs/ARCHITECTURE_DIAGRAM.md @@ -12,650 +12,24 @@ category: "ARCH" > **SSOT**: This document is the single source of truth for **Thread System - Architecture Diagrams & Visual Reference**. -## 1. System Architecture Overview - -``` -┌─────────────────────────────────────────────────────────────────────────────┐ -│ EXTERNAL SYSTEMS & APPLICATIONS │ -│ │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ Network │ │ Data │ │ Game │ │ Scientific │ │ -│ │ System │ │ Processing │ │ Engine │ │ Computing │ │ -│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ -└─────────┼──────────────────┼──────────────────┼──────────────────┼──────────┘ - │ │ │ │ - │ (via interfaces) │ (via interfaces) │ (via interfaces) │ - └──────────────────┼──────────────────┼──────────────────┘ - │ - ┌────────────────────▼────────────────────┐ - │ THREAD SYSTEM (Core Foundation) │ - │ ~2,700 lines of code │ - │ │ - │ No external dependencies (standalone) │ - └────┬────────────────────┬──────────────┘ - │ │ - ┌────────▼────────┐ ┌───────▼──────────┐ - │ CORE MODULE │ │ IMPLEMENTATIONS │ - │ │ │ │ - │ • thread_base │ │ • thread_pool │ - │ • thread_worker │ │ • typed_pool │ - │ • job │ │ • lockfree_queue │ - │ • job_queue │ │ • adaptive_queue │ - │ • sync primitives │ │ - │ • cancellation │ │ │ - │ • error handling│ │ │ - │ • service reg │ │ │ - └────────┬────────┘ └─────────┬────────┘ - │ │ - ┌────────▼─────────────────────▼────────┐ - │ PUBLIC INTERFACES │ - │ │ - │ • executor_interface │ - │ • scheduler_interface │ - │ • logger_interface (deprecated) │ - │ • monitoring_interface (deprecated) │ - └────────┬──────────────────────────────┘ - │ - ┌───────┴──────────┬──────────────┐ - │ │ │ - ┌───▼────┐ ┌─────▼────┐ ┌────▼──────┐ - │Logger │ │Monitoring│ │Integrated │ - │System │ │System │ │Examples │ - │(opt) │ │(opt) │ │ │ - └────────┘ └───────────┘ └───────────┘ -``` - -## 2. Core Module Component Hierarchy - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ CORE MODULE │ -├──────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ BASE THREADING │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ thread_base ──────┐ │ │ -│ │ [start/stop] │ │ │ -│ │ [do_work] │ │ │ -│ │ [lifecycle] │ │ │ -│ │ │ │ │ -│ │ └──▶ thread_worker │ │ -│ │ [process jobs] │ │ -│ │ [idle tracking] │ │ -│ │ [worker ID] │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ JOB & QUEUE SYSTEM │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ job ◀─────────────┐ │ │ -│ │ [abstract] │ │ │ -│ │ [do_work] │ │ │ -│ │ ├──▶ callback_job ◀─── [wrap callables] │ │ -│ │ │ │ │ -│ │ └──▶ typed_job ◀────── [type metadata] │ │ -│ │ │ │ -│ │ job_queue │ │ -│ │ [FIFO, mutex-based] │ │ -│ │ [enqueue/dequeue] │ │ -│ │ [condition_variable] │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ SYNCHRONIZATION & CONTROL │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ sync_primitives ──┬──▶ scoped_lock_guard │ │ -│ │ [RAII wrappers] │ [timeout support] │ │ -│ │ │ │ │ -│ │ ├──▶ condition_variable_wrapper │ │ -│ │ │ [predicates, notify] │ │ -│ │ │ │ │ -│ │ ├──▶ atomic_flag_wrapper │ │ -│ │ │ [wait/notify ops] │ │ -│ │ │ │ │ -│ │ └──▶ shared_mutex_wrapper │ │ -│ │ [reader-writer locks] │ │ -│ │ │ │ -│ │ cancellation_token │ │ -│ │ [cooperative cancellation] │ │ -│ │ [linked tokens] │ │ -│ │ [callbacks] │ │ -│ │ │ │ -│ │ hazard_pointer │ │ -│ │ [lock-free memory safety] │ │ -│ │ [per-thread lists] │ │ -│ │ [global registry] │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ ERROR HANDLING │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ result ◀────────── [C++23 expected pattern] │ │ -│ │ [is_success] │ │ -│ │ [map, and_then] │ │ -│ │ [unwrap] │ │ -│ │ │ │ -│ │ error_code ◀────────── [100+ typed codes] │ │ -│ │ [thread, queue, job, resource errors] │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ DEPENDENCY INJECTION │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ service_registry │ │ -│ │ [static DI container] │ │ -│ │ [thread-safe (shared_mutex)] │ │ -│ │ [type-safe via std::any] │ │ -│ │ │ │ -│ │ thread_context │ │ -│ │ [logger_interface] │ │ -│ │ [monitoring_interface] │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -└──────────────────────────────────────────────────────────────────┘ -``` - -## 3. Threading & Job Execution Flow - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ APPLICATION THREAD │ -│ │ -│ // Create pool │ -│ auto pool = std::make_shared("pool"); │ -│ │ -│ // Enqueue job (non-blocking) │ -│ pool->execute(std::make_unique()); │ -│ │ -│ // Job moves to queue │ -│ └──────────────┬──────────────────────────────────────────┐ │ -└─────────────────┼──────────────────────────────────────────┼───┘ - │ │ - ┌─────────────▼──────────────────────┐ │ - │ │ │ - │ SHARED JOB QUEUE │ │ - │ │ │ - │ ┌──────────┐ ┌──────────┐ │ │ - │ │ Job A │ │ Job B │ ◀────┘ (enqueue) │ - │ └──────────┘ └──────────┘ │ │ - │ │ │ - └─────────────┬──────────────────────┘ │ - │ (dequeue) │ - ┌───────────┴──────────────┬─────────────┐ │ - │ │ │ │ - ▼ ▼ ▼ │ - ┌─────────┐ ┌─────────┐ ┌─────────┐ │ - │ Worker1 │ │ Worker2 │ │ WorkerN │ │ - │ (thread)│ │ (thread)│ │ (thread)│ │ - │ │ │ │ │ │ │ - │ while() │ │ while() │ │ while() │ │ - │ loop: │ │ loop: │ │ loop: │ │ - │ │ │ │ │ │ │ - │ if job:◄┼─────────────┤─────────┼────┼─────────┼──┐ │ - │ run │ │ │ │ │ │ │ - │ it │ │ │ │ │ │ │ - │ │ │ │ │ │ │ │ - │ else: │ │ │ │ │ │ │ - │ wait │ │ │ │ │ │ │ - │ on CV │ │ │ │ │ │ │ - └─────────┘ └─────────┘ └─────────┘ │ │ - │ │ │ │ │ - │ (run) (run) │ (run) │ │ │ - └─────┬──────────────────┼─────────────┘ │ │ - │ │ │ │ - ▼ ▼ │ │ - [Job A] [Job B] [dequeue] │ │ - result result cycles...└───┘ -``` - -## 4. Queue Capabilities Architecture - -This diagram shows the interface hierarchy for queue implementations after Phase 1-5 enhancements. - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Queue Architecture (v2.0) │ -│ │ -│ ┌─────────────────────────────────────────────────────────────┐│ -│ │ scheduler_interface (UNCHANGED) ││ -│ │ + schedule(job) -> result_void ││ -│ │ + get_next_job() -> result> ││ -│ │ + has_pending() -> bool ││ -│ └────────────────────────────┬────────────────────────────────┘│ -│ │ │ -│ ┌────────────────────────────┴────────────────────────────────┐│ -│ │ queue_capabilities_interface (NEW - MIXIN) ││ -│ │ + get_capabilities() -> queue_capabilities ││ -│ │ + has_exact_size() / has_atomic_empty() / is_lock_free() ││ -│ │ + supports_batch() / supports_blocking_wait() ││ -│ └────────────────────────────┬────────────────────────────────┘│ -│ │ │ -│ ┌────────────────────┼────────────────────┐ │ -│ │ │ │ │ -│ ▼ ▼ ▼ │ -│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ -│ │ job_queue │ │ lockfree_ │ │ adaptive_ │ │ -│ │ (UNCHANGED) │ │ job_queue │ │ job_queue │ │ -│ │ │ │ (EXTENDED) │ │ (NEW) │ │ -│ ├─────────────┤ ├─────────────┤ ├─────────────┤ │ -│ │ exact_size │ │ lock_free │ │ auto-tune │ │ -│ │ ~300K ops/s │ │ ~1.2M ops/s │ │ best-of-both│ │ -│ │ batch: yes │ │ batch: no │ │ policy-based│ │ -│ │ blocking:yes│ │ blocking:no │ │ RAII guard │ │ -│ └─────────────┘ └─────────────┘ └─────────────┘ │ -│ │ -│ ┌─────────────────────────────────────────────────────────────┐│ -│ │ queue_factory (NEW - OPTIONAL) ││ -│ │ + create_standard_queue() -> shared_ptr ││ -│ │ + create_lockfree_queue() -> unique_ptr││ -│ │ + create_adaptive_queue() -> unique_ptr││ -│ │ + create_for_requirements(reqs) -> unique_ptr ││ -│ │ + create_optimal() -> unique_ptr ││ -│ └─────────────────────────────────────────────────────────────┘│ -│ │ -│ ┌─────────────────────────────────────────────────────────────┐│ -│ │ Compile-Time Type Selection (NEW) ││ -│ │ ││ -│ │ queue_t → job_queue (exact size) ││ -│ │ queue_t → lockfree_job_queue (fast) ││ -│ │ queue_t → adaptive_job_queue (balanced) ││ -│ │ ││ -│ │ Type Aliases: ││ -│ │ - accurate_queue_t = queue_t ││ -│ │ - fast_queue_t = queue_t ││ -│ │ - balanced_queue_t = queue_t ││ -│ └─────────────────────────────────────────────────────────────┘│ -│ │ -└─────────────────────────────────────────────────────────────────┘ -``` - -**Key Points:** -- All existing queue classes remain backward compatible -- `queue_capabilities_interface` is a mixin (additive inheritance) -- `queue_factory` is optional utility for convenient queue creation -- Compile-time selection provides zero runtime overhead - -## 5. Queue Implementation Strategy Comparison - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ QUEUE IMPLEMENTATIONS │ -├──────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ job_queue (Mutex-Based FIFO) │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ ENQUEUE DEQUEUE │ │ -│ │ ┌─────────────────┐ ┌──────────────────┐ │ │ -│ │ │ lock_guard │ │ lock_guard │ │ │ -│ │ │ queue.push(job) │ │ if queue.empty │ │ │ -│ │ │ notify_one() │ │ wait(cond_var) │ │ │ -│ │ │ unlock │ │ job = queue.pop()│ │ │ -│ │ └─────────────────┘ │ unlock │ │ │ -│ │ │ return job │ │ │ -│ │ ✅ Simple, reliable └──────────────────┘ │ │ -│ │ ✅ Baseline performance │ │ -│ │ ⚠️ Contention under high load │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ lockfree_job_queue (Michael-Scott + Hazard Pointers) │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ ALGORITHM: Michael-Scott Queue (1996) │ │ -│ │ MEMORY: Hazard Pointers (Michael, 2004) │ │ -│ │ │ │ -│ │ ENQUEUE (wait-free) DEQUEUE (lock-free) │ │ -│ │ ┌──────────────────┐ ┌──────────────────┐ │ │ -│ │ │ create node │ │ get hazard ptr │ │ │ -│ │ │ CAS tail.next │ │ read head │ │ │ -│ │ │ CAS tail pointer │ │ CAS head pointer │ │ │ -│ │ │ no locks! │ │ retire old node │ │ │ -│ │ └──────────────────┘ │ return job │ │ │ -│ │ └──────────────────┘ │ │ -│ │ MEMORY RECLAMATION: │ │ -│ │ • Per-thread hazard list (MAX 4) │ │ -│ │ • Global hazard registry │ │ -│ │ • Scan all threads before deletion │ │ -│ │ │ │ -│ │ ✅ 4x faster (71 μs vs 291 μs per op) │ │ -│ │ ✅ Production-safe (no TLS bugs) │ │ -│ │ ✅ Zero contention overhead │ │ -│ │ ⚠️ ~256 bytes per thread │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ adaptive_job_queue (Auto-Switching Strategy) │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ MONITOR: SELECT STRATEGY: │ │ -│ │ • Contention ratio ──┐ │ │ -│ │ • Operation latency ├──▶ Low contention: │ │ -│ │ • Operation count │ Use Mutex (simpler) │ │ -│ │ │ │ │ -│ │ └──▶ High contention: │ │ -│ │ Use Lock-Free (faster) │ │ -│ │ │ │ -│ │ ✅ Up to 7.7x improvement under contention │ │ -│ │ ✅ Automatic optimization │ │ -│ │ ✅ Zero configuration │ │ -│ │ ✅ Best of both worlds │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ bounded_job_queue (Ring Buffer with Size Limit) │ │ -│ ├────────────────────────────────────────────────────────────┤ │ -│ │ • Fixed-size ring buffer │ │ -│ │ • Memory-bounded (predictable) │ │ -│ │ • Wraparound on full │ │ -│ │ │ │ -│ │ ✅ Memory-predictable │ │ -│ │ ⚠️ Bounded capacity (enqueue may fail) │ │ -│ │ │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -└──────────────────────────────────────────────────────────────────┘ -``` - -## 6. Type-Based Thread Pool Architecture - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ TYPED THREAD POOL (Type-Routed Scheduling) │ -├─────────────────────────────────────────────────────────────────┤ -│ │ -│ APPLICATION LAYER │ -│ ┌──────────────────────┐ ┌──────────────────────┐ │ -│ │ enqueue(RealTime job)│ │ enqueue(Batch job) │ │ -│ └──────────┬───────────┘ └──────────┬───────────┘ │ -│ │ │ │ -│ ┌────────▼─────────────────────────▼────────┐ │ -│ │ TYPE-BASED JOB DISPATCH │ │ -│ │ (Enqueue-time decision) │ │ -│ └────┬─────────────┬───────────┬─────────┘ │ -│ │ │ │ │ -│ ┌────▼────┐ ┌────▼────┐ ┌──▼────┐ │ -│ │RealTime │ │ Normal │ │Background │ -│ │Queue │ │ Queue │ │Queue │ │ -│ │(FIFO) │ │ (FIFO) │ │(FIFO) │ │ -│ └────┬────┘ └────┬────┘ └──┬────┘ │ -│ │ │ │ │ -│ ┌────▼─────────────▼──────────▼────┐ │ -│ │ WORKER ASSIGNMENT │ │ -│ │ (Type-Aware Pool Partitioning) │ │ -│ └────┬─────────────┬────────────────┘ │ -│ │ │ │ │ -│ ┌────▼────┐ ┌────▼────┐ ┌──▼────────┐ │ -│ │Workers │ │Workers │ │Workers │ │ -│ │Set A │ │Set B │ │Set C │ │ -│ │(R.Time) │ │(Normal) │ │(Background │ -│ │ │ │ │ │) │ │ -│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ -│ │ │W₁ │ │ │ │W₄ │ │ │ │W₇ │ │ │ -│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ -│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ -│ │ │W₂ │ │ │ │W₅ │ │ │ │W₈ │ │ │ -│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ -│ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌──────┐ │ │ -│ │ │W₃ │ │ │ │W₆ │ │ │ │W₉ │ │ │ -│ │ └─────┘ │ │ └─────┘ │ │ └──────┘ │ │ -│ │ │ │ │ │ │ │ -│ └─────────┘ └─────────┘ └───────────┘ │ -│ │ -│ BENEFITS: │ -│ ✅ Type-accuracy >99% under all conditions │ -│ ✅ Per-type FIFO ordering preserved │ -│ ✅ Optimal resource allocation │ -│ ✅ Priority-aware scheduling │ -│ │ -└─────────────────────────────────────────────────────────────┘ -``` - -## 7. Hazard Pointer Memory Reclamation Flow - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ HAZARD POINTER MEMORY RECLAMATION MECHANISM │ -├──────────────────────────────────────────────────────────────────┤ -│ │ -│ THREAD 1 THREAD 2 THREAD 3 │ -│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ -│ │Hazard List│ │Hazard List│ │Hazard List│ │ -│ │ │ │ │ │ │ │ -│ │ [ptr→node]│ │ [NULL] │ │ [NULL] │ │ -│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ -│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ -│ │ [NULL] │ │ [NULL] │ │ [NULL] │ │ -│ │ │ │ │ │ │ │ -│ │ active=T │ │ active=T │ │ active=T │ │ -│ └───────┬───┘ └────┬──────┘ └────┬──────┘ │ -│ │ │ │ │ -│ └────────────────┼──────────────────┘ │ -│ │ │ -│ ┌─────────▼─────────┐ │ -│ │Global Hazard Ptr │ │ -│ │ Registry │ │ -│ │ │ │ -│ │ Head ──▶ [T1] ──▶ │ │ -│ │ ↓ │ │ -│ │ [T2] ──▶ │ │ -│ │ ↓ │ │ -│ │ [T3] │ │ -│ │ ↓ │ │ -│ │ NULL │ │ -│ └────────┬──────────┘ │ -│ │ │ -│ ┌───────────────────────┴────────────────────────┐ │ -│ │ DELETION PROCESS │ │ -│ ├─────────────────────────────────────────────────┤ │ -│ │ │ │ -│ │ 1. Mark node for retirement: │ │ -│ │ retire_list.push(node_ptr) │ │ -│ │ │ │ -│ │ 2. Scan all thread hazard lists: │ │ -│ │ protected = scan_hazards() │ │ -│ │ (Check: is node in any thread's list?) │ │ -│ │ │ │ -│ │ 3. Reclaim if safe: │ │ -│ │ for (auto node : retire_list) { │ │ -│ │ if (node not in protected) { │ │ -│ │ delete node; // SAFE TO DELETE │ │ -│ │ } else { │ │ -│ │ defer_delete(node); // Try later │ │ -│ │ } │ │ -│ │ } │ │ -│ │ │ │ -│ │ 4. Thread cleanup: │ │ -│ │ When thread exits: │ │ -│ │ mark_inactive() // Sets active=false │ │ -│ │ (TLS destructor no longer blocks delete) │ │ -│ │ │ │ -│ └────────────────────────────────────────────────┘ │ -│ │ -│ KEY BENEFITS: │ -│ ✅ True ABA prevention │ -│ ✅ No use-after-free │ -│ ✅ No memory leaks │ -│ ✅ Safe TLS destruction (no ordering dependency) │ -│ ✅ Production-safe │ -│ │ -└──────────────────────────────────────────────────────────────────┘ -``` - -## 8. Cancellation Token Hierarchy - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ CANCELLATION TOKEN HIERARCHY & PROPAGATION │ -├──────────────────────────────────────────────────────────────────┤ -│ │ -│ SCOPE 1: System-level Shutdown │ -│ ┌──────────────────────────┐ │ -│ │ system_shutdown_token │ │ -│ │ │ │ -│ │ is_cancelled() = false │ │ -│ │ [callbacks = {}] │ │ -│ └──────────┬───────────────┘ │ -│ │ │ -│ │ (create_linked) │ -│ │ │ -│ ┌──────┴──────┬─────────────┬─────────────┐ │ -│ │ │ │ │ │ -│ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐ │ -│ │Token 1 │ │Token 2 │ │Token 3 │ │Token 4 │ │ -│ │(DB Op) │ │(Network│ │(Compute) │(Cache) │ │ -│ │ │ │ Op) │ │ │ │ │ │ -│ │callback│ │callback│ │callback │callback │ │ -│ │cleanup │ │cleanup │ │cleanup │ │cleanup │ │ -│ │database│ │conns │ │threads │ │entries │ │ -│ └────────┘ └────────┘ └────────┘ └────────┘ │ -│ │ -│ PROPAGATION FLOW: │ -│ ┌─────────────────────────────────────────────────────────┐ │ -│ │ User: system_shutdown_token.cancel() │ │ -│ │ │ │ │ -│ │ └─▶ set atomic flag │ │ -│ │ │ │ │ -│ │ └─▶ notify() on all registered callbacks │ │ -│ │ │ │ │ -│ │ ├─▶ Token 1: cleanup() │ │ -│ │ │ └─▶ close DB connections │ │ -│ │ │ │ │ -│ │ ├─▶ Token 2: cleanup() │ │ -│ │ │ └─▶ close network sockets │ │ -│ │ │ │ │ -│ │ ├─▶ Token 3: cleanup() │ │ -│ │ │ └─▶ terminate worker threads │ │ -│ │ │ │ │ -│ │ └─▶ Token 4: cleanup() │ │ -│ │ └─▶ flush cache entries │ │ -│ │ │ │ -│ │ All linked tokens observe is_cancelled() = true │ │ -│ │ │ │ -│ └─────────────────────────────────────────────────────────┘ │ -│ │ -│ WEAK POINTER SAFETY: │ -│ • Token stores shared_ptr │ -│ • Callbacks store weak_ptr │ -│ ✅ Prevents circular reference cycles │ -│ ✅ Safe automatic cleanup │ -│ │ -└──────────────────────────────────────────────────────────────────┘ -``` - -## 9. Error Handling Flow - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ RESULT ERROR HANDLING PATTERN │ -├──────────────────────────────────────────────────────────────────┤ -│ │ -│ OPERATION WITH ERROR HANDLING: │ -│ ┌─────────────────────────────────────────────────────────┐ │ -│ │ result compute_with_error() { │ │ -│ │ // Try operation │ │ -│ │ if (invalid_input) { │ │ -│ │ return error{ │ │ -│ │ error_code::invalid_argument, │ │ -│ │ "Input value out of range" │ │ -│ │ }; │ │ -│ │ } │ │ -│ │ │ │ -│ │ // Success │ │ -│ │ return result(42); │ │ -│ │ } │ │ -│ └─────────────────────────────────────────────────────────┘ │ -│ │ -│ CONSUMING THE RESULT: │ -│ ┌─────────────────────────────────────────────────────────┐ │ -│ │ auto result = compute_with_error(); │ │ -│ │ │ │ -│ │ if (result.is_success()) { │ │ -│ │ int value = result.value(); // Use value │ │ -│ │ } else { │ │ -│ │ auto err = result.error_value(); │ │ -│ │ handle_error(err); // Handle error │ │ -│ │ } │ │ -│ └─────────────────────────────────────────────────────────┘ │ -│ │ -│ MONADIC OPERATIONS (Chaining): │ -│ ┌─────────────────────────────────────────────────────────┐ │ -│ │ // map: Transform value if success │ │ -│ │ auto squared = result │ │ -│ │ .map([](int x) { return x * x; }) // 42*42=1764 │ │ -│ │ .map([](int x) { return x + 1; }); // 1765 │ │ -│ │ │ │ -│ │ // and_then: Chain dependent operations │ │ -│ │ auto chained = parse_input() │ │ -│ │ .and_then([](auto val) { │ │ -│ │ return validate(val); // result │ │ -│ │ }) │ │ -│ │ .and_then([](auto val) { │ │ -│ │ return process(val); // result │ │ -│ │ }); │ │ -│ │ │ │ -│ │ // unwrap: Extract value or throw on error │ │ -│ │ int value = result.unwrap(); // Throws if error │ │ -│ │ │ │ -│ └─────────────────────────────────────────────────────────┘ │ -│ │ -│ ERROR CODE HIERARCHY: │ -│ ┌─────────────────────────────────────────────────────────┐ │ -│ │ enum class error_code { │ │ -│ │ success = 0, │ │ -│ │ unknown_error, │ │ -│ │ operation_canceled, │ │ -│ │ operation_timeout, │ │ -│ │ │ │ -│ │ thread_already_running = 100, // Thread (100+) │ │ -│ │ thread_not_running, │ │ -│ │ thread_start_failure, │ │ -│ │ thread_join_failure, │ │ -│ │ │ │ -│ │ queue_full = 200, // Queue (200+) │ │ -│ │ queue_empty, │ │ -│ │ queue_stopped, │ │ -│ │ │ │ -│ │ job_creation_failed = 300, // Job (300+) │ │ -│ │ job_execution_failed, │ │ -│ │ job_invalid, │ │ -│ │ │ │ -│ │ resource_allocation_failed = 400, // Resource (400+) │ │ -│ │ resource_limit_reached, │ │ -│ │ │ │ -│ │ mutex_error = 500, // Sync (500+) │ │ -│ │ deadlock_detected, │ │ -│ │ condition_variable_error, │ │ -│ │ │ │ -│ │ io_error = 600, // IO (600+) │ │ -│ │ ... │ │ -│ │ }; │ │ -│ └─────────────────────────────────────────────────────────┘ │ -│ │ -│ BENEFITS: │ -│ ✅ No exceptions in critical paths │ -│ ✅ Type-safe error information │ -│ ✅ Explicit error handling │ -│ ✅ Composable & chainable operations │ -│ ✅ Clear error propagation paths │ -│ │ -└──────────────────────────────────────────────────────────────────┘ -``` - ---- - -*Last Updated: 2025-12-04* -*Visual diagrams for Thread System architecture and key mechanisms* +This guide has been split into two focused documents for easier navigation: + +| Document | Description | Sections | +|----------|-------------|----------| +| **[Architecture Overview](ARCHITECTURE_OVERVIEW.md)** | High-level architecture | System architecture, core module hierarchy, threading & job execution flow | +| **[Architecture Details](ARCHITECTURE_DETAILS.md)** | Component deep-dives | Queue architecture, queue implementations, typed thread pool, hazard pointers, cancellation tokens, error handling | + +## Quick Links + +### Overview +- [System Architecture Overview](ARCHITECTURE_OVERVIEW.md#1-system-architecture-overview) +- [Core Module Component Hierarchy](ARCHITECTURE_OVERVIEW.md#2-core-module-component-hierarchy) +- [Threading & Job Execution Flow](ARCHITECTURE_OVERVIEW.md#3-threading--job-execution-flow) + +### Details +- [Queue Capabilities Architecture](ARCHITECTURE_DETAILS.md#4-queue-capabilities-architecture) +- [Queue Implementation Strategy Comparison](ARCHITECTURE_DETAILS.md#5-queue-implementation-strategy-comparison) +- [Type-Based Thread Pool Architecture](ARCHITECTURE_DETAILS.md#6-type-based-thread-pool-architecture) +- [Hazard Pointer Memory Reclamation Flow](ARCHITECTURE_DETAILS.md#7-hazard-pointer-memory-reclamation-flow) +- [Cancellation Token Hierarchy](ARCHITECTURE_DETAILS.md#8-cancellation-token-hierarchy) +- [Error Handling Flow](ARCHITECTURE_DETAILS.md#9-error-handling-flow) diff --git a/docs/ARCHITECTURE_OVERVIEW.md b/docs/ARCHITECTURE_OVERVIEW.md new file mode 100644 index 0000000000..abd3357f7e --- /dev/null +++ b/docs/ARCHITECTURE_OVERVIEW.md @@ -0,0 +1,227 @@ +--- +doc_id: "THR-ARCH-003a" +doc_title: "Thread System - Architecture Overview" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "ARCH" +--- + +# Thread System - Architecture Overview + +> **SSOT**: This document is the single source of truth for **Thread System - Architecture Overview** (high-level architecture diagrams). + +> **See also**: [Architecture Details](ARCHITECTURE_DETAILS.md) for component deep-dives (queues, hazard pointers, cancellation, error handling). + +## 1. System Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────────────┐ +│ EXTERNAL SYSTEMS & APPLICATIONS │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Network │ │ Data │ │ Game │ │ Scientific │ │ +│ │ System │ │ Processing │ │ Engine │ │ Computing │ │ +│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ +└─────────┼──────────────────┼──────────────────┼──────────────────┼──────────┘ + │ │ │ │ + │ (via interfaces) │ (via interfaces) │ (via interfaces) │ + └──────────────────┼──────────────────┼──────────────────┘ + │ + ┌────────────────────▼────────────────────┐ + │ THREAD SYSTEM (Core Foundation) │ + │ ~2,700 lines of code │ + │ │ + │ No external dependencies (standalone) │ + └────┬────────────────────┬──────────────┘ + │ │ + ┌────────▼────────┐ ┌───────▼──────────┐ + │ CORE MODULE │ │ IMPLEMENTATIONS │ + │ │ │ │ + │ • thread_base │ │ • thread_pool │ + │ • thread_worker │ │ • typed_pool │ + │ • job │ │ • lockfree_queue │ + │ • job_queue │ │ • adaptive_queue │ + │ • sync primitives │ │ + │ • cancellation │ │ │ + │ • error handling│ │ │ + │ • service reg │ │ │ + └────────┬────────┘ └─────────┬────────┘ + │ │ + ┌────────▼─────────────────────▼────────┐ + │ PUBLIC INTERFACES │ + │ │ + │ • executor_interface │ + │ • scheduler_interface │ + │ • logger_interface (deprecated) │ + │ • monitoring_interface (deprecated) │ + └────────┬──────────────────────────────┘ + │ + ┌───────┴──────────┬──────────────┐ + │ │ │ + ┌───▼────┐ ┌─────▼────┐ ┌────▼──────┐ + │Logger │ │Monitoring│ │Integrated │ + │System │ │System │ │Examples │ + │(opt) │ │(opt) │ │ │ + └────────┘ └───────────┘ └───────────┘ +``` + +## 2. Core Module Component Hierarchy + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ CORE MODULE │ +├──────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ BASE THREADING │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ thread_base ──────┐ │ │ +│ │ [start/stop] │ │ │ +│ │ [do_work] │ │ │ +│ │ [lifecycle] │ │ │ +│ │ │ │ │ +│ │ └──▶ thread_worker │ │ +│ │ [process jobs] │ │ +│ │ [idle tracking] │ │ +│ │ [worker ID] │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ JOB & QUEUE SYSTEM │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ job ◀─────────────┐ │ │ +│ │ [abstract] │ │ │ +│ │ [do_work] │ │ │ +│ │ ├──▶ callback_job ◀─── [wrap callables] │ │ +│ │ │ │ │ +│ │ └──▶ typed_job ◀────── [type metadata] │ │ +│ │ │ │ +│ │ job_queue │ │ +│ │ [FIFO, mutex-based] │ │ +│ │ [enqueue/dequeue] │ │ +│ │ [condition_variable] │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ SYNCHRONIZATION & CONTROL │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ sync_primitives ──┬──▶ scoped_lock_guard │ │ +│ │ [RAII wrappers] │ [timeout support] │ │ +│ │ │ │ │ +│ │ ├──▶ condition_variable_wrapper │ │ +│ │ │ [predicates, notify] │ │ +│ │ │ │ │ +│ │ ├──▶ atomic_flag_wrapper │ │ +│ │ │ [wait/notify ops] │ │ +│ │ │ │ │ +│ │ └──▶ shared_mutex_wrapper │ │ +│ │ [reader-writer locks] │ │ +│ │ │ │ +│ │ cancellation_token │ │ +│ │ [cooperative cancellation] │ │ +│ │ [linked tokens] │ │ +│ │ [callbacks] │ │ +│ │ │ │ +│ │ hazard_pointer │ │ +│ │ [lock-free memory safety] │ │ +│ │ [per-thread lists] │ │ +│ │ [global registry] │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ ERROR HANDLING │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ result ◀────────── [C++23 expected pattern] │ │ +│ │ [is_success] │ │ +│ │ [map, and_then] │ │ +│ │ [unwrap] │ │ +│ │ │ │ +│ │ error_code ◀────────── [100+ typed codes] │ │ +│ │ [thread, queue, job, resource errors] │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ DEPENDENCY INJECTION │ │ +│ ├────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ service_registry │ │ +│ │ [static DI container] │ │ +│ │ [thread-safe (shared_mutex)] │ │ +│ │ [type-safe via std::any] │ │ +│ │ │ │ +│ │ thread_context │ │ +│ │ [logger_interface] │ │ +│ │ [monitoring_interface] │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +## 3. Threading & Job Execution Flow + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ APPLICATION THREAD │ +│ │ +│ // Create pool │ +│ auto pool = std::make_shared("pool"); │ +│ │ +│ // Enqueue job (non-blocking) │ +│ pool->execute(std::make_unique()); │ +│ │ +│ // Job moves to queue │ +│ └──────────────┬──────────────────────────────────────────┐ │ +└─────────────────┼──────────────────────────────────────────┼───┘ + │ │ + ┌─────────────▼──────────────────────┐ │ + │ │ │ + │ SHARED JOB QUEUE │ │ + │ │ │ + │ ┌──────────┐ ┌──────────┐ │ │ + │ │ Job A │ │ Job B │ ◀────┘ (enqueue) │ + │ └──────────┘ └──────────┘ │ │ + │ │ │ + └─────────────┬──────────────────────┘ │ + │ (dequeue) │ + ┌───────────┴──────────────┬─────────────┐ │ + │ │ │ │ + ▼ ▼ ▼ │ + ┌─────────┐ ┌─────────┐ ┌─────────┐ │ + │ Worker1 │ │ Worker2 │ │ WorkerN │ │ + │ (thread)│ │ (thread)│ │ (thread)│ │ + │ │ │ │ │ │ │ + │ while() │ │ while() │ │ while() │ │ + │ loop: │ │ loop: │ │ loop: │ │ + │ │ │ │ │ │ │ + │ if job:◄┼─────────────┤─────────┼────┼─────────┼──┐ │ + │ run │ │ │ │ │ │ │ + │ it │ │ │ │ │ │ │ + │ │ │ │ │ │ │ │ + │ else: │ │ │ │ │ │ │ + │ wait │ │ │ │ │ │ │ + │ on CV │ │ │ │ │ │ │ + └─────────┘ └─────────┘ └─────────┘ │ │ + │ │ │ │ │ + │ (run) (run) │ (run) │ │ │ + └─────┬──────────────────┼─────────────┘ │ │ + │ │ │ │ + ▼ ▼ │ │ + [Job A] [Job B] [dequeue] │ │ + result result cycles...└───┘ +``` + +--- + +*Last Updated: 2025-12-04* +*Visual diagrams for Thread System architecture and key mechanisms* diff --git a/docs/DIAGNOSTICS_GUIDE.md b/docs/DIAGNOSTICS_GUIDE.md new file mode 100644 index 0000000000..ec57830fc4 --- /dev/null +++ b/docs/DIAGNOSTICS_GUIDE.md @@ -0,0 +1,531 @@ +--- +doc_id: "THR-GUID-002a" +doc_title: "Diagnostics Guide" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "GUID" +--- + +# Diagnostics Guide + +> **SSOT**: This document is the single source of truth for **Diagnostics Guide** (thread dump, job inspection, bottleneck detection, event tracing). + +> **Language:** **English** + +> **See also**: [Metrics Guide](METRICS_GUIDE.md) for health monitoring, Prometheus integration, and metrics backends. + +A comprehensive guide to thread_system's diagnostics subsystem, covering thread dumps, job inspection, bottleneck detection, and event tracing. + +## Table of Contents + +1. [Overview](#overview) +2. [Diagnostics Subsystem](#diagnostics-subsystem) +3. [Bottleneck Detection](#bottleneck-detection) +4. [Event Tracing](#event-tracing) + +--- + +## Overview + +The diagnostics and metrics subsystems provide comprehensive observability for thread_system's thread pools. They operate as two complementary layers: + +``` ++-----------------------------------------------------------+ +| Application Code | ++-----------------------------------------------------------+ +| thread_pool_diagnostics (qualitative) | +| [health check] [bottleneck detect] [event trace] [dump] | ++-----------------------------------------------------------+ +| metrics_service (quantitative) | +| [basic metrics] [enhanced metrics] [export backends] | ++----------------------------+------------------------------+ +| ThreadPoolMetrics | EnhancedThreadPoolMetrics | +| (6 atomic counters) | (histograms + counters) | ++----------------------------+------------------------------+ +| MetricsBase (5 atomic counters) | ++-----------------------------------------------------------+ +``` + +### Key Features + +| Feature | Description | +|---------|-------------| +| **Health Checks** | Component-level assessment with HTTP status codes and Kubernetes integration | +| **Bottleneck Detection** | Automatic identification of 7 bottleneck types with severity and recommendations | +| **Event Tracing** | Observer-pattern job lifecycle tracing with < 1us overhead | +| **Basic Metrics** | Lock-free atomic counters with < 50ns overhead per record | +| **Enhanced Metrics** | Latency histograms, throughput counters, per-worker tracking | +| **Pluggable Backends** | Prometheus, JSON, and Logging export with custom backend support | +| **Thread Dump** | Point-in-time worker state snapshots | + +### Source Files + +**Diagnostics** (`kcenon::thread::diagnostics` namespace): + +| Header | Description | +|--------|-------------| +| [`thread_pool_diagnostics.h`](../include/kcenon/thread/diagnostics/thread_pool_diagnostics.h) | Central diagnostics API | +| [`health_status.h`](../include/kcenon/thread/diagnostics/health_status.h) | Health state, thresholds, component health | +| [`bottleneck_report.h`](../include/kcenon/thread/diagnostics/bottleneck_report.h) | Bottleneck types, severity, recommendations | +| [`execution_event.h`](../include/kcenon/thread/diagnostics/execution_event.h) | Event types, listener interface | +| [`job_info.h`](../include/kcenon/thread/diagnostics/job_info.h) | Job lifecycle and timing | +| [`thread_info.h`](../include/kcenon/thread/diagnostics/thread_info.h) | Worker state and statistics | + +**Metrics** (`kcenon::thread::metrics` namespace): + +| Header | Description | +|--------|-------------| +| [`metrics_service.h`](../include/kcenon/thread/metrics/metrics_service.h) | Centralized metrics management | +| [`metrics_base.h`](../include/kcenon/thread/metrics/metrics_base.h) | Abstract base with 5 atomic counters | +| [`thread_pool_metrics.h`](../include/kcenon/thread/metrics/thread_pool_metrics.h) | Lightweight metrics (6 counters) | +| [`enhanced_metrics.h`](../include/kcenon/thread/metrics/enhanced_metrics.h) | Histograms, percentiles, per-worker tracking | +| [`latency_histogram.h`](../include/kcenon/thread/metrics/latency_histogram.h) | HDR-style 64-bucket histogram | +| [`sliding_window_counter.h`](../include/kcenon/thread/metrics/sliding_window_counter.h) | Lock-free throughput counter | +| [`metrics_backend.h`](../include/kcenon/thread/metrics/metrics_backend.h) | Backend interface and built-in implementations | + +### When to Use + +| Scenario | Recommendation | +|----------|---------------| +| Quick health check in HTTP endpoint | `diagnostics().health_check()` | +| Production monitoring with Prometheus | Enable enhanced metrics + `PrometheusBackend` | +| Debugging slow task execution | `diagnostics().detect_bottlenecks()` + event tracing | +| Lightweight embedded deployment | Basic `ThreadPoolMetrics` only (48 bytes) | +| Per-worker performance analysis | `EnhancedThreadPoolMetrics` with `worker_metrics()` | +| Custom monitoring integration | Implement `MetricsBackend` interface | + +--- + +## Diagnostics Subsystem + +### Architecture + +The `thread_pool_diagnostics` class provides a non-intrusive, read-only view into thread pool state: + +``` +thread_pool_diagnostics + | + +-- Thread Dump (dump_thread_states, format_thread_dump) + +-- Job Inspection (get_active_jobs, get_pending_jobs, get_recent_jobs) + +-- Bottleneck Detect (detect_bottlenecks -> bottleneck_report) + +-- Health Check (health_check -> health_status) + +-- Event Tracing (enable_tracing, add_event_listener) + +-- Export (to_json, to_string, to_prometheus) +``` + +### Design Principles + +| Principle | Description | +|-----------|-------------| +| **Non-intrusive** | Minimal overhead when not actively queried | +| **Thread-safe** | All methods callable from any thread | +| **Read-only** | Never modifies thread pool state | +| **Snapshot-based** | Returns point-in-time copies, not live references | + +### Construction + +```cpp +#include +using namespace kcenon::thread; +using namespace kcenon::thread::diagnostics; + +auto pool = std::make_shared("MyPool"); +pool->start(); + +// Access diagnostics via the pool's built-in diagnostics +auto& diag = pool->diagnostics(); + +// Or create standalone diagnostics with custom config +diagnostics_config config; +config.recent_jobs_capacity = 5000; +config.enable_tracing = true; +thread_pool_diagnostics custom_diag(*pool, config); +``` + +### Thread Dump + +Thread dump provides a snapshot of all worker states: + +```cpp +// Structured data +auto threads = diag.dump_thread_states(); +for (const auto& t : threads) { + LOG_INFO("Worker {} ({}): {} jobs done, {:.1f}% utilization", + t.thread_name, worker_state_to_string(t.state), + t.jobs_completed, t.utilization * 100.0); +} + +// Human-readable format +std::cout << diag.format_thread_dump() << std::endl; +``` + +Output format: + +``` +=== Thread Pool Dump: MyPool === +Time: 2025-01-08T10:30:00Z +Workers: 8, Active: 5, Idle: 3 + +Worker-0 [tid:12345] ACTIVE (2.5s) + Current Job: ProcessOrder#1234 (running 150ms) + Jobs: 1523 completed, 2 failed + Utilization: 87.3% +``` + +#### Worker States + +| State | Meaning | +|-------|---------| +| `idle` | Worker is waiting for jobs | +| `active` | Worker is executing a job | +| `stopping` | Worker is in the process of stopping | +| `stopped` | Worker has stopped | + +#### thread_info Fields + +| Field | Type | Description | +|-------|------|-------------| +| `thread_id` | `std::thread::id` | System thread ID | +| `thread_name` | `std::string` | Human-readable name (e.g., "Worker-0") | +| `worker_id` | `std::size_t` | Pool-assigned worker ID | +| `state` | `worker_state` | Current operational state | +| `state_since` | `steady_clock::time_point` | When the current state began | +| `current_job` | `std::optional` | Current job (if `active`) | +| `jobs_completed` | `std::uint64_t` | Total successful completions | +| `jobs_failed` | `std::uint64_t` | Total failures | +| `total_busy_time` | `std::chrono::nanoseconds` | Cumulative busy time | +| `total_idle_time` | `std::chrono::nanoseconds` | Cumulative idle time | +| `utilization` | `double` | `busy / (busy + idle)`, 0.0 to 1.0 | + +### Job Inspection + +```cpp +// Currently executing jobs +auto active = diag.get_active_jobs(); + +// Pending jobs in queue (default limit: 100) +auto pending = diag.get_pending_jobs(50); + +// Recent completed/failed jobs (ring buffer, default limit: 100) +auto recent = diag.get_recent_jobs(200); + +for (const auto& job : active) { + if (job.status == job_status::running) { + LOG_INFO("Job {} running for {:.1f}ms", + job.job_name, job.execution_time_ms()); + } +} +``` + +#### Job Lifecycle + +``` +enqueue_time start_time end_time + | | | + v v v + [=======wait_time=========][====execution_time==========] + |<----- pending ---------->|<-------- running --------->| +``` + +#### job_status Values + +| Status | Description | +|--------|-------------| +| `pending` | Job is waiting in the queue | +| `running` | Job is currently being executed | +| `completed` | Job completed successfully | +| `failed` | Job failed with an error | +| `cancelled` | Job was cancelled before completion | +| `timed_out` | Job exceeded its timeout limit | + +### Export Formats + +```cpp +// JSON export +std::string json = diag.to_json(); + +// Human-readable export +std::string text = diag.to_string(); + +// Prometheus exposition format +std::string prom = diag.to_prometheus(); +``` + +### Performance Characteristics + +| Operation | Complexity | Latency | +|-----------|-----------|---------| +| Thread dump | O(n) workers | < 1ms for 64 workers | +| Job inspection (active) | O(1) | < 100ns | +| Job inspection (history) | O(n) | < 1ms for 1000 entries | +| Bottleneck detection | O(n) workers | < 1ms for 64 workers | +| Health check | O(n) all components | < 1ms | +| Event tracing | O(1) per event | < 1us | + +--- + +## Bottleneck Detection + +### Overview + +The `detect_bottlenecks()` method analyzes thread pool metrics to identify performance bottlenecks and provide actionable recommendations. + +```cpp +auto report = pool->diagnostics().detect_bottlenecks(); +if (report.has_bottleneck) { + LOG_WARN("Bottleneck: {} - {}", + bottleneck_type_to_string(report.type), + report.description); + for (const auto& rec : report.recommendations) { + LOG_INFO(" Recommendation: {}", rec); + } +} +``` + +### Bottleneck Types + +| Type | Description | Typical Cause | +|------|-------------|---------------| +| `none` | No bottleneck detected | System operating normally | +| `queue_full` | Queue is at capacity | Production rate exceeds consumption | +| `slow_consumer` | Workers can't keep up | Tasks take too long to execute | +| `worker_starvation` | Not enough workers | Pool too small for workload | +| `lock_contention` | High mutex wait times | Shared resource contention | +| `uneven_distribution` | Work not evenly spread | Work stealing needed | +| `memory_pressure` | Excessive allocations | Memory-intensive tasks | + +### Detection Logic + +The detection algorithm evaluates metrics in priority order: + +``` +Step 1: Check queue_saturation + queue_saturation > 0.9 --> queue_full + +Step 2: Check consumer speed + avg_wait_time > threshold AND worker_utilization > 0.9 --> slow_consumer + +Step 3: Check worker capacity + worker_utilization > 0.95 AND queue_saturation > 0.5 --> worker_starvation + +Step 4: Check distribution + utilization variance high --> uneven_distribution +``` + +### Severity Levels + +| Level | Name | Condition | Action | +|-------|------|-----------|--------| +| 0 | `none` | No bottleneck | No action needed | +| 1 | `low` | Bottleneck detected, metrics below warning thresholds | Monitor | +| 2 | `medium` | `queue_saturation > 0.8` OR `worker_utilization > 0.9` | Investigate | +| 3 | `critical` | `queue_saturation > 0.95` OR `worker_utilization > 0.98` | Immediate action | + +### Interpreting the Report + +The `bottleneck_report` contains supporting metrics for diagnosis: + +| Field | Type | Description | +|-------|------|-------------| +| `has_bottleneck` | `bool` | Whether any bottleneck was detected | +| `type` | `bottleneck_type` | Classification of the bottleneck | +| `description` | `std::string` | Human-readable explanation | +| `queue_saturation` | `double` | Queue depth / capacity (0.0 to 1.0+) | +| `avg_wait_time_ms` | `double` | Average queue wait time | +| `worker_utilization` | `double` | Average worker busy ratio (0.0 to 1.0) | +| `utilization_variance` | `double` | Variance in worker utilization | +| `estimated_backlog_time_ms` | `std::size_t` | Estimated time to clear queue | +| `queue_depth` | `std::size_t` | Current items in queue | +| `idle_workers` | `std::size_t` | Number of idle workers | +| `total_workers` | `std::size_t` | Total worker count | +| `jobs_rejected` | `std::uint64_t` | Jobs rejected due to full queue | +| `recommendations` | `vector` | Actionable suggestions | + +### Response Guide + +| Bottleneck | Recommended Actions | +|------------|-------------------| +| `queue_full` | Increase queue capacity, add workers, enable autoscaling | +| `slow_consumer` | Optimize task execution time, break large tasks, add workers | +| `worker_starvation` | Increase `min_workers`, enable autoscaling, reduce queue capacity | +| `lock_contention` | Reduce shared state, use lock-free algorithms, partition work | +| `uneven_distribution` | Enable work stealing, check task affinity, rebalance queue | +| `memory_pressure` | Pool objects, reduce allocations, use arena allocators | + +### Bottleneck Report Export + +```cpp +auto report = diag.detect_bottlenecks(); + +// JSON format +std::string json = report.to_json(); + +// Human-readable format +std::string text = report.to_string(); + +// Severity check +if (report.requires_immediate_action()) { + alert_on_call_team(report.to_json()); +} +``` + +Human-readable output: + +``` +=== Bottleneck Report === +Status: DETECTED (medium severity) +Type: slow_consumer +Description: Workers cannot keep up with job submission rate + +Metrics: + Queue: 100 items (75.0% saturated) + Workers: 7/8 active (1 idle) + Utilization: 92.0% (variance: 0.0500) + Wait time: 150.500ms avg + Backlog: ~5000ms to clear + +Recommendations: + - Add more worker threads + - Optimize job execution time +``` + +--- + +## Event Tracing + +### Overview + +Event tracing provides lifecycle tracking for individual jobs with the observer pattern. When enabled, every job state transition generates a `job_execution_event`. + +### Event Lifecycle + +``` +enqueued --> dequeued --> started --> completed/failed/cancelled + | + retried --> started --> ... +``` + +### Event Types + +| Type | Description | Timing Fields | +|------|-------------|---------------| +| `enqueued` | Job added to queue | `timestamp` only | +| `dequeued` | Job taken from queue | `wait_time` populated | +| `started` | Execution started | `wait_time` populated | +| `completed` | Successful completion | `wait_time`, `execution_time` | +| `failed` | Failed with error | `wait_time`, `execution_time`, `error_*` | +| `cancelled` | Cancelled before completion | `wait_time`, optionally `execution_time` | +| `retried` | Being retried after failure | `wait_time` | + +### Enabling Tracing + +```cpp +// Enable with default history size (1000 events) +pool->diagnostics().enable_tracing(true); + +// Enable with custom history size +pool->diagnostics().enable_tracing(true, 5000); + +// Check if tracing is enabled +if (pool->diagnostics().is_tracing_enabled()) { + // Tracing is active +} + +// Disable tracing +pool->diagnostics().enable_tracing(false); +``` + +### Custom Event Listener + +Implement the `execution_event_listener` interface to receive real-time events: + +```cpp +#include +using namespace kcenon::thread::diagnostics; + +class JsonEventLogger : public execution_event_listener { +public: + void on_event(const job_execution_event& event) override { + // IMPORTANT: This is called from worker threads. + // Keep processing fast (< 1us) to avoid impacting performance. + + if (event.is_terminal()) { + LOG_INFO("[{}] job:{} {} wait:{:.3f}ms exec:{:.3f}ms", + event.format_timestamp(), + event.job_name, + event_type_to_string(event.type), + event.wait_time_ms(), + event.execution_time_ms()); + } + + if (event.is_error() && event.error_message.has_value()) { + LOG_ERROR("Job {} failed: {}", + event.job_name, + event.error_message.value()); + } + } +}; + +// Register the listener +auto logger = std::make_shared(); +pool->diagnostics().add_event_listener(logger); + +// Later: remove the listener +pool->diagnostics().remove_event_listener(logger); +``` + +### Querying Event History + +```cpp +// Get recent events (default limit: 100) +auto events = pool->diagnostics().get_recent_events(50); +for (const auto& event : events) { + std::cout << event.to_string() << std::endl; +} +``` + +Event string output: + +``` +[2025-01-08T10:30:00.123Z] Event#123 job:ProcessOrder#456 type:completed + worker:0 thread:12345 wait:1.500ms exec:10.200ms +``` + +### job_execution_event Fields + +| Field | Type | Description | +|-------|------|-------------| +| `event_id` | `std::uint64_t` | Monotonically increasing ID | +| `job_id` | `std::uint64_t` | Job identifier | +| `job_name` | `std::string` | Human-readable job name | +| `type` | `event_type` | Event classification | +| `timestamp` | `steady_clock::time_point` | Monotonic timestamp | +| `system_timestamp` | `system_clock::time_point` | Wall-clock timestamp | +| `thread_id` | `std::thread::id` | Processing thread ID | +| `worker_id` | `std::size_t` | Worker identifier | +| `wait_time` | `std::chrono::nanoseconds` | Queue wait duration | +| `execution_time` | `std::chrono::nanoseconds` | Execution duration | +| `error_code` | `std::optional` | Error code (failures only) | +| `error_message` | `std::optional` | Error message (failures only) | + +### Performance Impact + +| Tracing Mode | Overhead per Event | +|--------------|-------------------| +| Disabled | 0 (no overhead) | +| Enabled, no listeners | < 1us (history recording only) | +| Enabled, with listeners | < 1us + listener processing time | + +> **Warning**: Listener `on_event()` implementations must be fast (< 1us). Use async logging or buffering for expensive operations. + +--- + +## Related Documentation + +- [Metrics Guide](METRICS_GUIDE.md) - Health monitoring, Prometheus, metrics backends +- [Architecture Overview](ARCHITECTURE_OVERVIEW.md) - System-wide architecture +- [Policy Queue Combinations Guide](POLICY_QUEUE_GUIDE.md) - Queue and policy configurations +- [NUMA Work-Stealing Guide](NUMA_GUIDE.md) - NUMA topology and work stealing +- [Autoscaler Guide](AUTOSCALER_GUIDE.md) - Dynamic worker pool sizing diff --git a/docs/DIAGNOSTICS_METRICS_GUIDE.md b/docs/DIAGNOSTICS_METRICS_GUIDE.md index 85a067819c..edcbc6e9d6 100644 --- a/docs/DIAGNOSTICS_METRICS_GUIDE.md +++ b/docs/DIAGNOSTICS_METRICS_GUIDE.md @@ -14,1519 +14,30 @@ category: "GUID" > **Language:** **English** -A comprehensive guide to thread_system's diagnostics and metrics subsystems, covering health checks, bottleneck detection, event tracing, latency histograms, throughput counters, and pluggable export backends. - -## Table of Contents - -1. [Overview](#overview) -2. [Diagnostics Subsystem](#diagnostics-subsystem) -3. [Health Monitoring](#health-monitoring) -4. [Bottleneck Detection](#bottleneck-detection) -5. [Event Tracing](#event-tracing) -6. [Metrics Framework](#metrics-framework) -7. [Latency Histograms](#latency-histograms) -8. [Sliding Window Counters](#sliding-window-counters) -9. [Metrics Service](#metrics-service) -10. [Metrics Backends](#metrics-backends) -11. [Usage Examples](#usage-examples) -12. [Configuration Reference](#configuration-reference) -13. [Anti-Patterns](#anti-patterns) -14. [Troubleshooting](#troubleshooting) - ---- - -## Overview - -The diagnostics and metrics subsystems provide comprehensive observability for thread_system's thread pools. They operate as two complementary layers: - -``` -+-----------------------------------------------------------+ -| Application Code | -+-----------------------------------------------------------+ -| thread_pool_diagnostics (qualitative) | -| [health check] [bottleneck detect] [event trace] [dump] | -+-----------------------------------------------------------+ -| metrics_service (quantitative) | -| [basic metrics] [enhanced metrics] [export backends] | -+----------------------------+------------------------------+ -| ThreadPoolMetrics | EnhancedThreadPoolMetrics | -| (6 atomic counters) | (histograms + counters) | -+----------------------------+------------------------------+ -| MetricsBase (5 atomic counters) | -+-----------------------------------------------------------+ -``` - -### Key Features - -| Feature | Description | -|---------|-------------| -| **Health Checks** | Component-level assessment with HTTP status codes and Kubernetes integration | -| **Bottleneck Detection** | Automatic identification of 7 bottleneck types with severity and recommendations | -| **Event Tracing** | Observer-pattern job lifecycle tracing with < 1us overhead | -| **Basic Metrics** | Lock-free atomic counters with < 50ns overhead per record | -| **Enhanced Metrics** | Latency histograms, throughput counters, per-worker tracking | -| **Pluggable Backends** | Prometheus, JSON, and Logging export with custom backend support | -| **Thread Dump** | Point-in-time worker state snapshots | - -### Source Files - -**Diagnostics** (`kcenon::thread::diagnostics` namespace): - -| Header | Description | -|--------|-------------| -| [`thread_pool_diagnostics.h`](../include/kcenon/thread/diagnostics/thread_pool_diagnostics.h) | Central diagnostics API | -| [`health_status.h`](../include/kcenon/thread/diagnostics/health_status.h) | Health state, thresholds, component health | -| [`bottleneck_report.h`](../include/kcenon/thread/diagnostics/bottleneck_report.h) | Bottleneck types, severity, recommendations | -| [`execution_event.h`](../include/kcenon/thread/diagnostics/execution_event.h) | Event types, listener interface | -| [`job_info.h`](../include/kcenon/thread/diagnostics/job_info.h) | Job lifecycle and timing | -| [`thread_info.h`](../include/kcenon/thread/diagnostics/thread_info.h) | Worker state and statistics | - -**Metrics** (`kcenon::thread::metrics` namespace): - -| Header | Description | -|--------|-------------| -| [`metrics_service.h`](../include/kcenon/thread/metrics/metrics_service.h) | Centralized metrics management | -| [`metrics_base.h`](../include/kcenon/thread/metrics/metrics_base.h) | Abstract base with 5 atomic counters | -| [`thread_pool_metrics.h`](../include/kcenon/thread/metrics/thread_pool_metrics.h) | Lightweight metrics (6 counters) | -| [`enhanced_metrics.h`](../include/kcenon/thread/metrics/enhanced_metrics.h) | Histograms, percentiles, per-worker tracking | -| [`latency_histogram.h`](../include/kcenon/thread/metrics/latency_histogram.h) | HDR-style 64-bucket histogram | -| [`sliding_window_counter.h`](../include/kcenon/thread/metrics/sliding_window_counter.h) | Lock-free throughput counter | -| [`metrics_backend.h`](../include/kcenon/thread/metrics/metrics_backend.h) | Backend interface and built-in implementations | - -### When to Use - -| Scenario | Recommendation | -|----------|---------------| -| Quick health check in HTTP endpoint | `diagnostics().health_check()` | -| Production monitoring with Prometheus | Enable enhanced metrics + `PrometheusBackend` | -| Debugging slow task execution | `diagnostics().detect_bottlenecks()` + event tracing | -| Lightweight embedded deployment | Basic `ThreadPoolMetrics` only (48 bytes) | -| Per-worker performance analysis | `EnhancedThreadPoolMetrics` with `worker_metrics()` | -| Custom monitoring integration | Implement `MetricsBackend` interface | - ---- - -## Diagnostics Subsystem - -### Architecture - -The `thread_pool_diagnostics` class provides a non-intrusive, read-only view into thread pool state: - -``` -thread_pool_diagnostics - | - +-- Thread Dump (dump_thread_states, format_thread_dump) - +-- Job Inspection (get_active_jobs, get_pending_jobs, get_recent_jobs) - +-- Bottleneck Detect (detect_bottlenecks -> bottleneck_report) - +-- Health Check (health_check -> health_status) - +-- Event Tracing (enable_tracing, add_event_listener) - +-- Export (to_json, to_string, to_prometheus) -``` - -### Design Principles - -| Principle | Description | -|-----------|-------------| -| **Non-intrusive** | Minimal overhead when not actively queried | -| **Thread-safe** | All methods callable from any thread | -| **Read-only** | Never modifies thread pool state | -| **Snapshot-based** | Returns point-in-time copies, not live references | - -### Construction - -```cpp -#include -using namespace kcenon::thread; -using namespace kcenon::thread::diagnostics; - -auto pool = std::make_shared("MyPool"); -pool->start(); - -// Access diagnostics via the pool's built-in diagnostics -auto& diag = pool->diagnostics(); - -// Or create standalone diagnostics with custom config -diagnostics_config config; -config.recent_jobs_capacity = 5000; -config.enable_tracing = true; -thread_pool_diagnostics custom_diag(*pool, config); -``` - -### Thread Dump - -Thread dump provides a snapshot of all worker states: - -```cpp -// Structured data -auto threads = diag.dump_thread_states(); -for (const auto& t : threads) { - LOG_INFO("Worker {} ({}): {} jobs done, {:.1f}% utilization", - t.thread_name, worker_state_to_string(t.state), - t.jobs_completed, t.utilization * 100.0); -} - -// Human-readable format -std::cout << diag.format_thread_dump() << std::endl; -``` - -Output format: - -``` -=== Thread Pool Dump: MyPool === -Time: 2025-01-08T10:30:00Z -Workers: 8, Active: 5, Idle: 3 - -Worker-0 [tid:12345] ACTIVE (2.5s) - Current Job: ProcessOrder#1234 (running 150ms) - Jobs: 1523 completed, 2 failed - Utilization: 87.3% -``` - -#### Worker States - -| State | Meaning | -|-------|---------| -| `idle` | Worker is waiting for jobs | -| `active` | Worker is executing a job | -| `stopping` | Worker is in the process of stopping | -| `stopped` | Worker has stopped | - -#### thread_info Fields - -| Field | Type | Description | -|-------|------|-------------| -| `thread_id` | `std::thread::id` | System thread ID | -| `thread_name` | `std::string` | Human-readable name (e.g., "Worker-0") | -| `worker_id` | `std::size_t` | Pool-assigned worker ID | -| `state` | `worker_state` | Current operational state | -| `state_since` | `steady_clock::time_point` | When the current state began | -| `current_job` | `std::optional` | Current job (if `active`) | -| `jobs_completed` | `std::uint64_t` | Total successful completions | -| `jobs_failed` | `std::uint64_t` | Total failures | -| `total_busy_time` | `std::chrono::nanoseconds` | Cumulative busy time | -| `total_idle_time` | `std::chrono::nanoseconds` | Cumulative idle time | -| `utilization` | `double` | `busy / (busy + idle)`, 0.0 to 1.0 | - -### Job Inspection - -```cpp -// Currently executing jobs -auto active = diag.get_active_jobs(); - -// Pending jobs in queue (default limit: 100) -auto pending = diag.get_pending_jobs(50); - -// Recent completed/failed jobs (ring buffer, default limit: 100) -auto recent = diag.get_recent_jobs(200); - -for (const auto& job : active) { - if (job.status == job_status::running) { - LOG_INFO("Job {} running for {:.1f}ms", - job.job_name, job.execution_time_ms()); - } -} -``` - -#### Job Lifecycle - -``` -enqueue_time start_time end_time - | | | - v v v - [=======wait_time=========][====execution_time==========] - |<----- pending ---------->|<-------- running --------->| -``` - -#### job_status Values - -| Status | Description | -|--------|-------------| -| `pending` | Job is waiting in the queue | -| `running` | Job is currently being executed | -| `completed` | Job completed successfully | -| `failed` | Job failed with an error | -| `cancelled` | Job was cancelled before completion | -| `timed_out` | Job exceeded its timeout limit | - -### Export Formats - -```cpp -// JSON export -std::string json = diag.to_json(); - -// Human-readable export -std::string text = diag.to_string(); - -// Prometheus exposition format -std::string prom = diag.to_prometheus(); -``` - -### Performance Characteristics - -| Operation | Complexity | Latency | -|-----------|-----------|---------| -| Thread dump | O(n) workers | < 1ms for 64 workers | -| Job inspection (active) | O(1) | < 100ns | -| Job inspection (history) | O(n) | < 1ms for 1000 entries | -| Bottleneck detection | O(n) workers | < 1ms for 64 workers | -| Health check | O(n) all components | < 1ms | -| Event tracing | O(1) per event | < 1us | - ---- - -## Health Monitoring - -### Health States - -The health system uses four states compatible with Kubernetes probes and standard health check frameworks: - -| State | HTTP Code | Meaning | -|-------|-----------|---------| -| `healthy` | 200 | Component is fully operational | -| `degraded` | 200 | Operational but with reduced capacity/performance | -| `unhealthy` | 503 | Not operational or failing | -| `unknown` | 503 | Health state cannot be determined | - -### Health Check API - -```cpp -auto health = pool->diagnostics().health_check(); - -// Quick boolean check -if (pool->diagnostics().is_healthy()) { - // All good -} - -// HTTP endpoint integration -auto status_code = health.http_status_code(); -auto body = health.to_json(); -return http_response(status_code, body); -``` - -### Health Components - -The health check evaluates three independent components: - -| Component | What It Checks | -|-----------|---------------| -| **workers** | Worker utilization, idle worker count | -| **queue** | Queue saturation level | -| **metrics** | Success rate, average latency | - -### Overall Status Aggregation - -``` -calculate_overall_status(): - If any component unhealthy -> overall = unhealthy - Else if any component degraded -> overall = degraded - Else if any component unknown -> overall = degraded - Else all healthy -> overall = healthy -``` - -### Health Thresholds - -Default thresholds (all configurable via `health_thresholds`): - -| Threshold | Default | Description | -|-----------|---------|-------------| -| `min_success_rate` | `0.95` | Below this, pool is degraded | -| `unhealthy_success_rate` | `0.8` | Below this, pool is unhealthy | -| `max_healthy_latency_ms` | `100.0` | Max latency for healthy status | -| `degraded_latency_ms` | `500.0` | Latency above which pool is degraded | -| `queue_saturation_warning` | `0.8` | Queue saturation for degraded status | -| `queue_saturation_critical` | `0.95` | Queue saturation for unhealthy status | -| `worker_utilization_warning` | `0.9` | Worker utilization for degraded status | -| `min_idle_workers` | `0` | Minimum idle workers for healthy (0 = disabled) | - -### Customizing Thresholds - -```cpp -diagnostics_config config; -config.health_thresholds_config.min_success_rate = 0.99; -config.health_thresholds_config.max_healthy_latency_ms = 50.0; -config.health_thresholds_config.queue_saturation_warning = 0.7; -config.health_thresholds_config.min_idle_workers = 2; - -thread_pool_diagnostics diag(*pool, config); -auto health = diag.health_check(); -``` - -### Kubernetes Integration - -```cpp -// Liveness probe -auto handle_liveness = [&]() { - auto health = pool->diagnostics().health_check(); - return http_response(health.http_status_code(), health.to_json()); -}; - -// Readiness probe (stricter: only 200 if fully healthy) -auto handle_readiness = [&]() { - auto health = pool->diagnostics().health_check(); - int code = (health.overall_status == health_state::healthy) ? 200 : 503; - return http_response(code, health.to_json()); -}; -``` - -### Prometheus Health Export - -```cpp -std::string prom = health.to_prometheus("my_pool"); -``` - -Output: - -``` -# HELP thread_pool_health_status Health status (1=healthy, 0.5=degraded, 0=unhealthy) -# TYPE thread_pool_health_status gauge -thread_pool_health_status{pool="my_pool"} 1 - -# HELP thread_pool_uptime_seconds Total uptime in seconds -# TYPE thread_pool_uptime_seconds counter -thread_pool_uptime_seconds{pool="my_pool"} 3600.50 - -# HELP thread_pool_success_rate Ratio of successful jobs (0.0 to 1.0) -# TYPE thread_pool_success_rate gauge -thread_pool_success_rate{pool="my_pool"} 0.9987 - -# HELP thread_pool_workers_active Number of active workers -# TYPE thread_pool_workers_active gauge -thread_pool_workers_active{pool="my_pool"} 5 - -# HELP thread_pool_queue_depth Current queue depth -# TYPE thread_pool_queue_depth gauge -thread_pool_queue_depth{pool="my_pool"} 42 -``` - -### JSON Health Output - -```json -{ - "status": "healthy", - "message": "All components are healthy", - "http_code": 200, - "metrics": { - "uptime_seconds": 3600.50, - "total_jobs_processed": 125000, - "success_rate": 0.9987, - "avg_latency_ms": 2.450 - }, - "workers": { - "total": 8, - "active": 5, - "idle": 3 - }, - "queue": { - "depth": 42, - "capacity": 1024 - }, - "components": [ - { - "name": "workers", - "status": "healthy", - "message": "All workers operational" - }, - { - "name": "queue", - "status": "healthy", - "message": "Queue within normal limits" - }, - { - "name": "metrics", - "status": "healthy", - "message": "Success rate and latency within thresholds" - } - ] -} -``` - ---- - -## Bottleneck Detection - -### Overview - -The `detect_bottlenecks()` method analyzes thread pool metrics to identify performance bottlenecks and provide actionable recommendations. - -```cpp -auto report = pool->diagnostics().detect_bottlenecks(); -if (report.has_bottleneck) { - LOG_WARN("Bottleneck: {} - {}", - bottleneck_type_to_string(report.type), - report.description); - for (const auto& rec : report.recommendations) { - LOG_INFO(" Recommendation: {}", rec); - } -} -``` - -### Bottleneck Types - -| Type | Description | Typical Cause | -|------|-------------|---------------| -| `none` | No bottleneck detected | System operating normally | -| `queue_full` | Queue is at capacity | Production rate exceeds consumption | -| `slow_consumer` | Workers can't keep up | Tasks take too long to execute | -| `worker_starvation` | Not enough workers | Pool too small for workload | -| `lock_contention` | High mutex wait times | Shared resource contention | -| `uneven_distribution` | Work not evenly spread | Work stealing needed | -| `memory_pressure` | Excessive allocations | Memory-intensive tasks | - -### Detection Logic - -The detection algorithm evaluates metrics in priority order: - -``` -Step 1: Check queue_saturation - queue_saturation > 0.9 --> queue_full - -Step 2: Check consumer speed - avg_wait_time > threshold AND worker_utilization > 0.9 --> slow_consumer - -Step 3: Check worker capacity - worker_utilization > 0.95 AND queue_saturation > 0.5 --> worker_starvation - -Step 4: Check distribution - utilization variance high --> uneven_distribution -``` - -### Severity Levels - -| Level | Name | Condition | Action | -|-------|------|-----------|--------| -| 0 | `none` | No bottleneck | No action needed | -| 1 | `low` | Bottleneck detected, metrics below warning thresholds | Monitor | -| 2 | `medium` | `queue_saturation > 0.8` OR `worker_utilization > 0.9` | Investigate | -| 3 | `critical` | `queue_saturation > 0.95` OR `worker_utilization > 0.98` | Immediate action | - -### Interpreting the Report - -The `bottleneck_report` contains supporting metrics for diagnosis: - -| Field | Type | Description | -|-------|------|-------------| -| `has_bottleneck` | `bool` | Whether any bottleneck was detected | -| `type` | `bottleneck_type` | Classification of the bottleneck | -| `description` | `std::string` | Human-readable explanation | -| `queue_saturation` | `double` | Queue depth / capacity (0.0 to 1.0+) | -| `avg_wait_time_ms` | `double` | Average queue wait time | -| `worker_utilization` | `double` | Average worker busy ratio (0.0 to 1.0) | -| `utilization_variance` | `double` | Variance in worker utilization | -| `estimated_backlog_time_ms` | `std::size_t` | Estimated time to clear queue | -| `queue_depth` | `std::size_t` | Current items in queue | -| `idle_workers` | `std::size_t` | Number of idle workers | -| `total_workers` | `std::size_t` | Total worker count | -| `jobs_rejected` | `std::uint64_t` | Jobs rejected due to full queue | -| `recommendations` | `vector` | Actionable suggestions | - -### Response Guide - -| Bottleneck | Recommended Actions | -|------------|-------------------| -| `queue_full` | Increase queue capacity, add workers, enable autoscaling | -| `slow_consumer` | Optimize task execution time, break large tasks, add workers | -| `worker_starvation` | Increase `min_workers`, enable autoscaling, reduce queue capacity | -| `lock_contention` | Reduce shared state, use lock-free algorithms, partition work | -| `uneven_distribution` | Enable work stealing, check task affinity, rebalance queue | -| `memory_pressure` | Pool objects, reduce allocations, use arena allocators | - -### Bottleneck Report Export - -```cpp -auto report = diag.detect_bottlenecks(); - -// JSON format -std::string json = report.to_json(); - -// Human-readable format -std::string text = report.to_string(); - -// Severity check -if (report.requires_immediate_action()) { - alert_on_call_team(report.to_json()); -} -``` - -Human-readable output: - -``` -=== Bottleneck Report === -Status: DETECTED (medium severity) -Type: slow_consumer -Description: Workers cannot keep up with job submission rate - -Metrics: - Queue: 100 items (75.0% saturated) - Workers: 7/8 active (1 idle) - Utilization: 92.0% (variance: 0.0500) - Wait time: 150.500ms avg - Backlog: ~5000ms to clear - -Recommendations: - - Add more worker threads - - Optimize job execution time -``` - ---- - -## Event Tracing - -### Overview - -Event tracing provides lifecycle tracking for individual jobs with the observer pattern. When enabled, every job state transition generates a `job_execution_event`. - -### Event Lifecycle - -``` -enqueued --> dequeued --> started --> completed/failed/cancelled - | - retried --> started --> ... -``` - -### Event Types - -| Type | Description | Timing Fields | -|------|-------------|---------------| -| `enqueued` | Job added to queue | `timestamp` only | -| `dequeued` | Job taken from queue | `wait_time` populated | -| `started` | Execution started | `wait_time` populated | -| `completed` | Successful completion | `wait_time`, `execution_time` | -| `failed` | Failed with error | `wait_time`, `execution_time`, `error_*` | -| `cancelled` | Cancelled before completion | `wait_time`, optionally `execution_time` | -| `retried` | Being retried after failure | `wait_time` | - -### Enabling Tracing - -```cpp -// Enable with default history size (1000 events) -pool->diagnostics().enable_tracing(true); - -// Enable with custom history size -pool->diagnostics().enable_tracing(true, 5000); - -// Check if tracing is enabled -if (pool->diagnostics().is_tracing_enabled()) { - // Tracing is active -} - -// Disable tracing -pool->diagnostics().enable_tracing(false); -``` - -### Custom Event Listener - -Implement the `execution_event_listener` interface to receive real-time events: - -```cpp -#include -using namespace kcenon::thread::diagnostics; - -class JsonEventLogger : public execution_event_listener { -public: - void on_event(const job_execution_event& event) override { - // IMPORTANT: This is called from worker threads. - // Keep processing fast (< 1us) to avoid impacting performance. - - if (event.is_terminal()) { - LOG_INFO("[{}] job:{} {} wait:{:.3f}ms exec:{:.3f}ms", - event.format_timestamp(), - event.job_name, - event_type_to_string(event.type), - event.wait_time_ms(), - event.execution_time_ms()); - } - - if (event.is_error() && event.error_message.has_value()) { - LOG_ERROR("Job {} failed: {}", - event.job_name, - event.error_message.value()); - } - } -}; - -// Register the listener -auto logger = std::make_shared(); -pool->diagnostics().add_event_listener(logger); - -// Later: remove the listener -pool->diagnostics().remove_event_listener(logger); -``` - -### Querying Event History - -```cpp -// Get recent events (default limit: 100) -auto events = pool->diagnostics().get_recent_events(50); -for (const auto& event : events) { - std::cout << event.to_string() << std::endl; -} -``` - -Event string output: - -``` -[2025-01-08T10:30:00.123Z] Event#123 job:ProcessOrder#456 type:completed - worker:0 thread:12345 wait:1.500ms exec:10.200ms -``` - -### job_execution_event Fields - -| Field | Type | Description | -|-------|------|-------------| -| `event_id` | `std::uint64_t` | Monotonically increasing ID | -| `job_id` | `std::uint64_t` | Job identifier | -| `job_name` | `std::string` | Human-readable job name | -| `type` | `event_type` | Event classification | -| `timestamp` | `steady_clock::time_point` | Monotonic timestamp | -| `system_timestamp` | `system_clock::time_point` | Wall-clock timestamp | -| `thread_id` | `std::thread::id` | Processing thread ID | -| `worker_id` | `std::size_t` | Worker identifier | -| `wait_time` | `std::chrono::nanoseconds` | Queue wait duration | -| `execution_time` | `std::chrono::nanoseconds` | Execution duration | -| `error_code` | `std::optional` | Error code (failures only) | -| `error_message` | `std::optional` | Error message (failures only) | - -### Performance Impact - -| Tracing Mode | Overhead per Event | -|--------------|-------------------| -| Disabled | 0 (no overhead) | -| Enabled, no listeners | < 1us (history recording only) | -| Enabled, with listeners | < 1us + listener processing time | - -> **Warning**: Listener `on_event()` implementations must be fast (< 1us). Use async logging or buffering for expensive operations. - ---- - -## Metrics Framework - -### Class Hierarchy - -``` -MetricsBase (abstract) - | - +-- ThreadPoolMetrics (lightweight: 6 atomic counters) - | - +-- EnhancedThreadPoolMetrics (full-featured: histograms + counters) -``` - -### MetricsBase - -The abstract base class providing 5 lock-free atomic counters: - -| Counter | Method | Description | -|---------|--------|-------------| -| `tasks_submitted_` | `record_submission(count)` | Tasks submitted to pool | -| `tasks_executed_` | `record_execution(ns, success)` | Successfully completed tasks | -| `tasks_failed_` | `record_execution(ns, false)` | Failed tasks | -| `total_busy_time_ns_` | `record_execution(ns, *)` | Cumulative execution time | -| `total_idle_time_ns_` | `record_idle_time(ns)` | Cumulative idle time | - -Computed properties: -- `utilization()`: `busy_time / (busy_time + idle_time)`, 0.0 to 1.0 -- `success_rate()`: `executed / (executed + failed)`, 0.0 to 1.0 -- `base_snapshot()`: Returns `BaseSnapshot` struct - -**Performance**: < 50ns per record, 40 bytes memory (5 atomic counters). - -### ThreadPoolMetrics - -Extends `MetricsBase` with one additional counter: - -```cpp -auto metrics = std::make_shared(); - -// Record operations -metrics->record_submission(); -metrics->record_enqueue(); -metrics->record_execution(50000, true); // 50us execution, success -metrics->record_execution(100000, false); // 100us execution, failure -metrics->record_idle_time(200000); // 200us idle - -// Query individual values -auto submitted = metrics->tasks_submitted(); -auto enqueued = metrics->tasks_enqueued(); -auto executed = metrics->tasks_executed(); -auto failed = metrics->tasks_failed(); -auto util = metrics->utilization(); -auto rate = metrics->success_rate(); - -// Point-in-time snapshot -auto snap = metrics->snapshot(); -// snap.tasks_submitted, snap.tasks_enqueued, snap.tasks_executed, -// snap.tasks_failed, snap.total_busy_time_ns, snap.total_idle_time_ns -``` - -**Performance**: < 50ns per record, 48 bytes memory (6 atomic counters). - -### EnhancedThreadPoolMetrics - -Full-featured metrics with histograms, throughput counters, and per-worker tracking: - -```cpp -auto metrics = std::make_shared(8); // 8 workers - -// Record metrics (called internally by thread_pool) -metrics->record_submission(); -metrics->record_enqueue(std::chrono::nanoseconds{1000}); -metrics->record_execution(std::chrono::nanoseconds{50000}, true); -metrics->record_wait_time(std::chrono::nanoseconds{5000}); -metrics->record_queue_depth(42); -metrics->record_worker_state(0, true, 50000); // worker 0 busy for 50us -metrics->set_active_workers(6); - -// Get comprehensive snapshot -auto snap = metrics->snapshot(); -LOG_INFO("P99 execution: {:.2f}us", snap.execution_latency_p99_us); -LOG_INFO("Throughput: {:.1f} ops/sec", snap.throughput_1s); -LOG_INFO("Queue depth: {} (peak: {})", snap.current_queue_depth, snap.peak_queue_depth); -LOG_INFO("Worker utilization: {:.1f}%", snap.worker_utilization * 100.0); - -// Access individual histograms -const auto& exec_hist = metrics->execution_latency(); -LOG_INFO("Exec P50={:.0f}ns P99={:.0f}ns mean={:.0f}ns", - exec_hist.p50(), exec_hist.p99(), exec_hist.mean()); - -// Per-worker analysis -auto workers = metrics->worker_metrics(); -for (const auto& w : workers) { - LOG_INFO("Worker {}: {} tasks, busy={:.1f}%", - w.worker_id, w.tasks_executed, - static_cast(w.busy_time_ns) / - (w.busy_time_ns + w.idle_time_ns + 1) * 100.0); -} - -// Export -std::string json = metrics->to_json(); -std::string prom = metrics->to_prometheus("my_pool"); - -// Scale worker tracking -metrics->update_worker_count(12); // Pool scaled to 12 workers -``` - -**Performance**: < 100ns per record, < 10us snapshot, < 1KB per histogram, < 4KB per counter (60s window). - -### EnhancedSnapshot Fields - -| Category | Field | Unit | Description | -|----------|-------|------|-------------| -| **Counters** | `tasks_submitted` | count | Total tasks submitted | -| | `tasks_executed` | count | Successful completions | -| | `tasks_failed` | count | Failed tasks | -| **Enqueue Latency** | `enqueue_latency_p50_us` | us | Median enqueue time | -| | `enqueue_latency_p90_us` | us | P90 enqueue time | -| | `enqueue_latency_p99_us` | us | P99 enqueue time | -| **Execution Latency** | `execution_latency_p50_us` | us | Median execution time | -| | `execution_latency_p90_us` | us | P90 execution time | -| | `execution_latency_p99_us` | us | P99 execution time | -| **Wait Time** | `wait_time_p50_us` | us | Median queue wait | -| | `wait_time_p90_us` | us | P90 queue wait | -| | `wait_time_p99_us` | us | P99 queue wait | -| **Throughput** | `throughput_1s` | ops/sec | 1-second window rate | -| | `throughput_1m` | ops/sec | 1-minute window average | -| **Queue** | `current_queue_depth` | count | Current queue size | -| | `peak_queue_depth` | count | Peak since reset | -| | `avg_queue_depth` | count | Sampled average | -| **Workers** | `worker_utilization` | ratio | Overall utilization (0.0-1.0) | -| | `per_worker_utilization` | vector | Per-worker ratios | -| | `active_workers` | count | Currently active workers | -| **Timing** | `total_busy_time_ns` | ns | Cumulative busy time | -| | `total_idle_time_ns` | ns | Cumulative idle time | -| | `snapshot_time` | time_point | When snapshot was taken | - ---- - -## Latency Histograms - -### Design - -`LatencyHistogram` is an HDR-style histogram using 64 logarithmic buckets to efficiently capture latency distributions across a wide range (nanoseconds to seconds): - -``` -Bucket Range (nanoseconds) Resolution - 0 [0, 1) 1 ns - 1 [1, 2) 1 ns - 2 [2, 4) 2 ns - 3 [4, 8) 4 ns - ... - 10 [512, 1024) 512 ns (~1us) - 20 [524288, 1048576) ~500us (~1ms) - 30 [~500M, ~1B) ~500ms (~1s) - ... - 63 [2^62, 2^63) ~10^18 ns -``` - -### Properties - -| Property | Value | -|----------|-------| -| Bucket count | 64 (fixed) | -| Memory footprint | < 1KB | -| Record overhead | < 100ns (lock-free) | -| Range | 0 to ~10^19 nanoseconds | -| Accuracy | Within 1% for percentiles | -| Thread safety | Fully lock-free | - -### API - -```cpp -LatencyHistogram histogram; - -// Record values -histogram.record(std::chrono::nanoseconds{1500}); -histogram.record_ns(2500); - -// Percentiles (returned in nanoseconds) -double median = histogram.p50(); -double p90 = histogram.p90(); -double p95 = histogram.p95(); -double p99 = histogram.p99(); -double p999 = histogram.p999(); -double custom = histogram.percentile(0.75); // P75 - -// Statistics -double avg = histogram.mean(); -double sd = histogram.stddev(); -auto min_val = histogram.min(); -auto max_val = histogram.max(); -auto total = histogram.count(); -auto sum_val = histogram.sum(); - -// State management -bool is_empty = histogram.empty(); -histogram.reset(); - -// Merge another histogram -LatencyHistogram other; -// ... record into other ... -histogram.merge(other); - -// Bucket inspection -for (std::size_t i = 0; i < LatencyHistogram::BUCKET_COUNT; ++i) { - auto count = histogram.bucket_count(i); - if (count > 0) { - LOG_DEBUG("Bucket {} [{}, {}): {} samples", - i, - LatencyHistogram::bucket_lower_bound(i), - LatencyHistogram::bucket_upper_bound(i), - count); - } -} -``` - ---- - -## Sliding Window Counters - -### Design - -`SlidingWindowCounter` tracks event rates using a lock-free circular buffer of time buckets. As time advances, old buckets are automatically invalidated and reused. - -``` -Window: 1 second, 10 buckets per second - -Time: [0-100ms] [100-200ms] [200-300ms] ... [900-1000ms] -Buckets: [ 5 ] [ 3 ] [ 7 ] ... [ 4 ] - ^ - current bucket -``` - -### Properties - -| Property | Value | -|----------|-------| -| Default buckets per second | 10 (`DEFAULT_BUCKETS_PER_SECOND`) | -| Increment overhead | O(1), lock-free | -| Rate calculation | O(bucket_count) | -| Memory (1s window) | 10 buckets x 16 bytes = 160 bytes | -| Memory (60s window) | 600 buckets x 16 bytes = ~10KB | -| Thread safety | Fully lock-free | - -### API - -```cpp -using namespace std::chrono_literals; - -// 1-second window with default 10 buckets/second -SlidingWindowCounter counter_1s(1s); - -// 1-minute window with 10 buckets/second -SlidingWindowCounter counter_1m(60s); - -// Custom precision: 100 buckets/second for 5-second window -SlidingWindowCounter precise_counter(5s, 100); - -// Increment -counter_1s.increment(); // Add 1 -counter_1s.increment(10); // Add 10 - -// Query rates -double rate = counter_1s.rate_per_second(); -auto in_window = counter_1s.total_in_window(); -auto all_time = counter_1s.all_time_total(); - -// Metadata -auto window = counter_1s.window_size(); // 1s -auto buckets = counter_1s.bucket_count(); // 10 - -// Reset -counter_1s.reset(); -``` - ---- - -## Metrics Service - -### Overview - -`metrics_service` is the centralized entry point for metrics management, owned by `thread_pool` and shared with `thread_worker` instances. - -### Ownership Model - -``` -thread_pool - | - +-- owns --> metrics_service (shared_ptr) - | - +-- basic_metrics_ (always initialized) - +-- enhanced_metrics_ (lazily initialized) - -thread_worker - | - +-- non-owning pointer --> metrics_service -``` - -### API - -```cpp -auto svc = std::make_shared(); - -// === Recording (called by thread_pool and thread_worker) === -svc->record_submission(); -svc->record_enqueue(); -svc->record_enqueue_with_latency(std::chrono::nanoseconds{500}); -svc->record_execution(50000, true); // 50us, success -svc->record_execution_with_wait_time( - std::chrono::nanoseconds{50000}, // execution - std::chrono::nanoseconds{5000}, // wait - true); // success -svc->record_idle_time(200000); -svc->record_queue_depth(42); -svc->record_worker_state(0, true, 50000); - -// === Enhanced Metrics Control === -svc->set_enhanced_metrics_enabled(true, 8); // Enable with 8 workers -bool enabled = svc->is_enhanced_metrics_enabled(); -svc->update_worker_count(12); // Pool scaled -svc->set_active_workers(10); - -// === Query === -const auto& basic = svc->basic_metrics(); // Always available -auto basic_ptr = svc->get_basic_metrics(); // shared_ptr for workers -const auto& enhanced = svc->enhanced_metrics(); // Throws if not enabled -auto snap = svc->enhanced_snapshot(); // Empty snapshot if disabled - -// === Management === -svc->reset(); // Reset all metrics -``` - -### Thread Safety - -| Property | Mechanism | -|----------|-----------| -| Atomic counters | `std::memory_order_relaxed` for writes | -| Enhanced toggle | `std::atomic` | -| Lazy initialization | `std::mutex` for one-time init | -| Ownership | Non-copyable, non-movable | - ---- - -## Metrics Backends - -### MetricsBackend Interface - -The `MetricsBackend` abstract class defines the contract for exporting metrics to monitoring systems: - -```cpp -class MetricsBackend { -public: - virtual ~MetricsBackend() = default; - - // Required overrides - virtual std::string name() const = 0; - virtual std::string export_base(const BaseSnapshot& snapshot) const = 0; - virtual std::string export_enhanced(const EnhancedSnapshot& snapshot) const = 0; - - // Optional overrides (with defaults) - virtual void set_prefix(const std::string& prefix); // default: "thread_pool" - virtual void add_label(const std::string& key, const std::string& value); -}; -``` - -### Built-in Backends - -#### PrometheusBackend - -Exports metrics in Prometheus/OpenMetrics exposition format: - -```cpp -auto backend = std::make_shared(); -backend->set_prefix("myapp_pool"); -backend->add_label("instance", "worker-01"); - -auto snap = metrics->base_snapshot(); -std::string output = backend->export_base(snap); -``` - -Output: - -``` -# HELP myapp_pool_tasks_submitted_total Total tasks submitted -# TYPE myapp_pool_tasks_submitted_total counter -myapp_pool_tasks_submitted_total{instance="worker-01"} 1234 -``` - -#### JsonBackend - -Exports metrics as structured JSON: - -```cpp -auto backend = std::make_shared(); -backend->set_pretty(true); // Indented output (default) -backend->set_pretty(false); // Compact output - -auto snap = metrics->base_snapshot(); -std::string output = backend->export_base(snap); -``` - -Output (pretty): - -```json -{ - "tasks": { - "submitted": 1234, - "executed": 1200, - "failed": 5 - } -} -``` - -#### LoggingBackend - -Exports metrics in human-readable format for log files: - -```cpp -auto backend = std::make_shared(); -std::string output = backend->export_base(snap); -``` - -### BackendRegistry - -The singleton `BackendRegistry` manages available backends: - -```cpp -auto& registry = BackendRegistry::instance(); - -// All 3 defaults are auto-registered -bool has_prom = registry.has("prometheus"); // true -bool has_json = registry.has("json"); // true -bool has_log = registry.has("logging"); // true - -// Get a backend by name -auto prom = registry.get("prometheus"); -if (prom) { - std::string output = prom->export_base(snap); -} - -// Register a custom backend -registry.register_backend(std::make_shared()); -``` - -### Implementing a Custom Backend - -```cpp -#include -using namespace kcenon::thread::metrics; - -class StatsDBBackend : public MetricsBackend { -public: - std::string name() const override { - return "statsdb"; - } - - std::string export_base(const BaseSnapshot& snap) const override { - std::ostringstream oss; - oss << prefix() << ".submitted " << snap.tasks_submitted << "\n"; - oss << prefix() << ".executed " << snap.tasks_executed << "\n"; - oss << prefix() << ".failed " << snap.tasks_failed << "\n"; - - // Include configured labels - for (const auto& [key, value] : labels()) { - oss << "# label " << key << "=" << value << "\n"; - } - return oss.str(); - } - - std::string export_enhanced(const EnhancedSnapshot& snap) const override { - std::ostringstream oss; - oss << prefix() << ".exec_p99_us " << snap.execution_latency_p99_us << "\n"; - oss << prefix() << ".throughput_1s " << snap.throughput_1s << "\n"; - oss << prefix() << ".queue_depth " << snap.current_queue_depth << "\n"; - return oss.str(); - } -}; - -// Register it -BackendRegistry::instance().register_backend( - std::make_shared()); -``` - ---- - -## Usage Examples - -### Complete Diagnostics + Metrics Setup - -```cpp -#include -#include -#include - -using namespace kcenon::thread; -using namespace kcenon::thread::diagnostics; -using namespace kcenon::thread::metrics; - -// === 1. Create and configure the pool === -auto pool = std::make_shared("WorkerPool"); -pool->start(8); // 8 workers - -// === 2. Enable enhanced metrics === -pool->set_enhanced_metrics_enabled(true); - -// === 3. Configure diagnostics === -diagnostics_config diag_config; -diag_config.enable_tracing = true; -diag_config.recent_jobs_capacity = 5000; -diag_config.queue_saturation_high = 0.75; -diag_config.health_thresholds_config.min_success_rate = 0.99; - -// === 4. Add event listener === -class AlertListener : public execution_event_listener { -public: - void on_event(const job_execution_event& event) override { - if (event.type == event_type::failed) { - // Alert on failures - LOG_ERROR("Job {} failed: {}", - event.job_name, - event.error_message.value_or("unknown error")); - } - } -}; -pool->diagnostics().add_event_listener(std::make_shared()); - -// === 5. Submit work === -for (int i = 0; i < 1000; ++i) { - pool->enqueue(make_job() - .name("Task-" + std::to_string(i)) - .work([]() -> common::VoidResult { - // ... do work ... - return common::ok(); - }) - .build()); -} - -// === 6. Monitor health === -auto health = pool->diagnostics().health_check(); -LOG_INFO("Pool health: {} (HTTP {})", - health_state_to_string(health.overall_status), - health.http_status_code()); - -// === 7. Check for bottlenecks === -auto report = pool->diagnostics().detect_bottlenecks(); -if (report.has_bottleneck) { - LOG_WARN("Bottleneck detected: {} (severity: {})", - bottleneck_type_to_string(report.type), - report.severity_string()); -} - -// === 8. Export metrics === -auto& registry = BackendRegistry::instance(); - -// Prometheus format -auto prom = registry.get("prometheus"); -prom->set_prefix("myapp_worker_pool"); -prom->add_label("service", "order-processor"); -// ... use prom->export_enhanced(snap) in your /metrics endpoint - -// JSON format for dashboards -auto json_backend = registry.get("json"); -// ... use json_backend->export_enhanced(snap) in your REST API - -// === 9. Thread dump for debugging === -std::cout << pool->diagnostics().format_thread_dump() << std::endl; -``` - -### Periodic Monitoring Loop - -```cpp -void monitoring_loop(std::shared_ptr pool) { - while (pool->is_running()) { - // Health check - auto health = pool->diagnostics().health_check(); - - if (!health.is_healthy()) { - LOG_WARN("Pool degraded: {}", health.status_message); - - // Detailed bottleneck analysis - auto report = pool->diagnostics().detect_bottlenecks(); - if (report.severity() >= 2) { - for (const auto& rec : report.recommendations) { - LOG_WARN(" Action: {}", rec); - } - } - } - - // Metrics snapshot - auto snap = pool->metrics_service()->enhanced_snapshot(); - LOG_INFO("Throughput: {:.0f}/s P99: {:.1f}us Queue: {}", - snap.throughput_1s, - snap.execution_latency_p99_us, - snap.current_queue_depth); - - std::this_thread::sleep_for(std::chrono::seconds(5)); - } -} -``` - -### HTTP Health Endpoint - -```cpp -// Kubernetes liveness probe -auto handle_healthz = [&pool]() -> HttpResponse { - auto health = pool->diagnostics().health_check(); - return HttpResponse{ - health.http_status_code(), - "application/json", - health.to_json() - }; -}; - -// Prometheus metrics scrape endpoint -auto handle_metrics = [&pool]() -> HttpResponse { - auto snap = pool->metrics_service()->enhanced_snapshot(); - auto prom = BackendRegistry::instance().get("prometheus"); - return HttpResponse{ - 200, - "text/plain; version=0.0.4; charset=utf-8", - prom->export_enhanced(snap) - }; -}; -``` - ---- - -## Configuration Reference - -### diagnostics_config Defaults - -| Field | Default | Description | -|-------|---------|-------------| -| `recent_jobs_capacity` | `1000` | Max recent jobs in ring buffer | -| `event_history_size` | `1000` | Max events in history buffer | -| `enable_tracing` | `false` | Auto-enable event tracing | -| `queue_saturation_high` | `0.8` | High watermark for queue saturation | -| `utilization_high_threshold` | `0.9` | Worker utilization warning threshold | -| `wait_time_threshold_ms` | `100.0` | Wait time for slow consumer detection | -| `health_thresholds_config` | `{}` | Nested health threshold defaults | - -### health_thresholds Defaults - -| Field | Default | Description | -|-------|---------|-------------| -| `min_success_rate` | `0.95` | Below = degraded | -| `unhealthy_success_rate` | `0.8` | Below = unhealthy | -| `max_healthy_latency_ms` | `100.0` | Above = not fully healthy | -| `degraded_latency_ms` | `500.0` | Above = degraded | -| `queue_saturation_warning` | `0.8` | Queue warning threshold | -| `queue_saturation_critical` | `0.95` | Queue critical threshold | -| `worker_utilization_warning` | `0.9` | Worker warning threshold | -| `min_idle_workers` | `0` | Min idle workers (0 = disabled) | - -### Memory Budget - -| Component | Memory Usage | -|-----------|-------------| -| `MetricsBase` | 40 bytes (5 atomic counters) | -| `ThreadPoolMetrics` | 48 bytes (6 atomic counters) | -| `LatencyHistogram` | < 1KB (64 buckets + counters) | -| `SlidingWindowCounter` (1s) | ~160 bytes (10 buckets) | -| `SlidingWindowCounter` (60s) | ~10KB (600 buckets) | -| `EnhancedThreadPoolMetrics` | ~25KB base + ~32 bytes per worker | -| `diagnostics_config` | ~100 bytes | - ---- - -## Anti-Patterns - -### 1. Polling Diagnostics in a Tight Loop - -```cpp -// BAD: Wastes CPU on constant diagnostics polling -while (true) { - auto health = diag.health_check(); // O(n) per call - // No sleep! -} - -// GOOD: Poll at reasonable intervals -while (true) { - auto health = diag.health_check(); - std::this_thread::sleep_for(std::chrono::seconds(5)); -} -``` - -### 2. Slow Event Listeners - -```cpp -// BAD: Blocking I/O in event listener (blocks worker thread) -class SlowListener : public execution_event_listener { - void on_event(const job_execution_event& event) override { - write_to_database(event); // 10ms+ blocking call - } -}; - -// GOOD: Buffer events and process asynchronously -class AsyncListener : public execution_event_listener { - void on_event(const job_execution_event& event) override { - event_queue_.push(event.to_json()); // < 1us - } -private: - concurrent_queue event_queue_; -}; -``` - -### 3. Ignoring Enhanced Metrics Lifecycle - -```cpp -// BAD: Accessing enhanced metrics without checking -try { - const auto& em = svc->enhanced_metrics(); // Throws if not enabled! -} catch (...) { } - -// GOOD: Check first or use snapshot -if (svc->is_enhanced_metrics_enabled()) { - const auto& em = svc->enhanced_metrics(); -} -// Or: always-safe snapshot (returns empty if disabled) -auto snap = svc->enhanced_snapshot(); -``` - -### 4. Creating Multiple Diagnostics Instances - -```cpp -// BAD: Multiple diagnostics for the same pool (wasted memory) -thread_pool_diagnostics diag1(*pool); -thread_pool_diagnostics diag2(*pool); - -// GOOD: Use the pool's built-in diagnostics -auto& diag = pool->diagnostics(); -``` - -### 5. Not Resetting Metrics After Scale Events - -```cpp -// BAD: Metrics from old worker count skew per-worker stats -pool->resize(16); -// ... per_worker_utilization still has 8 entries ... - -// GOOD: Update worker count in metrics -pool->resize(16); -svc->update_worker_count(16); -``` - ---- - -## Troubleshooting - -### Health Check Returns `unknown` - -**Cause**: No components registered, or pool not started. - -**Solution**: -```cpp -// Ensure pool is started before health checks -pool->start(); -// Then check health -auto health = pool->diagnostics().health_check(); -``` - -### Bottleneck Detection Shows `none` Despite Slow Tasks - -**Cause**: Thresholds may be too lenient for your workload. - -**Solution**: -```cpp -diagnostics_config config; -config.queue_saturation_high = 0.5; // More sensitive -config.utilization_high_threshold = 0.7; // More sensitive -config.wait_time_threshold_ms = 10.0; // Stricter wait time -``` - -### Enhanced Metrics Snapshot Returns All Zeros - -**Cause**: Enhanced metrics not enabled, or no data recorded yet. - -**Solution**: -```cpp -// 1. Enable enhanced metrics BEFORE submitting work -svc->set_enhanced_metrics_enabled(true, worker_count); - -// 2. Submit work and wait for processing - -// 3. Then take snapshot -auto snap = svc->enhanced_snapshot(); -``` - -### Prometheus Metrics Missing Labels - -**Cause**: Labels must be set on the backend before exporting. - -**Solution**: -```cpp -auto prom = BackendRegistry::instance().get("prometheus"); -prom->set_prefix("myapp_pool"); -prom->add_label("service", "api"); -prom->add_label("env", "production"); -// NOW export -auto output = prom->export_enhanced(snap); -``` - -### Event Listener Not Receiving Events - -**Cause**: Tracing may not be enabled, or listener was not registered. - -**Solution**: -```cpp -// 1. Enable tracing -pool->diagnostics().enable_tracing(true); - -// 2. Register listener BEFORE submitting work -pool->diagnostics().add_event_listener(my_listener); - -// 3. Verify tracing is enabled -assert(pool->diagnostics().is_tracing_enabled()); -``` - -### Per-Worker Utilization Shows 0% for All Workers - -**Cause**: `record_worker_state()` is not being called with duration data. - -**Solution**: Ensure the thread pool implementation calls `record_worker_state(worker_id, busy, duration_ns)` on each state transition with the duration spent in the previous state. - ---- - -## Related Documentation - -- [Architecture Overview](ARCHITECTURE.md) - System-wide architecture -- [Policy Queue Combinations Guide](POLICY_QUEUE_GUIDE.md) - Queue and policy configurations -- [NUMA Work-Stealing Guide](NUMA_GUIDE.md) - NUMA topology and work stealing -- [Autoscaler Guide](AUTOSCALER_GUIDE.md) - Dynamic worker pool sizing +This guide has been split into two focused documents for easier navigation: + +| Document | Description | Topics | +|----------|-------------|--------| +| **[Diagnostics Guide](DIAGNOSTICS_GUIDE.md)** | Thread dump, job inspection, diagnostics subsystem | Diagnostics architecture, thread dump, job inspection, bottleneck detection, event tracing | +| **[Metrics Guide](METRICS_GUIDE.md)** | Health monitoring, Prometheus, Kubernetes integration | Health states, health thresholds, Kubernetes probes, Prometheus export, metrics framework, latency histograms, sliding window counters, metrics service, metrics backends, usage examples, configuration reference, anti-patterns, troubleshooting | + +## Quick Links + +### Diagnostics +- [Diagnostics Subsystem](DIAGNOSTICS_GUIDE.md#diagnostics-subsystem) +- [Thread Dump](DIAGNOSTICS_GUIDE.md#thread-dump) +- [Job Inspection](DIAGNOSTICS_GUIDE.md#job-inspection) +- [Bottleneck Detection](DIAGNOSTICS_GUIDE.md#bottleneck-detection) +- [Event Tracing](DIAGNOSTICS_GUIDE.md#event-tracing) + +### Metrics +- [Health Monitoring](METRICS_GUIDE.md#health-monitoring) +- [Kubernetes Integration](METRICS_GUIDE.md#kubernetes-integration) +- [Prometheus Health Export](METRICS_GUIDE.md#prometheus-health-export) +- [Metrics Framework](METRICS_GUIDE.md#metrics-framework) +- [Latency Histograms](METRICS_GUIDE.md#latency-histograms) +- [Sliding Window Counters](METRICS_GUIDE.md#sliding-window-counters) +- [Metrics Backends](METRICS_GUIDE.md#metrics-backends) +- [Configuration Reference](METRICS_GUIDE.md#configuration-reference) +- [Anti-Patterns](METRICS_GUIDE.md#anti-patterns) +- [Troubleshooting](METRICS_GUIDE.md#troubleshooting) diff --git a/docs/METRICS_GUIDE.md b/docs/METRICS_GUIDE.md new file mode 100644 index 0000000000..b76fe115ef --- /dev/null +++ b/docs/METRICS_GUIDE.md @@ -0,0 +1,1036 @@ +--- +doc_id: "THR-GUID-002b" +doc_title: "Metrics Guide" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "GUID" +--- + +# Metrics Guide + +> **SSOT**: This document is the single source of truth for **Metrics Guide** (health monitoring, Prometheus integration, Kubernetes probes, metrics backends). + +> **Language:** **English** + +> **See also**: [Diagnostics Guide](DIAGNOSTICS_GUIDE.md) for thread dump, job inspection, bottleneck detection, and event tracing. + +A comprehensive guide to thread_system's health monitoring, metrics framework, latency histograms, throughput counters, and pluggable export backends. + +## Table of Contents + +1. [Health Monitoring](#health-monitoring) +2. [Metrics Framework](#metrics-framework) +3. [Latency Histograms](#latency-histograms) +4. [Sliding Window Counters](#sliding-window-counters) +5. [Metrics Service](#metrics-service) +6. [Metrics Backends](#metrics-backends) +7. [Usage Examples](#usage-examples) +8. [Configuration Reference](#configuration-reference) +9. [Anti-Patterns](#anti-patterns) +10. [Troubleshooting](#troubleshooting) + +--- + +## Health Monitoring + +### Health States + +The health system uses four states compatible with Kubernetes probes and standard health check frameworks: + +| State | HTTP Code | Meaning | +|-------|-----------|---------| +| `healthy` | 200 | Component is fully operational | +| `degraded` | 200 | Operational but with reduced capacity/performance | +| `unhealthy` | 503 | Not operational or failing | +| `unknown` | 503 | Health state cannot be determined | + +### Health Check API + +```cpp +auto health = pool->diagnostics().health_check(); + +// Quick boolean check +if (pool->diagnostics().is_healthy()) { + // All good +} + +// HTTP endpoint integration +auto status_code = health.http_status_code(); +auto body = health.to_json(); +return http_response(status_code, body); +``` + +### Health Components + +The health check evaluates three independent components: + +| Component | What It Checks | +|-----------|---------------| +| **workers** | Worker utilization, idle worker count | +| **queue** | Queue saturation level | +| **metrics** | Success rate, average latency | + +### Overall Status Aggregation + +``` +calculate_overall_status(): + If any component unhealthy -> overall = unhealthy + Else if any component degraded -> overall = degraded + Else if any component unknown -> overall = degraded + Else all healthy -> overall = healthy +``` + +### Health Thresholds + +Default thresholds (all configurable via `health_thresholds`): + +| Threshold | Default | Description | +|-----------|---------|-------------| +| `min_success_rate` | `0.95` | Below this, pool is degraded | +| `unhealthy_success_rate` | `0.8` | Below this, pool is unhealthy | +| `max_healthy_latency_ms` | `100.0` | Max latency for healthy status | +| `degraded_latency_ms` | `500.0` | Latency above which pool is degraded | +| `queue_saturation_warning` | `0.8` | Queue saturation for degraded status | +| `queue_saturation_critical` | `0.95` | Queue saturation for unhealthy status | +| `worker_utilization_warning` | `0.9` | Worker utilization for degraded status | +| `min_idle_workers` | `0` | Minimum idle workers for healthy (0 = disabled) | + +### Customizing Thresholds + +```cpp +diagnostics_config config; +config.health_thresholds_config.min_success_rate = 0.99; +config.health_thresholds_config.max_healthy_latency_ms = 50.0; +config.health_thresholds_config.queue_saturation_warning = 0.7; +config.health_thresholds_config.min_idle_workers = 2; + +thread_pool_diagnostics diag(*pool, config); +auto health = diag.health_check(); +``` + +### Kubernetes Integration + +```cpp +// Liveness probe +auto handle_liveness = [&]() { + auto health = pool->diagnostics().health_check(); + return http_response(health.http_status_code(), health.to_json()); +}; + +// Readiness probe (stricter: only 200 if fully healthy) +auto handle_readiness = [&]() { + auto health = pool->diagnostics().health_check(); + int code = (health.overall_status == health_state::healthy) ? 200 : 503; + return http_response(code, health.to_json()); +}; +``` + +### Prometheus Health Export + +```cpp +std::string prom = health.to_prometheus("my_pool"); +``` + +Output: + +``` +# HELP thread_pool_health_status Health status (1=healthy, 0.5=degraded, 0=unhealthy) +# TYPE thread_pool_health_status gauge +thread_pool_health_status{pool="my_pool"} 1 + +# HELP thread_pool_uptime_seconds Total uptime in seconds +# TYPE thread_pool_uptime_seconds counter +thread_pool_uptime_seconds{pool="my_pool"} 3600.50 + +# HELP thread_pool_success_rate Ratio of successful jobs (0.0 to 1.0) +# TYPE thread_pool_success_rate gauge +thread_pool_success_rate{pool="my_pool"} 0.9987 + +# HELP thread_pool_workers_active Number of active workers +# TYPE thread_pool_workers_active gauge +thread_pool_workers_active{pool="my_pool"} 5 + +# HELP thread_pool_queue_depth Current queue depth +# TYPE thread_pool_queue_depth gauge +thread_pool_queue_depth{pool="my_pool"} 42 +``` + +### JSON Health Output + +```json +{ + "status": "healthy", + "message": "All components are healthy", + "http_code": 200, + "metrics": { + "uptime_seconds": 3600.50, + "total_jobs_processed": 125000, + "success_rate": 0.9987, + "avg_latency_ms": 2.450 + }, + "workers": { + "total": 8, + "active": 5, + "idle": 3 + }, + "queue": { + "depth": 42, + "capacity": 1024 + }, + "components": [ + { + "name": "workers", + "status": "healthy", + "message": "All workers operational" + }, + { + "name": "queue", + "status": "healthy", + "message": "Queue within normal limits" + }, + { + "name": "metrics", + "status": "healthy", + "message": "Success rate and latency within thresholds" + } + ] +} +``` + +--- + +## Metrics Framework + +### Class Hierarchy + +``` +MetricsBase (abstract) + | + +-- ThreadPoolMetrics (lightweight: 6 atomic counters) + | + +-- EnhancedThreadPoolMetrics (full-featured: histograms + counters) +``` + +### MetricsBase + +The abstract base class providing 5 lock-free atomic counters: + +| Counter | Method | Description | +|---------|--------|-------------| +| `tasks_submitted_` | `record_submission(count)` | Tasks submitted to pool | +| `tasks_executed_` | `record_execution(ns, success)` | Successfully completed tasks | +| `tasks_failed_` | `record_execution(ns, false)` | Failed tasks | +| `total_busy_time_ns_` | `record_execution(ns, *)` | Cumulative execution time | +| `total_idle_time_ns_` | `record_idle_time(ns)` | Cumulative idle time | + +Computed properties: +- `utilization()`: `busy_time / (busy_time + idle_time)`, 0.0 to 1.0 +- `success_rate()`: `executed / (executed + failed)`, 0.0 to 1.0 +- `base_snapshot()`: Returns `BaseSnapshot` struct + +**Performance**: < 50ns per record, 40 bytes memory (5 atomic counters). + +### ThreadPoolMetrics + +Extends `MetricsBase` with one additional counter: + +```cpp +auto metrics = std::make_shared(); + +// Record operations +metrics->record_submission(); +metrics->record_enqueue(); +metrics->record_execution(50000, true); // 50us execution, success +metrics->record_execution(100000, false); // 100us execution, failure +metrics->record_idle_time(200000); // 200us idle + +// Query individual values +auto submitted = metrics->tasks_submitted(); +auto enqueued = metrics->tasks_enqueued(); +auto executed = metrics->tasks_executed(); +auto failed = metrics->tasks_failed(); +auto util = metrics->utilization(); +auto rate = metrics->success_rate(); + +// Point-in-time snapshot +auto snap = metrics->snapshot(); +// snap.tasks_submitted, snap.tasks_enqueued, snap.tasks_executed, +// snap.tasks_failed, snap.total_busy_time_ns, snap.total_idle_time_ns +``` + +**Performance**: < 50ns per record, 48 bytes memory (6 atomic counters). + +### EnhancedThreadPoolMetrics + +Full-featured metrics with histograms, throughput counters, and per-worker tracking: + +```cpp +auto metrics = std::make_shared(8); // 8 workers + +// Record metrics (called internally by thread_pool) +metrics->record_submission(); +metrics->record_enqueue(std::chrono::nanoseconds{1000}); +metrics->record_execution(std::chrono::nanoseconds{50000}, true); +metrics->record_wait_time(std::chrono::nanoseconds{5000}); +metrics->record_queue_depth(42); +metrics->record_worker_state(0, true, 50000); // worker 0 busy for 50us +metrics->set_active_workers(6); + +// Get comprehensive snapshot +auto snap = metrics->snapshot(); +LOG_INFO("P99 execution: {:.2f}us", snap.execution_latency_p99_us); +LOG_INFO("Throughput: {:.1f} ops/sec", snap.throughput_1s); +LOG_INFO("Queue depth: {} (peak: {})", snap.current_queue_depth, snap.peak_queue_depth); +LOG_INFO("Worker utilization: {:.1f}%", snap.worker_utilization * 100.0); + +// Access individual histograms +const auto& exec_hist = metrics->execution_latency(); +LOG_INFO("Exec P50={:.0f}ns P99={:.0f}ns mean={:.0f}ns", + exec_hist.p50(), exec_hist.p99(), exec_hist.mean()); + +// Per-worker analysis +auto workers = metrics->worker_metrics(); +for (const auto& w : workers) { + LOG_INFO("Worker {}: {} tasks, busy={:.1f}%", + w.worker_id, w.tasks_executed, + static_cast(w.busy_time_ns) / + (w.busy_time_ns + w.idle_time_ns + 1) * 100.0); +} + +// Export +std::string json = metrics->to_json(); +std::string prom = metrics->to_prometheus("my_pool"); + +// Scale worker tracking +metrics->update_worker_count(12); // Pool scaled to 12 workers +``` + +**Performance**: < 100ns per record, < 10us snapshot, < 1KB per histogram, < 4KB per counter (60s window). + +### EnhancedSnapshot Fields + +| Category | Field | Unit | Description | +|----------|-------|------|-------------| +| **Counters** | `tasks_submitted` | count | Total tasks submitted | +| | `tasks_executed` | count | Successful completions | +| | `tasks_failed` | count | Failed tasks | +| **Enqueue Latency** | `enqueue_latency_p50_us` | us | Median enqueue time | +| | `enqueue_latency_p90_us` | us | P90 enqueue time | +| | `enqueue_latency_p99_us` | us | P99 enqueue time | +| **Execution Latency** | `execution_latency_p50_us` | us | Median execution time | +| | `execution_latency_p90_us` | us | P90 execution time | +| | `execution_latency_p99_us` | us | P99 execution time | +| **Wait Time** | `wait_time_p50_us` | us | Median queue wait | +| | `wait_time_p90_us` | us | P90 queue wait | +| | `wait_time_p99_us` | us | P99 queue wait | +| **Throughput** | `throughput_1s` | ops/sec | 1-second window rate | +| | `throughput_1m` | ops/sec | 1-minute window average | +| **Queue** | `current_queue_depth` | count | Current queue size | +| | `peak_queue_depth` | count | Peak since reset | +| | `avg_queue_depth` | count | Sampled average | +| **Workers** | `worker_utilization` | ratio | Overall utilization (0.0-1.0) | +| | `per_worker_utilization` | vector | Per-worker ratios | +| | `active_workers` | count | Currently active workers | +| **Timing** | `total_busy_time_ns` | ns | Cumulative busy time | +| | `total_idle_time_ns` | ns | Cumulative idle time | +| | `snapshot_time` | time_point | When snapshot was taken | + +--- + +## Latency Histograms + +### Design + +`LatencyHistogram` is an HDR-style histogram using 64 logarithmic buckets to efficiently capture latency distributions across a wide range (nanoseconds to seconds): + +``` +Bucket Range (nanoseconds) Resolution + 0 [0, 1) 1 ns + 1 [1, 2) 1 ns + 2 [2, 4) 2 ns + 3 [4, 8) 4 ns + ... + 10 [512, 1024) 512 ns (~1us) + 20 [524288, 1048576) ~500us (~1ms) + 30 [~500M, ~1B) ~500ms (~1s) + ... + 63 [2^62, 2^63) ~10^18 ns +``` + +### Properties + +| Property | Value | +|----------|-------| +| Bucket count | 64 (fixed) | +| Memory footprint | < 1KB | +| Record overhead | < 100ns (lock-free) | +| Range | 0 to ~10^19 nanoseconds | +| Accuracy | Within 1% for percentiles | +| Thread safety | Fully lock-free | + +### API + +```cpp +LatencyHistogram histogram; + +// Record values +histogram.record(std::chrono::nanoseconds{1500}); +histogram.record_ns(2500); + +// Percentiles (returned in nanoseconds) +double median = histogram.p50(); +double p90 = histogram.p90(); +double p95 = histogram.p95(); +double p99 = histogram.p99(); +double p999 = histogram.p999(); +double custom = histogram.percentile(0.75); // P75 + +// Statistics +double avg = histogram.mean(); +double sd = histogram.stddev(); +auto min_val = histogram.min(); +auto max_val = histogram.max(); +auto total = histogram.count(); +auto sum_val = histogram.sum(); + +// State management +bool is_empty = histogram.empty(); +histogram.reset(); + +// Merge another histogram +LatencyHistogram other; +// ... record into other ... +histogram.merge(other); + +// Bucket inspection +for (std::size_t i = 0; i < LatencyHistogram::BUCKET_COUNT; ++i) { + auto count = histogram.bucket_count(i); + if (count > 0) { + LOG_DEBUG("Bucket {} [{}, {}): {} samples", + i, + LatencyHistogram::bucket_lower_bound(i), + LatencyHistogram::bucket_upper_bound(i), + count); + } +} +``` + +--- + +## Sliding Window Counters + +### Design + +`SlidingWindowCounter` tracks event rates using a lock-free circular buffer of time buckets. As time advances, old buckets are automatically invalidated and reused. + +``` +Window: 1 second, 10 buckets per second + +Time: [0-100ms] [100-200ms] [200-300ms] ... [900-1000ms] +Buckets: [ 5 ] [ 3 ] [ 7 ] ... [ 4 ] + ^ + current bucket +``` + +### Properties + +| Property | Value | +|----------|-------| +| Default buckets per second | 10 (`DEFAULT_BUCKETS_PER_SECOND`) | +| Increment overhead | O(1), lock-free | +| Rate calculation | O(bucket_count) | +| Memory (1s window) | 10 buckets x 16 bytes = 160 bytes | +| Memory (60s window) | 600 buckets x 16 bytes = ~10KB | +| Thread safety | Fully lock-free | + +### API + +```cpp +using namespace std::chrono_literals; + +// 1-second window with default 10 buckets/second +SlidingWindowCounter counter_1s(1s); + +// 1-minute window with 10 buckets/second +SlidingWindowCounter counter_1m(60s); + +// Custom precision: 100 buckets/second for 5-second window +SlidingWindowCounter precise_counter(5s, 100); + +// Increment +counter_1s.increment(); // Add 1 +counter_1s.increment(10); // Add 10 + +// Query rates +double rate = counter_1s.rate_per_second(); +auto in_window = counter_1s.total_in_window(); +auto all_time = counter_1s.all_time_total(); + +// Metadata +auto window = counter_1s.window_size(); // 1s +auto buckets = counter_1s.bucket_count(); // 10 + +// Reset +counter_1s.reset(); +``` + +--- + +## Metrics Service + +### Overview + +`metrics_service` is the centralized entry point for metrics management, owned by `thread_pool` and shared with `thread_worker` instances. + +### Ownership Model + +``` +thread_pool + | + +-- owns --> metrics_service (shared_ptr) + | + +-- basic_metrics_ (always initialized) + +-- enhanced_metrics_ (lazily initialized) + +thread_worker + | + +-- non-owning pointer --> metrics_service +``` + +### API + +```cpp +auto svc = std::make_shared(); + +// === Recording (called by thread_pool and thread_worker) === +svc->record_submission(); +svc->record_enqueue(); +svc->record_enqueue_with_latency(std::chrono::nanoseconds{500}); +svc->record_execution(50000, true); // 50us, success +svc->record_execution_with_wait_time( + std::chrono::nanoseconds{50000}, // execution + std::chrono::nanoseconds{5000}, // wait + true); // success +svc->record_idle_time(200000); +svc->record_queue_depth(42); +svc->record_worker_state(0, true, 50000); + +// === Enhanced Metrics Control === +svc->set_enhanced_metrics_enabled(true, 8); // Enable with 8 workers +bool enabled = svc->is_enhanced_metrics_enabled(); +svc->update_worker_count(12); // Pool scaled +svc->set_active_workers(10); + +// === Query === +const auto& basic = svc->basic_metrics(); // Always available +auto basic_ptr = svc->get_basic_metrics(); // shared_ptr for workers +const auto& enhanced = svc->enhanced_metrics(); // Throws if not enabled +auto snap = svc->enhanced_snapshot(); // Empty snapshot if disabled + +// === Management === +svc->reset(); // Reset all metrics +``` + +### Thread Safety + +| Property | Mechanism | +|----------|-----------| +| Atomic counters | `std::memory_order_relaxed` for writes | +| Enhanced toggle | `std::atomic` | +| Lazy initialization | `std::mutex` for one-time init | +| Ownership | Non-copyable, non-movable | + +--- + +## Metrics Backends + +### MetricsBackend Interface + +The `MetricsBackend` abstract class defines the contract for exporting metrics to monitoring systems: + +```cpp +class MetricsBackend { +public: + virtual ~MetricsBackend() = default; + + // Required overrides + virtual std::string name() const = 0; + virtual std::string export_base(const BaseSnapshot& snapshot) const = 0; + virtual std::string export_enhanced(const EnhancedSnapshot& snapshot) const = 0; + + // Optional overrides (with defaults) + virtual void set_prefix(const std::string& prefix); // default: "thread_pool" + virtual void add_label(const std::string& key, const std::string& value); +}; +``` + +### Built-in Backends + +#### PrometheusBackend + +Exports metrics in Prometheus/OpenMetrics exposition format: + +```cpp +auto backend = std::make_shared(); +backend->set_prefix("myapp_pool"); +backend->add_label("instance", "worker-01"); + +auto snap = metrics->base_snapshot(); +std::string output = backend->export_base(snap); +``` + +Output: + +``` +# HELP myapp_pool_tasks_submitted_total Total tasks submitted +# TYPE myapp_pool_tasks_submitted_total counter +myapp_pool_tasks_submitted_total{instance="worker-01"} 1234 +``` + +#### JsonBackend + +Exports metrics as structured JSON: + +```cpp +auto backend = std::make_shared(); +backend->set_pretty(true); // Indented output (default) +backend->set_pretty(false); // Compact output + +auto snap = metrics->base_snapshot(); +std::string output = backend->export_base(snap); +``` + +Output (pretty): + +```json +{ + "tasks": { + "submitted": 1234, + "executed": 1200, + "failed": 5 + } +} +``` + +#### LoggingBackend + +Exports metrics in human-readable format for log files: + +```cpp +auto backend = std::make_shared(); +std::string output = backend->export_base(snap); +``` + +### BackendRegistry + +The singleton `BackendRegistry` manages available backends: + +```cpp +auto& registry = BackendRegistry::instance(); + +// All 3 defaults are auto-registered +bool has_prom = registry.has("prometheus"); // true +bool has_json = registry.has("json"); // true +bool has_log = registry.has("logging"); // true + +// Get a backend by name +auto prom = registry.get("prometheus"); +if (prom) { + std::string output = prom->export_base(snap); +} + +// Register a custom backend +registry.register_backend(std::make_shared()); +``` + +### Implementing a Custom Backend + +```cpp +#include +using namespace kcenon::thread::metrics; + +class StatsDBBackend : public MetricsBackend { +public: + std::string name() const override { + return "statsdb"; + } + + std::string export_base(const BaseSnapshot& snap) const override { + std::ostringstream oss; + oss << prefix() << ".submitted " << snap.tasks_submitted << "\n"; + oss << prefix() << ".executed " << snap.tasks_executed << "\n"; + oss << prefix() << ".failed " << snap.tasks_failed << "\n"; + + // Include configured labels + for (const auto& [key, value] : labels()) { + oss << "# label " << key << "=" << value << "\n"; + } + return oss.str(); + } + + std::string export_enhanced(const EnhancedSnapshot& snap) const override { + std::ostringstream oss; + oss << prefix() << ".exec_p99_us " << snap.execution_latency_p99_us << "\n"; + oss << prefix() << ".throughput_1s " << snap.throughput_1s << "\n"; + oss << prefix() << ".queue_depth " << snap.current_queue_depth << "\n"; + return oss.str(); + } +}; + +// Register it +BackendRegistry::instance().register_backend( + std::make_shared()); +``` + +--- + +## Usage Examples + +### Complete Diagnostics + Metrics Setup + +```cpp +#include +#include +#include + +using namespace kcenon::thread; +using namespace kcenon::thread::diagnostics; +using namespace kcenon::thread::metrics; + +// === 1. Create and configure the pool === +auto pool = std::make_shared("WorkerPool"); +pool->start(8); // 8 workers + +// === 2. Enable enhanced metrics === +pool->set_enhanced_metrics_enabled(true); + +// === 3. Configure diagnostics === +diagnostics_config diag_config; +diag_config.enable_tracing = true; +diag_config.recent_jobs_capacity = 5000; +diag_config.queue_saturation_high = 0.75; +diag_config.health_thresholds_config.min_success_rate = 0.99; + +// === 4. Add event listener === +class AlertListener : public execution_event_listener { +public: + void on_event(const job_execution_event& event) override { + if (event.type == event_type::failed) { + // Alert on failures + LOG_ERROR("Job {} failed: {}", + event.job_name, + event.error_message.value_or("unknown error")); + } + } +}; +pool->diagnostics().add_event_listener(std::make_shared()); + +// === 5. Submit work === +for (int i = 0; i < 1000; ++i) { + pool->enqueue(make_job() + .name("Task-" + std::to_string(i)) + .work([]() -> common::VoidResult { + // ... do work ... + return common::ok(); + }) + .build()); +} + +// === 6. Monitor health === +auto health = pool->diagnostics().health_check(); +LOG_INFO("Pool health: {} (HTTP {})", + health_state_to_string(health.overall_status), + health.http_status_code()); + +// === 7. Check for bottlenecks === +auto report = pool->diagnostics().detect_bottlenecks(); +if (report.has_bottleneck) { + LOG_WARN("Bottleneck detected: {} (severity: {})", + bottleneck_type_to_string(report.type), + report.severity_string()); +} + +// === 8. Export metrics === +auto& registry = BackendRegistry::instance(); + +// Prometheus format +auto prom = registry.get("prometheus"); +prom->set_prefix("myapp_worker_pool"); +prom->add_label("service", "order-processor"); +// ... use prom->export_enhanced(snap) in your /metrics endpoint + +// JSON format for dashboards +auto json_backend = registry.get("json"); +// ... use json_backend->export_enhanced(snap) in your REST API + +// === 9. Thread dump for debugging === +std::cout << pool->diagnostics().format_thread_dump() << std::endl; +``` + +### Periodic Monitoring Loop + +```cpp +void monitoring_loop(std::shared_ptr pool) { + while (pool->is_running()) { + // Health check + auto health = pool->diagnostics().health_check(); + + if (!health.is_healthy()) { + LOG_WARN("Pool degraded: {}", health.status_message); + + // Detailed bottleneck analysis + auto report = pool->diagnostics().detect_bottlenecks(); + if (report.severity() >= 2) { + for (const auto& rec : report.recommendations) { + LOG_WARN(" Action: {}", rec); + } + } + } + + // Metrics snapshot + auto snap = pool->metrics_service()->enhanced_snapshot(); + LOG_INFO("Throughput: {:.0f}/s P99: {:.1f}us Queue: {}", + snap.throughput_1s, + snap.execution_latency_p99_us, + snap.current_queue_depth); + + std::this_thread::sleep_for(std::chrono::seconds(5)); + } +} +``` + +### HTTP Health Endpoint + +```cpp +// Kubernetes liveness probe +auto handle_healthz = [&pool]() -> HttpResponse { + auto health = pool->diagnostics().health_check(); + return HttpResponse{ + health.http_status_code(), + "application/json", + health.to_json() + }; +}; + +// Prometheus metrics scrape endpoint +auto handle_metrics = [&pool]() -> HttpResponse { + auto snap = pool->metrics_service()->enhanced_snapshot(); + auto prom = BackendRegistry::instance().get("prometheus"); + return HttpResponse{ + 200, + "text/plain; version=0.0.4; charset=utf-8", + prom->export_enhanced(snap) + }; +}; +``` + +--- + +## Configuration Reference + +### diagnostics_config Defaults + +| Field | Default | Description | +|-------|---------|-------------| +| `recent_jobs_capacity` | `1000` | Max recent jobs in ring buffer | +| `event_history_size` | `1000` | Max events in history buffer | +| `enable_tracing` | `false` | Auto-enable event tracing | +| `queue_saturation_high` | `0.8` | High watermark for queue saturation | +| `utilization_high_threshold` | `0.9` | Worker utilization warning threshold | +| `wait_time_threshold_ms` | `100.0` | Wait time for slow consumer detection | +| `health_thresholds_config` | `{}` | Nested health threshold defaults | + +### health_thresholds Defaults + +| Field | Default | Description | +|-------|---------|-------------| +| `min_success_rate` | `0.95` | Below = degraded | +| `unhealthy_success_rate` | `0.8` | Below = unhealthy | +| `max_healthy_latency_ms` | `100.0` | Above = not fully healthy | +| `degraded_latency_ms` | `500.0` | Above = degraded | +| `queue_saturation_warning` | `0.8` | Queue warning threshold | +| `queue_saturation_critical` | `0.95` | Queue critical threshold | +| `worker_utilization_warning` | `0.9` | Worker warning threshold | +| `min_idle_workers` | `0` | Min idle workers (0 = disabled) | + +### Memory Budget + +| Component | Memory Usage | +|-----------|-------------| +| `MetricsBase` | 40 bytes (5 atomic counters) | +| `ThreadPoolMetrics` | 48 bytes (6 atomic counters) | +| `LatencyHistogram` | < 1KB (64 buckets + counters) | +| `SlidingWindowCounter` (1s) | ~160 bytes (10 buckets) | +| `SlidingWindowCounter` (60s) | ~10KB (600 buckets) | +| `EnhancedThreadPoolMetrics` | ~25KB base + ~32 bytes per worker | +| `diagnostics_config` | ~100 bytes | + +--- + +## Anti-Patterns + +### 1. Polling Diagnostics in a Tight Loop + +```cpp +// BAD: Wastes CPU on constant diagnostics polling +while (true) { + auto health = diag.health_check(); // O(n) per call + // No sleep! +} + +// GOOD: Poll at reasonable intervals +while (true) { + auto health = diag.health_check(); + std::this_thread::sleep_for(std::chrono::seconds(5)); +} +``` + +### 2. Slow Event Listeners + +```cpp +// BAD: Blocking I/O in event listener (blocks worker thread) +class SlowListener : public execution_event_listener { + void on_event(const job_execution_event& event) override { + write_to_database(event); // 10ms+ blocking call + } +}; + +// GOOD: Buffer events and process asynchronously +class AsyncListener : public execution_event_listener { + void on_event(const job_execution_event& event) override { + event_queue_.push(event.to_json()); // < 1us + } +private: + concurrent_queue event_queue_; +}; +``` + +### 3. Ignoring Enhanced Metrics Lifecycle + +```cpp +// BAD: Accessing enhanced metrics without checking +try { + const auto& em = svc->enhanced_metrics(); // Throws if not enabled! +} catch (...) { } + +// GOOD: Check first or use snapshot +if (svc->is_enhanced_metrics_enabled()) { + const auto& em = svc->enhanced_metrics(); +} +// Or: always-safe snapshot (returns empty if disabled) +auto snap = svc->enhanced_snapshot(); +``` + +### 4. Creating Multiple Diagnostics Instances + +```cpp +// BAD: Multiple diagnostics for the same pool (wasted memory) +thread_pool_diagnostics diag1(*pool); +thread_pool_diagnostics diag2(*pool); + +// GOOD: Use the pool's built-in diagnostics +auto& diag = pool->diagnostics(); +``` + +### 5. Not Resetting Metrics After Scale Events + +```cpp +// BAD: Metrics from old worker count skew per-worker stats +pool->resize(16); +// ... per_worker_utilization still has 8 entries ... + +// GOOD: Update worker count in metrics +pool->resize(16); +svc->update_worker_count(16); +``` + +--- + +## Troubleshooting + +### Health Check Returns `unknown` + +**Cause**: No components registered, or pool not started. + +**Solution**: +```cpp +// Ensure pool is started before health checks +pool->start(); +// Then check health +auto health = pool->diagnostics().health_check(); +``` + +### Bottleneck Detection Shows `none` Despite Slow Tasks + +**Cause**: Thresholds may be too lenient for your workload. + +**Solution**: +```cpp +diagnostics_config config; +config.queue_saturation_high = 0.5; // More sensitive +config.utilization_high_threshold = 0.7; // More sensitive +config.wait_time_threshold_ms = 10.0; // Stricter wait time +``` + +### Enhanced Metrics Snapshot Returns All Zeros + +**Cause**: Enhanced metrics not enabled, or no data recorded yet. + +**Solution**: +```cpp +// 1. Enable enhanced metrics BEFORE submitting work +svc->set_enhanced_metrics_enabled(true, worker_count); + +// 2. Submit work and wait for processing + +// 3. Then take snapshot +auto snap = svc->enhanced_snapshot(); +``` + +### Prometheus Metrics Missing Labels + +**Cause**: Labels must be set on the backend before exporting. + +**Solution**: +```cpp +auto prom = BackendRegistry::instance().get("prometheus"); +prom->set_prefix("myapp_pool"); +prom->add_label("service", "api"); +prom->add_label("env", "production"); +// NOW export +auto output = prom->export_enhanced(snap); +``` + +### Event Listener Not Receiving Events + +**Cause**: Tracing may not be enabled, or listener was not registered. + +**Solution**: +```cpp +// 1. Enable tracing +pool->diagnostics().enable_tracing(true); + +// 2. Register listener BEFORE submitting work +pool->diagnostics().add_event_listener(my_listener); + +// 3. Verify tracing is enabled +assert(pool->diagnostics().is_tracing_enabled()); +``` + +### Per-Worker Utilization Shows 0% for All Workers + +**Cause**: `record_worker_state()` is not being called with duration data. + +**Solution**: Ensure the thread pool implementation calls `record_worker_state(worker_id, busy, duration_ns)` on each state transition with the duration spent in the previous state. + +--- + +## Related Documentation + +- [Diagnostics Guide](DIAGNOSTICS_GUIDE.md) - Thread dump, job inspection, bottleneck detection, event tracing +- [Architecture Overview](ARCHITECTURE_OVERVIEW.md) - System-wide architecture +- [Policy Queue Combinations Guide](POLICY_QUEUE_GUIDE.md) - Queue and policy configurations +- [NUMA Work-Stealing Guide](NUMA_GUIDE.md) - NUMA topology and work stealing +- [Autoscaler Guide](AUTOSCALER_GUIDE.md) - Dynamic worker pool sizing diff --git a/docs/advanced/PERFORMANCE.md b/docs/advanced/PERFORMANCE.md index 168d0c6efc..daee2ed184 100644 --- a/docs/advanced/PERFORMANCE.md +++ b/docs/advanced/PERFORMANCE.md @@ -14,1304 +14,29 @@ category: "PERF" > **Language:** **English** | [한국어](PERFORMANCE.kr.md) -This comprehensive guide covers performance benchmarks, tuning strategies, and optimization techniques for the Thread System framework. All measurements are based on real benchmark data and extensive testing. - -## Table of Contents - -1. [Performance Overview](#performance-overview) -2. [Benchmark Environment](#benchmark-environment) -3. [Core Performance Metrics](#core-performance-metrics) -4. [Data Race Fix Impact](#data-race-fix-impact) -5. [Detailed Benchmark Results](#detailed-benchmark-results) -6. [Typed Lock-Free Thread Pool Benchmarks](#typed-lock-free-thread-pool-benchmarks) -7. [Scalability Analysis](#scalability-analysis) -8. [Memory Performance](#memory-performance) -9. [Comparison with Other Libraries](#comparison-with-other-libraries) -10. [Optimization Strategies](#optimization-strategies) -11. [Platform-Specific Optimizations](#platform-specific-optimizations) -12. [Best Practices](#best-practices) - -## Performance Overview - -The Thread System framework delivers exceptional performance across various workload patterns: - -### Key Performance Highlights (Current Architecture) - -- **Peak Throughput**: Up to 13.0M jobs/second (1 worker, empty jobs - theoretical) -- **Real-world Throughput**: - - Standard thread pool: 1.16M jobs/s (10 workers) - - Typed thread pool: 1.24M jobs/s (6 workers) - - Adaptive job queue: Automatically selects optimal strategy -- **Low Latency**: - - Standard pool: ~77 nanoseconds job scheduling latency - - Adaptive queues: 96-580ns based on contention level -- **Scaling Efficiency**: 96% at 8 cores (theoretical), 55-56% real-world -- **Memory Efficient**: <1MB baseline memory usage -- **Cross-Platform**: Consistent performance across Windows, Linux, and macOS -- **Streamlined Architecture**: - - Adaptive queue strategy for automatic optimization - - Reduced code complexity from ~8,700 to ~2,700 lines - - Separated logger and monitoring into independent projects - - Enhanced synchronization primitives and cancellation support - - Service registry for dependency injection - -## Benchmark Environment - -### Test Hardware -- **CPU**: Apple M1 (8-core) - 4 performance + 4 efficiency cores -- **Memory**: 16GB unified memory -- **Storage**: NVMe SSD -- **OS**: macOS Sonoma 14.x - -### Compiler Configuration -- **Compiler**: Apple Clang 17.0.0 -- **C++ Standard**: C++20 -- **Optimization**: -O3 Release mode -- **Features**: std::format enabled, std::thread fallback (std::jthread not available) - -### Thread System Version -- **Version**: Latest development build with streamlined architecture -- **Build Date**: 2025-09-07 (latest update - modularized architecture) -- **Configuration**: Release build with adaptive queue support -- **Benchmark Tool**: Google Benchmark -- **Architecture Changes**: - - Streamlined from ~8,700 to ~2,700 lines of code - - Logger and monitoring moved to separate projects - - Clean interface-based architecture - - Added sync_primitives, cancellation_token, service_registry -- **Performance**: Maintained with automatic optimization via adaptive queues - -## Core Performance Metrics - -### Component Overhead - -| Component | Operation | Overhead | Notes | -|-----------|-----------|----------|-------| -| Thread Base | Thread creation | ~10-15 μs | Per thread initialization | -| Thread Base | Job scheduling | ~77 ns | 10x improvement from previous ~1-2 μs | -| Thread Base | Wake interval access | +5% | Mutex protection added | -| Thread Pool | Pool creation (1 worker) | ~162 ns | Measured with Google Benchmark | -| Thread Pool | Pool creation (8 workers) | ~578 ns | Linear scaling | -| Thread Pool | Pool creation (16 workers) | ~1041 ns | Consistent overhead | -| Adaptive Queue | Mutex mode enqueue | ~96 ns | Default strategy | -| Adaptive Queue | Lock-free mode enqueue | ~320 ns | High contention mode | -| Adaptive Queue | Batch operations | ~212 ns/job | Optimized processing | -| Sync Primitives | Scoped lock with timeout | ~15 ns | RAII wrapper overhead | -| Cancellation Token | Registration | +3% | Thread-safe callbacks | -| Service Registry | Service lookup | ~25 ns | Type-safe retrieval | -| Job Queue | Operations | -4% | Optimized atomics | - -### Thread Pool Creation Performance - -| Worker Count | Creation Time | Items/sec | Notes | -|-------------|---------------|-----------|-------| -| 1 | 162 ns | 6.19M/s | Minimal overhead | -| 2 | 227 ns | 4.43M/s | Good scaling | -| 4 | 347 ns | 2.89M/s | Linear increase | -| 8 | 578 ns | 1.73M/s | Expected overhead | -| 16 | 1041 ns | 960K/s | Still sub-microsecond | - -## Data Race Fix Impact - -### Overview -The recent data race fixes addressed three critical concurrency issues while maintaining excellent performance characteristics: - -1. **thread_base::wake_interval** - Added mutex protection for thread-safe access -2. **cancellation_token** - Fixed double-check pattern and circular references -3. **job_queue::queue_size_** - Removed redundant atomic counter - -### Performance Impact Analysis - -| Fix | Performance Impact | Benefit | Trade-off | -|-----|-------------------|---------|-----------| -| Wake interval mutex | +5% overhead | Thread-safe access | Minimal impact on most workloads | -| Cancellation token fix | +3% overhead | Prevents race conditions | Safer callback registration | -| Job queue optimization | -4% (improvement) | Better cache locality | None - pure win | -| **Net Impact** | **+4% overall** | **100% thread safety** | **Excellent trade-off** | - -### Before vs After Comparison - -| Metric | Before Fixes | After Fixes | Change | -|--------|--------------|-------------|--------| -| Peak throughput | ~12.5M jobs/s | 13.0M jobs/s | +4% | -| Job submission latency | ~80 ns | ~77 ns | -4% | -| Thread safety | 3 data races | 0 data races | ✅ | -| Memory ordering | Weak | Strong | ✅ | - -### Real-World Impact -- **Production Safety**: All data races eliminated, ensuring reliable operation under high concurrency -- **Performance**: Slight net improvement due to job queue optimization offsetting mutex overhead -- **Maintainability**: Cleaner code with proper synchronization primitives - -## Detailed Benchmark Results - -### Job Submission Latency - -#### Standard Thread Pool (Mutex-based) -| Queue State | Avg Latency | 50th Percentile | 95th Percentile | 99th Percentile | -|------------|-------------|-----------------|-----------------|-----------------| -| Empty | 0.8 μs | 0.7 μs | 1.1 μs | 1.2 μs | -| 100 jobs | 0.9 μs | 0.8 μs | 1.3 μs | 1.5 μs | -| 1K jobs | 1.1 μs | 1.0 μs | 1.8 μs | 2.1 μs | -| 10K jobs | 1.3 μs | 1.2 μs | 2.8 μs | 3.5 μs | -| 100K jobs | 1.6 μs | 1.4 μs | 3.2 μs | 4.8 μs | - -#### Adaptive Queue (Lock-free Mode) -| Queue State | Avg Latency | 50th Percentile | 95th Percentile | 99th Percentile | -|------------|-------------|-----------------|-----------------|-----------------| -| Empty | 0.32 μs | 0.28 μs | 0.45 μs | 0.52 μs | -| 100 jobs | 0.35 μs | 0.31 μs | 0.48 μs | 0.58 μs | -| 1K jobs | 0.38 μs | 0.34 μs | 0.52 μs | 0.65 μs | -| 10K jobs | 0.42 μs | 0.38 μs | 0.58 μs | 0.72 μs | -| 100K jobs | 0.48 μs | 0.43 μs | 0.68 μs | 0.85 μs | - -### Throughput by Job Complexity - -#### Standard Thread Pool Performance - -| Job Duration | 1 Worker | 2 Workers | 4 Workers | 8 Workers | Notes | -|-------------|----------|-----------|-----------|-----------|-------| -| Empty job | 13.0M/s | 10.4M/s | 8.3M/s | 6.6M/s | High contention | -| 1 μs work | 890K/s | 1.6M/s | 3.0M/s | 5.5M/s | Good scaling | -| 10 μs work | 95K/s | 180K/s | 350K/s | 680K/s | Near-linear | -| 100 μs work | 9.9K/s | 19.8K/s | 39.5K/s | 78K/s | Excellent scaling | -| 1 ms work | 990/s | 1.98K/s | 3.95K/s | 7.8K/s | CPU-bound | -| 10 ms work | 99/s | 198/s | 395/s | 780/s | I/O-bound territory | -| **Real workload** | **1.16M/s** | - | - | - | **10 workers, measured** | - -#### Adaptive Job Queue Performance (Lock-free Mode) - -| Job Duration | 1 Worker | 2 Workers | 4 Workers | 8 Workers | vs Standard | -|-------------|----------|-----------|-----------|-----------|-------------| -| Empty job | 15.2M/s | 14.8M/s | 13.5M/s | 12.1M/s | +83% avg | -| 1 μs work | 1.2M/s | 2.3M/s | 4.4M/s | 8.2M/s | +49% avg | -| 10 μs work | 112K/s | 218K/s | 425K/s | 820K/s | +21% avg | -| 100 μs work | 10.2K/s | 20.3K/s | 40.5K/s | 80K/s | +3% avg | -| 1 ms work | 995/s | 1.99K/s | 3.97K/s | 7.9K/s | +1% avg | -| 10 ms work | 99/s | 198/s | 396/s | 781/s | ~0% avg | -| **Real workload** | **Available via adaptive strategy** | - | - | - | **Dynamic selection** | - -#### Type Thread Pool Performance - -| Type Mix | Basic Pool | Type Pool | Performance | Type Accuracy | -|-------------|------------|-----------|-------------|---------------| -| Single (High) | 540K/s | 525K/s | -3% | 100% | -| 2 Levels | 540K/s | 510K/s | -6% | 99.8% | -| 3 Levels | 540K/s | 495K/s | -9% | 99.6% | -| 5 Levels | 540K/s | 470K/s | -15% | 99.3% | -| 10 Levels | 540K/s | 420K/s | -29% | 98.8% | - -#### Real-World Measurements (Lock-Free Implementation) - -| Configuration | Throughput | Time (1M jobs) | Workers | CPU Usage | Improvement | -|--------------|------------|----------------|---------|-----------|-------------| -| Basic Pool | 1.16M/s | 862 ms | 10 | 559% | Baseline | -| Type Pool | 1.24M/s | 806 ms | 6 | 330% | +6.9% | - -#### Type Thread Pool with Adaptive Queues - -The Type Thread Pool features adaptive job queue implementation: - -##### typed_thread_pool (Current Implementation) -- **Architecture**: Type-specific job queues with adaptive strategy selection -- **Synchronization**: Dynamic mutex/lock-free switching based on contention -- **Memory**: Adaptive queue allocation per job type -- **Best for**: All scenarios with automatic optimization - -##### Adaptive Queue Strategy -- **Architecture**: Automatic switching between mutex and lock-free based on load -- **Synchronization**: Seamless fallback between strategies -- **Memory**: Optimized allocation based on queue usage patterns -- **Best for**: Dynamic workloads with varying contention patterns - -**Performance Characteristics**: - -| Metric | Adaptive Implementation | Benefits | -|--------|-------------------------|----------| -| Simple jobs (100-10K) | 540K/s baseline | Optimal strategy selection | -| High contention scenarios | Auto lock-free mode | Maintains performance | -| Priority scheduling | Type-based routing | Efficient job distribution | -| Job dequeue latency | ~571 ns (lock-free mode) | Automatic optimization | -| Memory per type | Dynamic allocation | Scalable resource usage | - -**Implementation Features**: -- Automatic strategy selection based on contention detection -- Type-based job routing and worker specialization -- Dynamic queue creation and lifecycle management -- Per-type statistics collection for monitoring and tuning -- Compatible API with seamless optimization - -*Note: The adaptive implementation automatically selects the optimal queue strategy based on runtime conditions, providing both simplicity and performance.* - -## Adaptive Job Queue Benchmarks - -### Overview - -Comprehensive benchmarks demonstrating adaptive job queue performance with automatic strategy selection across multiple dimensions: - -### Thread Pool Level Benchmarks - -#### Simple Job Processing -*Jobs with minimal computation (10 iterations)* - -| Queue Strategy | Job Count | Execution Time | Throughput | Relative Performance | -|---------------|-----------|----------------|------------|---------------------| -| Mutex (low load) | 100 | ~45 μs | 2.22M/s | Baseline | -| Adaptive | 100 | ~42 μs | 2.38M/s | **+7.2%** | -| Mutex (med load) | 1,000 | ~380 μs | 2.63M/s | Baseline | -| Adaptive | 1,000 | ~365 μs | 2.74M/s | **+4.2%** | -| Mutex (high load) | 10,000 | ~3.2 ms | 3.13M/s | Baseline | -| Adaptive | 10,000 | ~3.0 ms | 3.33M/s | **+6.4%** | - -#### Medium Workload Processing -*Jobs with moderate computation (100 iterations)* - -| Queue Strategy | Job Count | Execution Time | Throughput | Relative Performance | -|---------------|-----------|----------------|------------|---------------------| -| Mutex-based | 100 | ~125 μs | 800K/s | Baseline | -| Adaptive | 100 | ~118 μs | 847K/s | **+5.9%** | -| Mutex-based | 1,000 | ~1.1 ms | 909K/s | Baseline | -| Adaptive | 1,000 | ~1.0 ms | 1.00M/s | **+10.0%** | - -#### Priority Scheduling Performance -*Type-based job routing with adaptive queue selection* - -| Jobs per Type | Total Jobs | Processing Time | Routing Accuracy | Priority Handling | -|---------------|------------|-----------------|------------------|-------------------| -| 100 | 300 | ~285 μs | 99.7% | Optimal | -| 500 | 1,500 | ~1.35 ms | 99.4% | Efficient | -| 1,000 | 3,000 | ~2.65 ms | 99.1% | Stable | - -#### High Contention Scenarios -*Multiple producer threads simultaneously submitting jobs* - -| Thread Count | Standard Logger | Adaptive Logger | Performance Gain | -|-------------|---------------------|-------------------|------------------| -| 1 | 1,000 jobs/μs | 1,000 jobs/μs | 0% (baseline) | -| 2 | 850 jobs/μs | 920 jobs/μs | **+8.2%** | -| 4 | 620 jobs/μs | 780 jobs/μs | **+25.8%** | -| 8 | 380 jobs/μs | 650 jobs/μs | **+71.1%** | -| 16 | 190 jobs/μs | 520 jobs/μs | **+173.7%** | - -### Queue Level Benchmarks - -#### Basic Queue Operations -*Raw enqueue/dequeue performance* - -| Operation | Mutex Queue | Adaptive Queue | Improvement | -|-----------|-------------|----------------|-------------| -| Enqueue (single) | ~85 ns | ~78 ns | **+8.2%** | -| Dequeue (single) | ~195 ns | ~142 ns | **+37.3%** | -| Enqueue/Dequeue pair | ~280 ns | ~220 ns | **+27.3%** | - -#### Batch Operations -*Processing multiple items at once* - -| Batch Size | Mutex Queue (μs) | Adaptive Queue (μs) | Improvement | -|-----------|------------------|---------------------|-------------| -| 8 | 2.8 | 2.1 | **+33.3%** | -| 32 | 9.2 | 6.8 | **+35.3%** | -| 128 | 34.1 | 24.7 | **+38.0%** | -| 512 | 128.4 | 91.2 | **+41.0%** | -| 1024 | 248.7 | 175.3 | **+41.9%** | - -#### Contention Stress Tests -*Multiple threads competing for queue access* - -| Concurrent Threads | Mutex Queue (μs) | Adaptive Queue (μs) | Scalability Factor | -|-------------------|------------------|---------------------|-------------------| -| 1 | 28.5 | 29.1 | 0.98x | -| 2 | 65.2 | 42.3 | **1.54x** | -| 4 | 156.8 | 73.5 | **2.13x** | -| 8 | 387.2 | 125.8 | **3.08x** | -| 16 | 892.5 | 218.6 | **4.08x** | - -#### Job Type Routing Features -*Type-based job queue selection and routing* - -| Job Type Mix | Type-specific Jobs | Routing Time | Standard Time | Routing Benefit | -|--------------|-------------------|--------------|---------------|-----------------| -| 33% each type | 1,000 | 142 ns | 168 ns | **+18.3%** | -| 50% High priority | 1,500 | 138 ns | 175 ns | **+26.8%** | -| 80% High priority | 2,400 | 135 ns | 182 ns | **+34.8%** | - -#### Memory Usage Comparison - -| Queue Type | Job Count | Memory Usage | Per-Job Memory | Notes | -|------------|-----------|--------------|----------------|-------| -| Mutex Queue | 100 | 8.2 KB | 82 bytes | Shared data structures | -| Adaptive Queue | 100 | 12.5 KB | 125 bytes | Dynamic allocation | -| Mutex Queue | 1,000 | 24.1 KB | 24 bytes | Memory efficiency improves | -| Adaptive Queue | 1,000 | 31.8 KB | 32 bytes | Good scaling properties | -| Mutex Queue | 10,000 | 195.2 KB | 20 bytes | Excellent density | -| Adaptive Queue | 10,000 | 248.7 KB | 25 bytes | Acceptable overhead | - -### Benchmark Environment Details - -- **Hardware**: Apple M1 (8-core), 16GB RAM -- **Software**: macOS Sonoma, Apple Clang 17.0.0, C++20 -- **Build**: Release mode (-O3), Google Benchmark framework -- **Test Duration**: 10 seconds per benchmark with warmup -- **Iterations**: Auto-determined by Google Benchmark for statistical significance -- **Thread Configuration**: 4 workers (1 per type + 1 universal) -- **Latest Update**: 2025-06-29 with enhanced lock-free algorithms - -### Available Benchmarks - -The Thread System includes comprehensive benchmarks for performance testing: - -#### Thread Pool Benchmarks (`tests/benchmarks/thread_pool_benchmarks/`) -- **gbench_thread_pool**: Basic Google Benchmark integration -- **thread_pool_benchmark**: Core thread pool performance metrics -- **memory_benchmark**: Memory usage and allocation patterns (logger benchmark removed) -- **real_world_benchmark**: Realistic workload simulations -- **stress_test_benchmark**: Extreme load and contention testing -- **scalability_benchmark**: Multi-core scaling analysis -- **contention_benchmark**: Contention-specific scenarios -- **comparison_benchmark**: Cross-library comparisons -- **throughput_detailed_benchmark**: Detailed throughput analysis - -#### Queue Benchmarks (`tests/benchmarks/thread_base_benchmarks/`) -- **mpmc_performance_test**: MPMC queue performance analysis -- **simple_mpmc_benchmark**: Basic queue operations -- **quick_mpmc_test**: Fast queue validation - -#### Other Benchmarks -- **data_race_benchmark**: Data race fix impact analysis - -#### Running Benchmarks -```bash -# Build with benchmarks enabled -./build.sh --clean --benchmark - -# Run specific benchmark -./build/bin/thread_pool_benchmark - -# Run with custom parameters -./build/bin/thread_pool_benchmark --benchmark_time_unit=ms --benchmark_min_time=1s - -# Filter specific tests -./build/bin/thread_pool_benchmark --benchmark_filter="BM_ThreadPool/*" - -# Export results -./build/bin/thread_pool_benchmark --benchmark_format=json > results.json -``` - -### Key Performance Insights - -1. **Adaptive Queue Advantages**: - - Automatic strategy selection based on contention - - Better queue operation latency (20-40% faster) when needed - - Supports both mutex and lock-free modes - - Consistent performance scaling - -2. **Simplified Architecture Benefits**: - - Lower memory overhead for typical scenarios - - Cleaner codebase with automatic optimization - - Predictable performance characteristics - - Maintains lock-free capability when beneficial - -3. **Recommended Usage**: - - **Adaptive queues**: Automatic optimization for all scenarios - - **Type-based routing**: Specialized job handling - - **Dynamic scaling**: Automatic resource allocation - -4. **Performance Characteristics**: - - Adaptive queues show 2-4x better scalability under contention - - Memory overhead: Optimized allocation based on usage - - Type routing adds 15-35% efficiency for specialized jobs - -## Scalability Analysis - -### Worker Thread Scaling Efficiency - -| Workers | Speedup | Efficiency | Queue Depth (avg) | CPU Utilization | -|---------|---------|------------|-------------------|-----------------| -| 1 | 1.0x | 100% | 0.1 | 98% | -| 2 | 2.0x | 99% | 0.2 | 97% | -| 4 | 3.9x | 97.5% | 0.5 | 96% | -| 8 | 7.7x | 96.25% | 1.2 | 95% | -| 16 | 15.0x | 93.75% | 3.1 | 92% | -| 32 | 28.3x | 88.4% | 8.7 | 86% | -| 64 | 52.1x | 81.4% | 22.4 | 78% | - -### Workload-Specific Scaling - -#### CPU-Bound Tasks -- **Optimal Workers**: Hardware core count -- **Peak Efficiency**: 96% at 8 cores -- **Scaling Limit**: Physical cores (performance cores on ARM) -- **Recommended**: Use exact core count for CPU-intensive work - -#### I/O-Bound Tasks -- **Optimal Workers**: 2-3x hardware core count -- **Peak Efficiency**: 85% at 16+ workers -- **Scaling Benefit**: Continues beyond core count -- **Recommended**: Start with 2x cores, tune based on I/O wait time - -#### Mixed Workloads -- **Optimal Workers**: 1.5x hardware core count -- **Peak Efficiency**: 90% at 12 workers -- **Balance Point**: Between CPU and I/O characteristics -- **Recommended**: Profile workload to find optimal balance - -## Memory Performance - -### Memory Usage by Configuration - -| Configuration | Virtual Memory | Resident Memory | Peak Memory | Per-Worker | -|--------------|----------------|-----------------|-------------|------------| -| Base System | 45.2 MB | 12.8 MB | 12.8 MB | - | -| 1 Worker | 46.4 MB | 14.0 MB | 14.2 MB | 1.2 MB | -| 4 Workers | 48.1 MB | 14.6 MB | 15.1 MB | 450 KB | -| 8 Workers | 50.4 MB | 15.4 MB | 16.3 MB | 325 KB | -| 16 Workers | 54.8 MB | 16.6 MB | 18.7 MB | 262 KB | -| 32 Workers | 63.2 MB | 20.2 MB | 25.1 MB | 231 KB | - -### Memory Optimization Impact (v2.0) - -With the latest memory optimizations: - -| Component | Before | After | Savings | Notes | -|-----------|--------|-------|---------|-------| -| Adaptive Queue (idle) | 8.2 MB | 0.4 MB | 95% | Lazy initialization | -| Node Pool (256 nodes) | 16 KB | 1 KB | 93.75% | Reduced initial chunks | -| Thread Pool (8 workers) | 15.4 MB | 12.1 MB | 21% | Combined optimizations | -| Peak Memory (unchanged) | 16.3 MB | 16.3 MB | 0% | Same maximum capacity | - -### Startup Memory Profile - -| Phase | Memory Usage | Time | Description | -|-------|-------------|------|-------------| -| Binary Load | 8.2 MB | 0 ms | Base executable | -| Library Init | 10.4 MB | 2 ms | Dynamic libraries | -| Thread Pool Create | 10.8 MB | 0.3 ms | Pool structure only | -| Worker Spawn (8) | 12.1 MB | 1.2 ms | Thread stack allocation | -| First Job | 12.3 MB | 0.1 ms | Queue initialization | -| Steady State | 12.8 MB | - | Normal operation | - -### Memory Allocation Impact on Performance - -| Memory Pattern | Allocation Size | Jobs/sec | vs No Alloc | P99 Latency | Memory Overhead | -|---------------|----------------|----------|-------------|-------------|-----------------| -| None | 0 | 1,160,000| 100% | 1.8μs | 0 | -| Small | <1KB | 1,044,000| 90% | 2.2μs | +15% | -| Medium | 1-100KB | 684,000 | 59% | 3.8μs | +45% | -| Large | 100KB-1MB | 267,000 | 23% | 9.5μs | +120% | -| Very Large | >1MB | 58,000 | 5% | 42μs | +300% | - -### Potential Memory Pool Optimization - -*Note: Thread System does not currently implement built-in memory pools. The following represents potential improvements with custom memory pool implementations:* - -| Pool Type | Current | With Pool (Estimated) | Potential Improvement | Memory Savings | -|-----------|---------|----------------------|----------------------|----------------| -| Small Jobs | 1.04M/s | 1.11M/s (estimated) | +7% | 60% | -| Medium Jobs | 684K/s | 848K/s (estimated) | +24% | 75% | -| Large Jobs | 267K/s | 385K/s (estimated) | +44% | 80% | - -## Adaptive MPMC Queue Performance - -### Overview -The adaptive MPMC (Multiple Producer Multiple Consumer) queue implementation provides automatic strategy selection for optimal performance: - -- **Architecture**: Dynamic switching between mutex and lock-free strategies -- **Memory Management**: Efficient allocation based on queue usage patterns -- **Contention Handling**: Automatic detection and strategy switching -- **Cache Optimization**: Optimized memory layout for performance - -### Performance Comparison - -| Configuration | Mutex-only Queue | Adaptive MPMC | Improvement | -|--------------|-------------------|----------------|-------------| -| 1P-1C (10K ops) | 2.03 ms | 1.87 ms | +8.6% | -| 2P-2C (10K ops) | 5.21 ms | 3.42 ms | +52.3% | -| 4P-4C (10K ops) | 12.34 ms | 5.67 ms | +117.6% | -| 8P-8C (10K ops) | 28.91 ms | 9.23 ms | +213.4% | -| **Raw operation** | **12.2 μs** | **2.8 μs** | **+431%** | -| **Real workload** | **950 ms/1M** | **865 ms/1M** | **+10%** | - -### Scalability Analysis - -| Workers | Mutex-only Efficiency | Adaptive Efficiency | Efficiency Gain | -|---------|----------------------|-------------------|-----------------| -| 1 | 100% | 100% | 0% | -| 2 | 81% | 95% | +14% | -| 4 | 52% | 88% | +36% | -| 8 | 29% | 82% | +53% | - -### Implementation Details - -## Adaptive Logger Performance - -### Overview -The adaptive logger implementation provides automatic optimization for high-throughput logging scenarios: - -- **Architecture**: Adaptive job queue for log message submission -- **Contention Handling**: Dynamic strategy selection based on load -- **Scalability**: Optimal performance scaling with thread count -- **Compatibility**: Seamless integration with existing code - -### Single-Threaded Performance -*Message throughput comparison* - -| Message Size | Standard Logger | Adaptive Logger | Improvement | -|--------------|-----------------|------------------|-------------| -| Short (17 chars) | 7.64 M/s | 7.42 M/s | -2.9% | -| Medium (123 chars) | 5.73 M/s | 5.61 M/s | -2.1% | -| Long (1024 chars) | 2.59 M/s | 2.55 M/s | -1.5% | - -*Note: Single-threaded performance shows minimal overhead. Benefits appear under contention.* - -### Multi-Threaded Scalability -*Throughput with concurrent logging threads* - -| Threads | Standard Logger | Adaptive Logger | Improvement | -|---------|-----------------|------------------|-------------| -| 2 | 1.91 M/s | 1.95 M/s | **+2.1%** | -| 4 | 0.74 M/s | 1.07 M/s | **+44.6%** | -| 8 | 0.22 M/s | 0.63 M/s | **+186.4%** | -| 16 | 0.16 M/s | 0.54 M/s | **+237.5%** | - -### Formatted Logging Performance -*Complex format string with multiple parameters* - -| Logger Type | Throughput | Latency (ns) | -|-------------|------------|--------------| -| Standard | 2.94 M/s | 340 | -| Adaptive | 2.89 M/s | 346 | - -### Burst Logging Performance -*Handling sudden log bursts* - -| Burst Size | Standard Logger | Adaptive Logger | Improvement | -|------------|-----------------|------------------|-------------| -| 10 messages | 1.90 M/s | 1.88 M/s | -1.1% | -| 100 messages | 5.33 M/s | 5.15 M/s | -3.4% | - -### Mixed Log Types Performance -*Different log levels (Info, Debug, Error, Exception)* - -| Logger Type | Throughput | CPU Efficiency | -|-------------|------------|----------------| -| Standard | 6.51 M/s | 100% | -| Adaptive | 6.42 M/s | 98% | - -### Key Findings - -1. **High Contention Benefits**: Adaptive logger shows significant advantages with 4+ threads -2. **Scalability**: Up to 237% improvement at 16 threads -3. **Minimal Overhead**: Single-threaded performance nearly identical -4. **Use Cases**: Ideal for all multi-threaded applications with automatic optimization - -### Recommendations - -- **Use Adaptive Logger**: Automatic optimization for all scenarios -- **Dynamic Scaling**: Logger adapts to application's threading patterns -- **Batch Processing**: Automatically enabled when beneficial for throughput -- **Buffer Management**: Dynamic queue sizing based on workload - -### Implementation Details - -- **Adaptive Strategy**: Automatic switching between mutex and lock-free based on contention -- **Dynamic Allocation**: Efficient memory usage based on queue patterns -- **Smart Retry Logic**: Intelligent backoff strategies to prevent contention -- **Queue Optimization**: Automatic batching and buffer management -- **Performance Monitoring**: Built-in metrics for optimization decisions - -### Current Status - -- Modularized architecture with adaptive queue selection -- Logger and monitoring separated into independent projects -- All stress tests enabled and passing reliably -- Adaptive implementation provides optimal performance for all scenarios -- Average operation latencies: - - Enqueue: ~96 ns (low contention), ~320 ns (high contention) - - Dequeue: ~571 ns with adaptive optimization - -### Recent Benchmark Results (2025-07-25) - -#### Data Race Fix Verification -| Threads | Wake Interval Access | Cancellation Token | Job Queue Consistency | -|---------|---------------------|--------------------|-----------------------| -| 1 | 163μs/10K | 25μs/100 | - | -| 4 | 272μs/10K | 59μs/400 | 842μs/2K dequeued | -| 8 | 438μs/10K | 111μs/800 | 2.18ms/4K dequeued | -| 16 | 750μs/10K | 210μs/1.6K | 4.81ms/8K dequeued | - -*Note: All data race issues have been resolved with proper synchronization.* - -### Usage Recommendations - -1. **Adaptive Queue Benefits**: - - All contention scenarios automatically optimized - - Latency-sensitive applications benefit from automatic switching - - Systems with varying CPU load patterns - - Real-time applications with dynamic requirements - -2. **Configuration Guidelines**: - ```cpp - // Adaptive behavior (recommended) - adaptive_job_queue queue(adaptive_job_queue::queue_strategy::ADAPTIVE); - - // Force specific strategy when needed - adaptive_job_queue mutex_queue(adaptive_job_queue::queue_strategy::FORCE_MUTEX); - adaptive_job_queue lockfree_queue(adaptive_job_queue::queue_strategy::FORCE_LOCKFREE); - ``` - -### Performance Tuning Tips - -1. **Batch Operations**: Use batch enqueue/dequeue for better throughput -2. **CPU Affinity**: Pin threads to specific cores for consistent performance -3. **Memory Alignment**: Ensure job objects are cache-line aligned -4. **Retry Handling**: Operations may fail under extreme contention - implement retry logic -5. **Monitoring**: Use built-in statistics to track performance metrics including retry counts - -## Logger Performance (Now Separate Project) - -*Note: Logger has been moved to a separate project. The following benchmarks are from when logger was integrated with Thread System.* - -### Logger Comparison with Industry Standards - -### Overview -This section compares Thread System's logging performance against industry-standard logging libraries. The logger uses adaptive job queues for automatic optimization. - -### Single-Threaded Performance Comparison -*Baseline measurements on Apple M1 (8-core)* - -| Logger | Throughput | Latency (ns) | Relative Performance | -|--------|------------|--------------|---------------------| -| Console Output | 542.8K/s | 1,842 | Baseline | -| Thread System Logger | 4.41M/s | 227 | **8.1x** faster | -| Thread System (Adaptive) | 4.34M/s | 240 | **8.0x** faster | - -### Multi-Threaded Scalability -*Concurrent logging performance with adaptive optimization* - -| Threads | Standard Mode | Adaptive Mode | Improvement | -|---------|---------------|---------------|-------------| -| 2 | 2.61M/s | 2.58M/s | -1% | -| 4 | 859K/s | 1.07M/s | **+25%** | -| 8 | 234K/s | 412K/s | **+76%** | -| 16 | 177K/s | 385K/s | **+118%** | - -### Latency Characteristics -*End-to-end logging latency* - -| Logger Type | Mean Latency | P99 Latency | Notes | -|-------------|--------------|-------------|-------| -| Thread System Logger | 144 ns | ~200 ns | Adaptive queue | -| Console Output | 1,880 ns | ~2,500 ns | System call overhead | - -### Key Findings - -1. **Logger Excellence**: - - 8.1x faster than console output - - Excellent single-threaded performance - - Predictable latency characteristics - -2. **Adaptive Scalability**: - - Automatic optimization at 4+ threads - - Up to 118% improvement at high contention - - Minimal overhead in single-threaded scenarios - -3. **Usage Recommendations**: - - Use Thread System Logger for all scenarios - - Adaptive queues automatically optimize based on load - - No manual configuration needed - -### Comparison with spdlog - -*Comprehensive performance comparison with the popular spdlog library* - -#### Single-Threaded Performance -| Logger | Throughput | Latency | vs Console | Notes | -|--------|------------|---------|------------|-------| -| Console Output | 583K/s | 1,716 ns | Baseline | System call overhead | -| **Thread System Logger** | **4.34M/s** | **148 ns** | **7.4x** | Best latency | -| spdlog (sync) | 515K/s | 2,333 ns | 0.88x | Poor performance | -| **spdlog (async)** | **5.35M/s** | - | **9.2x** | Best throughput | - -#### Multi-Threaded Performance (4 Threads) -| Logger | Throughput | vs Single-thread | Scalability | -|--------|------------|------------------|-------------| -| Thread System (Standard) | 599K/s | -86% | Moderate | -| **Thread System (Adaptive)** | **1.07M/s** | -75% | **Good** | -| spdlog (sync) | 210K/s | -59% | Very Poor | -| spdlog (async) | 785K/s | -85% | Poor | - -#### High Contention (8 Threads) -| Logger | Throughput | vs Console | Notes | -|--------|------------|------------|-------| -| Thread System (Standard) | 198K/s | 0.34x | High contention | -| **Thread System (Adaptive)** | **412K/s** | **0.71x** | Auto-optimized | -| spdlog (sync) | 52K/s | 0.09x | Severe degradation | -| spdlog (async) | 240K/s | 0.41x | Queue saturation | - -#### Key Findings - -1. **Single-threaded Champion**: spdlog async (5.35M/s) edges out Thread System (4.34M/s) -2. **Multi-threaded Champion**: Thread System with adaptive queues shows consistent performance -3. **Latency Champion**: Thread System with 148ns, **15.8x lower** than spdlog sync (2333/148 = 15.76) -4. **Scalability**: Thread System adaptive mode provides automatic optimization - -### Recommendations - -1. **For All Applications**: Use Thread System Logger - - Excellent performance with adaptive optimization - - Simple API with type safety - - Built-in file rotation and callbacks - - Automatic optimization for high concurrency - -2. **Configuration Tips**: - - Logger automatically adapts to workload - - No manual tuning required - - Adaptive queues handle burst patterns efficiently - -3. **Usage Example**: - ```cpp - // Simple usage - automatic optimization - log_module::start(); - log_module::write_information("Message: {}", value); - log_module::write_error("Error: {}", error); - log_module::stop(); - ``` - -## Comparison with Other Libraries - -### Throughput Comparison (Real-world measurements) - -| Library | Throughput | Relative Performance | Features | -|---------------------------|------------|---------------------|------------------------| -| **Thread System** | 1.16M/s | 100% (baseline) | Type, logging, C++20, lock-free | -| Intel TBB | ~1.24M/s | ~107% | Industry standard, work stealing | -| Boost.Thread Pool | ~1.09M/s | ~94% | Header-only, portable | -| std::async | ~267K/s | ~23% | Standard library, basic | -| Custom (naive) | ~684K/s | ~59% | Simple mutex-based impl | -| OpenMP | ~1.06M/s | ~92% | Compiler directives | -| Microsoft PPL | ~1.02M/s | ~88% | Windows-specific | - -### Feature Comparison - -| Library | Type Support | Logging | C++20 | Cross-Platform | Memory Pool | Error Handling | -|---------|-----------------|---------|-------|----------------|-------------|----------------| -| Thread System | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ✅ Comprehensive | -| Intel TBB | ✅ Yes | ❌ No | ⚠️ Partial | ✅ Yes | ✅ Yes | ⚠️ Basic | -| Boost.Thread Pool | ❌ No | ❌ No | ⚠️ Partial | ✅ Yes | ❌ No | ⚠️ Basic | -| std::async | ❌ No | ❌ No | ✅ Yes | ✅ Yes | ❌ No | ⚠️ Basic | - -### Latency Comparison (μs) - -| Library | Submission | Execution Start | Total Overhead | -|---------|------------|-----------------|----------------| -| Thread System | 77 ns | 96 ns | 173 ns | -| Intel TBB | ~100 ns | ~90 ns | ~190 ns | -| Boost.Thread Pool | ~150 ns | ~120 ns | ~270 ns | -| std::async | ~15.2 μs | ~12.8 μs | ~28.0 μs | - -## Optimization Strategies - -### 1. Optimal Thread Count Selection - -```cpp -uint16_t determine_optimal_thread_count(WorkloadType workload) { - uint16_t hardware_threads = std::thread::hardware_concurrency(); - - switch (workload) { - case WorkloadType::CpuBound: - return hardware_threads; - - case WorkloadType::MemoryBound: - return std::max(1u, hardware_threads / 2); - - case WorkloadType::IoBlocking: - return hardware_threads * 2; - - case WorkloadType::Mixed: - return static_cast(hardware_threads * 1.5); - - case WorkloadType::RealTime: - return hardware_threads - 1; // Reserve one core for OS - } - - return hardware_threads; -} -``` - -### 2. Job Batching for Performance - -Batching reduces scheduling overhead significantly: - -| Batch Size | Overhead per Job | Recommended Use Case | -|------------|-----------------|---------------------| -| 1 | 77 ns | Real-time tasks | -| 10 | 25 ns | Interactive tasks | -| 100 | 8 ns | Background processing| -| 1000 | 3 ns | Batch processing | -| 10000 | 2 ns | Bulk operations | - -```cpp -// Efficient job batching -std::vector> jobs; -jobs.reserve(batch_size); - -for (int i = 0; i < batch_size; ++i) { - jobs.push_back(create_job(data[i])); -} - -pool->enqueue_batch(std::move(jobs)); -``` - -### 3. Job Granularity Optimization - -| Job Execution Time | Recommended Action | Reason | -|--------------------|-------------------|--------| -| < 10μs | Batch 1000+ operations | Overhead dominates | -| 10-100μs | Batch 100 operations | Balance overhead/parallelism | -| 100μs-1ms | Batch 10 operations | Minimize coordination | -| 1ms-10ms | Individual jobs | Good granularity | -| > 10ms | Consider subdivision | Improve responsiveness | - -### 4. Type Pool Configuration - -```cpp -void configure_type_pool(std::shared_ptr pool, - const WorkloadProfile& profile) { - const uint16_t hw_threads = std::thread::hardware_concurrency(); - - // Allocate workers based on type distribution - uint16_t high_workers = static_cast(hw_threads * profile.high_type_ratio); - uint16_t normal_workers = static_cast(hw_threads * profile.normal_type_ratio); - uint16_t low_workers = static_cast(hw_threads * profile.low_type_ratio); - - // Ensure minimum coverage - high_workers = std::max(1u, high_workers); - normal_workers = std::max(1u, normal_workers); - low_workers = std::max(1u, low_workers); - - // Add specialized workers - add_type_workers(pool, job_types::High, high_workers); - add_type_workers(pool, job_types::Normal, normal_workers); - add_type_workers(pool, job_types::Low, low_workers); -} -``` - -### 4b. Adaptive Queue Configuration - -```cpp -#include "typed_thread_pool/pool/typed_thread_pool.h" - -auto create_optimal_pool(const std::string& name, - size_t expected_concurrency, - bool priority_sensitive) -> std::shared_ptr> { - - // Create typed thread pool with adaptive queue strategy - auto pool = std::make_shared>(name); - - // Configure adaptive queue strategy based on expected usage - if (expected_concurrency > 4 || priority_sensitive) { - // High contention - adaptive queue will automatically optimize - pool->set_queue_strategy(queue_strategy::ADAPTIVE); - } - - // Add specialized workers for each priority - auto realtime_worker = std::make_unique>(); - realtime_worker->set_responsibilities({job_types::RealTime}); - pool->add_worker(std::move(realtime_worker)); - - auto batch_worker = std::make_unique>(); - batch_worker->set_responsibilities({job_types::Batch}); - pool->add_worker(std::move(batch_worker)); - - auto background_worker = std::make_unique>(); - background_worker->set_responsibilities({job_types::Background}); - pool->add_worker(std::move(background_worker)); - - // Universal worker for load balancing - auto universal_worker = std::make_unique>(); - universal_worker->set_responsibilities({job_types::RealTime, job_types::Batch, job_types::Background}); - pool->add_worker(std::move(universal_worker)); - - return pool; -} - -// Usage examples -auto high_concurrency_pool = create_optimal_pool( - "HighConcurrency", 8, true); - -auto simple_pool = create_optimal_pool( - "Simple", 2, false); -``` - -### 5. Memory Optimization - -#### Cache-Line Alignment -```cpp -// Prevent false sharing -struct alignas(64) WorkerData { - std::atomic processed_jobs{0}; - std::atomic execution_time{0}; - char padding[64 - 2 * sizeof(std::atomic)]; -}; -``` - -#### Memory Pool Implementation (Suggested Optimization) - -*Note: This is a suggested optimization pattern for users who need memory pool functionality. Thread System does not currently include built-in memory pools.* - -```cpp -template -class JobPool { -public: - auto acquire() -> std::unique_ptr { - std::lock_guard lock(mutex_); - if (!pool_.empty()) { - auto job = std::move(pool_.back()); - pool_.pop_back(); - return job; - } - return std::make_unique(); - } - - auto release(std::unique_ptr job) -> void { - if (!job) return; - job->reset(); - - std::lock_guard lock(mutex_); - if (pool_.size() < PoolSize) { - pool_.push_back(std::move(job)); - } - } - -private: - std::vector> pool_; - std::mutex mutex_; -}; -``` - -## Platform-Specific Optimizations - -### macOS/ARM64 Optimizations - -```cpp -#ifdef __APPLE__ -// Leverage performance cores on Apple Silicon -void configure_for_apple_silicon(thread_pool_module::thread_pool& pool) { - size_t performance_cores = 4; // M1 has 4 performance cores - size_t efficiency_cores = 4; // M1 has 4 efficiency cores - - // Prioritize performance cores for CPU-intensive work - for (size_t i = 0; i < performance_cores; ++i) { - auto worker = std::make_unique(pool.get_job_queue()); - pthread_set_qos_class_self_np(QOS_CLASS_USER_INITIATED, 0); - pool.enqueue(std::move(worker)); - } -} -#endif -``` - -### Linux Optimizations - -```cpp -#ifdef __linux__ -void set_thread_affinity(std::thread& thread, uint32_t core_id) { - cpu_set_t cpuset; - CPU_ZERO(&cpuset); - CPU_SET(core_id, &cpuset); - pthread_setaffinity_np(thread.native_handle(), sizeof(cpu_set_t), &cpuset); -} - -void configure_numa_awareness(thread_pool_module::thread_pool& pool) { - // Distribute workers across NUMA nodes - int numa_nodes = numa_max_node() + 1; - auto workers = pool.get_workers(); - - for (size_t i = 0; i < workers.size(); ++i) { - int node = i % numa_nodes; - numa_run_on_node(node); - set_thread_affinity(workers[i]->get_thread(), i); - } -} -#endif -``` - -### Windows Optimizations - -```cpp -#ifdef _WIN32 -void configure_windows_type(thread_pool_module::thread_pool& pool) { - auto workers = pool.get_workers(); - - for (auto& worker : workers) { - SetThreadType(worker->get_thread_handle(), THREAD_PRIORITY_ABOVE_NORMAL); - } -} - -void configure_processor_groups(thread_pool_module::thread_pool& pool) { - DWORD num_groups = GetActiveProcessorGroupCount(); - if (num_groups <= 1) return; - - auto workers = pool.get_workers(); - for (size_t i = 0; i < workers.size(); ++i) { - WORD group = i % num_groups; - GROUP_AFFINITY affinity = {0}; - affinity.Group = group; - affinity.Mask = 1ULL << (i / num_groups); - - SetThreadGroupAffinity(workers[i]->get_thread_handle(), &affinity, nullptr); - } -} -#endif -``` - -## Best Practices - -### Performance Tuning Checklist - -#### Measurement and Analysis -- [x] Establish performance baseline with benchmarks -- [x] Profile actual workload patterns -- [x] Measure thread utilization and queue depths -- [x] Identify bottlenecks through systematic analysis - -#### Thread Pool Configuration -- [x] Set optimal thread count based on workload type -- [x] Configure type-specific workers appropriately -- [x] Consider thread affinity for critical applications -- [x] Adjust for platform-specific characteristics - -#### Job Design -- [x] Batch job submission where possible -- [x] Ensure appropriate job granularity (>100μs recommended) -- [x] Balance workload across job types -- [x] Minimize memory allocation in job execution - -#### Memory Considerations -- [x] Prevent false sharing with proper alignment -- [x] Consider implementing memory pools for frequently allocated objects (not built-in) -- [x] Consider thread-local storage for worker data -- [x] Monitor memory growth under sustained load - -#### Advanced Techniques -- [x] Implement backpressure mechanisms for overload protection -- [x] Consider work-stealing for load balancing -- [x] Use lock-free data structures where appropriate -- [x] Implement circuit breakers for fault tolerance - -### Real-World Performance Guidelines - -#### Web Server Applications -- **Thread Count**: 2x hardware threads for I/O-heavy workloads -- **Job Granularity**: Keep request processing > 100μs -- **Type Usage**: High for interactive requests, Normal for API calls, Low for analytics -- **Memory**: Use connection pools and request object pools - -#### Data Processing Pipelines -- **Thread Count**: Match physical core count -- **Batch Size**: Use large batches (1000+ items) -- **Memory**: Pre-allocate buffers, use memory-mapped files for large datasets -- **Optimization**: Pipeline stages with different thread pools - -#### Real-Time Systems -- **Thread Count**: Reserve 1 core for OS, use remaining cores -- **Latency**: Target <10μs scheduling latency -- **Type**: Strict type separation with dedicated workers -- **Memory**: Pre-allocate all memory, avoid runtime allocation - -#### Scientific Computing -- **Thread Count**: Use all available cores -- **Job Granularity**: Balance computation size with coordination overhead -- **Memory**: Consider NUMA topology and memory bandwidth -- **Optimization**: Use CPU-specific optimizations (SIMD, cache optimization) - -### Monitoring and Diagnostics - -#### Key Performance Indicators - -| Metric | Target Range | Warning Threshold | Critical Threshold | -|--------|-------------|------------------|-------------------| -| Jobs/sec | >100K | <50K | <10K | -| Queue Depth | 0-10 | >50 | >200 | -| CPU Utilization | 80-95% | >98% | 100% sustained | -| Memory Growth | <1% per hour | >5% per hour | >10% per hour | -| Error Rate | <0.1% | >1% | >5% | - -#### Diagnostic Tools - -```cpp -class PerformanceMonitor { -public: - struct Metrics { - std::atomic jobs_submitted{0}; - std::atomic jobs_completed{0}; - std::atomic total_execution_time{0}; - std::atomic current_queue_depth{0}; - std::atomic peak_queue_depth{0}; - }; - - auto get_throughput() const -> double { - auto duration = std::chrono::steady_clock::now() - start_time_; - auto seconds = std::chrono::duration(duration).count(); - return metrics_.jobs_completed.load() / seconds; - } - - auto get_average_latency() const -> double { - uint64_t completed = metrics_.jobs_completed.load(); - if (completed == 0) return 0.0; - return static_cast(metrics_.total_execution_time.load()) / completed; - } - -private: - Metrics metrics_; - std::chrono::steady_clock::time_point start_time_{std::chrono::steady_clock::now()}; -}; -``` - -## Future Performance Improvements - -### Planned Optimizations - -1. **Type Thread Pool Optimizations**: - - Work stealing between type queues for better load balancing - - Batch dequeue operations for reduced overhead - - Type-aware scheduling policies - -2. **Memory Pool Integration**: - - Built-in memory pools for job objects - - Reduce allocation overhead by 60-80% - - Thread-local pools for cache efficiency - -3. **Work Stealing for Type Pools**: - - Allow idle workers to steal from other type queues - - Better CPU utilization under uneven load - - Configurable stealing policies - -## Performance Recommendations Summary (2025) - -### Quick Configuration Guide - -#### 1. **For General Applications** -```cpp -// Use standard thread pool with adaptive queues -auto pool = std::make_shared("MyPool"); - -// Add workers (hardware_concurrency for CPU-bound) -for (int i = 0; i < std::thread::hardware_concurrency(); ++i) { - pool->enqueue(std::make_unique()); -} -pool->start(); -``` - -#### 2. **For Priority-Sensitive Applications** -```cpp -// Use typed thread pool with adaptive queues -auto pool = std::make_shared>("PriorityPool"); - -// Add specialized workers -for (auto priority : {job_types::RealTime, job_types::Batch, job_types::Background}) { - auto worker = std::make_unique>(); - worker->set_responsibilities({priority}); - pool->enqueue(std::move(worker)); -} - -// Add universal workers for load balancing -for (int i = 0; i < 2; ++i) { - auto worker = std::make_unique>(); - worker->set_responsibilities({job_types::RealTime, job_types::Batch, job_types::Background}); - pool->enqueue(std::move(worker)); -} -pool->start(); -``` - -#### 3. **For High-Concurrency Scenarios** -```cpp -// Standard pool with batch processing -auto pool = std::make_shared("HighConcurrency"); - -// Configure workers for batch processing -std::vector> workers; -for (int i = 0; i < std::thread::hardware_concurrency() * 2; ++i) { - auto worker = std::make_unique(); - worker->set_batch_processing(true, 32); // Process up to 32 jobs at once - workers.push_back(std::move(worker)); -} -pool->enqueue_batch(std::move(workers)); -pool->start(); -``` - -### Performance Tuning Quick Reference - -| Scenario | Configuration | Expected Performance | -|----------|---------------|---------------------| -| **CPU-Bound Tasks** | Workers = hardware_concurrency() | 96% efficiency at 8 cores | -| **I/O-Bound Tasks** | Workers = hardware_concurrency() × 2 | Good overlap of I/O waits | -| **Mixed Workload** | Workers = hardware_concurrency() × 1.5 | Balanced performance | -| **Low Latency** | Standard pool, single jobs | ~77ns submission latency | -| **High Throughput** | Batch processing enabled | Up to 13M jobs/s theoretical | -| **Priority Scheduling** | Typed pool with 3-4 workers per type | 99.6% type accuracy | - -### Common Pitfalls to Avoid - -1. **Over-Threading**: Don't create more workers than 2× hardware threads -2. **Small Jobs**: Batch jobs < 10μs for better efficiency -3. **Memory Allocation**: Pre-allocate job objects when possible -4. **Queue Depth**: Monitor queue depth; > 1000 indicates backpressure needed -5. **Type Proliferation**: Keep priority types to 3-5 for optimal performance - - -## Conclusion - -The Thread System framework provides exceptional performance characteristics with the simplified adaptive architecture: - -1. **High Throughput**: - - Standard pool: 1.16M jobs/second (proven in production) - - Adaptive queues: Automatic optimization for all scenarios - - Typed pools: 1.24M jobs/second with priority specialization -2. **Low Latency**: - - Standard pool: 77ns scheduling overhead - - Adaptive queues: 96-580ns with automatic strategy selection - - Consistent performance across varying workloads -3. **Excellent Scalability**: - - Standard pool: 96% efficiency at 8 cores - - Adaptive queues: Maintain performance under any contention level - - Up to **3.46x improvement** under high contention -4. **Memory Efficiency**: - - Standard pool: <1MB baseline memory usage - - Dynamic allocation based on actual usage - - Reduced codebase by ~8,700+ lines without performance loss -5. **Platform Optimization**: - - Consistent performance across Windows, Linux, and macOS - - Platform-specific optimizations where beneficial -6. **Simplified Architecture**: - - Removed duplicate code and unused features - - Maintained all performance capabilities - - Cleaner, more maintainable codebase - -### Key Success Factors - -1. **Simplified Usage**: - - Standard pools with adaptive queues work optimally out-of-the-box - - No manual configuration required - - Automatic optimization for all scenarios -2. **Profile Your Workload**: - - Use built-in benchmarks for baseline measurements - - Monitor actual performance characteristics - - Let adaptive queues handle optimization -3. **Clean Architecture Benefits**: - - Reduced code complexity improves maintainability - - Removed ~8,700+ lines (logger, monitoring, unused utilities) - - Modular design with interface-based architecture - - Performance maintained through smart design -4. **Monitor Performance**: - - Track job throughput and latency - - Monitor worker utilization - - Observe adaptive queue behavior -5. **Best Practices**: - - Use typed pools for priority-based workloads - - Leverage batch operations for small jobs - - Trust automatic optimization - -By following the guidelines and techniques in this comprehensive performance guide, you can achieve optimal performance for your specific application requirements. The simplified adaptive architecture provides powerful optimization capabilities while maintaining the simplicity and reliability of the Thread System framework. ---- - -*Last Updated: 2025-10-20* +This guide has been split into two focused documents for easier navigation: + +| Document | Description | Topics | +|----------|-------------|--------| +| **[Performance Benchmarks](PERFORMANCE_BENCHMARKS.md)** | Benchmark results and data | Performance overview, benchmark environment, core metrics, data race fix impact, detailed benchmarks, typed pool benchmarks, adaptive queue benchmarks, scalability analysis, memory performance, MPMC queue, logger performance, library comparisons | +| **[Performance Tuning](PERFORMANCE_TUNING.md)** | Tuning guidance and best practices | Optimization strategies (thread count, batching, granularity, type pool, adaptive queue, memory), platform-specific optimizations (macOS, Linux, Windows), best practices, performance recommendations, conclusion | + +## Quick Links + +### Benchmarks +- [Performance Overview](PERFORMANCE_BENCHMARKS.md#performance-overview) +- [Benchmark Environment](PERFORMANCE_BENCHMARKS.md#benchmark-environment) +- [Core Performance Metrics](PERFORMANCE_BENCHMARKS.md#core-performance-metrics) +- [Data Race Fix Impact](PERFORMANCE_BENCHMARKS.md#data-race-fix-impact) +- [Detailed Benchmark Results](PERFORMANCE_BENCHMARKS.md#detailed-benchmark-results) +- [Adaptive Job Queue Benchmarks](PERFORMANCE_BENCHMARKS.md#adaptive-job-queue-benchmarks) +- [Scalability Analysis](PERFORMANCE_BENCHMARKS.md#scalability-analysis) +- [Memory Performance](PERFORMANCE_BENCHMARKS.md#memory-performance) +- [Comparison with Other Libraries](PERFORMANCE_BENCHMARKS.md#comparison-with-other-libraries) + +### Tuning +- [Optimization Strategies](PERFORMANCE_TUNING.md#optimization-strategies) +- [Platform-Specific Optimizations](PERFORMANCE_TUNING.md#platform-specific-optimizations) +- [Best Practices](PERFORMANCE_TUNING.md#best-practices) +- [Performance Recommendations Summary](PERFORMANCE_TUNING.md#performance-recommendations-summary-2025) +- [Conclusion](PERFORMANCE_TUNING.md#conclusion) diff --git a/docs/advanced/PERFORMANCE_BENCHMARKS.md b/docs/advanced/PERFORMANCE_BENCHMARKS.md new file mode 100644 index 0000000000..7c65825fb6 --- /dev/null +++ b/docs/advanced/PERFORMANCE_BENCHMARKS.md @@ -0,0 +1,819 @@ +--- +doc_id: "THR-PERF-006a" +doc_title: "Thread System Performance Benchmarks" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "PERF" +--- + +# Thread System Performance Benchmarks + +> **SSOT**: This document is the single source of truth for **Thread System Performance Benchmarks** (benchmark results and data). + +> **Language:** **English** | [한국어](PERFORMANCE.kr.md) + +> **See also**: [Performance Tuning](PERFORMANCE_TUNING.md) for optimization strategies, platform-specific guidance, and best practices. + +This guide covers performance benchmarks, scalability analysis, and comparison data for the Thread System framework. All measurements are based on real benchmark data and extensive testing. + +## Table of Contents + +1. [Performance Overview](#performance-overview) +2. [Benchmark Environment](#benchmark-environment) +3. [Core Performance Metrics](#core-performance-metrics) +4. [Data Race Fix Impact](#data-race-fix-impact) +5. [Detailed Benchmark Results](#detailed-benchmark-results) +6. [Typed Lock-Free Thread Pool Benchmarks](#typed-lock-free-thread-pool-benchmarks) +7. [Adaptive Job Queue Benchmarks](#adaptive-job-queue-benchmarks) +8. [Scalability Analysis](#scalability-analysis) +9. [Memory Performance](#memory-performance) +10. [Adaptive MPMC Queue Performance](#adaptive-mpmc-queue-performance) +11. [Adaptive Logger Performance](#adaptive-logger-performance) +12. [Logger Performance (Now Separate Project)](#logger-performance-now-separate-project) +13. [Comparison with Other Libraries](#comparison-with-other-libraries) + +## Performance Overview + +The Thread System framework delivers exceptional performance across various workload patterns: + +### Key Performance Highlights (Current Architecture) + +- **Peak Throughput**: Up to 13.0M jobs/second (1 worker, empty jobs - theoretical) +- **Real-world Throughput**: + - Standard thread pool: 1.16M jobs/s (10 workers) + - Typed thread pool: 1.24M jobs/s (6 workers) + - Adaptive job queue: Automatically selects optimal strategy +- **Low Latency**: + - Standard pool: ~77 nanoseconds job scheduling latency + - Adaptive queues: 96-580ns based on contention level +- **Scaling Efficiency**: 96% at 8 cores (theoretical), 55-56% real-world +- **Memory Efficient**: <1MB baseline memory usage +- **Cross-Platform**: Consistent performance across Windows, Linux, and macOS +- **Streamlined Architecture**: + - Adaptive queue strategy for automatic optimization + - Reduced code complexity from ~8,700 to ~2,700 lines + - Separated logger and monitoring into independent projects + - Enhanced synchronization primitives and cancellation support + - Service registry for dependency injection + +## Benchmark Environment + +### Test Hardware +- **CPU**: Apple M1 (8-core) - 4 performance + 4 efficiency cores +- **Memory**: 16GB unified memory +- **Storage**: NVMe SSD +- **OS**: macOS Sonoma 14.x + +### Compiler Configuration +- **Compiler**: Apple Clang 17.0.0 +- **C++ Standard**: C++20 +- **Optimization**: -O3 Release mode +- **Features**: std::format enabled, std::thread fallback (std::jthread not available) + +### Thread System Version +- **Version**: Latest development build with streamlined architecture +- **Build Date**: 2025-09-07 (latest update - modularized architecture) +- **Configuration**: Release build with adaptive queue support +- **Benchmark Tool**: Google Benchmark +- **Architecture Changes**: + - Streamlined from ~8,700 to ~2,700 lines of code + - Logger and monitoring moved to separate projects + - Clean interface-based architecture + - Added sync_primitives, cancellation_token, service_registry +- **Performance**: Maintained with automatic optimization via adaptive queues + +## Core Performance Metrics + +### Component Overhead + +| Component | Operation | Overhead | Notes | +|-----------|-----------|----------|-------| +| Thread Base | Thread creation | ~10-15 μs | Per thread initialization | +| Thread Base | Job scheduling | ~77 ns | 10x improvement from previous ~1-2 μs | +| Thread Base | Wake interval access | +5% | Mutex protection added | +| Thread Pool | Pool creation (1 worker) | ~162 ns | Measured with Google Benchmark | +| Thread Pool | Pool creation (8 workers) | ~578 ns | Linear scaling | +| Thread Pool | Pool creation (16 workers) | ~1041 ns | Consistent overhead | +| Adaptive Queue | Mutex mode enqueue | ~96 ns | Default strategy | +| Adaptive Queue | Lock-free mode enqueue | ~320 ns | High contention mode | +| Adaptive Queue | Batch operations | ~212 ns/job | Optimized processing | +| Sync Primitives | Scoped lock with timeout | ~15 ns | RAII wrapper overhead | +| Cancellation Token | Registration | +3% | Thread-safe callbacks | +| Service Registry | Service lookup | ~25 ns | Type-safe retrieval | +| Job Queue | Operations | -4% | Optimized atomics | + +### Thread Pool Creation Performance + +| Worker Count | Creation Time | Items/sec | Notes | +|-------------|---------------|-----------|-------| +| 1 | 162 ns | 6.19M/s | Minimal overhead | +| 2 | 227 ns | 4.43M/s | Good scaling | +| 4 | 347 ns | 2.89M/s | Linear increase | +| 8 | 578 ns | 1.73M/s | Expected overhead | +| 16 | 1041 ns | 960K/s | Still sub-microsecond | + +## Data Race Fix Impact + +### Overview +The recent data race fixes addressed three critical concurrency issues while maintaining excellent performance characteristics: + +1. **thread_base::wake_interval** - Added mutex protection for thread-safe access +2. **cancellation_token** - Fixed double-check pattern and circular references +3. **job_queue::queue_size_** - Removed redundant atomic counter + +### Performance Impact Analysis + +| Fix | Performance Impact | Benefit | Trade-off | +|-----|-------------------|---------|-----------| +| Wake interval mutex | +5% overhead | Thread-safe access | Minimal impact on most workloads | +| Cancellation token fix | +3% overhead | Prevents race conditions | Safer callback registration | +| Job queue optimization | -4% (improvement) | Better cache locality | None - pure win | +| **Net Impact** | **+4% overall** | **100% thread safety** | **Excellent trade-off** | + +### Before vs After Comparison + +| Metric | Before Fixes | After Fixes | Change | +|--------|--------------|-------------|--------| +| Peak throughput | ~12.5M jobs/s | 13.0M jobs/s | +4% | +| Job submission latency | ~80 ns | ~77 ns | -4% | +| Thread safety | 3 data races | 0 data races | ✅ | +| Memory ordering | Weak | Strong | ✅ | + +### Real-World Impact +- **Production Safety**: All data races eliminated, ensuring reliable operation under high concurrency +- **Performance**: Slight net improvement due to job queue optimization offsetting mutex overhead +- **Maintainability**: Cleaner code with proper synchronization primitives + +## Detailed Benchmark Results + +### Job Submission Latency + +#### Standard Thread Pool (Mutex-based) +| Queue State | Avg Latency | 50th Percentile | 95th Percentile | 99th Percentile | +|------------|-------------|-----------------|-----------------|-----------------| +| Empty | 0.8 μs | 0.7 μs | 1.1 μs | 1.2 μs | +| 100 jobs | 0.9 μs | 0.8 μs | 1.3 μs | 1.5 μs | +| 1K jobs | 1.1 μs | 1.0 μs | 1.8 μs | 2.1 μs | +| 10K jobs | 1.3 μs | 1.2 μs | 2.8 μs | 3.5 μs | +| 100K jobs | 1.6 μs | 1.4 μs | 3.2 μs | 4.8 μs | + +#### Adaptive Queue (Lock-free Mode) +| Queue State | Avg Latency | 50th Percentile | 95th Percentile | 99th Percentile | +|------------|-------------|-----------------|-----------------|-----------------| +| Empty | 0.32 μs | 0.28 μs | 0.45 μs | 0.52 μs | +| 100 jobs | 0.35 μs | 0.31 μs | 0.48 μs | 0.58 μs | +| 1K jobs | 0.38 μs | 0.34 μs | 0.52 μs | 0.65 μs | +| 10K jobs | 0.42 μs | 0.38 μs | 0.58 μs | 0.72 μs | +| 100K jobs | 0.48 μs | 0.43 μs | 0.68 μs | 0.85 μs | + +### Throughput by Job Complexity + +#### Standard Thread Pool Performance + +| Job Duration | 1 Worker | 2 Workers | 4 Workers | 8 Workers | Notes | +|-------------|----------|-----------|-----------|-----------|-------| +| Empty job | 13.0M/s | 10.4M/s | 8.3M/s | 6.6M/s | High contention | +| 1 μs work | 890K/s | 1.6M/s | 3.0M/s | 5.5M/s | Good scaling | +| 10 μs work | 95K/s | 180K/s | 350K/s | 680K/s | Near-linear | +| 100 μs work | 9.9K/s | 19.8K/s | 39.5K/s | 78K/s | Excellent scaling | +| 1 ms work | 990/s | 1.98K/s | 3.95K/s | 7.8K/s | CPU-bound | +| 10 ms work | 99/s | 198/s | 395/s | 780/s | I/O-bound territory | +| **Real workload** | **1.16M/s** | - | - | - | **10 workers, measured** | + +#### Adaptive Job Queue Performance (Lock-free Mode) + +| Job Duration | 1 Worker | 2 Workers | 4 Workers | 8 Workers | vs Standard | +|-------------|----------|-----------|-----------|-----------|-------------| +| Empty job | 15.2M/s | 14.8M/s | 13.5M/s | 12.1M/s | +83% avg | +| 1 μs work | 1.2M/s | 2.3M/s | 4.4M/s | 8.2M/s | +49% avg | +| 10 μs work | 112K/s | 218K/s | 425K/s | 820K/s | +21% avg | +| 100 μs work | 10.2K/s | 20.3K/s | 40.5K/s | 80K/s | +3% avg | +| 1 ms work | 995/s | 1.99K/s | 3.97K/s | 7.9K/s | +1% avg | +| 10 ms work | 99/s | 198/s | 396/s | 781/s | ~0% avg | +| **Real workload** | **Available via adaptive strategy** | - | - | - | **Dynamic selection** | + +#### Type Thread Pool Performance + +| Type Mix | Basic Pool | Type Pool | Performance | Type Accuracy | +|-------------|------------|-----------|-------------|---------------| +| Single (High) | 540K/s | 525K/s | -3% | 100% | +| 2 Levels | 540K/s | 510K/s | -6% | 99.8% | +| 3 Levels | 540K/s | 495K/s | -9% | 99.6% | +| 5 Levels | 540K/s | 470K/s | -15% | 99.3% | +| 10 Levels | 540K/s | 420K/s | -29% | 98.8% | + +#### Real-World Measurements (Lock-Free Implementation) + +| Configuration | Throughput | Time (1M jobs) | Workers | CPU Usage | Improvement | +|--------------|------------|----------------|---------|-----------|-------------| +| Basic Pool | 1.16M/s | 862 ms | 10 | 559% | Baseline | +| Type Pool | 1.24M/s | 806 ms | 6 | 330% | +6.9% | + +## Typed Lock-Free Thread Pool Benchmarks + +#### Type Thread Pool with Adaptive Queues + +The Type Thread Pool features adaptive job queue implementation: + +##### typed_thread_pool (Current Implementation) +- **Architecture**: Type-specific job queues with adaptive strategy selection +- **Synchronization**: Dynamic mutex/lock-free switching based on contention +- **Memory**: Adaptive queue allocation per job type +- **Best for**: All scenarios with automatic optimization + +##### Adaptive Queue Strategy +- **Architecture**: Automatic switching between mutex and lock-free based on load +- **Synchronization**: Seamless fallback between strategies +- **Memory**: Optimized allocation based on queue usage patterns +- **Best for**: Dynamic workloads with varying contention patterns + +**Performance Characteristics**: + +| Metric | Adaptive Implementation | Benefits | +|--------|-------------------------|----------| +| Simple jobs (100-10K) | 540K/s baseline | Optimal strategy selection | +| High contention scenarios | Auto lock-free mode | Maintains performance | +| Priority scheduling | Type-based routing | Efficient job distribution | +| Job dequeue latency | ~571 ns (lock-free mode) | Automatic optimization | +| Memory per type | Dynamic allocation | Scalable resource usage | + +**Implementation Features**: +- Automatic strategy selection based on contention detection +- Type-based job routing and worker specialization +- Dynamic queue creation and lifecycle management +- Per-type statistics collection for monitoring and tuning +- Compatible API with seamless optimization + +*Note: The adaptive implementation automatically selects the optimal queue strategy based on runtime conditions, providing both simplicity and performance.* + +## Adaptive Job Queue Benchmarks + +### Overview + +Comprehensive benchmarks demonstrating adaptive job queue performance with automatic strategy selection across multiple dimensions: + +### Thread Pool Level Benchmarks + +#### Simple Job Processing +*Jobs with minimal computation (10 iterations)* + +| Queue Strategy | Job Count | Execution Time | Throughput | Relative Performance | +|---------------|-----------|----------------|------------|---------------------| +| Mutex (low load) | 100 | ~45 μs | 2.22M/s | Baseline | +| Adaptive | 100 | ~42 μs | 2.38M/s | **+7.2%** | +| Mutex (med load) | 1,000 | ~380 μs | 2.63M/s | Baseline | +| Adaptive | 1,000 | ~365 μs | 2.74M/s | **+4.2%** | +| Mutex (high load) | 10,000 | ~3.2 ms | 3.13M/s | Baseline | +| Adaptive | 10,000 | ~3.0 ms | 3.33M/s | **+6.4%** | + +#### Medium Workload Processing +*Jobs with moderate computation (100 iterations)* + +| Queue Strategy | Job Count | Execution Time | Throughput | Relative Performance | +|---------------|-----------|----------------|------------|---------------------| +| Mutex-based | 100 | ~125 μs | 800K/s | Baseline | +| Adaptive | 100 | ~118 μs | 847K/s | **+5.9%** | +| Mutex-based | 1,000 | ~1.1 ms | 909K/s | Baseline | +| Adaptive | 1,000 | ~1.0 ms | 1.00M/s | **+10.0%** | + +#### Priority Scheduling Performance +*Type-based job routing with adaptive queue selection* + +| Jobs per Type | Total Jobs | Processing Time | Routing Accuracy | Priority Handling | +|---------------|------------|-----------------|------------------|-------------------| +| 100 | 300 | ~285 μs | 99.7% | Optimal | +| 500 | 1,500 | ~1.35 ms | 99.4% | Efficient | +| 1,000 | 3,000 | ~2.65 ms | 99.1% | Stable | + +#### High Contention Scenarios +*Multiple producer threads simultaneously submitting jobs* + +| Thread Count | Standard Logger | Adaptive Logger | Performance Gain | +|-------------|---------------------|-------------------|------------------| +| 1 | 1,000 jobs/μs | 1,000 jobs/μs | 0% (baseline) | +| 2 | 850 jobs/μs | 920 jobs/μs | **+8.2%** | +| 4 | 620 jobs/μs | 780 jobs/μs | **+25.8%** | +| 8 | 380 jobs/μs | 650 jobs/μs | **+71.1%** | +| 16 | 190 jobs/μs | 520 jobs/μs | **+173.7%** | + +### Queue Level Benchmarks + +#### Basic Queue Operations +*Raw enqueue/dequeue performance* + +| Operation | Mutex Queue | Adaptive Queue | Improvement | +|-----------|-------------|----------------|-------------| +| Enqueue (single) | ~85 ns | ~78 ns | **+8.2%** | +| Dequeue (single) | ~195 ns | ~142 ns | **+37.3%** | +| Enqueue/Dequeue pair | ~280 ns | ~220 ns | **+27.3%** | + +#### Batch Operations +*Processing multiple items at once* + +| Batch Size | Mutex Queue (μs) | Adaptive Queue (μs) | Improvement | +|-----------|------------------|---------------------|-------------| +| 8 | 2.8 | 2.1 | **+33.3%** | +| 32 | 9.2 | 6.8 | **+35.3%** | +| 128 | 34.1 | 24.7 | **+38.0%** | +| 512 | 128.4 | 91.2 | **+41.0%** | +| 1024 | 248.7 | 175.3 | **+41.9%** | + +#### Contention Stress Tests +*Multiple threads competing for queue access* + +| Concurrent Threads | Mutex Queue (μs) | Adaptive Queue (μs) | Scalability Factor | +|-------------------|------------------|---------------------|-------------------| +| 1 | 28.5 | 29.1 | 0.98x | +| 2 | 65.2 | 42.3 | **1.54x** | +| 4 | 156.8 | 73.5 | **2.13x** | +| 8 | 387.2 | 125.8 | **3.08x** | +| 16 | 892.5 | 218.6 | **4.08x** | + +#### Job Type Routing Features +*Type-based job queue selection and routing* + +| Job Type Mix | Type-specific Jobs | Routing Time | Standard Time | Routing Benefit | +|--------------|-------------------|--------------|---------------|-----------------| +| 33% each type | 1,000 | 142 ns | 168 ns | **+18.3%** | +| 50% High priority | 1,500 | 138 ns | 175 ns | **+26.8%** | +| 80% High priority | 2,400 | 135 ns | 182 ns | **+34.8%** | + +#### Memory Usage Comparison + +| Queue Type | Job Count | Memory Usage | Per-Job Memory | Notes | +|------------|-----------|--------------|----------------|-------| +| Mutex Queue | 100 | 8.2 KB | 82 bytes | Shared data structures | +| Adaptive Queue | 100 | 12.5 KB | 125 bytes | Dynamic allocation | +| Mutex Queue | 1,000 | 24.1 KB | 24 bytes | Memory efficiency improves | +| Adaptive Queue | 1,000 | 31.8 KB | 32 bytes | Good scaling properties | +| Mutex Queue | 10,000 | 195.2 KB | 20 bytes | Excellent density | +| Adaptive Queue | 10,000 | 248.7 KB | 25 bytes | Acceptable overhead | + +### Benchmark Environment Details + +- **Hardware**: Apple M1 (8-core), 16GB RAM +- **Software**: macOS Sonoma, Apple Clang 17.0.0, C++20 +- **Build**: Release mode (-O3), Google Benchmark framework +- **Test Duration**: 10 seconds per benchmark with warmup +- **Iterations**: Auto-determined by Google Benchmark for statistical significance +- **Thread Configuration**: 4 workers (1 per type + 1 universal) +- **Latest Update**: 2025-06-29 with enhanced lock-free algorithms + +### Available Benchmarks + +The Thread System includes comprehensive benchmarks for performance testing: + +#### Thread Pool Benchmarks (`tests/benchmarks/thread_pool_benchmarks/`) +- **gbench_thread_pool**: Basic Google Benchmark integration +- **thread_pool_benchmark**: Core thread pool performance metrics +- **memory_benchmark**: Memory usage and allocation patterns (logger benchmark removed) +- **real_world_benchmark**: Realistic workload simulations +- **stress_test_benchmark**: Extreme load and contention testing +- **scalability_benchmark**: Multi-core scaling analysis +- **contention_benchmark**: Contention-specific scenarios +- **comparison_benchmark**: Cross-library comparisons +- **throughput_detailed_benchmark**: Detailed throughput analysis + +#### Queue Benchmarks (`tests/benchmarks/thread_base_benchmarks/`) +- **mpmc_performance_test**: MPMC queue performance analysis +- **simple_mpmc_benchmark**: Basic queue operations +- **quick_mpmc_test**: Fast queue validation + +#### Other Benchmarks +- **data_race_benchmark**: Data race fix impact analysis + +#### Running Benchmarks +```bash +# Build with benchmarks enabled +./build.sh --clean --benchmark + +# Run specific benchmark +./build/bin/thread_pool_benchmark + +# Run with custom parameters +./build/bin/thread_pool_benchmark --benchmark_time_unit=ms --benchmark_min_time=1s + +# Filter specific tests +./build/bin/thread_pool_benchmark --benchmark_filter="BM_ThreadPool/*" + +# Export results +./build/bin/thread_pool_benchmark --benchmark_format=json > results.json +``` + +### Key Performance Insights + +1. **Adaptive Queue Advantages**: + - Automatic strategy selection based on contention + - Better queue operation latency (20-40% faster) when needed + - Supports both mutex and lock-free modes + - Consistent performance scaling + +2. **Simplified Architecture Benefits**: + - Lower memory overhead for typical scenarios + - Cleaner codebase with automatic optimization + - Predictable performance characteristics + - Maintains lock-free capability when beneficial + +3. **Recommended Usage**: + - **Adaptive queues**: Automatic optimization for all scenarios + - **Type-based routing**: Specialized job handling + - **Dynamic scaling**: Automatic resource allocation + +4. **Performance Characteristics**: + - Adaptive queues show 2-4x better scalability under contention + - Memory overhead: Optimized allocation based on usage + - Type routing adds 15-35% efficiency for specialized jobs + +## Scalability Analysis + +### Worker Thread Scaling Efficiency + +| Workers | Speedup | Efficiency | Queue Depth (avg) | CPU Utilization | +|---------|---------|------------|-------------------|-----------------| +| 1 | 1.0x | 100% | 0.1 | 98% | +| 2 | 2.0x | 99% | 0.2 | 97% | +| 4 | 3.9x | 97.5% | 0.5 | 96% | +| 8 | 7.7x | 96.25% | 1.2 | 95% | +| 16 | 15.0x | 93.75% | 3.1 | 92% | +| 32 | 28.3x | 88.4% | 8.7 | 86% | +| 64 | 52.1x | 81.4% | 22.4 | 78% | + +### Workload-Specific Scaling + +#### CPU-Bound Tasks +- **Optimal Workers**: Hardware core count +- **Peak Efficiency**: 96% at 8 cores +- **Scaling Limit**: Physical cores (performance cores on ARM) +- **Recommended**: Use exact core count for CPU-intensive work + +#### I/O-Bound Tasks +- **Optimal Workers**: 2-3x hardware core count +- **Peak Efficiency**: 85% at 16+ workers +- **Scaling Benefit**: Continues beyond core count +- **Recommended**: Start with 2x cores, tune based on I/O wait time + +#### Mixed Workloads +- **Optimal Workers**: 1.5x hardware core count +- **Peak Efficiency**: 90% at 12 workers +- **Balance Point**: Between CPU and I/O characteristics +- **Recommended**: Profile workload to find optimal balance + +## Memory Performance + +### Memory Usage by Configuration + +| Configuration | Virtual Memory | Resident Memory | Peak Memory | Per-Worker | +|--------------|----------------|-----------------|-------------|------------| +| Base System | 45.2 MB | 12.8 MB | 12.8 MB | - | +| 1 Worker | 46.4 MB | 14.0 MB | 14.2 MB | 1.2 MB | +| 4 Workers | 48.1 MB | 14.6 MB | 15.1 MB | 450 KB | +| 8 Workers | 50.4 MB | 15.4 MB | 16.3 MB | 325 KB | +| 16 Workers | 54.8 MB | 16.6 MB | 18.7 MB | 262 KB | +| 32 Workers | 63.2 MB | 20.2 MB | 25.1 MB | 231 KB | + +### Memory Optimization Impact (v2.0) + +With the latest memory optimizations: + +| Component | Before | After | Savings | Notes | +|-----------|--------|-------|---------|-------| +| Adaptive Queue (idle) | 8.2 MB | 0.4 MB | 95% | Lazy initialization | +| Node Pool (256 nodes) | 16 KB | 1 KB | 93.75% | Reduced initial chunks | +| Thread Pool (8 workers) | 15.4 MB | 12.1 MB | 21% | Combined optimizations | +| Peak Memory (unchanged) | 16.3 MB | 16.3 MB | 0% | Same maximum capacity | + +### Startup Memory Profile + +| Phase | Memory Usage | Time | Description | +|-------|-------------|------|-------------| +| Binary Load | 8.2 MB | 0 ms | Base executable | +| Library Init | 10.4 MB | 2 ms | Dynamic libraries | +| Thread Pool Create | 10.8 MB | 0.3 ms | Pool structure only | +| Worker Spawn (8) | 12.1 MB | 1.2 ms | Thread stack allocation | +| First Job | 12.3 MB | 0.1 ms | Queue initialization | +| Steady State | 12.8 MB | - | Normal operation | + +### Memory Allocation Impact on Performance + +| Memory Pattern | Allocation Size | Jobs/sec | vs No Alloc | P99 Latency | Memory Overhead | +|---------------|----------------|----------|-------------|-------------|-----------------| +| None | 0 | 1,160,000| 100% | 1.8μs | 0 | +| Small | <1KB | 1,044,000| 90% | 2.2μs | +15% | +| Medium | 1-100KB | 684,000 | 59% | 3.8μs | +45% | +| Large | 100KB-1MB | 267,000 | 23% | 9.5μs | +120% | +| Very Large | >1MB | 58,000 | 5% | 42μs | +300% | + +### Potential Memory Pool Optimization + +*Note: Thread System does not currently implement built-in memory pools. The following represents potential improvements with custom memory pool implementations:* + +| Pool Type | Current | With Pool (Estimated) | Potential Improvement | Memory Savings | +|-----------|---------|----------------------|----------------------|----------------| +| Small Jobs | 1.04M/s | 1.11M/s (estimated) | +7% | 60% | +| Medium Jobs | 684K/s | 848K/s (estimated) | +24% | 75% | +| Large Jobs | 267K/s | 385K/s (estimated) | +44% | 80% | + +## Adaptive MPMC Queue Performance + +### Overview +The adaptive MPMC (Multiple Producer Multiple Consumer) queue implementation provides automatic strategy selection for optimal performance: + +- **Architecture**: Dynamic switching between mutex and lock-free strategies +- **Memory Management**: Efficient allocation based on queue usage patterns +- **Contention Handling**: Automatic detection and strategy switching +- **Cache Optimization**: Optimized memory layout for performance + +### Performance Comparison + +| Configuration | Mutex-only Queue | Adaptive MPMC | Improvement | +|--------------|-------------------|----------------|-------------| +| 1P-1C (10K ops) | 2.03 ms | 1.87 ms | +8.6% | +| 2P-2C (10K ops) | 5.21 ms | 3.42 ms | +52.3% | +| 4P-4C (10K ops) | 12.34 ms | 5.67 ms | +117.6% | +| 8P-8C (10K ops) | 28.91 ms | 9.23 ms | +213.4% | +| **Raw operation** | **12.2 μs** | **2.8 μs** | **+431%** | +| **Real workload** | **950 ms/1M** | **865 ms/1M** | **+10%** | + +### Scalability Analysis + +| Workers | Mutex-only Efficiency | Adaptive Efficiency | Efficiency Gain | +|---------|----------------------|-------------------|-----------------| +| 1 | 100% | 100% | 0% | +| 2 | 81% | 95% | +14% | +| 4 | 52% | 88% | +36% | +| 8 | 29% | 82% | +53% | + +### Implementation Details + +## Adaptive Logger Performance + +### Overview +The adaptive logger implementation provides automatic optimization for high-throughput logging scenarios: + +- **Architecture**: Adaptive job queue for log message submission +- **Contention Handling**: Dynamic strategy selection based on load +- **Scalability**: Optimal performance scaling with thread count +- **Compatibility**: Seamless integration with existing code + +### Single-Threaded Performance +*Message throughput comparison* + +| Message Size | Standard Logger | Adaptive Logger | Improvement | +|--------------|-----------------|------------------|-------------| +| Short (17 chars) | 7.64 M/s | 7.42 M/s | -2.9% | +| Medium (123 chars) | 5.73 M/s | 5.61 M/s | -2.1% | +| Long (1024 chars) | 2.59 M/s | 2.55 M/s | -1.5% | + +*Note: Single-threaded performance shows minimal overhead. Benefits appear under contention.* + +### Multi-Threaded Scalability +*Throughput with concurrent logging threads* + +| Threads | Standard Logger | Adaptive Logger | Improvement | +|---------|-----------------|------------------|-------------| +| 2 | 1.91 M/s | 1.95 M/s | **+2.1%** | +| 4 | 0.74 M/s | 1.07 M/s | **+44.6%** | +| 8 | 0.22 M/s | 0.63 M/s | **+186.4%** | +| 16 | 0.16 M/s | 0.54 M/s | **+237.5%** | + +### Formatted Logging Performance +*Complex format string with multiple parameters* + +| Logger Type | Throughput | Latency (ns) | +|-------------|------------|--------------| +| Standard | 2.94 M/s | 340 | +| Adaptive | 2.89 M/s | 346 | + +### Burst Logging Performance +*Handling sudden log bursts* + +| Burst Size | Standard Logger | Adaptive Logger | Improvement | +|------------|-----------------|------------------|-------------| +| 10 messages | 1.90 M/s | 1.88 M/s | -1.1% | +| 100 messages | 5.33 M/s | 5.15 M/s | -3.4% | + +### Mixed Log Types Performance +*Different log levels (Info, Debug, Error, Exception)* + +| Logger Type | Throughput | CPU Efficiency | +|-------------|------------|----------------| +| Standard | 6.51 M/s | 100% | +| Adaptive | 6.42 M/s | 98% | + +### Key Findings + +1. **High Contention Benefits**: Adaptive logger shows significant advantages with 4+ threads +2. **Scalability**: Up to 237% improvement at 16 threads +3. **Minimal Overhead**: Single-threaded performance nearly identical +4. **Use Cases**: Ideal for all multi-threaded applications with automatic optimization + +### Recommendations + +- **Use Adaptive Logger**: Automatic optimization for all scenarios +- **Dynamic Scaling**: Logger adapts to application's threading patterns +- **Batch Processing**: Automatically enabled when beneficial for throughput +- **Buffer Management**: Dynamic queue sizing based on workload + +### Implementation Details + +- **Adaptive Strategy**: Automatic switching between mutex and lock-free based on contention +- **Dynamic Allocation**: Efficient memory usage based on queue patterns +- **Smart Retry Logic**: Intelligent backoff strategies to prevent contention +- **Queue Optimization**: Automatic batching and buffer management +- **Performance Monitoring**: Built-in metrics for optimization decisions + +### Current Status + +- Modularized architecture with adaptive queue selection +- Logger and monitoring separated into independent projects +- All stress tests enabled and passing reliably +- Adaptive implementation provides optimal performance for all scenarios +- Average operation latencies: + - Enqueue: ~96 ns (low contention), ~320 ns (high contention) + - Dequeue: ~571 ns with adaptive optimization + +### Recent Benchmark Results (2025-07-25) + +#### Data Race Fix Verification +| Threads | Wake Interval Access | Cancellation Token | Job Queue Consistency | +|---------|---------------------|--------------------|-----------------------| +| 1 | 163μs/10K | 25μs/100 | - | +| 4 | 272μs/10K | 59μs/400 | 842μs/2K dequeued | +| 8 | 438μs/10K | 111μs/800 | 2.18ms/4K dequeued | +| 16 | 750μs/10K | 210μs/1.6K | 4.81ms/8K dequeued | + +*Note: All data race issues have been resolved with proper synchronization.* + +### Usage Recommendations + +1. **Adaptive Queue Benefits**: + - All contention scenarios automatically optimized + - Latency-sensitive applications benefit from automatic switching + - Systems with varying CPU load patterns + - Real-time applications with dynamic requirements + +2. **Configuration Guidelines**: + ```cpp + // Adaptive behavior (recommended) + adaptive_job_queue queue(adaptive_job_queue::queue_strategy::ADAPTIVE); + + // Force specific strategy when needed + adaptive_job_queue mutex_queue(adaptive_job_queue::queue_strategy::FORCE_MUTEX); + adaptive_job_queue lockfree_queue(adaptive_job_queue::queue_strategy::FORCE_LOCKFREE); + ``` + +### Performance Tuning Tips + +1. **Batch Operations**: Use batch enqueue/dequeue for better throughput +2. **CPU Affinity**: Pin threads to specific cores for consistent performance +3. **Memory Alignment**: Ensure job objects are cache-line aligned +4. **Retry Handling**: Operations may fail under extreme contention - implement retry logic +5. **Monitoring**: Use built-in statistics to track performance metrics including retry counts + +## Logger Performance (Now Separate Project) + +*Note: Logger has been moved to a separate project. The following benchmarks are from when logger was integrated with Thread System.* + +### Logger Comparison with Industry Standards + +### Overview +This section compares Thread System's logging performance against industry-standard logging libraries. The logger uses adaptive job queues for automatic optimization. + +### Single-Threaded Performance Comparison +*Baseline measurements on Apple M1 (8-core)* + +| Logger | Throughput | Latency (ns) | Relative Performance | +|--------|------------|--------------|---------------------| +| Console Output | 542.8K/s | 1,842 | Baseline | +| Thread System Logger | 4.41M/s | 227 | **8.1x** faster | +| Thread System (Adaptive) | 4.34M/s | 240 | **8.0x** faster | + +### Multi-Threaded Scalability +*Concurrent logging performance with adaptive optimization* + +| Threads | Standard Mode | Adaptive Mode | Improvement | +|---------|---------------|---------------|-------------| +| 2 | 2.61M/s | 2.58M/s | -1% | +| 4 | 859K/s | 1.07M/s | **+25%** | +| 8 | 234K/s | 412K/s | **+76%** | +| 16 | 177K/s | 385K/s | **+118%** | + +### Latency Characteristics +*End-to-end logging latency* + +| Logger Type | Mean Latency | P99 Latency | Notes | +|-------------|--------------|-------------|-------| +| Thread System Logger | 144 ns | ~200 ns | Adaptive queue | +| Console Output | 1,880 ns | ~2,500 ns | System call overhead | + +### Key Findings + +1. **Logger Excellence**: + - 8.1x faster than console output + - Excellent single-threaded performance + - Predictable latency characteristics + +2. **Adaptive Scalability**: + - Automatic optimization at 4+ threads + - Up to 118% improvement at high contention + - Minimal overhead in single-threaded scenarios + +3. **Usage Recommendations**: + - Use Thread System Logger for all scenarios + - Adaptive queues automatically optimize based on load + - No manual configuration needed + +### Comparison with spdlog + +*Comprehensive performance comparison with the popular spdlog library* + +#### Single-Threaded Performance +| Logger | Throughput | Latency | vs Console | Notes | +|--------|------------|---------|------------|-------| +| Console Output | 583K/s | 1,716 ns | Baseline | System call overhead | +| **Thread System Logger** | **4.34M/s** | **148 ns** | **7.4x** | Best latency | +| spdlog (sync) | 515K/s | 2,333 ns | 0.88x | Poor performance | +| **spdlog (async)** | **5.35M/s** | - | **9.2x** | Best throughput | + +#### Multi-Threaded Performance (4 Threads) +| Logger | Throughput | vs Single-thread | Scalability | +|--------|------------|------------------|-------------| +| Thread System (Standard) | 599K/s | -86% | Moderate | +| **Thread System (Adaptive)** | **1.07M/s** | -75% | **Good** | +| spdlog (sync) | 210K/s | -59% | Very Poor | +| spdlog (async) | 785K/s | -85% | Poor | + +#### High Contention (8 Threads) +| Logger | Throughput | vs Console | Notes | +|--------|------------|------------|-------| +| Thread System (Standard) | 198K/s | 0.34x | High contention | +| **Thread System (Adaptive)** | **412K/s** | **0.71x** | Auto-optimized | +| spdlog (sync) | 52K/s | 0.09x | Severe degradation | +| spdlog (async) | 240K/s | 0.41x | Queue saturation | + +#### Key Findings + +1. **Single-threaded Champion**: spdlog async (5.35M/s) edges out Thread System (4.34M/s) +2. **Multi-threaded Champion**: Thread System with adaptive queues shows consistent performance +3. **Latency Champion**: Thread System with 148ns, **15.8x lower** than spdlog sync (2333/148 = 15.76) +4. **Scalability**: Thread System adaptive mode provides automatic optimization + +### Recommendations + +1. **For All Applications**: Use Thread System Logger + - Excellent performance with adaptive optimization + - Simple API with type safety + - Built-in file rotation and callbacks + - Automatic optimization for high concurrency + +2. **Configuration Tips**: + - Logger automatically adapts to workload + - No manual tuning required + - Adaptive queues handle burst patterns efficiently + +3. **Usage Example**: + ```cpp + // Simple usage - automatic optimization + log_module::start(); + log_module::write_information("Message: {}", value); + log_module::write_error("Error: {}", error); + log_module::stop(); + ``` + +## Comparison with Other Libraries + +### Throughput Comparison (Real-world measurements) + +| Library | Throughput | Relative Performance | Features | +|---------------------------|------------|---------------------|------------------------| +| **Thread System** | 1.16M/s | 100% (baseline) | Type, logging, C++20, lock-free | +| Intel TBB | ~1.24M/s | ~107% | Industry standard, work stealing | +| Boost.Thread Pool | ~1.09M/s | ~94% | Header-only, portable | +| std::async | ~267K/s | ~23% | Standard library, basic | +| Custom (naive) | ~684K/s | ~59% | Simple mutex-based impl | +| OpenMP | ~1.06M/s | ~92% | Compiler directives | +| Microsoft PPL | ~1.02M/s | ~88% | Windows-specific | + +### Feature Comparison + +| Library | Type Support | Logging | C++20 | Cross-Platform | Memory Pool | Error Handling | +|---------|-----------------|---------|-------|----------------|-------------|----------------| +| Thread System | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ✅ Comprehensive | +| Intel TBB | ✅ Yes | ❌ No | ⚠️ Partial | ✅ Yes | ✅ Yes | ⚠️ Basic | +| Boost.Thread Pool | ❌ No | ❌ No | ⚠️ Partial | ✅ Yes | ❌ No | ⚠️ Basic | +| std::async | ❌ No | ❌ No | ✅ Yes | ✅ Yes | ❌ No | ⚠️ Basic | + +### Latency Comparison (μs) + +| Library | Submission | Execution Start | Total Overhead | +|---------|------------|-----------------|----------------| +| Thread System | 77 ns | 96 ns | 173 ns | +| Intel TBB | ~100 ns | ~90 ns | ~190 ns | +| Boost.Thread Pool | ~150 ns | ~120 ns | ~270 ns | +| std::async | ~15.2 μs | ~12.8 μs | ~28.0 μs | + +--- + +*Last Updated: 2025-10-20* diff --git a/docs/advanced/PERFORMANCE_TUNING.md b/docs/advanced/PERFORMANCE_TUNING.md new file mode 100644 index 0000000000..fd066ba15c --- /dev/null +++ b/docs/advanced/PERFORMANCE_TUNING.md @@ -0,0 +1,534 @@ +--- +doc_id: "THR-PERF-006b" +doc_title: "Thread System Performance Tuning" +doc_version: "1.0.0" +doc_date: "2026-04-04" +doc_status: "Released" +project: "thread_system" +category: "PERF" +--- + +# Thread System Performance Tuning + +> **SSOT**: This document is the single source of truth for **Thread System Performance Tuning** (optimization strategies, platform-specific guidance, best practices). + +> **Language:** **English** | [한국어](PERFORMANCE.kr.md) + +> **See also**: [Performance Benchmarks](PERFORMANCE_BENCHMARKS.md) for benchmark results, scalability analysis, and comparison data. + +This guide covers optimization strategies, platform-specific tuning, and best practices for achieving optimal performance with the Thread System framework. + +## Table of Contents + +1. [Optimization Strategies](#optimization-strategies) +2. [Platform-Specific Optimizations](#platform-specific-optimizations) +3. [Best Practices](#best-practices) +4. [Performance Recommendations Summary (2025)](#performance-recommendations-summary-2025) +5. [Conclusion](#conclusion) + +## Optimization Strategies + +### 1. Optimal Thread Count Selection + +```cpp +uint16_t determine_optimal_thread_count(WorkloadType workload) { + uint16_t hardware_threads = std::thread::hardware_concurrency(); + + switch (workload) { + case WorkloadType::CpuBound: + return hardware_threads; + + case WorkloadType::MemoryBound: + return std::max(1u, hardware_threads / 2); + + case WorkloadType::IoBlocking: + return hardware_threads * 2; + + case WorkloadType::Mixed: + return static_cast(hardware_threads * 1.5); + + case WorkloadType::RealTime: + return hardware_threads - 1; // Reserve one core for OS + } + + return hardware_threads; +} +``` + +### 2. Job Batching for Performance + +Batching reduces scheduling overhead significantly: + +| Batch Size | Overhead per Job | Recommended Use Case | +|------------|-----------------|---------------------| +| 1 | 77 ns | Real-time tasks | +| 10 | 25 ns | Interactive tasks | +| 100 | 8 ns | Background processing| +| 1000 | 3 ns | Batch processing | +| 10000 | 2 ns | Bulk operations | + +```cpp +// Efficient job batching +std::vector> jobs; +jobs.reserve(batch_size); + +for (int i = 0; i < batch_size; ++i) { + jobs.push_back(create_job(data[i])); +} + +pool->enqueue_batch(std::move(jobs)); +``` + +### 3. Job Granularity Optimization + +| Job Execution Time | Recommended Action | Reason | +|--------------------|-------------------|--------| +| < 10μs | Batch 1000+ operations | Overhead dominates | +| 10-100μs | Batch 100 operations | Balance overhead/parallelism | +| 100μs-1ms | Batch 10 operations | Minimize coordination | +| 1ms-10ms | Individual jobs | Good granularity | +| > 10ms | Consider subdivision | Improve responsiveness | + +### 4. Type Pool Configuration + +```cpp +void configure_type_pool(std::shared_ptr pool, + const WorkloadProfile& profile) { + const uint16_t hw_threads = std::thread::hardware_concurrency(); + + // Allocate workers based on type distribution + uint16_t high_workers = static_cast(hw_threads * profile.high_type_ratio); + uint16_t normal_workers = static_cast(hw_threads * profile.normal_type_ratio); + uint16_t low_workers = static_cast(hw_threads * profile.low_type_ratio); + + // Ensure minimum coverage + high_workers = std::max(1u, high_workers); + normal_workers = std::max(1u, normal_workers); + low_workers = std::max(1u, low_workers); + + // Add specialized workers + add_type_workers(pool, job_types::High, high_workers); + add_type_workers(pool, job_types::Normal, normal_workers); + add_type_workers(pool, job_types::Low, low_workers); +} +``` + +### 4b. Adaptive Queue Configuration + +```cpp +#include "typed_thread_pool/pool/typed_thread_pool.h" + +auto create_optimal_pool(const std::string& name, + size_t expected_concurrency, + bool priority_sensitive) -> std::shared_ptr> { + + // Create typed thread pool with adaptive queue strategy + auto pool = std::make_shared>(name); + + // Configure adaptive queue strategy based on expected usage + if (expected_concurrency > 4 || priority_sensitive) { + // High contention - adaptive queue will automatically optimize + pool->set_queue_strategy(queue_strategy::ADAPTIVE); + } + + // Add specialized workers for each priority + auto realtime_worker = std::make_unique>(); + realtime_worker->set_responsibilities({job_types::RealTime}); + pool->add_worker(std::move(realtime_worker)); + + auto batch_worker = std::make_unique>(); + batch_worker->set_responsibilities({job_types::Batch}); + pool->add_worker(std::move(batch_worker)); + + auto background_worker = std::make_unique>(); + background_worker->set_responsibilities({job_types::Background}); + pool->add_worker(std::move(background_worker)); + + // Universal worker for load balancing + auto universal_worker = std::make_unique>(); + universal_worker->set_responsibilities({job_types::RealTime, job_types::Batch, job_types::Background}); + pool->add_worker(std::move(universal_worker)); + + return pool; +} + +// Usage examples +auto high_concurrency_pool = create_optimal_pool( + "HighConcurrency", 8, true); + +auto simple_pool = create_optimal_pool( + "Simple", 2, false); +``` + +### 5. Memory Optimization + +#### Cache-Line Alignment +```cpp +// Prevent false sharing +struct alignas(64) WorkerData { + std::atomic processed_jobs{0}; + std::atomic execution_time{0}; + char padding[64 - 2 * sizeof(std::atomic)]; +}; +``` + +#### Memory Pool Implementation (Suggested Optimization) + +*Note: This is a suggested optimization pattern for users who need memory pool functionality. Thread System does not currently include built-in memory pools.* + +```cpp +template +class JobPool { +public: + auto acquire() -> std::unique_ptr { + std::lock_guard lock(mutex_); + if (!pool_.empty()) { + auto job = std::move(pool_.back()); + pool_.pop_back(); + return job; + } + return std::make_unique(); + } + + auto release(std::unique_ptr job) -> void { + if (!job) return; + job->reset(); + + std::lock_guard lock(mutex_); + if (pool_.size() < PoolSize) { + pool_.push_back(std::move(job)); + } + } + +private: + std::vector> pool_; + std::mutex mutex_; +}; +``` + +## Platform-Specific Optimizations + +### macOS/ARM64 Optimizations + +```cpp +#ifdef __APPLE__ +// Leverage performance cores on Apple Silicon +void configure_for_apple_silicon(thread_pool_module::thread_pool& pool) { + size_t performance_cores = 4; // M1 has 4 performance cores + size_t efficiency_cores = 4; // M1 has 4 efficiency cores + + // Prioritize performance cores for CPU-intensive work + for (size_t i = 0; i < performance_cores; ++i) { + auto worker = std::make_unique(pool.get_job_queue()); + pthread_set_qos_class_self_np(QOS_CLASS_USER_INITIATED, 0); + pool.enqueue(std::move(worker)); + } +} +#endif +``` + +### Linux Optimizations + +```cpp +#ifdef __linux__ +void set_thread_affinity(std::thread& thread, uint32_t core_id) { + cpu_set_t cpuset; + CPU_ZERO(&cpuset); + CPU_SET(core_id, &cpuset); + pthread_setaffinity_np(thread.native_handle(), sizeof(cpu_set_t), &cpuset); +} + +void configure_numa_awareness(thread_pool_module::thread_pool& pool) { + // Distribute workers across NUMA nodes + int numa_nodes = numa_max_node() + 1; + auto workers = pool.get_workers(); + + for (size_t i = 0; i < workers.size(); ++i) { + int node = i % numa_nodes; + numa_run_on_node(node); + set_thread_affinity(workers[i]->get_thread(), i); + } +} +#endif +``` + +### Windows Optimizations + +```cpp +#ifdef _WIN32 +void configure_windows_type(thread_pool_module::thread_pool& pool) { + auto workers = pool.get_workers(); + + for (auto& worker : workers) { + SetThreadType(worker->get_thread_handle(), THREAD_PRIORITY_ABOVE_NORMAL); + } +} + +void configure_processor_groups(thread_pool_module::thread_pool& pool) { + DWORD num_groups = GetActiveProcessorGroupCount(); + if (num_groups <= 1) return; + + auto workers = pool.get_workers(); + for (size_t i = 0; i < workers.size(); ++i) { + WORD group = i % num_groups; + GROUP_AFFINITY affinity = {0}; + affinity.Group = group; + affinity.Mask = 1ULL << (i / num_groups); + + SetThreadGroupAffinity(workers[i]->get_thread_handle(), &affinity, nullptr); + } +} +#endif +``` + +## Best Practices + +### Performance Tuning Checklist + +#### Measurement and Analysis +- [x] Establish performance baseline with benchmarks +- [x] Profile actual workload patterns +- [x] Measure thread utilization and queue depths +- [x] Identify bottlenecks through systematic analysis + +#### Thread Pool Configuration +- [x] Set optimal thread count based on workload type +- [x] Configure type-specific workers appropriately +- [x] Consider thread affinity for critical applications +- [x] Adjust for platform-specific characteristics + +#### Job Design +- [x] Batch job submission where possible +- [x] Ensure appropriate job granularity (>100μs recommended) +- [x] Balance workload across job types +- [x] Minimize memory allocation in job execution + +#### Memory Considerations +- [x] Prevent false sharing with proper alignment +- [x] Consider implementing memory pools for frequently allocated objects (not built-in) +- [x] Consider thread-local storage for worker data +- [x] Monitor memory growth under sustained load + +#### Advanced Techniques +- [x] Implement backpressure mechanisms for overload protection +- [x] Consider work-stealing for load balancing +- [x] Use lock-free data structures where appropriate +- [x] Implement circuit breakers for fault tolerance + +### Real-World Performance Guidelines + +#### Web Server Applications +- **Thread Count**: 2x hardware threads for I/O-heavy workloads +- **Job Granularity**: Keep request processing > 100μs +- **Type Usage**: High for interactive requests, Normal for API calls, Low for analytics +- **Memory**: Use connection pools and request object pools + +#### Data Processing Pipelines +- **Thread Count**: Match physical core count +- **Batch Size**: Use large batches (1000+ items) +- **Memory**: Pre-allocate buffers, use memory-mapped files for large datasets +- **Optimization**: Pipeline stages with different thread pools + +#### Real-Time Systems +- **Thread Count**: Reserve 1 core for OS, use remaining cores +- **Latency**: Target <10μs scheduling latency +- **Type**: Strict type separation with dedicated workers +- **Memory**: Pre-allocate all memory, avoid runtime allocation + +#### Scientific Computing +- **Thread Count**: Use all available cores +- **Job Granularity**: Balance computation size with coordination overhead +- **Memory**: Consider NUMA topology and memory bandwidth +- **Optimization**: Use CPU-specific optimizations (SIMD, cache optimization) + +### Monitoring and Diagnostics + +#### Key Performance Indicators + +| Metric | Target Range | Warning Threshold | Critical Threshold | +|--------|-------------|------------------|-------------------| +| Jobs/sec | >100K | <50K | <10K | +| Queue Depth | 0-10 | >50 | >200 | +| CPU Utilization | 80-95% | >98% | 100% sustained | +| Memory Growth | <1% per hour | >5% per hour | >10% per hour | +| Error Rate | <0.1% | >1% | >5% | + +#### Diagnostic Tools + +```cpp +class PerformanceMonitor { +public: + struct Metrics { + std::atomic jobs_submitted{0}; + std::atomic jobs_completed{0}; + std::atomic total_execution_time{0}; + std::atomic current_queue_depth{0}; + std::atomic peak_queue_depth{0}; + }; + + auto get_throughput() const -> double { + auto duration = std::chrono::steady_clock::now() - start_time_; + auto seconds = std::chrono::duration(duration).count(); + return metrics_.jobs_completed.load() / seconds; + } + + auto get_average_latency() const -> double { + uint64_t completed = metrics_.jobs_completed.load(); + if (completed == 0) return 0.0; + return static_cast(metrics_.total_execution_time.load()) / completed; + } + +private: + Metrics metrics_; + std::chrono::steady_clock::time_point start_time_{std::chrono::steady_clock::now()}; +}; +``` + +## Future Performance Improvements + +### Planned Optimizations + +1. **Type Thread Pool Optimizations**: + - Work stealing between type queues for better load balancing + - Batch dequeue operations for reduced overhead + - Type-aware scheduling policies + +2. **Memory Pool Integration**: + - Built-in memory pools for job objects + - Reduce allocation overhead by 60-80% + - Thread-local pools for cache efficiency + +3. **Work Stealing for Type Pools**: + - Allow idle workers to steal from other type queues + - Better CPU utilization under uneven load + - Configurable stealing policies + +## Performance Recommendations Summary (2025) + +### Quick Configuration Guide + +#### 1. **For General Applications** +```cpp +// Use standard thread pool with adaptive queues +auto pool = std::make_shared("MyPool"); + +// Add workers (hardware_concurrency for CPU-bound) +for (int i = 0; i < std::thread::hardware_concurrency(); ++i) { + pool->enqueue(std::make_unique()); +} +pool->start(); +``` + +#### 2. **For Priority-Sensitive Applications** +```cpp +// Use typed thread pool with adaptive queues +auto pool = std::make_shared>("PriorityPool"); + +// Add specialized workers +for (auto priority : {job_types::RealTime, job_types::Batch, job_types::Background}) { + auto worker = std::make_unique>(); + worker->set_responsibilities({priority}); + pool->enqueue(std::move(worker)); +} + +// Add universal workers for load balancing +for (int i = 0; i < 2; ++i) { + auto worker = std::make_unique>(); + worker->set_responsibilities({job_types::RealTime, job_types::Batch, job_types::Background}); + pool->enqueue(std::move(worker)); +} +pool->start(); +``` + +#### 3. **For High-Concurrency Scenarios** +```cpp +// Standard pool with batch processing +auto pool = std::make_shared("HighConcurrency"); + +// Configure workers for batch processing +std::vector> workers; +for (int i = 0; i < std::thread::hardware_concurrency() * 2; ++i) { + auto worker = std::make_unique(); + worker->set_batch_processing(true, 32); // Process up to 32 jobs at once + workers.push_back(std::move(worker)); +} +pool->enqueue_batch(std::move(workers)); +pool->start(); +``` + +### Performance Tuning Quick Reference + +| Scenario | Configuration | Expected Performance | +|----------|---------------|---------------------| +| **CPU-Bound Tasks** | Workers = hardware_concurrency() | 96% efficiency at 8 cores | +| **I/O-Bound Tasks** | Workers = hardware_concurrency() × 2 | Good overlap of I/O waits | +| **Mixed Workload** | Workers = hardware_concurrency() × 1.5 | Balanced performance | +| **Low Latency** | Standard pool, single jobs | ~77ns submission latency | +| **High Throughput** | Batch processing enabled | Up to 13M jobs/s theoretical | +| **Priority Scheduling** | Typed pool with 3-4 workers per type | 99.6% type accuracy | + +### Common Pitfalls to Avoid + +1. **Over-Threading**: Don't create more workers than 2× hardware threads +2. **Small Jobs**: Batch jobs < 10μs for better efficiency +3. **Memory Allocation**: Pre-allocate job objects when possible +4. **Queue Depth**: Monitor queue depth; > 1000 indicates backpressure needed +5. **Type Proliferation**: Keep priority types to 3-5 for optimal performance + + +## Conclusion + +The Thread System framework provides exceptional performance characteristics with the simplified adaptive architecture: + +1. **High Throughput**: + - Standard pool: 1.16M jobs/second (proven in production) + - Adaptive queues: Automatic optimization for all scenarios + - Typed pools: 1.24M jobs/second with priority specialization +2. **Low Latency**: + - Standard pool: 77ns scheduling overhead + - Adaptive queues: 96-580ns with automatic strategy selection + - Consistent performance across varying workloads +3. **Excellent Scalability**: + - Standard pool: 96% efficiency at 8 cores + - Adaptive queues: Maintain performance under any contention level + - Up to **3.46x improvement** under high contention +4. **Memory Efficiency**: + - Standard pool: <1MB baseline memory usage + - Dynamic allocation based on actual usage + - Reduced codebase by ~8,700+ lines without performance loss +5. **Platform Optimization**: + - Consistent performance across Windows, Linux, and macOS + - Platform-specific optimizations where beneficial +6. **Simplified Architecture**: + - Removed duplicate code and unused features + - Maintained all performance capabilities + - Cleaner, more maintainable codebase + +### Key Success Factors + +1. **Simplified Usage**: + - Standard pools with adaptive queues work optimally out-of-the-box + - No manual configuration required + - Automatic optimization for all scenarios +2. **Profile Your Workload**: + - Use built-in benchmarks for baseline measurements + - Monitor actual performance characteristics + - Let adaptive queues handle optimization +3. **Clean Architecture Benefits**: + - Reduced code complexity improves maintainability + - Removed ~8,700+ lines (logger, monitoring, unused utilities) + - Modular design with interface-based architecture + - Performance maintained through smart design +4. **Monitor Performance**: + - Track job throughput and latency + - Monitor worker utilization + - Observe adaptive queue behavior +5. **Best Practices**: + - Use typed pools for priority-based workloads + - Leverage batch operations for small jobs + - Trust automatic optimization + +By following the guidelines and techniques in this comprehensive performance guide, you can achieve optimal performance for your specific application requirements. The simplified adaptive architecture provides powerful optimization capabilities while maintaining the simplicity and reliability of the Thread System framework. +--- + +*Last Updated: 2025-10-20*