Crate tracing_limit 

Rate limiting for tracing events.

This crate provides a tracing-subscriber layer that rate limits log events to prevent log flooding. Events are grouped by their callsite and contextual fields, with each unique combination rate limited independently.

How it works

Within each rate limit window (default 10 seconds):

• 1st occurrence: Event is emitted normally
• 2nd occurrence: Emits a “suppressing” warning
• 3rd+ occurrences: Silent until window expires
• After window: Emits a summary of suppressed count, then next event normally

Rate limit grouping

Events are rate limited independently based on a combination of:

• Callsite: The code location where the log statement appears
• Contextual fields: Any fields attached to the event or its parent spans

How fields contribute to grouping

Only these fields create distinct rate limit groups:

• component_id - Different components are rate limited independently

All other fields are ignored for grouping, including:

• fanout_id, input_id, output_id - Not used for grouping to avoid resource/cost implications from high-cardinality tags
• message - The log message itself doesn’t differentiate groups
• internal_log_rate_limit - Control field for enabling/disabling rate limiting
• internal_log_rate_secs - Control field for customizing the rate limit window
• Any custom fields you add

Examples

// Example 1: Different component_id values create separate rate limit groups
info!(component_id = "transform_1", "Processing event");  // Group A
info!(component_id = "transform_2", "Processing event");  // Group B
// Even though the message is identical, these are rate limited independently

// Example 2: Only component_id matters for grouping
info!(component_id = "router", fanout_id = "output_1", "Routing event");  // Group C
info!(component_id = "router", fanout_id = "output_2", "Routing event");  // Group C (same group!)
info!(component_id = "router", fanout_id = "output_1", "Routing event");  // Group C (same group!)
info!(component_id = "router", fanout_id = "output_1", input_id = "kafka", "Routing event");  // Group C (same!)
// All of these share the same group because they have the same component_id
// The fanout_id and input_id fields are ignored to avoid resource/cost implications

// Example 3: Span fields contribute to grouping
let span = info_span!("process", component_id = "transform_1");
let _enter = span.enter();
info!("Processing event");  // Group E: callsite + component_id from span
drop(_enter);

let span = info_span!("process", component_id = "transform_2");
let _enter = span.enter();
info!("Processing event");  // Group F: same callsite but different component_id

// Example 4: Nested spans - child span fields take precedence
let outer = info_span!("outer", component_id = "parent");
let _outer_guard = outer.enter();
let inner = info_span!("inner", component_id = "child");
let _inner_guard = inner.enter();
info!("Nested event");  // Grouped by component_id = "child"

// Example 5: Same callsite with no fields = single rate limit group
info!("Simple message");  // Group G
info!("Simple message");  // Group G
info!("Simple message");  // Group G

// Example 6: Custom fields are ignored for grouping
info!(component_id = "source", input_id = "in_1", "Received data");  // Group H
info!(component_id = "source", input_id = "in_2", "Received data");  // Group H (same group!)
// The input_id field is ignored - only component_id matters

// Example 7: Disabling rate limiting for specific logs
// Rate limiting is ON by default - explicitly disable for important logs
warn!(
    component_id = "critical_component",
    message = "Fatal error occurred",
    internal_log_rate_limit = false
);
// This event will NEVER be rate limited, regardless of how often it fires

// Example 8: Custom rate limit window for specific events
info!(
    component_id = "noisy_component",
    message = "Frequent status update",
    internal_log_rate_secs = 60  // Only log once per minute
);
// Override the default window for this specific log

This ensures logs from different components are rate limited independently, while avoiding resource/cost implications from high-cardinality tags.

Structs

RateLimitedLayer