Struct vector_core::metrics::AgentDDSketch

pub struct AgentDDSketch { /* private fields */ }

DDSketch implementation based on the Datadog Agent.

This implementation is subtly different from the open-source implementations of DDSketch, as Datadog made some slight tweaks to configuration values and in-memory layout to optimize it for insertion performance within the agent.

We’ve mimicked the agent version of DDSketch here in order to support a future where we can take sketches shipped by the agent, handle them internally, merge them, and so on, without any loss of accuracy, eventually forwarding them to Datadog ourselves.

As such, this implementation is constrained in the same ways: the configuration parameters cannot be changed, the collapsing strategy is fixed, and we support a limited number of methods for inserting into the sketch.

Importantly, we have a special function, again taken from the agent version, to allow us to interpolate histograms, specifically our own aggregated histograms, into a sketch so that we can emit useful default quantiles, rather than having to ship the buckets – upper bound and count – to a downstream system that might have no native way to do the same thing, basically providing no value as they have no way to render useful data from them.

Implementations

impl AgentDDSketch

pub fn with_agent_defaults() -> Self

Creates a new AgentDDSketch based on a configuration that is identical to the one used by the Datadog agent itself.

pub fn from_raw( count: u32, min: f64, max: f64, sum: f64, avg: f64, keys: &[i16], counts: &[u16], ) -> Option<AgentDDSketch>

Creates a new AgentDDSketch based on the given inputs.

This is only useful for constructing a sketch from the raw components when the sketch has passed through the transform boundary into Lua/VRL and needs to be reconstructed.

This is a light code smell, as our intention is to rigorously mediate access and mutation of a sketch through AgentDDSketch and the provided methods.

pub fn gamma(&self) -> f64

pub fn bin_index_offset(&self) -> i32

pub fn bins(&self) -> &[Bin]

pub fn bin_map(&self) -> BinMap

pub fn config(&self) -> &Config

pub fn is_empty(&self) -> bool

Whether or not this sketch is empty.

pub fn count(&self) -> u32

Number of samples currently represented by this sketch.

pub fn min(&self) -> Option<f64>

Minimum value seen by this sketch.

Returns None if the sketch is empty.

pub fn max(&self) -> Option<f64>

Maximum value seen by this sketch.

Returns None if the sketch is empty.

pub fn sum(&self) -> Option<f64>

Sum of all values seen by this sketch.

Returns None if the sketch is empty.

pub fn avg(&self) -> Option<f64>

Average value seen by this sketch.

Returns None if the sketch is empty.

pub fn clear(&mut self)

Clears the sketch, removing all bins and resetting all statistics.

pub fn insert(&mut self, v: f64)

pub fn insert_many(&mut self, vs: &[f64])

pub fn insert_n(&mut self, v: f64, n: u32)

pub fn insert_interpolate_buckets( &mut self, buckets: Vec<Bucket>, ) -> Result<(), &'static str>

Errors

Returns an error if a bucket size is greater that u32::MAX.

pub fn quantile(&self, q: f64) -> Option<f64>

pub fn merge(&mut self, other: &AgentDDSketch) -> Result<(), MergeError>

Merges another sketch into this sketch, without a loss of accuracy.

All samples present in the other sketch will be correctly represented in this sketch, and summary statistics such as the sum, average, count, min, and max, will represent the sum of samples from both sketches.

Errors

If there is an error while merging the two sketches together, an error variant will be returned that describes the issue.

pub fn transform_to_sketch(metric: Metric) -> Result<Metric, &'static str>

Converts a Metric to a sketch representation, if possible, using AgentDDSketch.

For certain types of metric values, such as distributions or aggregated histograms, we can easily convert them to a sketch-based representation. Rather than push the logic of how to do that up to callers that wish to use a sketch-based representation, we bundle it here as a free function on AgentDDSketch itself.

If the metric value cannot be represented as a sketch – essentially, everything that isn’t a distribution or aggregated histogram – then the metric is passed back unmodified. All existing metadata – series name, tags, timestamp, etc – is left unmodified, even if the metric is converted to a sketch internally.

Errors

Returns an error if a bucket size is greater that u32::MAX.

Trait Implementations

impl ByteSizeOf for AgentDDSketch

fn allocated_bytes(&self) -> usize

Returns the allocated bytes of this type

fn size_of(&self) -> usize

Returns the in-memory size of this type

impl Clone for AgentDDSketch

fn clone(&self) -> AgentDDSketch

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Configurable for AgentDDSketch

fn referenceable_name() -> Option<&'static str>

Gets the referenceable name of this value, if any.

fn metadata() -> Metadata

Gets the metadata for this value.

fn generate_schema( schema_gen: &RefCell<SchemaGenerator>, ) -> Result<SchemaObject, GenerateError>

Generates the schema for this value.

fn is_optional() -> bool

where Self: Sized,

Whether or not this value is optional.

fn validate_metadata(_metadata: &Metadata) -> Result<(), GenerateError>

where Self: Sized,

Validates the given metadata against this type.

fn as_configurable_ref() -> ConfigurableRef

where Self: Sized + 'static,

Create a new configurable reference table.

impl Debug for AgentDDSketch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for AgentDDSketch

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<AgentDDSketch> for MetricValue

fn from(ddsketch: AgentDDSketch) -> Self

Converts to this type from the input type.

impl From<AgentDDSketch> for Sketch

fn from(ddsketch: AgentDDSketch) -> Self

Converts to this type from the input type.

impl PartialEq for AgentDDSketch

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for AgentDDSketch

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl ToValue for AgentDDSketch

fn to_value(&self) -> Value

Convert this value into a serde_json::Value. Must not fail.

impl Eq for AgentDDSketch