Enum vector_buffers::config::BufferConfig

pub enum BufferConfig {
    Single(BufferType),
    Chained(Vec<BufferType>),
}

Buffer configuration.

Buffers are compromised of stages(*) that form a buffer topology, with input items being subject to configurable behavior when each stage reaches configured limits. Buffers are configured for sinks, where backpressure from the sink can be handled by the buffer. This allows absorbing temporary load, or potentially adding write-ahead-log behavior to a sink to increase the durability of a given Vector pipeline.

While we use the term “buffer topology” here, a buffer topology is referred to by the more common “buffer” or “buffers” shorthand. This is related to buffers originally being a single component, where you could only choose which buffer type to use. As we expand buffer functionality to allow chaining buffers together, you’ll see “buffer topology” used in internal documentation to correctly reflect the internal structure.

Variants

Single(BufferType)

A single stage buffer topology.

Chained(Vec<BufferType>)

A chained buffer topology.

Implementations

impl BufferConfig

pub fn stages(&self) -> &[BufferType]

Gets all of the configured stages for this buffer.

pub async fn build<T>( &self, data_dir: Option<PathBuf>, buffer_id: String, span: Span, ) -> Result<(BufferSender<T>, BufferReceiver<T>), BufferBuildError>

where T: Bufferable + Clone + Finalizable,

Builds the buffer components represented by this configuration.

The caller gets back a Sink and Stream implementation that represent a way to push items into the buffer, as well as pop items out of the buffer, respectively.

Errors

If the buffer is configured with anything other than a single stage, an error variant will be thrown.

If a disk buffer stage is configured and the data directory provided is None, an error variant will be thrown.

Trait Implementations

impl Clone for BufferConfig

fn clone(&self) -> BufferConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Configurable for BufferConfig

fn referenceable_name() -> Option<&'static str>

Gets the referenceable name of this value, if any.

fn metadata() -> Metadata

Gets the metadata for this value.

fn generate_schema( schema_gen: &RefCell<SchemaGenerator>, ) -> Result<SchemaObject, GenerateError>

Generates the schema for this value.

fn is_optional() -> bool

where Self: Sized,

Whether or not this value is optional.

fn validate_metadata(_metadata: &Metadata) -> Result<(), GenerateError>

where Self: Sized,

Validates the given metadata against this type.

fn as_configurable_ref() -> ConfigurableRef

where Self: Sized + 'static,

Create a new configurable reference table.

impl Debug for BufferConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BufferConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for BufferConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for BufferConfig

fn eq(&self, other: &BufferConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for BufferConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl ToValue for BufferConfig

fn to_value(&self) -> Value

Convert this value into a serde_json::Value. Must not fail.

impl Eq for BufferConfig

impl StructuralPartialEq for BufferConfig