vector_config/
lib.rs

1// High-level list of TODOS.
2//
3// TODO: `serde` supports defining a default at the struct level to fill in fields when no value is
4// present during serialization, but it also supports defaults on a per-field basis, which override
5// any defaults that would be applied by virtue of the struct-level default.
6//
7// Thus, we should mark a struct optional if it has a struct-level default _or_ if all fields are
8// optional: either literal `Option<T>` fields or if they all have defaults.
9//
10// This could clean up some of the required properties where we have a field-level/struct-level
11// default that we can check by looking at the metadata for the type implementing `T`.
12//
13// TODO: What happens if we try to stick in a field that has a struct with a lifetime attached to
14// it? How does the name of that get generated in terms of what ends up in the schema? Do we even
15// have fields with lifetime bounds in any of our configuration types in `vector`? :thinking:
16//
17// TODO: Is there a way that we could attempt to brute force detect the types of fields being used
18// with a validation to give a compile-time error when validators are used incorrectly? For example,
19// we throw a runtime error if you use a negative `min` range bound on an unsigned integer field,
20// but it's a bit opaque and hard to decipher.  Could we simply brute force match the qualified path
21// field to see if there's any known unsigned integer type in it -- i.e. `u8`, `u64`, etc -- and
22// then throw a compile-error from the macro? We would still end up throwing an error at runtime if
23// our heuristic to detect unsigned integers failed, but we might be able to give a meaningful error
24// closer to the problem, which would be much better.
25//
26// TODO: We may want to simply switch from using `description` as the baseline descriptive field to
27// using `title`.  While, by itself, I think `description` makes a little more sense than `title`,
28// it makes it hard to do split-location documentation.
29//
30// For example, it would be nice to have helper types (i.e. `BatchConfig`, `MultilineConfig`, etc)
31// define their own titles, and then allow other structs that have theor types as fields specify a
32// description. This would be very useful in cases where fields are optional, such that you want the
33// field's title to be the title of the underlying type (e.g.  "Multi-line parsing configuration.")
34// but you want the field's description to say something like "If not specified, then multiline
35// parsing is disabled". Including that description on `MultilineConfig` itself is kind of weird
36// because it forces that on everyone else using it, where, in some cases, it may not be optional at
37// all.
38//
39// TODO: Right now, we're manually generating a referenceable name where it makes sense by appending
40// the module path to the ident for structs/enums, and by crafting the name by hand for anything
41// like stdlib impls, or impls on external types.
42//
43// We do this because technically `std::any::type_name` says that it doesn't provide a stable
44// interface for getting the fully-qualified path of a type, which we would need (in general,
45// regardless of whether or not we used that function) because we don't want definition types
46// totally changing name between compiler versions, etc.
47//
48// This is obviously also tricky from a re-export standpoint i.e. what is the referenceable name of
49// a type that uses the derive macros for `Configurable` but is exported somewhere entirely
50// different? The path would refer to the source no matter what, as it's based on how
51// `std::module_path!()` works. Technically speaking, that's still correct from a "we shouldn't
52// create duplicate schemas for T" standpoint, but could manifest as a non-obvious divergence.
53//
54// TODO: We need to figure out how to handle aliases. Looking previously, it seemed like we might
55// need to do some very ugly combinatorial explosion stuff to define a schema per permutation of all
56// aliased fields in a config. We might be able to get away with using a combination of `allOf` and
57// `oneOf` where we define a subschema for the non-aliased fields, and then a subschema using
58// `oneOf`for each aliased field -- allowing it to match any of the possible field names for that
59// specific field -- and then combine them all with `allOf`, which keeps the schema as compact as
60// possible, I think, short of a new version of the specification coming out that adds native alias
61// support for properties.
62//
63// TODO: Should we add a way, and/or make it the default, that if you only supply a description of a
64// field, it concats the description of the type of the field? for example, you have:
65//
66// /// Predefined ACLs.
67// ///
68// /// For more information, see this link.
69// pub enum PredefinedAcl { ...
70// }
71//
72// and then somewhere else, you use it like this:
73//
74// struct Foo {
75// ...
76//     /// The Predefined ACL to apply to newly created objects.
77//     field: PredefinedAcl,
78// ...
79// }
80//
81// the resulting docs for `field` should look as if we wrote this directly:
82//
83// /// The Predefined ACL to apply to newly created objects.
84// ///
85// /// For more information, see this link.
86//
87// Basically, we're always documenting these shared types fully, but sometimes their title is
88// written in an intentionally generic way, and we may want to spice up the wording so it's
89// context-specific i.e. we're using predefined ACLs for new objects, or using it for new firewall
90// rules, or ... so on and so forth. and by concating the existing description on the shared type,
91// we can continue to include high-quality doc comments with contextual links, examples, etc and
92// avoid duplication.
93//
94// One question there would be: do we concat the description of the field _and_ the field's type
95// together? We would probably have to, since the unwritten rule is to use link references, which
96// are shoved at the end of the description like a footnote, and if we have a link reference in our
97// field's title, then we need the field's description to be concatenated so that it can be resolved.
98//
99// TODO: Should we always apply the transparent marker to fields when they're the only field in a
100// tuple struct/tuple variant? There's also some potential interplay with using the `derived` helper
101// attribute on the tuple struct/tuple variant itself to signal that we want to pull the
102// title/description from the field instead, which could be useful when using newtype wrappers
103// around existing/remote types for the purpose of making them `Configurable`.
104#![deny(warnings)]
105
106// Re-export of the various public dependencies required by the generated code to simplify the import requirements for
107// crates actually using the macros/derives.
108pub mod indexmap {
109    pub use indexmap::*;
110}
111
112pub use serde_json;
113
114pub mod component;
115mod configurable;
116pub use self::configurable::{Configurable, ConfigurableRef, ToValue};
117mod errors;
118pub use self::errors::{BoundDirection, GenerateError};
119mod external;
120mod http;
121mod metadata;
122pub use self::metadata::Metadata;
123mod named;
124pub use self::named::NamedComponent;
125mod num;
126pub use self::num::{ConfigurableNumber, NumberClass};
127pub mod schema;
128pub mod ser;
129mod stdlib;
130mod str;
131pub use self::str::ConfigurableString;
132
133// Re-export of the `#[configurable_component]` and `#[derive(Configurable)]` proc macros.
134pub use vector_config_macros::*;
135
136// Re-export of both `Format` and `Validation` from `vector_config_common`.
137//
138// The crate exists so that both `vector_config_macros` and `vector_config` can import the types and work with them
139// natively, but from a codegen and usage perspective, it's much cleaner to export everything needed to use
140// `Configurable` from `vector_config` itself, and not leak out the crate arrangement as an impl detail.
141pub use vector_config_common::{attributes, validation};
142
143#[doc(hidden)]
144pub fn __ensure_numeric_validation_bounds<N>(metadata: &Metadata) -> Result<(), GenerateError>
145where
146    N: Configurable + ConfigurableNumber,
147{
148    // In `Validation::ensure_conformance`, we do some checks on any supplied numeric bounds to try and ensure they're
149    // no larger than the largest f64 value where integer/floating-point conversions are still lossless.  What we
150    // cannot do there, however, is ensure that the bounds make sense for the type on the Rust side, such as a user
151    // supplying a negative bound which would be fine for `i64`/`f64` but not for `u64`. That's where this function
152    // comes in.
153    //
154    // We simply check the given metadata for any numeric validation bounds, and ensure they do not exceed the
155    // mechanical limits of the given numeric type `N`.  If they do, we panic, which is not as friendly as a contextual
156    // compile-time error emitted from the `Configurable` derive macro... but we're working with what we've got here.
157    let mechanical_min_bound = N::get_enforced_min_bound();
158    let mechanical_max_bound = N::get_enforced_max_bound();
159
160    for validation in metadata.validations() {
161        if let validation::Validation::Range { minimum, maximum } = validation {
162            if let Some(min_bound) = minimum {
163                if *min_bound < mechanical_min_bound {
164                    return Err(GenerateError::IncompatibleNumericBounds {
165                        numeric_type: std::any::type_name::<N>(),
166                        bound_direction: BoundDirection::Minimum,
167                        mechanical_bound: mechanical_min_bound,
168                        specified_bound: *min_bound,
169                    });
170                }
171            }
172
173            if let Some(max_bound) = maximum {
174                if *max_bound > mechanical_max_bound {
175                    return Err(GenerateError::IncompatibleNumericBounds {
176                        numeric_type: std::any::type_name::<N>(),
177                        bound_direction: BoundDirection::Maximum,
178                        mechanical_bound: mechanical_max_bound,
179                        specified_bound: *max_bound,
180                    });
181                }
182            }
183        }
184    }
185
186    Ok(())
187}