Module enrichment::tables

The Enrichment TableRegistry manages the collection of Tables loaded into Vector. Enrichment Tables go through two stages.

1. Writing

The tables are loaded. There are two elements that need loading. The first is the actual data. This is loaded at config load time, the actual loading is performed by the implementation of the EnrichmentTable trait. Next, the tables are passed through Vectors Transform components, particularly the Remap transform. These Transforms are able to determine which fields we will want to lookup whilst Vector is running. They can notify the tables of these fields so that the data can be indexed.

During this phase, the data is loaded within a single thread, so can be loaded directly into a HashMap.

2. Reading

Once all the data has been loaded we can move to the next stage. This is signified by calling the finish_load method. At this point all the data is swapped into the ArcSwap of the tables field. ArcSwap provides lock-free read-only access to the data. From this point on we have fast, efficient read-only access and can no longer add indexes or otherwise mutate the data.

This data within the ArcSwap is accessed through the TableSearch struct. Any transform that needs access to this can call TableRegistry::as_readonly. This returns a cheaply clonable struct that implements vrl:EnrichmentTableSearch through with the enrichment tables can be searched.

Structs

• TableRegistry
• TableSearch
  Provides read only access to the enrichment tables via the vrl::EnrichmentTableSearch trait. Cloning this object is designed to be cheap. The underlying data will be shared by all clones.