Clio develop
The XRP Ledger API server.
Loading...
Searching...
No Matches
etlng::ETLService Class Reference

This class is responsible for continuously extracting data from a p2p node, and writing that data to the databases. More...

#include <ETLService.hpp>

Inheritance diagram for etlng::ETLService:
Collaboration diagram for etlng::ETLService:

Public Member Functions

 ETLService (util::async::AnyExecutionContext ctx, std::reference_wrapper< util::config::ClioConfigDefinition const > config, std::shared_ptr< data::BackendInterface > backend, std::shared_ptr< LoadBalancerInterface > balancer, std::shared_ptr< etl::NetworkValidatedLedgersInterface > ledgers, std::shared_ptr< LedgerPublisherInterface > publisher, std::shared_ptr< CacheLoaderInterface > cacheLoader, std::shared_ptr< CacheUpdaterInterface > cacheUpdater, std::shared_ptr< ExtractorInterface > extractor, std::shared_ptr< LoaderInterface > loader, std::shared_ptr< InitialLoadObserverInterface > initialLoadObserver, std::shared_ptr< etlng::TaskManagerProviderInterface > taskManagerProvider, std::shared_ptr< etlng::MonitorProviderInterface > monitorProvider, std::shared_ptr< etl::SystemState > state)
 Create an instance of ETLService.
 
void run () override
 Start all components to run ETL service.
 
void stop () override
 Stop the ETL service.
 
boost::json::object getInfo () const override
 Get state of ETL as a JSON object.
 
bool isAmendmentBlocked () const override
 Check for the amendment blocked state.
 
bool isCorruptionDetected () const override
 Check whether Clio detected DB corruptions.
 
std::optional< etl::ETLStategetETLState () const override
 Get the etl nodes' state.
 
std::uint32_t lastCloseAgeSeconds () const override
 Get time passed since last ledger close, in seconds.
 
- Public Member Functions inherited from etlng::ETLServiceInterface

Detailed Description

This class is responsible for continuously extracting data from a p2p node, and writing that data to the databases.

Usually, multiple different processes share access to the same network accessible databases, in which case only one such process is performing ETL and writing to the database. The other processes simply monitor the database for new ledgers, and publish those ledgers to the various subscription streams. If a monitoring process determines that the ETL writer has failed (no new ledgers written for some time), the process will attempt to become the ETL writer.

If there are multiple monitoring processes that try to become the ETL writer at the same time, one will win out, and the others will fall back to monitoring/publishing. In this sense, this class dynamically transitions from monitoring to writing and from writing to monitoring, based on the activity of other processes running on different machines.

Constructor & Destructor Documentation

◆ ETLService()

etlng::ETLService::ETLService ( util::async::AnyExecutionContext ctx,
std::reference_wrapper< util::config::ClioConfigDefinition const > config,
std::shared_ptr< data::BackendInterface > backend,
std::shared_ptr< LoadBalancerInterface > balancer,
std::shared_ptr< etl::NetworkValidatedLedgersInterface > ledgers,
std::shared_ptr< LedgerPublisherInterface > publisher,
std::shared_ptr< CacheLoaderInterface > cacheLoader,
std::shared_ptr< CacheUpdaterInterface > cacheUpdater,
std::shared_ptr< ExtractorInterface > extractor,
std::shared_ptr< LoaderInterface > loader,
std::shared_ptr< InitialLoadObserverInterface > initialLoadObserver,
std::shared_ptr< etlng::TaskManagerProviderInterface > taskManagerProvider,
std::shared_ptr< etlng::MonitorProviderInterface > monitorProvider,
std::shared_ptr< etl::SystemState > state )

Create an instance of ETLService.

Parameters
ctxThe execution context for asynchronous operations
configThe Clio configuration definition
backendInterface to the backend database
balancerLoad balancer for distributing work
ledgersInterface for accessing network validated ledgers
publisherInterface for publishing ledger data
cacheLoaderInterface for loading cache data
cacheUpdaterInterface for updating cache data
extractorThe extractor to use
loaderInterface for loading data
initialLoadObserverThe observer for initial data loading
taskManagerProviderThe provider of the task manager instance
monitorProviderThe provider of the monitor instance
stateSystem state tracking object

Member Function Documentation

◆ getETLState()

std::optional< etl::ETLState > etlng::ETLService::getETLState ( ) const
overridevirtual

Get the etl nodes' state.

Returns
The etl nodes' state, nullopt if etl nodes are not connected

Implements etlng::ETLServiceInterface.

◆ getInfo()

boost::json::object etlng::ETLService::getInfo ( ) const
overridevirtual

Get state of ETL as a JSON object.

Returns
The state of ETL as a JSON object

Implements etlng::ETLServiceInterface.

◆ isAmendmentBlocked()

bool etlng::ETLService::isAmendmentBlocked ( ) const
overridevirtual

Check for the amendment blocked state.

Returns
true if currently amendment blocked; false otherwise

Implements etlng::ETLServiceInterface.

◆ isCorruptionDetected()

bool etlng::ETLService::isCorruptionDetected ( ) const
overridevirtual

Check whether Clio detected DB corruptions.

Returns
true if corruption of DB was detected and cache was stopped.

Implements etlng::ETLServiceInterface.

◆ lastCloseAgeSeconds()

std::uint32_t etlng::ETLService::lastCloseAgeSeconds ( ) const
overridevirtual

Get time passed since last ledger close, in seconds.

Returns
Time passed since last ledger close

Implements etlng::ETLServiceInterface.

◆ run()

void etlng::ETLService::run ( )
overridevirtual

Start all components to run ETL service.

Implements etlng::ETLServiceInterface.

◆ stop()

void etlng::ETLService::stop ( )
overridevirtual

Stop the ETL service.

Note
This method blocks until the ETL service has stopped.

Implements etlng::ETLServiceInterface.


The documentation for this class was generated from the following files: