xrpl::NodeStore::Database Class Reference

Persistency layer for NodeObject. More...

#include <Database.h>

Inheritance diagram for xrpl::NodeStore::Database:

Collaboration diagram for xrpl::NodeStore::Database:

Public Member Functions
	Database ()=delete
	Database (Scheduler &scheduler, int readThreads, Section const &config, beast::Journal j)
	Construct the node store.
virtual	~Database ()
	Destroy the node store.
virtual std::string	getName () const =0
	Retrieve the name associated with this backend.
virtual void	importDatabase (Database &source)=0
	Import objects from another database.
virtual std::int32_t	getWriteLoad () const =0
	Retrieve the estimated number of pending write operations.
virtual void	store (NodeObjectType type, Blob &&data, uint256 const &hash, std::uint32_t ledgerSeq)=0
	Store the object.
virtual bool	isSameDB (std::uint32_t s1, std::uint32_t s2)=0
virtual void	sync ()=0
std::shared_ptr< NodeObject >	fetchNodeObject (uint256 const &hash, std::uint32_t ledgerSeq=0, FetchType fetchType=FetchType::Synchronous, bool duplicate=false)
	Fetch a node object.
virtual void	asyncFetch (uint256 const &hash, std::uint32_t ledgerSeq, std::function< void(std::shared_ptr< NodeObject > const &)> &&callback)
	Fetch an object without waiting.
virtual void	sweep ()=0
	Remove expired entries from the positive and negative caches.
std::uint64_t	getStoreCount () const
	Gather statistics pertaining to read and write activities.
std::uint32_t	getFetchTotalCount () const
std::uint32_t	getFetchHitCount () const
std::uint64_t	getStoreSize () const
std::uint32_t	getFetchSize () const
void	getCountsJson (json::Value &obj)
int	fdRequired () const
	Returns the number of file descriptors the database expects to need.
virtual void	stop ()
bool	isStopping () const
std::uint32_t	earliestLedgerSeq () const noexcept

Protected Member Functions
void	storeStats (std::uint64_t count, std::uint64_t sz)
void	importInternal (Backend &dstBackend, Database &srcDB)
void	updateFetchMetrics (uint64_t fetches, uint64_t hits, uint64_t duration)

Protected Attributes
beast::Journal const	j_
Scheduler &	scheduler_
int	fdRequired_ {0}
std::atomic< std::uint32_t >	fetchHitCount_ {0}
std::atomic< std::uint32_t >	fetchSz_ {0}
std::uint32_t const	earliestLedgerSeq_
int const	requestBundle_

Private Member Functions
virtual std::shared_ptr< NodeObject >	fetchNodeObject (uint256 const &hash, std::uint32_t ledgerSeq, FetchReport &fetchReport, bool duplicate)=0
virtual void	forEach (std::function< void(std::shared_ptr< NodeObject >)> f)=0
	Visit every object in the database This is usually called during import.
void	threadEntry ()

Private Attributes
std::atomic< std::uint64_t >	storeCount_ {0}
std::atomic< std::uint64_t >	storeSz_ {0}
std::atomic< std::uint64_t >	fetchTotalCount_ {0}
std::atomic< std::uint64_t >	fetchDurationUs_ {0}
std::atomic< std::uint64_t >	storeDurationUs_ {0}
std::mutex	readLock_
std::condition_variable	readCondVar_
std::map< uint256, std::vector< std::pair< std::uint32_t, std::function< void(std::shared_ptr< NodeObject > const &)> > > >	read_
std::atomic< bool >	readStopping_ = false
std::atomic< int >	readThreads_ = 0
std::atomic< int >	runningThreads_ = 0

Detailed Description

Persistency layer for NodeObject.

A Node is a ledger object which is uniquely identified by a key, which is the 256-bit hash of the body of the node. The payload is a variable length block of serialized data.

All ledger data is stored as node objects and as such, needs to be persisted between launches. Furthermore, since the set of node objects will in general be larger than the amount of available memory, purged node objects which are later accessed must be retrieved from the node store.

See also

NodeObject

Definition at line 31 of file Database.h.

Constructor & Destructor Documentation

Database() [1/2]

xrpl::NodeStore::Database::Database ( )

delete

Database() [2/2]

xrpl::NodeStore::Database::Database	(	Scheduler &	scheduler,
		int	readThreads,
		Section const &	config,
		beast::Journal	j )

Construct the node store.

Parameters

scheduler	The scheduler to use for performing asynchronous tasks.
readThreads	The number of asynchronous read threads to create.
config	The configuration settings
journal	Destination for logging output.

Definition at line 35 of file Database.cpp.

~Database()

xrpl::NodeStore::Database::~Database ( )

virtual

Destroy the node store.

All pending operations are completed, pending writes flushed, and files closed before this returns.

Definition at line 126 of file Database.cpp.

Member Function Documentation

getName()

virtual std::string xrpl::NodeStore::Database::getName ( ) const

pure virtual

Retrieve the name associated with this backend.

This is used for diagnostics and may not reflect the actual path or paths used by the underlying backend.

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

importDatabase()

virtual void xrpl::NodeStore::Database::importDatabase ( Database & source )

pure virtual

Import objects from another database.

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

getWriteLoad()

virtual std::int32_t xrpl::NodeStore::Database::getWriteLoad ( ) const

pure virtual

Retrieve the estimated number of pending write operations.

This is used for diagnostics.

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

store()

virtual void xrpl::NodeStore::Database::store ( NodeObjectType type, Blob && data, uint256 const & hash, std::uint32_t ledgerSeq )

pure virtual

Store the object.

The caller's Blob parameter is overwritten.

Parameters

type	The type of object.
data	The payload of the object. The caller's variable is overwritten.
hash	The 256-bit hash of the payload data.
ledgerSeq	The sequence of the ledger the object belongs to.

Returns

true if the object was stored?

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

isSameDB()

virtual bool xrpl::NodeStore::Database::isSameDB ( std::uint32_t s1, std::uint32_t s2 )

pure virtual

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

sync()

virtual void xrpl::NodeStore::Database::sync ( )

pure virtual

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

fetchNodeObject() [1/2]

std::shared_ptr< NodeObject > xrpl::NodeStore::Database::fetchNodeObject	(	uint256 const &	hash,
		std::uint32_t	ledgerSeq = 0,
		FetchType	fetchType = FetchType::Synchronous,
		bool	duplicate = false )

Fetch a node object.

If the object is known to be not in the database, isn't found in the database during the fetch, or failed to load correctly during the fetch, nullptr is returned.

Note

This can be called concurrently.

Parameters

hash	The key of the object to retrieve.
ledgerSeq	The sequence of the ledger where the object is stored.
fetchType	the type of fetch, synchronous or asynchronous.

Returns

The object, or nullptr if it couldn't be retrieved.

Definition at line 231 of file Database.cpp.

asyncFetch()

void xrpl::NodeStore::Database::asyncFetch ( uint256 const & hash, std::uint32_t ledgerSeq, std::function< void(std::shared_ptr< NodeObject > const &)> && callback )

virtual

Fetch an object without waiting.

If I/O is required to determine whether or not the object is present, false is returned. Otherwise, true is returned and object is set to refer to the object, or nullptr if the object is not present. If I/O is required, the I/O is scheduled and true is returned

Note

This can be called concurrently.

Parameters

hash	The key of the object to retrieve
ledgerSeq	The sequence of the ledger where the object is stored.
callback	Callback function when read completes

Reimplemented in xrpl::NodeStore::DatabaseNodeImp.

Definition at line 178 of file Database.cpp.

sweep()

virtual void xrpl::NodeStore::Database::sweep ( )

pure virtual

Remove expired entries from the positive and negative caches.

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

getStoreCount()

std::uint64_t xrpl::NodeStore::Database::getStoreCount

(

)

const

Gather statistics pertaining to read and write activities.

Parameters

obj	Json object reference into which to place counters.

Definition at line 146 of file Database.h.

getFetchTotalCount()

std::uint32_t xrpl::NodeStore::Database::getFetchTotalCount

(

)

const

Definition at line 152 of file Database.h.

getFetchHitCount()

std::uint32_t xrpl::NodeStore::Database::getFetchHitCount

(

)

const

Definition at line 158 of file Database.h.

getStoreSize()

std::uint64_t xrpl::NodeStore::Database::getStoreSize

(

)

const

Definition at line 164 of file Database.h.

getFetchSize()

std::uint32_t xrpl::NodeStore::Database::getFetchSize

(

)

const

Definition at line 170 of file Database.h.

getCountsJson()

void xrpl::NodeStore::Database::getCountsJson

(

json::Value &

obj

)

Definition at line 258 of file Database.cpp.

fdRequired()

int xrpl::NodeStore::Database::fdRequired

(

)

const

Returns the number of file descriptors the database expects to need.

Definition at line 180 of file Database.h.

stop()

void xrpl::NodeStore::Database::stop ( )

virtual

Definition at line 144 of file Database.cpp.

isStopping()

bool xrpl::NodeStore::Database::isStopping

(

)

const

Definition at line 138 of file Database.cpp.

earliestLedgerSeq()

std::uint32_t xrpl::NodeStore::Database::earliestLedgerSeq ( ) const

nodiscardnoexcept

Returns

The earliest ledger sequence allowed

Definition at line 194 of file Database.h.

storeStats()

void xrpl::NodeStore::Database::storeStats ( std::uint64_t count, std::uint64_t sz )

protected

Definition at line 221 of file Database.h.

importInternal()

void xrpl::NodeStore::Database::importInternal ( Backend & dstBackend, Database & srcDB )

protected

Definition at line 193 of file Database.cpp.

updateFetchMetrics()

void xrpl::NodeStore::Database::updateFetchMetrics ( uint64_t fetches, uint64_t hits, uint64_t duration )

protected

Definition at line 233 of file Database.h.

fetchNodeObject() [2/2]

virtual std::shared_ptr< NodeObject > xrpl::NodeStore::Database::fetchNodeObject ( uint256 const & hash, std::uint32_t ledgerSeq, FetchReport & fetchReport, bool duplicate )

privatepure virtual

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

forEach()

virtual void xrpl::NodeStore::Database::forEach ( std::function< void(std::shared_ptr< NodeObject >)> f )

privatepure virtual

Visit every object in the database This is usually called during import.

Note

This routine will not be called concurrently with itself or other methods.

See also

import

Implemented in xrpl::NodeStore::DatabaseNodeImp, and xrpl::NodeStore::DatabaseRotatingImp.

threadEntry()

void xrpl::NodeStore::Database::threadEntry ( )

private

Member Data Documentation

j_

beast::Journal const xrpl::NodeStore::Database::j_

protected

Definition at line 200 of file Database.h.

scheduler_

Scheduler& xrpl::NodeStore::Database::scheduler_

protected

Definition at line 201 of file Database.h.

fdRequired_

int xrpl::NodeStore::Database::fdRequired_ {0}

protected

Definition at line 202 of file Database.h.

fetchHitCount_

std::atomic<std::uint32_t> xrpl::NodeStore::Database::fetchHitCount_ {0}

protected

Definition at line 204 of file Database.h.

fetchSz_

std::atomic<std::uint32_t> xrpl::NodeStore::Database::fetchSz_ {0}

protected

Definition at line 205 of file Database.h.

earliestLedgerSeq_

std::uint32_t const xrpl::NodeStore::Database::earliestLedgerSeq_

protected

Definition at line 213 of file Database.h.

requestBundle_

int const xrpl::NodeStore::Database::requestBundle_

protected

Definition at line 218 of file Database.h.

storeCount_

std::atomic<std::uint64_t> xrpl::NodeStore::Database::storeCount_ {0}

private

Definition at line 241 of file Database.h.

storeSz_

std::atomic<std::uint64_t> xrpl::NodeStore::Database::storeSz_ {0}

private

Definition at line 242 of file Database.h.

fetchTotalCount_

std::atomic<std::uint64_t> xrpl::NodeStore::Database::fetchTotalCount_ {0}

private

Definition at line 243 of file Database.h.

fetchDurationUs_

std::atomic<std::uint64_t> xrpl::NodeStore::Database::fetchDurationUs_ {0}

private

Definition at line 244 of file Database.h.

storeDurationUs_

std::atomic<std::uint64_t> xrpl::NodeStore::Database::storeDurationUs_ {0}

private

Definition at line 245 of file Database.h.

readLock_

std::mutex xrpl::NodeStore::Database::readLock_

mutableprivate

Definition at line 247 of file Database.h.

readCondVar_

std::condition_variable xrpl::NodeStore::Database::readCondVar_

private

Definition at line 248 of file Database.h.

read_

std::map< uint256, std::vector< std::pair<std::uint32_t, std::function<void(std::shared_ptr<NodeObject> const&)> > > > xrpl::NodeStore::Database::read_

private

Definition at line 255 of file Database.h.

readStopping_

std::atomic<bool> xrpl::NodeStore::Database::readStopping_ = false

private

Definition at line 257 of file Database.h.

readThreads_

std::atomic<int> xrpl::NodeStore::Database::readThreads_ = 0

private

Definition at line 258 of file Database.h.

runningThreads_

std::atomic<int> xrpl::NodeStore::Database::runningThreads_ = 0

private

Definition at line 259 of file Database.h.