Pipeline vector for running a group of aggregation operations on a family of vectors. More...

#include <bmaggregator.h>

Collaboration diagram for bm::aggregator< BV >::pipeline< Opt >:

[legend]

Public Types
typedef Opt	options_type

Public Member Functions
	pipeline ()
run_options &	options () BMNOEXCEPT
	Set pipeline run options.
const run_options &	get_options () const BMNOEXCEPT
	Get pipeline run options.
Aggregator traits and control constants
	~pipeline () BMNOEXCEPT
pipeline argument groups fill-in methods
arg_groups *	add ()
	Add new arguments group.
void	set_or_target (bvector_type *bv_or) BMNOEXCEPT
	Attach OR (aggregator vector).
void	set_search_count_limit (size_type limit) BMNOEXCEPT
	Set search limit for results.
void	complete ()
	Prepare pipeline for the execution (resize and init internal structures) Once complete, you cannot add() to it.
bool	is_complete () const BMNOEXCEPT
	return true if pipeline is ready for execution (complete)
size_type	size () const BMNOEXCEPT
	Return size() of pileine.
const arg_vector_type &	get_args_vector () const BMNOEXCEPT
	Return argument vector used for pipeline execution.
bvect_vector_type &	get_bv_res_vector () BMNOEXCEPT
	Return results vector used for pipeline execution.
bv_count_vector_type &	get_bv_count_vector () BMNOEXCEPT
	Return results vector count used for pipeline execution.

access to internals
class	aggregator
run_options	options_
	execution parameters
bool	is_complete_ = false
	ready to run state flag
size_t	total_vect_ = 0
	total number of vector mentions in all groups
arg_vector_type	arg_vect_
	input arg. groups
bvect_vector_type	bv_res_vect_
	results (bit-vector ptrs)
bv_count_vector_type	count_res_vect_
	results (counts)
size_type	search_count_limit_ {bm::id_max}
	search limit by count
pipeline_bcache	bcache_
	blocks cache structure
unsigned	top_blocks_ = 1
	top-level structure size, max of all bvectors
bvector_type *	bv_or_target_ = 0
	OR target bit-bector ptr.
const bv_vector_type &	get_all_input_vect () const BMNOEXCEPT
const count_vector_type &	get_all_input_cnt_vect () const BMNOEXCEPT
size_t	unique_vectors () const BMNOEXCEPT
	Return number of unique vectors in the pipeline (after complete()).
size_t	compute_run_batch () const BMNOEXCEPT
	Function looks at the pipeline to apply euristics to suggest optimal run_batch parameter.
pipeline_bcache &	get_bcache () BMNOEXCEPT
unsigned	get_top_blocks () const BMNOEXCEPT
	Return number of top blocks after complete.
	pipeline (const pipeline &)=delete
pipeline &	operator= (const pipeline &)=delete

Detailed Description

template<typename BV>
template<class Opt = bm::agg_run_options<>>
class bm::aggregator< BV >::pipeline< Opt >

Pipeline vector for running a group of aggregation operations on a family of vectors.

Pipeline is used to run multiple aggregation combinations (searches) for essencially same set of vectors (different combinations of ANDs and SUBs for example). Pipeline execution improves CPU cache reuse and caches the compressed blocks to re-use it for more efficient execution. Essencially it is a tool to run thousads of runs at once faster.

Definition at line 223 of file bmaggregator.h.

Member Typedef Documentation

◆ options_type

template<typename BV>

template<class Opt = bm::agg_run_options<>>

typedef Opt bm::aggregator< BV >::pipeline< Opt >::options_type

Definition at line 226 of file bmaggregator.h.

Constructor & Destructor Documentation

◆ pipeline() [1/2]

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bm::aggregator< BV >::pipeline< Opt >::pipeline ( )

inline

Definition at line 228 of file bmaggregator.h.

Referenced by operator=(), and pipeline().

◆ ~pipeline()

template<typename BV>

template<class Opt>

bm::aggregator< BV >::pipeline< Opt >::~pipeline ( )

Definition at line 2761 of file bmaggregator.h.

References bm::aligned_free(), arg_vect_, bcache_, BMNOEXCEPT, bv_res_vect_, bm::aggregator< BV >::free_arg_group(), and ~pipeline().

Referenced by ~pipeline().

◆ pipeline() [2/2]

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bm::aggregator< BV >::pipeline< Opt >::pipeline ( const pipeline< Opt > & )

protecteddelete

References pipeline().

Member Function Documentation

◆ add()

template<typename BV>

template<class Opt>

aggregator< BV >::arg_groups * bm::aggregator< BV >::pipeline< Opt >::add ( )

Add new arguments group.

Definition at line 2888 of file bmaggregator.h.

References arg_vect_, BM_ASSERT, bm::aggregator< BV >::construct_arg_group(), and is_complete_.

◆ complete()

template<typename BV>

template<class Opt>

void bm::aggregator< BV >::pipeline< Opt >::complete ( )

Prepare pipeline for the execution (resize and init internal structures) Once complete, you cannot add() to it.

Definition at line 2783 of file bmaggregator.h.

References bm::aggregator< BV >::arg_groups::arg_bv0, bm::aggregator< BV >::arg_groups::arg_bv1, arg_vect_, bcache_, BM_ASSERT, bv_res_vect_, complete(), count_res_vect_, get_args_vector(), is_complete_, bm::aggregator< BV >::max_top_blocks(), and top_blocks_.

Referenced by complete().

◆ compute_run_batch()

template<typename BV>

template<class Opt>

size_t bm::aggregator< BV >::pipeline< Opt >::compute_run_batch ( ) const

Function looks at the pipeline to apply euristics to suggest optimal run_batch parameter.

Definition at line 2899 of file bmaggregator.h.

References arg_vect_, BMNOEXCEPT, compute_run_batch(), bm::set_block_size, total_vect_, and unique_vectors().

Referenced by compute_run_batch().

◆ get_all_input_cnt_vect()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

const count_vector_type & bm::aggregator< BV >::pipeline< Opt >::get_all_input_cnt_vect ( ) const

inline

Definition at line 297 of file bmaggregator.h.

References bcache_, and BMNOEXCEPT.

◆ get_all_input_vect()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

const bv_vector_type & bm::aggregator< BV >::pipeline< Opt >::get_all_input_vect ( ) const

inline

Definition at line 295 of file bmaggregator.h.

References bcache_, and BMNOEXCEPT.

◆ get_args_vector()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

const arg_vector_type & bm::aggregator< BV >::pipeline< Opt >::get_args_vector ( ) const

inline

Return argument vector used for pipeline execution.

Definition at line 279 of file bmaggregator.h.

References arg_vect_, and BMNOEXCEPT.

Referenced by complete().

◆ get_bcache()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

pipeline_bcache & bm::aggregator< BV >::pipeline< Opt >::get_bcache ( )

inlineprotected

Definition at line 310 of file bmaggregator.h.

References bcache_, BMNOEXCEPT, and get_bcache().

Referenced by get_bcache().

◆ get_bv_count_vector()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bv_count_vector_type & bm::aggregator< BV >::pipeline< Opt >::get_bv_count_vector ( )

inline

Return results vector count used for pipeline execution.

Definition at line 287 of file bmaggregator.h.

References BMNOEXCEPT, and count_res_vect_.

◆ get_bv_res_vector()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bvect_vector_type & bm::aggregator< BV >::pipeline< Opt >::get_bv_res_vector ( )

inline

Return results vector used for pipeline execution.

Definition at line 283 of file bmaggregator.h.

References BMNOEXCEPT, and bv_res_vect_.

◆ get_options()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

const run_options & bm::aggregator< BV >::pipeline< Opt >::get_options ( ) const

inline

Get pipeline run options.

Definition at line 235 of file bmaggregator.h.

References BMNOEXCEPT, and options_.

◆ get_top_blocks()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

unsigned bm::aggregator< BV >::pipeline< Opt >::get_top_blocks ( ) const

inlineprotected

Return number of top blocks after complete.

Definition at line 315 of file bmaggregator.h.

References BMNOEXCEPT, and top_blocks_.

◆ is_complete()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bool bm::aggregator< BV >::pipeline< Opt >::is_complete ( ) const

inline

return true if pipeline is ready for execution (complete)

Definition at line 269 of file bmaggregator.h.

References BMNOEXCEPT, and is_complete_.

◆ operator=()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

pipeline & bm::aggregator< BV >::pipeline< Opt >::operator= ( const pipeline< Opt > & )

protecteddelete

References pipeline().

◆ options()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

run_options & bm::aggregator< BV >::pipeline< Opt >::options ( )

inline

Set pipeline run options.

Definition at line 232 of file bmaggregator.h.

References BMNOEXCEPT, options(), and options_.

Referenced by options().

◆ set_or_target()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

void bm::aggregator< BV >::pipeline< Opt >::set_or_target ( bvector_type * bv_or )

inline

Attach OR (aggregator vector).

Pipeline results all will be OR-ed (UNION) into the OR target vector

Parameters

bv_or - OR target vector

Definition at line 251 of file bmaggregator.h.

References BMNOEXCEPT, and bv_or_target_.

◆ set_search_count_limit()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

void bm::aggregator< BV >::pipeline< Opt >::set_search_count_limit ( size_type limit )

inline

Set search limit for results.

Requires that bit-counting to be enabled in the template parameters. Warning: search limit is approximate (for performance reasons) so it can occasinally find more than requested. It cannot find less.

Parameters

limit - search limit (target population count to search for)

Definition at line 260 of file bmaggregator.h.

References BMNOEXCEPT, and search_count_limit_.

◆ size()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

size_type bm::aggregator< BV >::pipeline< Opt >::size ( ) const

inline

Return size() of pileine.

Definition at line 272 of file bmaggregator.h.

References arg_vect_, and BMNOEXCEPT.

◆ unique_vectors()

template<typename BV>

template<class Opt = bm::agg_run_options<>>

size_t bm::aggregator< BV >::pipeline< Opt >::unique_vectors ( ) const

inline

Return number of unique vectors in the pipeline (after complete()).

Definition at line 301 of file bmaggregator.h.

References bcache_, and BMNOEXCEPT.

Referenced by compute_run_batch().

◆ aggregator

template<typename BV>

template<class Opt = bm::agg_run_options<>>

friend class aggregator

friend

Definition at line 323 of file bmaggregator.h.

References aggregator.

Referenced by aggregator.

Field Documentation

◆ arg_vect_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

arg_vector_type bm::aggregator< BV >::pipeline< Opt >::arg_vect_

protected

input arg. groups

Definition at line 332 of file bmaggregator.h.

Referenced by add(), complete(), compute_run_batch(), get_args_vector(), size(), and ~pipeline().

◆ bcache_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

pipeline_bcache bm::aggregator< BV >::pipeline< Opt >::bcache_

protected

blocks cache structure

Definition at line 338 of file bmaggregator.h.

Referenced by complete(), get_all_input_cnt_vect(), get_all_input_vect(), get_bcache(), unique_vectors(), and ~pipeline().

◆ bv_or_target_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bvector_type* bm::aggregator< BV >::pipeline< Opt >::bv_or_target_ = 0

protected

OR target bit-bector ptr.

Definition at line 340 of file bmaggregator.h.

Referenced by set_or_target().

◆ bv_res_vect_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bvect_vector_type bm::aggregator< BV >::pipeline< Opt >::bv_res_vect_

protected

results (bit-vector ptrs)

Definition at line 334 of file bmaggregator.h.

Referenced by complete(), get_bv_res_vector(), and ~pipeline().

◆ count_res_vect_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bv_count_vector_type bm::aggregator< BV >::pipeline< Opt >::count_res_vect_

protected

results (counts)

Definition at line 335 of file bmaggregator.h.

Referenced by complete(), and get_bv_count_vector().

◆ is_complete_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

bool bm::aggregator< BV >::pipeline< Opt >::is_complete_ = false

protected

ready to run state flag

Definition at line 330 of file bmaggregator.h.

Referenced by add(), complete(), and is_complete().

◆ options_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

run_options bm::aggregator< BV >::pipeline< Opt >::options_

protected

execution parameters

Definition at line 329 of file bmaggregator.h.

Referenced by get_options(), and options().

◆ search_count_limit_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

size_type bm::aggregator< BV >::pipeline< Opt >::search_count_limit_ {bm::id_max}

protected

search limit by count

Definition at line 336 of file bmaggregator.h.

Referenced by set_search_count_limit().

◆ top_blocks_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

unsigned bm::aggregator< BV >::pipeline< Opt >::top_blocks_ = 1

protected

top-level structure size, max of all bvectors

Definition at line 339 of file bmaggregator.h.

Referenced by complete(), and get_top_blocks().

◆ total_vect_

template<typename BV>

template<class Opt = bm::agg_run_options<>>

size_t bm::aggregator< BV >::pipeline< Opt >::total_vect_ = 0

protected

total number of vector mentions in all groups

Definition at line 331 of file bmaggregator.h.

Referenced by compute_run_batch().

The documentation for this class was generated from the following file:

bmaggregator.h

Public Types

Public Member Functions

access to internals

Detailed Description

Member Typedef Documentation

◆ options_type

Constructor & Destructor Documentation

◆ pipeline() [1/2]

◆ ~pipeline()

◆ pipeline() [2/2]

Member Function Documentation

◆ add()

◆ complete()

◆ compute_run_batch()

◆ get_all_input_cnt_vect()

◆ get_all_input_vect()

◆ get_args_vector()

◆ get_bcache()

◆ get_bv_count_vector()

◆ get_bv_res_vector()

◆ get_options()

◆ get_top_blocks()

◆ is_complete()

◆ operator=()

◆ options()

◆ set_or_target()

◆ set_search_count_limit()

◆ size()

◆ unique_vectors()

Friends And Related Symbol Documentation

◆ aggregator

Field Documentation

◆ arg_vect_

◆ bcache_

◆ bv_or_target_

◆ bv_res_vect_

◆ count_res_vect_

◆ is_complete_

◆ options_

◆ search_count_limit_

◆ top_blocks_

◆ total_vect_