Awkward Array: BitMaskedArray Class Reference

Loading...
Searching...
No Matches
Public Member Functions | List of all members
BitMaskedArray Class Reference

Represents potentially missing data by overlaying a bit mask over its content. More...

`#include <BitMaskedArray.h>`

Inheritance diagram for BitMaskedArray:

## Public Member Functions
| BitMaskedArray (const IdentitiesPtr &identities, const util::Parameters &parameters, const IndexU8 &mask, const ContentPtr &content, bool valid_when, int64_t length, bool lsb_order)
| Creates an BitMaskedArray from a full set of parameters. More...
const IndexU8 | mask () const
| Mask in which each bit represents a missing value (`None`) or a valid value (from content). More...
const ContentPtr | content () const
| Data to be masked; `mask[i]` corresponds to `content[i*8]` through `content[i*8 + 7]` (inclusive) for all `i`. More...
bool | valid_when () const
| Interpretation of the boolean bytes in mask as `None` or valid values from content. Only boolean bytes that are equal to valid_when are not `None`. (Any non-zero value of a boolean byte and `valid_when` are equivalent.) More...
bool | lsb_order () const
| If `true`, the bits in each byte of the mask are taken to be in Least Significant Bit (LSB) order; if `false`, they are taken to be in Most Significant Bit (MSB) order. More...
const ContentPtr | project () const
| Return an array with the same type as content with `None` values removed. More...
const ContentPtr | project (const Index8 &mask) const
| Performs a set-union of a given `mask` with the missing values and calls project. More...
const Index8 | bytemask () const
| Expands the mask to a byte-valued mask with a fixed interpretation: missing values are `1` and valid values are `0` (as though valid_when were `false`). More...
const ContentPtr | simplify_optiontype () const
| If the content also has OptionType, combine the mask with the content's indicator of missing values; also combine if the content is a non-OptionType `IndexedArray`. More...
const std::shared_ptr< ByteMaskedArray > | toByteMaskedArray () const
| Converts this array into a ByteMaskedArray. More...
const std::shared_ptr< IndexedArrayOf< int64_t, true > > | toIndexedOptionArray64 () const
| Converts this array into an `IndexedOptionArray` with the same missing values. More...
const std::string | classname () const override
| User-friendly name of this class: `"BitMaskedArray"`. More...
void | setidentities () override
| Assign a surrogate index of Identities to this array (in-place). More...
void | setidentities (const IdentitiesPtr &identities) override
| Assign a specified set of Identities to this array (in-place). More...
const TypePtr | type (const util::TypeStrs &typestrs) const override
| High-level Type describing this array. More...
const FormPtr | form (bool materialize) const override
| Low-level Form describing all the features of this array except the actual data buffers (`Index`, `Identities`, `NumpyArray::ptr`, and `RawArray::ptr`). More...
kernel::lib | kernels () const override
| Returns the kernel library enum for all nested `ptr_lib` within the array's tree structure. If different libraries are associated with nodes of the same tree, this returns `kernel::lib::size` to indicate that it is "mixed". More...
void | caches (std::vector< ArrayCachePtr > &out) const override
| Accumulates all the unique #ArrayCache objects from nested #VirtualArray nodes. (Uniqueness is determined by pointer value.) More...
const std::string | tostring_part (const std::string &indent, const std::string &pre, const std::string &post) const override
| Internal function to build an output string for tostring. More...
void | tojson_part (ToJson &builder, bool include_beginendlist) const override
| Internal function to produce a JSON representation one node at a time. More...
void | nbytes_part (std::map< size_t, int64_t > &largest) const override
| Internal function used to calculate nbytes. More...
int64_t | length () const override
| The number of elements in the array. More...
const ContentPtr | shallow_copy () const override
| Copies this node without copying any nodes hierarchically nested within it or any array/index/identity buffers. More...
const ContentPtr | deep_copy (bool copyarrays, bool copyindexes, bool copyidentities) const override
| Copies this node and all nodes hierarchically nested within it, optionally copying the associated arrays, indexes, and identities, too. More...
void | check_for_iteration () const override
| Performs up-front validity checks on an array so that they don't have to be checked in getitem_at_nowrap for each item. More...
const ContentPtr | getitem_nothing () const override
| Internal function to get an empty slice (with the correct type). More...
const ContentPtr | getitem_at (int64_t at) const override
| Returns the element at a given position in the array, handling negative indexing and bounds-checking like Python. More...
const ContentPtr | getitem_at_nowrap (int64_t at) const override
| Returns the element at a given position in the array, without handling negative indexing or bounds-checking. More...
const ContentPtr | getitem_range (int64_t start, int64_t stop) const override
| Subinterval of this array, handling negative indexing and bounds-checking like Python. More...
const ContentPtr | getitem_range_nowrap (int64_t start, int64_t stop) const override
| Subinterval of this array, without handling negative indexing or bounds-checking. More...
const ContentPtr | getitem_field (const std::string &key) const override
| This array with the first nested RecordArray replaced by the field at `key`. More...
const ContentPtr | getitem_field (const std::string &key, const Slice &only_fields) const override
const ContentPtr | getitem_fields (const std::vector< std::string > &keys) const override
| This array with the first nested RecordArray replaced by a RecordArray of a given subset of `keys`. More...
const ContentPtr | getitem_fields (const std::vector< std::string > &keys, const Slice &only_fields) const override
const ContentPtr | getitem_next (const SliceItemPtr &head, const Slice &tail, const Index64 &advanced) const override
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
const ContentPtr | carry (const Index64 &carry, bool allow_lazy) const override
| Returns an array of the same type with elements filtered, rearranged, and possibly duplicated by the `carry` array of integers. More...
int64_t | purelist_depth () const override
| The list-depth of this array, not counting any contained within a RecordArray. More...
const std::pair< int64_t, int64_t > | minmax_depth () const override
| Returns (a) the minimum list-depth and (b) the maximum list-depth of the array, which can differ if this array "branches" (differs when followed through different fields of a RecordArray or `UnionArray`). More...
const std::pair< bool, int64_t > | branch_depth () const override
| Returns (a) whether the list-depth of this array "branches," or differs when followed through different fields of a RecordArray or `UnionArray` and (b) the minimum list-depth. More...
int64_t | numfields () const override
| The number of fields in the first nested tuple or records or `-1` if this array does not contain a RecordArray. More...
int64_t | fieldindex (const std::string &key) const override
| The position of a tuple or record key name if this array contains a RecordArray. More...
const std::string | key (int64_t fieldindex) const override
| The record name associated with a given field index or the tuple index as a string (e.g. `"0"`, `"1"`, `"2"`) if a tuple. More...
bool | haskey (const std::string &key) const override
| Returns `true` if the array contains a RecordArray with the specified `key`; `false` otherwise. More...
const std::vector< std::string > | keys () const override
| A list of RecordArray keys or an empty list if this array does not contain a RecordArray. More...
bool | istuple () const override
| Returns `true` if the outermost RecordArray is a tuple. More...
const std::string | validityerror (const std::string &path) const override
| Returns an error message if this array is invalid; otherwise, returns an empty string. More...
const ContentPtr | shallow_simplify () const override
| Returns an equivalent array simplified at one level only using `simplify_optiontype` if an option-type array and `simplify_uniontype` if a union-type array. More...
const ContentPtr | num (int64_t axis, int64_t depth) const override
| The length of this array (as a NumpyArray scalar) if `axis = 0` or the lengths of subarrays (as an array or nested array) if `axis != 0`. More...
const std::pair< Index64, ContentPtr > | offsets_and_flattened (int64_t axis, int64_t depth) const override
| Returns (a) an offsets `Index` and (b) a flattened version of the array at some `axis` depth. More...
bool | mergeable (const ContentPtr &other, bool mergebool) const override
| Returns `true` if this array can be merged with the `other`; `false` otherwise. More...
bool | referentially_equal (const ContentPtr &other) const override
| Returns `true` if this array has all the same buffers and parameters as `other`; `false` otherwise. More...
const ContentPtr | reverse_merge (const ContentPtr &other) const override
| Merges a single `other` with this array in reverse order: `other` first, this last. More...
const ContentPtr | mergemany (const ContentPtrVec &others) const override
| Returns an array with this and the `others` concatenated (in order, this first, `others` last). More...
const SliceItemPtr | asslice () const override
| Converts this array into a SliceItem that can be used in getitem. More...
const ContentPtr | fillna (const ContentPtr &value) const override
| Returns this array with `None` values replaced by a given `value`. More...
const ContentPtr | rpad (int64_t target, int64_t axis, int64_t depth) const override
| If `axis = 0`, returns a view of this array padded on the right with `None` values to have a minimum length; otherwise, returns an array with nested lists all padded to the minimum length. More...
const ContentPtr | rpad_and_clip (int64_t target, int64_t axis, int64_t depth) const override
| If `axis = 0`, returns a view of this array padded on the right. More...
const ContentPtr | reduce_next (const Reducer &reducer, int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool mask, bool keepdims) const override
| This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all). More...
const ContentPtr | sort_next (int64_t negaxis, const Index64 &starts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const override
| This array sorted. More...
const ContentPtr | argsort_next (int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const override
| This array sorted indices. More...
const ContentPtr | localindex (int64_t axis, int64_t depth) const override
| A (possibly nested) array of integers indicating the positions of elements within each nested list. More...
const ContentPtr | combinations (int64_t n, bool replacement, const util::RecordLookupPtr &recordlookup, const util::Parameters &parameters, int64_t axis, int64_t depth) const override
| Tuples or records of all `n`-tuple combinations of list items at some `axis` depth. More...
const ContentPtr | getitem_next (const SliceAt &at, const Slice &tail, const Index64 &advanced) const override
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
const ContentPtr | getitem_next (const SliceRange &range, const Slice &tail, const Index64 &advanced) const override
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
const ContentPtr | getitem_next (const SliceArray64 &array, const Slice &tail, const Index64 &advanced) const override
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
const ContentPtr | getitem_next (const SliceJagged64 &jagged, const Slice &tail, const Index64 &advanced) const override
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceArray64 &slicecontent, const Slice &tail) const override
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceMissing64 &slicecontent, const Slice &tail) const override
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceJagged64 &slicecontent, const Slice &tail) const override
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
const ContentPtr | copy_to (kernel::lib ptr_lib) const override
| Recursively copies components of the array from main memory to a GPU (if `ptr_lib == kernel::lib::cuda`) or to main memory (if `ptr_lib == kernel::lib::cpu`) if those components are not already there. More...
const ContentPtr | numbers_to_type (const std::string &name) const override
| Change the leaf types to 'totype'. More...
bool | is_unique () const override
| Returns 'true' if all components of the array are unique. More...
const ContentPtr | unique () const override
| Returns an array where all components are unique. More...
bool | is_subrange_equal (const Index64 &start, const Index64 &stop) const override
| Returns 'true' if subranges are equal. More...
Public Member Functions inherited from Content
| Content (const IdentitiesPtr &identities, const util::Parameters &parameters)
| Called by all subclass constructors; assigns identities and parameters upon construction. More...
virtual | ~Content ()
| Empty destructor; required for some C++ reason. More...
virtual bool | isscalar () const
| Returns `true` if the data represented by this node is scalar, not a true array. More...
virtual const std::string | classname () const =0
| User-friendly name of this class, including integer-type specialization. More...
virtual const IdentitiesPtr | identities () const
| Optional Identities for each element of the array (may be `nullptr`). More...
virtual void | setidentities ()=0
| Assign a surrogate index of Identities to this array (in-place). More...
virtual void | setidentities (const IdentitiesPtr &identities)=0
| Assign a specified set of Identities to this array (in-place). More...
virtual const TypePtr | type (const util::TypeStrs &typestrs) const =0
| High-level Type describing this array. More...
virtual const FormPtr | form (bool materialize) const =0
| Low-level Form describing all the features of this array except the actual data buffers (`Index`, `Identities`, `NumpyArray::ptr`, and `RawArray::ptr`). More...
virtual kernel::lib | kernels () const =0
| Returns the kernel library enum for all nested `ptr_lib` within the array's tree structure. If different libraries are associated with nodes of the same tree, this returns `kernel::lib::size` to indicate that it is "mixed". More...
virtual void | caches (std::vector< ArrayCachePtr > &out) const =0
| Accumulates all the unique #ArrayCache objects from nested #VirtualArray nodes. (Uniqueness is determined by pointer value.) More...
virtual const std::string | tostring_part (const std::string &indent, const std::string &pre, const std::string &post) const =0
| Internal function to build an output string for tostring. More...
virtual void | tojson_part (ToJson &builder, bool include_beginendlist) const =0
| Internal function to produce a JSON representation one node at a time. More...
virtual void | nbytes_part (std::map< size_t, int64_t > &largest) const =0
| Internal function used to calculate nbytes. More...
virtual int64_t | length () const =0
| The number of elements in the array. More...
virtual const ContentPtr | shallow_copy () const =0
| Copies this node without copying any nodes hierarchically nested within it or any array/index/identity buffers. More...
virtual const ContentPtr | deep_copy (bool copyarrays, bool copyindexes, bool copyidentities) const =0
| Copies this node and all nodes hierarchically nested within it, optionally copying the associated arrays, indexes, and identities, too. More...
virtual void | check_for_iteration () const =0
| Performs up-front validity checks on an array so that they don't have to be checked in getitem_at_nowrap for each item. More...
virtual const ContentPtr | getitem_nothing () const =0
| Internal function to get an empty slice (with the correct type). More...
virtual const ContentPtr | getitem_at (int64_t at) const =0
| Returns the element at a given position in the array, handling negative indexing and bounds-checking like Python. More...
virtual const ContentPtr | getitem_at_nowrap (int64_t at) const =0
| Returns the element at a given position in the array, without handling negative indexing or bounds-checking. More...
virtual const ContentPtr | getitem_range (int64_t start, int64_t stop) const =0
| Subinterval of this array, handling negative indexing and bounds-checking like Python. More...
virtual const ContentPtr | getitem_range_nowrap (int64_t start, int64_t stop) const =0
| Subinterval of this array, without handling negative indexing or bounds-checking. More...
virtual const ContentPtr | getitem_field (const std::string &key) const =0
| This array with the first nested RecordArray replaced by the field at `key`. More...
virtual const ContentPtr | getitem_field (const std::string &key, const Slice &only_fields) const =0
virtual const ContentPtr | getitem_fields (const std::vector< std::string > &keys) const =0
| This array with the first nested RecordArray replaced by a RecordArray of a given subset of `keys`. More...
virtual const ContentPtr | getitem_fields (const std::vector< std::string > &keys, const Slice &only_fields) const =0
virtual const ContentPtr | getitem (const Slice &where) const
| Entry point for general slicing: Slice represents a tuple of SliceItem nodes applying to each level of nested lists. More...
virtual const ContentPtr | getitem_next (const SliceItemPtr &head, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceItemPtr &slicecontent, const Slice &tail) const
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
virtual const ContentPtr | carry (const Index64 &carry, bool allow_lazy) const =0
| Returns an array of the same type with elements filtered, rearranged, and possibly duplicated by the `carry` array of integers. More...
virtual const std::string | purelist_parameter (const std::string &key) const
| The parameter associated with `key` at the first level that has a non-null value, descending only as deep as the first RecordArray. More...
bool | purelist_isregular () const
| Returns `true` if all nested lists down to the first RecordArray are RegularArray nodes; `false` otherwise. More...
virtual bool | dimension_optiontype () const
| Returns `true` if this dimension has option-type; `false` otherwise. More...
virtual int64_t | purelist_depth () const =0
| The list-depth of this array, not counting any contained within a RecordArray. More...
virtual const std::pair< int64_t, int64_t > | minmax_depth () const =0
| Returns (a) the minimum list-depth and (b) the maximum list-depth of the array, which can differ if this array "branches" (differs when followed through different fields of a RecordArray or `UnionArray`). More...
virtual const std::pair< bool, int64_t > | branch_depth () const =0
| Returns (a) whether the list-depth of this array "branches," or differs when followed through different fields of a RecordArray or `UnionArray` and (b) the minimum list-depth. More...
virtual int64_t | numfields () const =0
| The number of fields in the first nested tuple or records or `-1` if this array does not contain a RecordArray. More...
virtual int64_t | fieldindex (const std::string &key) const =0
| The position of a tuple or record key name if this array contains a RecordArray. More...
virtual const std::string | key (int64_t fieldindex) const =0
| The record name associated with a given field index or the tuple index as a string (e.g. `"0"`, `"1"`, `"2"`) if a tuple. More...
virtual bool | haskey (const std::string &key) const =0
| Returns `true` if the array contains a RecordArray with the specified `key`; `false` otherwise. More...
virtual const std::vector< std::string > | keys () const =0
| A list of RecordArray keys or an empty list if this array does not contain a RecordArray. More...
virtual bool | istuple () const =0
| Returns `true` if the outermost RecordArray is a tuple. More...
virtual const std::string | validityerror (const std::string &path) const =0
| Returns an error message if this array is invalid; otherwise, returns an empty string. More...
virtual const ContentPtr | shallow_simplify () const =0
| Returns an equivalent array simplified at one level only using `simplify_optiontype` if an option-type array and `simplify_uniontype` if a union-type array. More...
virtual const ContentPtr | num (int64_t axis, int64_t depth) const =0
| The length of this array (as a NumpyArray scalar) if `axis = 0` or the lengths of subarrays (as an array or nested array) if `axis != 0`. More...
virtual const std::pair< Index64, ContentPtr > | offsets_and_flattened (int64_t axis, int64_t depth) const =0
| Returns (a) an offsets `Index` and (b) a flattened version of the array at some `axis` depth. More...
virtual bool | mergeable (const ContentPtr &other, bool mergebool) const =0
| Returns `true` if this array can be merged with the `other`; `false` otherwise. More...
virtual bool | referentially_equal (const ContentPtr &other) const =0
| Returns `true` if this array has all the same buffers and parameters as `other`; `false` otherwise. More...
virtual const std::pair< ContentPtrVec, ContentPtrVec > | merging_strategy (const ContentPtrVec &others) const
| Partitions `this` array plus a list of `others` into a `head` sequence and a `tail` sequence: More...
virtual const ContentPtr | reverse_merge (const ContentPtr &other) const
| Merges a single `other` with this array in reverse order: `other` first, this last. More...
const ContentPtr | merge (const ContentPtr &other) const
| An array with this and the `other` concatenated (this first, `other` last). More...
virtual const ContentPtr | mergemany (const ContentPtrVec &others) const =0
| Returns an array with this and the `others` concatenated (in order, this first, `others` last). More...
virtual const SliceItemPtr | asslice () const =0
| Converts this array into a SliceItem that can be used in getitem. More...
virtual const ContentPtr | fillna (const ContentPtr &value) const =0
| Returns this array with `None` values replaced by a given `value`. More...
virtual const ContentPtr | rpad (int64_t target, int64_t axis, int64_t depth) const =0
| If `axis = 0`, returns a view of this array padded on the right with `None` values to have a minimum length; otherwise, returns an array with nested lists all padded to the minimum length. More...
virtual const ContentPtr | rpad_and_clip (int64_t target, int64_t axis, int64_t depth) const =0
| If `axis = 0`, returns a view of this array padded on the right. More...
virtual const ContentPtr | reduce_next (const Reducer &reducer, int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool mask, bool keepdims) const =0
| This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all). More...
virtual const ContentPtr | sort_next (int64_t negaxis, const Index64 &starts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const =0
| This array sorted. More...
virtual const ContentPtr | argsort_next (int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const =0
| This array sorted indices. More...
virtual const ContentPtr | localindex (int64_t axis, int64_t depth) const =0
| A (possibly nested) array of integers indicating the positions of elements within each nested list. More...
virtual const ContentPtr | combinations (int64_t n, bool replacement, const util::RecordLookupPtr &recordlookup, const util::Parameters &parameters, int64_t axis, int64_t depth) const =0
| Tuples or records of all `n`-tuple combinations of list items at some `axis` depth. More...
const std::string | tostring () const
| Returns a string representation of this array (multi-line XML). More...
const std::string | tojson (bool pretty, int64_t maxdecimals, const char *nan_string=nullptr, const char *infinity_string=nullptr, const char *minus_infinity_string=nullptr, const char *complex_real_string=nullptr, const char *complex_imag_string=nullptr) const
| Returns a JSON representation of this array. More...
void | tojson (FILE *destination, bool pretty, int64_t maxdecimals, int64_t buffersize, const char *nan_string=nullptr, const char *infinity_string=nullptr, const char *minus_infinity_string=nullptr, const char *complex_real_string=nullptr, const char *complex_imag_string=nullptr) const
| Writes a JSON representation of this array to a `destination` file. More...
int64_t | nbytes () const
| The number of bytes contained in all array buffers, `Index` buffers, and Identities buffers, not including the lightweight node objects themselves. More...
const ContentPtr | reduce (const Reducer &reducer, int64_t axis, bool mask, bool keepdims) const
| This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all). More...
const ContentPtr | sort (int64_t axis, bool ascending, bool stable) const
| This array with one axis sorted by applying a sorting algorithm. More...
const ContentPtr | argsort (int64_t axis, bool ascending, bool stable) const
| This array indices with one axis sorted by applying a sorting algorithm. More...
const util::Parameters | parameters () const
| String-to-JSON map that augments the meaning of this array. More...
void | setparameters (const util::Parameters &parameters)
| Assign all the parameters for this array node (in-place). More...
const std::string | parameter (const std::string &key) const
| Get one parameter from this array node. More...
void | setparameter (const std::string &key, const std::string &value)
| Assign one parameter for this array node (in-place). More...
bool | parameter_equals (const std::string &key, const std::string &value) const
| Returns `true` if the parameter associated with `key` exists and is equal to `value`; `false` otherwise. More...
bool | parameters_equal (const util::Parameters &other, bool check_all) const
| Returns `true` if all parameters of this array node are equal to the `other` parameters. More...
bool | parameter_isstring (const std::string &key) const
| Returns `true` if the parameter associated with `key` is a string; `false` otherwise. More...
bool | parameter_isname (const std::string &key) const
| Returns `true` if the parameter associated with `key` is a string that matches `[A-Za-z_][A-Za-z_0-9]*`; `false` otherwise. More...
const std::string | parameter_asstring (const std::string &key) const
| Returns the parameter associated with `key` as a string if parameter_isstring; raises an error otherwise. More...
const ContentPtr | merge_as_union (const ContentPtr &other) const
| Concatenates this array with `other` by creating a `UnionArray` instead of actually merging the data. This does not call `UnionArray::simplify_uniontype`. More...
const ContentPtr | rpad_axis0 (int64_t target, bool clip) const
| Internal function to handle the `axis = 0` case of rpad and rpad_and_clip. More...
const ContentPtr | localindex_axis0 () const
| Internal function to handle the `axis = 0` case of localindex. More...
const ContentPtr | combinations_axis0 (int64_t n, bool replacement, const util::RecordLookupPtr &recordlookup, const util::Parameters &parameters) const
| Internal function to handle the `axis = 0` case of combinations. More...
virtual const ContentPtr | getitem_next (const SliceAt &at, const Slice &tail, const Index64 &advanced) const =0
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceRange &range, const Slice &tail, const Index64 &advanced) const =0
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceEllipsis &ellipsis, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceNewAxis &newaxis, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceArray64 &array, const Slice &tail, const Index64 &advanced) const =0
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceField &field, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceFields &fields, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceMissing64 &missing, const Slice &tail, const Index64 &advanced) const
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next (const SliceJagged64 &jagged, const Slice &tail, const Index64 &advanced) const =0
| Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...
virtual const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceArray64 &slicecontent, const Slice &tail) const =0
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
virtual const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceMissing64 &slicecontent, const Slice &tail) const =0
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
virtual const ContentPtr | getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceJagged64 &slicecontent, const Slice &tail) const =0
| Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...
const int64_t | axis_wrap_if_negative (int64_t axis) const
| Internal function defining the negative axis handling for many operations. More...
virtual const ContentPtr | copy_to (kernel::lib ptr_lib) const =0
| Recursively copies components of the array from main memory to a GPU (if `ptr_lib == kernel::lib::cuda`) or to main memory (if `ptr_lib == kernel::lib::cpu`) if those components are not already there. More...
virtual const ContentPtr | numbers_to_type (const std::string &name) const =0
| Change the leaf types to 'totype'. More...
virtual bool | is_unique () const =0
| Returns 'true' if all components of the array are unique. More...
virtual const ContentPtr | unique () const =0
| Returns sorted unique values. More...
virtual bool | is_subrange_equal (const Index64 &start, const Index64 &stop) const =0
| Returns 'true' if subranges are equal. More...

## Additional Inherited Members
Protected Member Functions inherited from Content
const std::string | validityerror_parameters (const std::string &path) const
| Internal function to determine if a particular set of parameters are allowed on a particular node (centralized by parameter, not by class). More...
const ContentPtr | getitem_next_array_wrap (const ContentPtr &outcontent, const std::vector< int64_t > &shape) const
| Internal function to wrap putative getitem output with enough RegularArray nodes to satisfy a given `shape`. More...
const std::string | parameters_tostring (const std::string &indent, const std::string &pre, const std::string &post) const
| Internal function to convert parameters into a string fragment for tostring. More...
kernel::lib | kernels_compare (kernel::lib from_index, const ContentPtr &content) const
| Internal function to support most of the logic in kernels. More...
Protected Attributes inherited from Content
IdentitiesPtr | identities_
| See identities. More...
util::Parameters | parameters_
| See parameters. More...

## Detailed Description

Represents potentially missing data by overlaying a bit mask over its content.

See BitMaskedArray for the meaning of each parameter.

## Constructor & Destructor Documentation

## ◆ BitMaskedArray()

BitMaskedArray | ( | const IdentitiesPtr & | identities,
|  | const util::Parameters & | parameters,
|  | const IndexU8 & | mask,
|  | const ContentPtr & | content,
|  | bool | valid_when,
|  | int64_t | length,
|  | bool | lsb_order
| ) |  |

Creates an BitMaskedArray from a full set of parameters.

Parameters

  identities | Optional Identities for each element of the array (may be `nullptr`).
  parameters | String-to-JSON map that augments the meaning of this array.
  mask | Mask in which each bit represents a missing value (`None`) or a valid value (from content). The interpretation depends on valid_when; only bits that are equal to valid_when are not `None`. (Any non-zero value of valid_when is equivalent to a `1` bit.)
  content | Data to be masked; `mask[i]` corresponds to `content[i]` for all `i`.
  valid_when | Interpretation of the boolean bytes in mask as `None` or valid values from content. Only boolean bytes that are equal to valid_when are not `None`.
  length | Length of the array, since it cannot be determined from the mask without an 8-fold ambiguity.
  lsb_order | If `true`, the bits in each byte of the mask are taken to be in Least Significant Bit (LSB) order; if `false`, they are taken to be in Most Significant Bit (MSB) order.

Any non-zero value of a boolean byte and valid_when are equivalent.

## Member Function Documentation

## ◆ argsort_next()

const ContentPtr argsort_next | ( | int64_t | negaxis,
|  | const Index64 & | starts,
|  | const Index64 & | shifts,
|  | const Index64 & | parents,
|  | int64_t | outlength,
|  | bool | ascending,
|  | bool | stable
| ) |  | const
| overridevirtual

This array sorted indices.

The user's entry point for this operation is argsort.

Parameters

  negaxis | The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
  starts | Staring positions of each group to combine as an `Index`. These are downward pointers from an outer structure into this structure with the same meaning as in `ListArray`.
  shifts | Per-element adjustments that allows for variable-length lists with axis != -1 and for missing values (None).
  parents | Groups to combine as an `Index` of upward pointers from this structure to the outer structure to sort.
  outlength | The length of the array, after the operation completes.
  ascending | If `true`, the values will be sorted in an ascending order.
  stable | If `true`, the values will be sorted by a stable sort algorithm to maintain the relative order of records with equal keys (i.e. values).

Implements Content.

## ◆ asslice()

const SliceItemPtr asslice | ( |  | ) | const
| overridevirtual

Converts this array into a SliceItem that can be used in getitem.

Implements Content.

## ◆ branch_depth()

const std::pair< bool, int64_t > branch_depth | ( |  | ) | const
| overridevirtual

Returns (a) whether the list-depth of this array "branches," or differs when followed through different fields of a RecordArray or `UnionArray` and (b) the minimum list-depth.

If the array does not contain any records or heterogeneous data, the `first` element is always `true` and the `second` is simply the depth.

Implements Content.

## ◆ bytemask()

const Index8 bytemask | ( |  | ) | const

Expands the mask to a byte-valued mask with a fixed interpretation: missing values are `1` and valid values are `0` (as though valid_when were `false`).

## ◆ caches()

void caches | ( | std::vector< ArrayCachePtr > & | out | ) | const
| overridevirtual

Accumulates all the unique #ArrayCache objects from nested #VirtualArray nodes. (Uniqueness is determined by pointer value.)

Implements Content.

## ◆ carry()

const ContentPtr carry | ( | const Index64 & | carry,
|  | bool | allow_lazy
| ) |  | const
| overridevirtual

Returns an array of the same type with elements filtered, rearranged, and possibly duplicated by the `carry` array of integers.

The output has the same length as the `carry` index, not the `array` that is being manipulated. For each item `i` in `carry`, the output is `array[index[i]]`.

This operation is called take in NumPy and Arrow, although this carry is a low-level function that does not handle negative indexes and is not exposed to the Python layer. It is used by many operations to pass filters/rearrangements/duplications from one typed array node to another without knowing the latter's type.

Taking getitem_at_nowrap as a function from integers to the array's item type, `A: [0, len(a)) → T`, and the `carry` array's `Index64::getitem_at_nowrap` as a function `C: [0, len(c)) → [0, len(a))`, this method represents function composition, `A ∘ C: [0, len(c)) → T`.

Note
  If the `carry` array has duplicate elements, the array is only duplicated one level deep. For instance, on a `ListArray`, only the `starts` and `stops` get duplicated, not the `content` (and similarly for all other node types).

Implements Content.

## ◆ check_for_iteration()

void check_for_iteration | ( |  | ) | const
| overridevirtual

Performs up-front validity checks on an array so that they don't have to be checked in getitem_at_nowrap for each item.

Implements Content.

## ◆ classname()

const std::string classname | ( |  | ) | const
| overridevirtual

User-friendly name of this class: `"BitMaskedArray"`.

Implements Content.

## ◆ combinations()

const ContentPtr combinations | ( | int64_t | n,
|  | bool | replacement,
|  | const util::RecordLookupPtr & | recordlookup,
|  | const util::Parameters & | parameters,
|  | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

Tuples or records of all `n`-tuple combinations of list items at some `axis` depth.

For example, the `n = 2` combinations at `axis = 0` of

[a, b, c, d, e]

would be

[[(a, b), (a, c), (a, d), (a, e)],
[(b, c), (b, d), (b, e)],
[(c, d), (c, e)],
[(d, e)]]

and the `n = 3` combinations at `axis = 1` of

[[a, b, c, d], [], [e, f], [g, h, i]]

would be

[[[(a, b, c), (a, b, d), (a, c, d)], [(b, c, d)]],
[],
[],
[[(g, h, i)]]]

Parameters

  n | The number of items in each tuple/record.
  replacement | If `true`, the tuples/records are allowed to include the same item more than once, such as `(a, a, a)` and `(a, a, b)`. In the above examples, `replacement = false`. (If the output of this function is thought of as the "upper triangle" elements of the Cartesian product of the input with itself `n` times, the `replacement` parameter determines whether elements on the diagonal are allowed.)
  recordlookup | If `nullptr`, the output consists of tuples, a RecordArray indexed by `"0"`, `"1"`, `"2"`, etc. If `recordlookup` is a `std::vector<std::string>`, the output consists of records, a RecordArray indexed by names (strings). The length of `recordlookup` must be equal to `n`.
  parameters | Parameters assigned to the new RecordArray. This can be used to set `"__record_" = "\"record_name\\""` to give the records a custom behavior in Python.
  axis | The nesting depth at which this operation is applied. At `axis = 0`, combinations are formed from elements of the whole array (see example above); at `axis = 1`, combinations are formed independently for each element. At a deeper `axis`, outer levels are left untouched. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Note that `axis = 0` is qualitatively different from any other `axis` because a dataset is typically much larger than any one of its elements. As such, `axis = 0` is lazily generated with an `IndexedArray`, while any other `axis` is eagerly generated by duplicating elements (with carry).

Implements Content.

## ◆ content()

const ContentPtr content | ( |  | ) | const

Data to be masked; `mask[i]` corresponds to `content[i*8]` through `content[i*8 + 7]` (inclusive) for all `i`.

## ◆ copy_to()

const ContentPtr copy_to | ( | kernel::lib | ptr_lib | ) | const
| overridevirtual

Recursively copies components of the array from main memory to a GPU (if `ptr_lib == kernel::lib::cuda`) or to main memory (if `ptr_lib == kernel::lib::cpu`) if those components are not already there.

Implements Content.

## ◆ deep_copy()

const ContentPtr deep_copy | ( | bool | copyarrays,
|  | bool | copyindexes,
|  | bool | copyidentities
| ) |  | const
| overridevirtual

Copies this node and all nodes hierarchically nested within it, optionally copying the associated arrays, indexes, and identities, too.

See also shallow_copy.

Parameters

  copyarrays | If `true`, copy the associated array buffers (in NumpyArray and `RawArray`), not just the lightweight objects that point to them.
  copyindexes | If `true`, copy the `Index` objects and their buffers as well.
  copyidentities | If `true`, copy the `Identities` objects and their buffers as well.

Implements Content.

## ◆ fieldindex()

int64_t fieldindex | ( | const std::string & | key | ) | const
| overridevirtual

The position of a tuple or record key name if this array contains a RecordArray.

Implements Content.

## ◆ fillna()

const ContentPtr fillna | ( | const ContentPtr & | value | ) | const
| overridevirtual

Returns this array with `None` values replaced by a given `value`.

Parameters

  value | An array of exactly one element, which need not have the same type as the missing values it's replacing.

Implements Content.

## ◆ form()

const FormPtr form | ( | bool | materialize | ) | const
| overridevirtual

Low-level Form describing all the features of this array except the actual data buffers (`Index`, `Identities`, `NumpyArray::ptr`, and `RawArray::ptr`).

Parameters

  materialize | If true, materialize any `VirtualArrays` encountered along the way.

Implements Content.

## ◆ getitem_at()

const ContentPtr getitem_at | ( | int64_t | at | ) | const
| overridevirtual

Returns the element at a given position in the array, handling negative indexing and bounds-checking like Python.

The first item in the array is at `0`, the second at `1`, the last at `-1`, the penultimate at `-2`, etc.

Implements Content.

## ◆ getitem_at_nowrap()

const ContentPtr getitem_at_nowrap | ( | int64_t | at | ) | const
| overridevirtual

Returns the element at a given position in the array, without handling negative indexing or bounds-checking.

If the array has Identities, the identity bounds are checked.

Implements Content.

## ◆ getitem_field() [1/2]

const ContentPtr getitem_field | ( | const std::string & | key | ) | const
| overridevirtual

This array with the first nested RecordArray replaced by the field at `key`.

Implements Content.

## ◆ getitem_field() [2/2]

const ContentPtr getitem_field | ( | const std::string & | key,
|  | const Slice & | only_fields
| ) |  | const
| overridevirtual

Implements Content.

## ◆ getitem_fields() [1/2]

const ContentPtr getitem_fields | ( | const std::vector< std::string > & | keys | ) | const
| overridevirtual

This array with the first nested RecordArray replaced by a RecordArray of a given subset of `keys`.

Implements Content.

## ◆ getitem_fields() [2/2]

const ContentPtr getitem_fields | ( | const std::vector< std::string > & | keys,
|  | const Slice & | only_fields
| ) |  | const
| overridevirtual

Implements Content.

## ◆ getitem_next() [1/5]

const ContentPtr getitem_next | ( | const SliceArray64 & | array,
|  | const Slice & | tail,
|  | const Index64 & | advanced
| ) |  | const
| overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

## ◆ getitem_next() [2/5]

const ContentPtr getitem_next | ( | const SliceAt & | at,
|  | const Slice & | tail,
|  | const Index64 & | advanced
| ) |  | const
| overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

## ◆ getitem_next() [3/5]

const ContentPtr getitem_next | ( | const SliceItemPtr & | head,
|  | const Slice & | tail,
|  | const Index64 & | advanced
| ) |  | const
| overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

Parameters

  head | First element of the Slice tuple.
  tail | The rest of the elements of the Slice tuple.
  advanced | If empty, no array slices (integer or boolean) have been encountered yet; otherwise, positions in any subsequent array slices to select.

In the NumPy documentation, advanced indexes are described as iterating "as one," which requires an `Index` to be propagated when implemented recursively.

Reimplemented from Content.

## ◆ getitem_next() [4/5]

const ContentPtr getitem_next | ( | const SliceJagged64 & | jagged,
|  | const Slice & | tail,
|  | const Index64 & | advanced
| ) |  | const
| overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

## ◆ getitem_next() [5/5]

const ContentPtr getitem_next | ( | const SliceRange & | range,
|  | const Slice & | tail,
|  | const Index64 & | advanced
| ) |  | const
| overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

## ◆ getitem_next_jagged() [1/3]

const ContentPtr getitem_next_jagged | ( | const Index64 & | slicestarts,
|  | const Index64 & | slicestops,
|  | const SliceArray64 & | slicecontent,
|  | const Slice & | tail
| ) |  | const
| overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

## ◆ getitem_next_jagged() [2/3]

const ContentPtr getitem_next_jagged | ( | const Index64 & | slicestarts,
|  | const Index64 & | slicestops,
|  | const SliceJagged64 & | slicecontent,
|  | const Slice & | tail
| ) |  | const
| overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

## ◆ getitem_next_jagged() [3/3]

const ContentPtr getitem_next_jagged | ( | const Index64 & | slicestarts,
|  | const Index64 & | slicestops,
|  | const SliceMissing64 & | slicecontent,
|  | const Slice & | tail
| ) |  | const
| overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

## ◆ getitem_nothing()

const ContentPtr getitem_nothing | ( |  | ) | const
| overridevirtual

Internal function to get an empty slice (with the correct type).

Implements Content.

## ◆ getitem_range()

const ContentPtr getitem_range | ( | int64_t | start,
|  | int64_t | stop
| ) |  | const
| overridevirtual

Subinterval of this array, handling negative indexing and bounds-checking like Python.

The first item in the array is at `0`, the second at `1`, the last at `-1`, the penultimate at `-2`, etc.

Ranges beyond the array are not an error; they are trimmed to `start = 0` on the left and `stop = length() - 1` on the right.

This operation only affects the node metadata; its calculation time does not scale with the size of the array.

Implements Content.

## ◆ getitem_range_nowrap()

const ContentPtr getitem_range_nowrap | ( | int64_t | start,
|  | int64_t | stop
| ) |  | const
| overridevirtual

Subinterval of this array, without handling negative indexing or bounds-checking.

If the array has Identities, the identity bounds are checked.

This operation only affects the node metadata; its calculation time does not scale with the size of the array.

Implements Content.

## ◆ haskey()

bool haskey | ( | const std::string & | key | ) | const
| overridevirtual

Returns `true` if the array contains a RecordArray with the specified `key`; `false` otherwise.

Implements Content.

## ◆ is_subrange_equal()

bool is_subrange_equal | ( | const Index64 & | start,
|  | const Index64 & | stop
| ) |  | const
| overridevirtual

Returns 'true' if subranges are equal.

Implements Content.

## ◆ is_unique()

bool is_unique | ( |  | ) | const
| overridevirtual

Returns 'true' if all components of the array are unique.

Implements Content.

## ◆ istuple()

bool istuple | ( |  | ) | const
| overridevirtual

Returns `true` if the outermost RecordArray is a tuple.

Implements Content.

## ◆ kernels()

kernel::lib kernels | ( |  | ) | const
| overridevirtual

Returns the kernel library enum for all nested `ptr_lib` within the array's tree structure. If different libraries are associated with nodes of the same tree, this returns `kernel::lib::size` to indicate that it is "mixed".

Implements Content.

## ◆ key()

const std::string key | ( | int64_t | fieldindex | ) | const
| overridevirtual

The record name associated with a given field index or the tuple index as a string (e.g. `"0"`, `"1"`, `"2"`) if a tuple.

Raises an error if the array does not contain a RecordArray.

Implements Content.

## ◆ keys()

const std::vector< std::string > keys | ( |  | ) | const
| overridevirtual

A list of RecordArray keys or an empty list if this array does not contain a RecordArray.

Implements Content.

## ◆ length()

int64_t length | ( |  | ) | const
| overridevirtual

The number of elements in the array.

Note that this is an input parameter.

Implements Content.

## ◆ localindex()

const ContentPtr localindex | ( | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

A (possibly nested) array of integers indicating the positions of elements within each nested list.

Parameters

  axis | The nesting depth at which this operation is applied. If `axis = 0`, the output is simply an array of integers from `0` to `length()`. If `axis = 1`, the output has one level of nesting containing integers from `0` to the length of the nested list. Higher values of `axis` leave outer layers of the structure untouched. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

## ◆ lsb_order()

bool lsb_order | ( |  | ) | const

If `true`, the bits in each byte of the mask are taken to be in Least Significant Bit (LSB) order; if `false`, they are taken to be in Most Significant Bit (MSB) order.

## ◆ mask()

const IndexU8 mask | ( |  | ) | const

Mask in which each bit represents a missing value (`None`) or a valid value (from content).

The interpretation depends on valid_when; only bits that are equal to valid_when are not `None`. (Any non-zero value of valid_when is equivalent to a `1` bit.)

## ◆ mergeable()

bool mergeable | ( | const ContentPtr & | other,
|  | bool | mergebool
| ) |  | const
| overridevirtual

Returns `true` if this array can be merged with the `other`; `false` otherwise.

The merge method will complete without errors if this function returns `true`.

Parameters

  other | The other array to merge with.
  mergebool | If `true`, consider boolean types to be equivalent to integers.

Implements Content.

## ◆ mergemany()

const ContentPtr mergemany | ( | const ContentPtrVec & | others | ) | const
| overridevirtual

Returns an array with this and the `others` concatenated (in order, this first, `others` last).

Implements Content.

## ◆ minmax_depth()

const std::pair< int64_t, int64_t > minmax_depth | ( |  | ) | const
| overridevirtual

Returns (a) the minimum list-depth and (b) the maximum list-depth of the array, which can differ if this array "branches" (differs when followed through different fields of a RecordArray or `UnionArray`).

Implements Content.

## ◆ nbytes_part()

void nbytes_part | ( | std::map< size_t, int64_t > & | largest | ) | const
| overridevirtual

Internal function used to calculate nbytes.

Parameters

  largest | The largest range of bytes used in each reference-counted pointer (`size_t`).

Note
  This method of accounting for overlapping buffers is insufficient: two nodes could use parts of the same buffer in which one doesn't completely overlap the other. It's not likely, but currently the calculation of nbytes is an underestimate.

Implements Content.

## ◆ num()

const ContentPtr num | ( | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

The length of this array (as a NumpyArray scalar) if `axis = 0` or the lengths of subarrays (as an array or nested array) if `axis != 0`.

Parameters

  axis | The axis whose length or lengths to quantify. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

## ◆ numbers_to_type()

const ContentPtr numbers_to_type | ( | const std::string & | name | ) | const
| overridevirtual

Change the leaf types to 'totype'.

Implements Content.

## ◆ numfields()

int64_t numfields | ( |  | ) | const
| overridevirtual

The number of fields in the first nested tuple or records or `-1` if this array does not contain a RecordArray.

Implements Content.

## ◆ offsets_and_flattened()

const std::pair< Index64, ContentPtr > offsets_and_flattened | ( | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

Returns (a) an offsets `Index` and (b) a flattened version of the array at some `axis` depth.

If `axis > 1` (or its negative equivalent), the offsets is empty.

Parameters

  axis | The axis to eliminate by flattening. `axis = 0` is invalid. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

## ◆ project() [1/2]

const ContentPtr project | ( |  | ) | const

Return an array with the same type as content with `None` values removed.

## ◆ project() [2/2]

const ContentPtr project | ( | const Index8 & | mask | ) | const

Performs a set-union of a given `mask` with the missing values and calls project.

Parameters

  mask | A byte mask that is valid when `0`, `None` when `1`.

## ◆ purelist_depth()

int64_t purelist_depth | ( |  | ) | const
| overridevirtual

The list-depth of this array, not counting any contained within a RecordArray.

The `purelist_depth` of a Record is `0`, and a RecordArray is `1` (regardless of what its fields contain).

If this array contains a `UnionArray` with different depths, the return value is `-1`.

Implements Content.

## ◆ reduce_next()

const ContentPtr reduce_next | ( | const Reducer & | reducer,
|  | int64_t | negaxis,
|  | const Index64 & | starts,
|  | const Index64 & | shifts,
|  | const Index64 & | parents,
|  | int64_t | outlength,
|  | bool | mask,
|  | bool | keepdims
| ) |  | const
| overridevirtual

This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all).

The user's entry point for this operation is reduce.

Parameters

  reducer | The choice of Reducer algorithm.
  negaxis | The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
  starts | Staring positions of each group to combine as an `Index`. These are downward pointers from an outer structure into this structure with the same meaning as in `ListArray`. Only used by a reducer that `returns_positions` (currently only argmin and argmax).
  shifts | Per-element adjustments for any reducer that `returns_positions`. Allows for variable-length lists with axis != -1 and for missing values (None) in argmin and argmax.
  parents | Groups to combine as an `Index` of upward pointers from this structure to the outer structure to reduce.
  outlength | The length of the reduced array, after the operation completes.
  mask | If `true`, the Reducer's identity values will be covered by `None` using a ByteMaskedArray. This is desirable for ReducerMin, ReducerMax, ReducerArgmin, and ReducerArgmax to indicate that empty lists have no minimum or maximum.
  keepdims | If `true`, the reduced values will be wrapped by a singleton RegularArray to maintain the same number of dimensions in the output.

Implements Content.

## ◆ referentially_equal()

bool referentially_equal | ( | const ContentPtr & | other | ) | const
| overridevirtual

Returns `true` if this array has all the same buffers and parameters as `other`; `false` otherwise.

Parameters

  other | The array to compare this with.

Implements Content.

## ◆ reverse_merge()

const ContentPtr reverse_merge | ( | const ContentPtr & | other | ) | const
| overridevirtual

Merges a single `other` with this array in reverse order: `other` first, this last.

Only arrays that need to be reversible have this function: option-type and union-type arrays. Others raise a runtime error.

Reimplemented from Content.

## ◆ rpad()

const ContentPtr rpad | ( | int64_t | target,
|  | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

If `axis = 0`, returns a view of this array padded on the right with `None` values to have a minimum length; otherwise, returns an array with nested lists all padded to the minimum length.

Parameters

  target | The intended length. The output may be longer than this target length, but not shorter (using `ListArray`).
  axis | The axis at which to apply padding. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

## ◆ rpad_and_clip()

const ContentPtr rpad_and_clip | ( | int64_t | target,
|  | int64_t | axis,
|  | int64_t | depth
| ) |  | const
| overridevirtual

If `axis = 0`, returns a view of this array padded on the right.

with `None` values to have exactly the specified length; otherwise, returns an array with nested lists all padded to the specified length.

Parameters

  target | The intended length; the output has exactly this target length (using RegularArray).
  axis | The axis at which to apply padding. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
  depth | The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

## ◆ setidentities() [1/2]

void setidentities | ( |  | ) |
| overridevirtual

Assign a surrogate index of Identities to this array (in-place).

This also assigns and possibly replaces Identities in nested arrays.

Note
  This mutability is temporary: scikit-hep/awkward-1.0#116 Eventually, this interface will be deprecated and all Content instances will be immutable.

Implements Content.

## ◆ setidentities() [2/2]

void setidentities | ( | const IdentitiesPtr & | identities | ) |
| overridevirtual

Assign a specified set of Identities to this array (in-place).

This also assigns and possibly replaces Identities in nested arrays.

Note
  This mutability is temporary: scikit-hep/awkward-1.0#116 Eventually, this interface will be deprecated and all Content instances will be immutable.

Implements Content.

## ◆ shallow_copy()

const ContentPtr shallow_copy | ( |  | ) | const
| overridevirtual

Copies this node without copying any nodes hierarchically nested within it or any array/index/identity buffers.

See also deep_copy.

Implements Content.

## ◆ shallow_simplify()

const ContentPtr shallow_simplify | ( |  | ) | const
| overridevirtual

Returns an equivalent array simplified at one level only using `simplify_optiontype` if an option-type array and `simplify_uniontype` if a union-type array.

For all other types of arrays, this operation is a pass-through.

For BitMaskedArray, this method returns simplify_optiontype.

Implements Content.

## ◆ simplify_optiontype()

const ContentPtr simplify_optiontype | ( |  | ) | const

If the content also has OptionType, combine the mask with the content's indicator of missing values; also combine if the content is a non-OptionType `IndexedArray`.

This is a shallow operation: it only checks the content one level deep.

## ◆ sort_next()

const ContentPtr sort_next | ( | int64_t | negaxis,
|  | const Index64 & | starts,
|  | const Index64 & | parents,
|  | int64_t | outlength,
|  | bool | ascending,
|  | bool | stable
| ) |  | const
| overridevirtual

This array sorted.

The user's entry point for this operation is sort.

Parameters

  negaxis | The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
  starts | Staring positions of each group to combine as an `Index`. These are downward pointers from an outer structure into this structure with the same meaning as in `ListArray`.
  parents | Groups to combine as an `Index` of upward pointers from this structure to the outer structure to sort.
  outlength | The length of the array, after the operation completes.
  ascending | If `true`, the values will be sorted in an ascending order.
  stable | If `true`, the values will be sorted by a stable sort algorithm to maintain the relative order of records with equal keys (i.e. values).

Implements Content.

## ◆ toByteMaskedArray()

const std::shared_ptr< ByteMaskedArray > toByteMaskedArray | ( |  | ) | const

Converts this array into a ByteMaskedArray.

## ◆ toIndexedOptionArray64()

const std::shared_ptr< IndexedArrayOf< int64_t, true > > toIndexedOptionArray64 | ( |  | ) | const

Converts this array into an `IndexedOptionArray` with the same missing values.

## ◆ tojson_part()

void tojson_part | ( | ToJson & | builder,
|  | bool | include_beginendlist
| ) |  | const
| overridevirtual

Internal function to produce a JSON representation one node at a time.

Implements Content.

## ◆ tostring_part()

const std::string tostring_part | ( | const std::string & | indent,
|  | const std::string & | pre,
|  | const std::string & | post
| ) |  | const
| overridevirtual

Internal function to build an output string for tostring.

Parameters

  indent | Indentation depth as a string of spaces.
  pre | Prefix string, usually an opening XML tag.
  post | Postfix string, usually a closing XML tag and carriage return.

Implements Content.

## ◆ type()

const TypePtr type | ( | const util::TypeStrs & | typestrs | ) | const
| overridevirtual

High-level Type describing this array.

Parameters

  typestrs | A mapping from `"__record__"` parameters to string representations of those types, to override the derived strings.

Implements Content.

## ◆ unique()

const ContentPtr unique | ( |  | ) | const
| overridevirtual

Returns an array where all components are unique.

Implements Content.

## ◆ valid_when()

bool valid_when | ( |  | ) | const

Interpretation of the boolean bytes in mask as `None` or valid values from content. Only boolean bytes that are equal to valid_when are not `None`. (Any non-zero value of a boolean byte and `valid_when` are equivalent.)

## ◆ validityerror()

const std::string validityerror | ( | const std::string & | path | ) | const
| overridevirtual

Returns an error message if this array is invalid; otherwise, returns an empty string.

Implements Content.

The documentation for this class was generated from the following file:

- include/awkward/array/BitMaskedArray.h