Medial Code Documentation
Loading...
Searching...
No Matches
Namespaces | Data Structures | Typedefs | Enumerations | Functions | Variables
xgboost::common Namespace Reference

Copyright 2017-2023, XGBoost Contributors. More...

Namespaces

namespace  base64
 namespace of base64 decoding and encoding table
 
namespace  cpu_impl
 Reduction with iterator.
 

Data Structures

struct  AFTLoss
 The AFT loss function. More...
 
struct  AFTParam
 Parameter structure for AFT loss and metric. More...
 
class  AlignedFileWriteStream
 Output stream backed by a file. More...
 
class  AlignedMemWriteStream
 Output stream backed by memory buffer. More...
 
class  AlignedResourceReadStream
 Wrap resource into a dmlc stream. More...
 
class  AlignedWriteStream
 Base class for write stream with alignment defined by IOAlignment(). More...
 
class  Base64InStream
 the stream that reads from base64, note we take from file pointers More...
 
class  Base64OutStream
 the stream that write to base64, note we take from file pointers More...
 
struct  BaseClass
 
class  BlockedSpace2d
 
class  Column
 a column storage, to be used with ApplySplit. Note that each bin id is stored as index[i] + index_base. Different types of column index for each column allow to reduce the memory usage. More...
 
class  ColumnMatrix
 Column major matrix for gradient index on CPU. More...
 
class  ColumnSampler
 Handles selection of columns due to colsample_bytree, colsample_bylevel and colsample_bynode parameters. More...
 
class  CompressedBufferWriter
 Writes bit compressed symbols to a memory buffer. More...
 
class  CompressedIterator
 Read symbols from a bit compressed memory buffer. More...
 
class  ConfigParser
 Implementation of config reader. More...
 
class  DenseColumnIter
 Column stored as a dense vector. More...
 
struct  DerivedClass
 
struct  ExtremeDistribution
 
class  FixedSizeStream
 A simple class used to consume ‘dmlc::Stream’ all at once. More...
 
class  GHistBuildingManager
 
class  HistCollection
 histogram of gradient statistics for multiple nodes More...
 
class  HistogramCuts
 
class  HostSketchContainer
 
class  Index
 Optionally compressed gradient index. More...
 
class  IndexTransformIter
 Transform iterator that takes an index and calls transform operator. More...
 
struct  IsCatOp
 
class  IterSpan
 A simple custom Span type that uses general iterator instead of pointer. More...
 
struct  LogisticDistribution
 
class  MallocResource
 
class  MemStackAllocator
 A C-style array with in-stack allocation. As long as the array is smaller than MaxStackSize, it will be allocated inside the stack. Otherwise, it will be heap-allocated. More...
 
struct  MMAPFile
 
class  MmapResource
 A class for wrapping mmap as a resource for RAII. More...
 
struct  Monitor
 Timing utility used to measure total method execution time over the lifetime of the containing object. More...
 
struct  NormalDistribution
 
struct  OptionalWeights
 
class  ParallelGHistBuilder
 Stores temporary histograms to compute them in parallel Supports processing multiple tree-nodes for nested parallelism Able to reduce histograms across threads in efficient way. More...
 
class  ParallelGroupBuilder
 multi-thread version of group builder More...
 
class  ParamFloatArray
 
class  PartitionBuilder
 
class  PeekableInStream
 Input stream that support additional PeekRead operation, besides read. More...
 
struct  Prefetch
 
class  PrivateMmapConstStream
 Private mmap file as a read-only stream. More...
 
struct  QuantileLossParam
 
class  QuantileSketchTemplate
 template for all quantile sketch algorithm that uses merge/prune scheme More...
 
struct  RandomThreadLocalEntry
 thread local entry for random. More...
 
class  Range
 
class  Range1d
 
class  ReallocVector
 
class  RefResourceView
 A vector-like type that holds a reference counted resource. More...
 
class  ResourceHandler
 Handler for one-shot resource. More...
 
class  RowSetCollection
 collection of rowset More...
 
struct  RuntimeFlags
 
struct  Sched
 OpenMP schedule. More...
 
class  SketchContainerImpl
 
struct  SortedQuantile
 Quantile structure accepts sorted data, extracted from histmaker. More...
 
class  SortedSketchContainer
 
class  Span
 span class implementation, based on ISO++20 span<T>. The interface should be the same. More...
 
class  SparseColumnIter
 
class  StreamBufferReader
 buffer reader of the stream that allows you to get More...
 
struct  TestAsBytes
 
struct  TestAssignment
 
struct  TestAsWritableBytes
 
struct  TestBeginEnd
 
struct  TestCompare
 
struct  TestIterCalculate
 
struct  TestIterCompare
 
struct  TestIterConstruct
 
struct  TestIterRef
 
struct  TestObservers
 
struct  TestRBeginREnd
 
struct  TestTestStatus
 
struct  TestTransformRange
 
struct  Timer
 
class  Transform
 Do Transformation on HostDeviceVectors. More...
 
struct  WQSummary
 experimental wsummary More...
 
class  WQuantileSketch
 Quantile sketch use WQSummary. More...
 
struct  WXQSummary
 try to do efficient pruning More...
 
class  WXQuantileSketch
 Quantile sketch use WXQSummary. More...
 

Typedefs

using CatBitField = LBitField32
 
using KCatBitField = CLBitField32
 
using RandomThreadLocalStore = dmlc::ThreadLocalStore< RandomThreadLocalEntry >
 
using CompressedByteT = unsigned char
 
using GHistIndexRow = Span< uint32_t const >
 A single row in global histogram index. Directly represent the global index in the histogram entry.
 
using GHistRow = Span< xgboost::GradientPairPrecise >
 
using ConstGHistRow = Span< xgboost::GradientPairPrecise const >
 
using MemoryFixSizeBuffer = rabit::utils::MemoryFixSizeBuffer
 
using MemoryBufferStream = rabit::utils::MemoryBufferStream
 
using RandomEngine = std::mt19937
 Define mt19937 as default type Random Engine.
 
using GlobalRandomEngine = RandomEngine
 global random engine
 

Enumerations

enum class  byte : unsigned char
 
enum  ColumnType : std::uint8_t { kDenseColumn , kSparseColumn }
 column type
 
enum  BinTypeSize : uint8_t { kUint8BinsTypeSize = 1 , kUint16BinsTypeSize = 2 , kUint32BinsTypeSize = 4 }
 
enum class  ProbabilityDistributionType : int { kNormal = 0 , kLogistic = 1 , kExtreme = 2 }
 Enum encoding possible choices of probability distribution.
 
enum class  CensoringType : uint8_t { kUncensored , kRightCensored , kLeftCensored , kIntervalCensored }
 

Functions

template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE bool operator== (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE constexpr bool operator!= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE constexpr bool operator< (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE constexpr bool operator<= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE constexpr bool operator> (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE constexpr bool operator>= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t E>
XGBOOST_DEVICE auto as_bytes (Span< T, E > s) __span_noexcept -> Span< const byte, detail::ExtentAsBytesValue< T, E >::value >
 
template<class T , std::size_t E>
XGBOOST_DEVICE auto as_writable_bytes (Span< T, E > s) __span_noexcept -> Span< byte, detail::ExtentAsBytesValue< T, E >::value >
 
bool CheckNAN (double v)
 
double LogGamma (double v)
 
template<typename It , typename Idx >
auto SegmentId (It first, It last, Idx idx)
 
template<typename Iter , typename Comp >
void StableSort (Context const *ctx, Iter begin, Iter end, Comp &&comp)
 
template<typename Iter , typename Comp >
void Sort (Context const *ctx, Iter begin, Iter end, Comp comp)
 
template<typename Idx , typename Iter , typename V = typename std::iterator_traits<Iter>::value_type, typename Comp = std::less<V>>
std::vector< Idx > ArgSort (Context const *ctx, Iter begin, Iter end, Comp comp=std::less< V >{})
 
template<typename T >
XGBOOST_DEVICE bst_cat_t AsCat (T const &v)
 
XGBOOST_DEVICE bool IsCat (Span< FeatureType const > ft, bst_feature_t fidx)
 
constexpr bst_cat_t OutOfRangeCat ()
 
XGBOOST_DEVICE bool InvalidCat (float cat)
 
XGBOOST_DEVICE bool Decision (common::Span< CatBitField::value_type const > cats, float cat)
 Whether should it traverse to left branch of a tree.
 
void InvalidCategory ()
 
void CheckMaxCat (float max_cat, size_t n_categories)
 
XGBOOST_DEVICE bool UseOneHot (uint32_t n_cats, uint32_t max_cat_to_onehot)
 Whether should we use onehot encoding for categorical data.
 
GlobalRandomEngineGlobalRandom ()
 global singleton of a random engine. This random engine is thread-local and only visible to current thread.
 
void EscapeU8 (std::string const &string, std::string *p_buffer)
 
int AllVisibleGPUs ()
 
std::vector< std::string > Split (const std::string &s, char delim)
 Split a string by delimiter.
 
template<typename T >
XGBOOST_DEVICEMax (T a, T b)
 
template<typename T1 , typename T2 >
XGBOOST_DEVICE T1 DivRoundUp (const T1 a, const T2 b)
 
template<class T , std::size_t N>
constexpr auto UnpackArr (std::array< T, N > &&arr)
 
void AssertGPUSupport ()
 
void AssertOneAPISupport ()
 
void SetDevice (std::int32_t device)
 
template<typename Indexable >
XGBOOST_DEVICE size_t LastOf (size_t group, Indexable const &indptr)
 Last index of a group in a CSR style of index pointer.
 
HistogramCuts SketchOnDMatrix (Context const *ctx, DMatrix *m, bst_bin_t max_bins, bool use_sorted=false, Span< float const > hessian={})
 Run CPU sketching on DMatrix.
 
void IncrementHist (GHistRow dst, ConstGHistRow add, std::size_t begin, std::size_t end)
 Increment hist as dst += add in range [begin, end)
 
void CopyHist (GHistRow dst, const GHistRow src, size_t begin, size_t end)
 Copy hist from src to dst in range [begin, end)
 
void SubtractionHist (GHistRow dst, const GHistRow src1, const GHistRow src2, size_t begin, size_t end)
 Compute Subtraction: dst = src1 - src2 in range [begin, end)
 
template<bool do_prefetch, class BuildingManager >
void RowsWiseBuildHistKernel (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist)
 
template<class BuildingManager >
void ColsWiseBuildHistKernel (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist)
 
template<class BuildingManager >
void BuildHistDispatch (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist)
 
template<bool any_missing>
void BuildHist (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist, bool force_read_by_column)
 
template void BuildHist< true > (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist, bool force_read_by_column)
 
template void BuildHist< false > (Span< GradientPair const > gpair, const RowSetCollection::Elem row_indices, const GHistIndexMatrix &gmat, GHistRow hist, bool force_read_by_column)
 
template<typename Fn >
auto DispatchBinType (BinTypeSize type, Fn &&fn)
 Dispatch for bin type, fn is a function that accepts a scalar of the bin type.
 
template<typename GradientIndex >
bst_bin_t XGBOOST_HOST_DEV_INLINE BinarySearchBin (std::size_t begin, std::size_t end, GradientIndex const &data, bst_feature_t const fidx_begin, bst_feature_t const fidx_end)
 
std::string LoadSequentialFile (std::string uri, bool stream=false)
 Helper function for loading consecutive file to avoid dmlc Stream when possible.
 
std::string FileExtension (std::string fname, bool lower=true)
 Get file extension from file name.
 
std::unique_ptr< MMAPFileOpen (std::string path, std::size_t offset, std::size_t length)
 
std::string ReadAll (dmlc::Stream *fi, PeekableInStream *fp)
 Read the whole buffer from dmlc stream.
 
std::string ReadAll (std::string const &path)
 Read the whole file content into a string.
 
constexpr std::size_t IOAlignment ()
 
template<typename T >
XGBOOST_DEVICESqr (T const &w)
 
XGBOOST_DEVICE float Sigmoid (float x)
 calculate the sigmoid of the input.
 
XGBOOST_DEVICE double Sigmoid (double x)
 
template<typename T , typename U >
XGBOOST_DEVICE constexpr bool CloseTo (T a, U b)
 Equality test for both integer and floating point.
 
template<typename Iterator >
XGBOOST_DEVICE void Softmax (Iterator start, Iterator end)
 Do inplace softmax transformaton on start to end.
 
template<typename Iterator >
XGBOOST_DEVICE Iterator FindMaxIndex (Iterator begin, Iterator end)
 Find the maximum iterator within the iterators.
 
float LogSum (float x, float y)
 perform numerically safe logsum
 
template<typename Iterator >
float LogSum (Iterator begin, Iterator end)
 perform numerically safe logsum
 
template<typename T >
XGBOOST_DEVICE std::enable_if< std::numeric_limits< T >::is_integer, bool >::type CheckNAN (T)
 
XGBOOST_DEVICE bool CheckNAN (float x)
 
template<typename T >
XGBOOST_DEVICELogGamma (T v)
 
double Reduce (Context const *ctx, HostDeviceVector< float > const &values)
 Reduction on host device vector.
 
template<typename Iter , typename Idx >
void RunLengthEncode (Iter begin, Iter end, std::vector< Idx > *p_out)
 Run length encode on CPU, input must be sorted.
 
template<typename InIt , typename OutIt , typename T >
void PartialSum (int32_t n_threads, InIt begin, InIt end, T init, OutIt out_it)
 Varient of std::partial_sum, out_it should point to a container that has n + 1 elements.
 
template<typename It >
void Iota (Context const *ctx, It first, It last, typename std::iterator_traits< It >::value_type const &value)
 
OptionalWeights MakeOptionalWeights (Context const *ctx, HostDeviceVector< float > const &weights)
 
template<typename SketchType >
void AddCutPoint (typename SketchType::SummaryContainer const &summary, int max_bin, HistogramCuts *cuts)
 
auto AddCategories (std::set< float > const &categories, HistogramCuts *cuts)
 
template<typename Batch , typename IsValid >
std::vector< bst_row_tCalcColumnSize (Batch const &batch, bst_feature_t const n_columns, size_t const n_threads, IsValid &&is_valid)
 
template<typename Batch , typename IsValid >
std::vector< bst_feature_tLoadBalance (Batch const &batch, size_t nnz, bst_feature_t n_columns, size_t const nthreads, IsValid &&is_valid)
 
std::ostream & operator<< (std::ostream &os, const ParamFloatArray &array)
 
std::istream & operator>> (std::istream &is, ParamFloatArray &array)
 
 DMLC_REGISTER_PARAMETER (QuantileLossParam)
 
template<typename T >
std::vector< T > WeightedSamplingWithoutReplacement (Context const *ctx, std::vector< T > const &array, std::vector< float > const &weights, size_t n)
 
template<typename Vec >
bool ReadVec (common::AlignedResourceReadStream *fi, Vec *vec)
 Read a vector from stream.
 
template<typename Vec >
std::size_t WriteVec (AlignedFileWriteStream *fo, Vec const &vec)
 Write a vector to stream.
 
template<typename T >
RefResourceView< T > MakeFixedVecWithMalloc (std::size_t n_elements, T const &init)
 Make a fixed size RefResourceView with malloc resource.
 
void Median (Context const *ctx, linalg::Tensor< float, 2 > const &t, HostDeviceVector< float > const &weights, linalg::Tensor< float, 1 > *out)
 Calculate medians for each column of the input matrix.
 
void Mean (Context const *ctx, linalg::Vector< float > const &v, linalg::Vector< float > *out)
 
template<typename Iter >
float Quantile (Context const *ctx, double alpha, Iter const &begin, Iter const &end)
 Quantile using linear interpolation.
 
template<typename Iter , typename WeightIter >
float WeightedQuantile (Context const *ctx, double alpha, Iter begin, Iter end, WeightIter w_begin)
 Calculate the weighted quantile with step function.
 
 DMLC_REGISTER_PARAMETER (AFTParam)
 
std::int32_t GetCGroupV1Count (std::filesystem::path const &quota_path, std::filesystem::path const &peroid_path)
 Modified from github.com/psiha/sweater/blob/master/include/boost/sweater/hardware_concurrency.hpp.
 
std::int32_t GetCGroupV2Count (std::filesystem::path const &bandwidth_path) noexcept(true)
 
std::int32_t GetCfsCPUCount () noexcept
 Get thread limit from CFS.
 
std::int32_t OmpGetNumThreads (std::int32_t n_threads)
 Get the number of available threads based on n_threads specified by users.
 
template<typename Func >
void ParallelFor2d (const BlockedSpace2d &space, int n_threads, Func &&func)
 
template<typename Index , typename Func >
void ParallelFor (Index size, int32_t n_threads, Sched sched, Func fn)
 
template<typename Index , typename Func >
void ParallelFor (Index size, int32_t n_threads, Func fn)
 
std::int32_t OmpGetThreadLimit ()
 
std::int32_t constexpr DefaultMaxThreads ()
 Constant that can be used for initializing static thread local memory.
 
template<typename Fn >
auto MakeIndexTransformIter (Fn &&fn)
 
 TEST (Algorithm, ArgSort)
 
 TEST (Algorithm, Sort)
 
 TEST (Categorical, Decision)
 
 TEST (Categorical, MinimalSet)
 Test for running inference with input category greater than the one stored in tree.
 
 TEST (ColumnMatrix, Basic)
 
template<typename BinIdxType >
void CheckSparseColumn (SparseColumnIter< BinIdxType > *p_col, const GHistIndexMatrix &gmat)
 
 TEST (ColumnMatrix, SparseColumn)
 
template<typename BinIdxType >
void CheckColumWithMissingValue (const DenseColumnIter< BinIdxType, true > &col, const GHistIndexMatrix &gmat)
 
 TEST (ColumnMatrix, DenseColumnWithMissing)
 
 TEST (ColumnMatrix, GrowMissing)
 
 TEST (CompressedIterator, Test)
 
 TEST (ConfigParser, NormalizeConfigEOL)
 
 TEST (ConfigParser, TrimWhitespace)
 
 TEST (ConfigParser, ParseKeyValuePair)
 
 TEST (GroupData, ParallelGroupBuilder)
 
void ParallelGHistBuilderReset ()
 
void ParallelGHistBuilderReduceHist ()
 
 TEST (ParallelGHistBuilder, Reset)
 
 TEST (ParallelGHistBuilder, ReduceHist)
 
 TEST (CutsBuilder, SearchGroupInd)
 
 TEST (HistUtil, DenseCutsCategorical)
 
 TEST (HistUtil, DenseCutsAccuracyTest)
 
 TEST (HistUtil, DenseCutsAccuracyTestWeights)
 
void TestQuantileWithHessian (bool use_sorted)
 
 TEST (HistUtil, QuantileWithHessian)
 
 TEST (HistUtil, DenseCutsExternalMemory)
 
 TEST (HistUtil, IndexBinBound)
 
template<typename T >
void CheckIndexData (T const *data_ptr, uint32_t const *offsets, const GHistIndexMatrix &hmat, size_t n_cols)
 
 TEST (HistUtil, IndexBinData)
 
void TestSketchFromWeights (bool with_group)
 
 TEST (HistUtil, SketchFromWeights)
 
 TEST (HistUtil, SketchCategoricalFeatures)
 
std::vector< float > GenerateRandom (int num_rows, int num_columns)
 
std::vector< float > GenerateRandomWeights (int num_rows)
 
std::shared_ptr< data::SimpleDMatrixGetDMatrixFromData (const std::vector< float > &x, int num_rows, int num_columns)
 
std::shared_ptr< DMatrixGetExternalMemoryDMatrixFromData (const std::vector< float > &x, int num_rows, int num_columns, const dmlc::TemporaryDirectory &tempdir)
 
void TestBinDistribution (const HistogramCuts &cuts, int column_idx, const std::vector< float > &sorted_column, const std::vector< float > &sorted_weights)
 
void TestRank (const std::vector< float > &column_cuts, const std::vector< float > &sorted_x, const std::vector< float > &sorted_weights)
 
void ValidateColumn (const HistogramCuts &cuts, int column_idx, const std::vector< float > &sorted_column, const std::vector< float > &sorted_weights, size_t num_bins)
 
void ValidateCuts (const HistogramCuts &cuts, DMatrix *dmat, int num_bins)
 
template<typename Fn >
void TestCategoricalSketch (size_t n, size_t num_categories, int32_t num_bins, bool weighted, Fn sketch)
 Test for sketching on categorical data.
 
 TEST (MemoryFixSizeBuffer, Seek)
 
 TEST (IO, FileExtension)
 
 TEST (IO, FixedSizeStream)
 
 TEST (IO, LoadSequentialFile)
 
 TEST (IO, Resource)
 
 TEST (IO, PrivateMmapStream)
 
 TEST (Monitor, Logging)
 
 TEST (Numeric, PartialSum)
 
 TEST (Numeric, Reduce)
 
 TEST (OptionalWeight, Basic)
 
 TEST (PartitionBuilder, BasicTest)
 
template<typename Distribution >
void RunDistributionGenericTest ()
 
 TEST (ProbabilityDistribution, DistributionGeneric)
 
 TEST (ProbabilityDistribution, NormalDist)
 
 TEST (ProbabilityDistribution, LogisticDist)
 
 TEST (ProbabilityDistribution, ExtremeDist)
 
 TEST (Quantile, LoadBalance)
 
 TEST (Quantile, DistributedBasic)
 
 TEST (Quantile, Distributed)
 
 TEST (Quantile, SortedDistributedBasic)
 
 TEST (Quantile, SortedDistributed)
 
 TEST (Quantile, ColSplitBasic)
 
 TEST (Quantile, ColSplit)
 
 TEST (Quantile, ColSplitSortedBasic)
 
 TEST (Quantile, ColSplitSorted)
 
 TEST (Quantile, SameOnAllWorkers)
 
template<typename Fn >
void RunWithSeedsAndBins (size_t rows, Fn fn)
 
 TEST (QuantileLossParam, Basic)
 
 TEST (ColumnSampler, Test)
 
 TEST (ColumnSampler, ThreadSynchronisation)
 
 TEST (ColumnSampler, WeightedSampling)
 
 TEST (ColumnSampler, WeightedMultiSampling)
 
 TEST (RefResourceView, Basic)
 
 TEST (RefResourceView, IO)
 
 TEST (RefResourceView, IOAligned)
 
 TEST (Span, TestStatus)
 
 TEST (Span, DlfConstructors)
 
 TEST (Span, FromNullPtr)
 
 TEST (Span, FromPtrLen)
 
 TEST (SpanDeathTest, FromPtrLen)
 
 TEST (Span, FromFirstLast)
 
 TEST (Span, FromOther)
 
 TEST (Span, FromArray)
 
 TEST (Span, FromContainer)
 
 TEST (Span, Assignment)
 
 TEST (SpanIter, Construct)
 
 TEST (SpanIter, Ref)
 
 TEST (SpanIter, Calculate)
 
 TEST (SpanIter, Compare)
 
 TEST (Span, BeginEnd)
 
 TEST (Span, RBeginREnd)
 
 TEST (Span, ElementAccess)
 
 TEST (SpanDeathTest, ElementAccess)
 
 TEST (Span, Obversers)
 
 TEST (Span, FrontBack)
 
 TEST (SpanDeathTest, FrontBack)
 
 TEST (Span, FirstLast)
 
 TEST (SpanDeathTest, FirstLast)
 
 TEST (Span, Subspan)
 
 TEST (SpanDeathTest, Subspan)
 
 TEST (Span, Compare)
 
 TEST (Span, AsBytes)
 
 TEST (Span, AsWritableBytes)
 
 TEST (Span, Empty)
 
 TEST (SpanDeathTest, Empty)
 
 TEST (IterSpan, Basic)
 
 TEST (Stats, Quantile)
 
 TEST (Stats, WeightedQuantile)
 
 TEST (Stats, Median)
 
 TEST (Stats, Mean)
 
 TEST (AFTLoss, RobustGradientPair)
 
 TEST (ParallelFor2d, CreateBlockedSpace2d)
 
 TEST (ParallelFor2d, Test)
 
 TEST (ParallelFor2d, NonUniform)
 
 TEST (ParallelFor, Basic)
 
 TEST (IndexTransformIter, Basic)
 
 TEST (Transform, DeclareUnifiedTest(Basic))
 
 TEST (TransformDeathTest, Exception)
 
 TEST (AFTLoss, Uncensored)
 
 TEST (AFTLoss, LeftCensored)
 
 TEST (AFTLoss, RightCensored)
 
 TEST (AFTLoss, IntervalCensored)
 
void CheckDeterministicMetricElementWise (StringView name, int32_t device)
 
void VerifyAFTNegLogLik (DataSplitMode data_split_mode=DataSplitMode::kRow)
 
void VerifyIntervalRegressionAccuracy (DataSplitMode data_split_mode=DataSplitMode::kRow)
 
 TEST (Objective, DeclareUnifiedTest(AFTObjConfiguration))
 
 TEST (Objective, DeclareUnifiedTest(AFTObjGPairUncensoredLabels))
 
 TEST (Objective, DeclareUnifiedTest(AFTObjGPairLeftCensoredLabels))
 
 TEST (Objective, DeclareUnifiedTest(AFTObjGPairRightCensoredLabels))
 
 TEST (Objective, DeclareUnifiedTest(AFTObjGPairIntervalCensoredLabels))
 
 TEST_F (FederatedMetricTest, AFTNegLogLikRowSplit)
 
 TEST_F (FederatedMetricTest, AFTNegLogLikColumnSplit)
 
 TEST_F (FederatedMetricTest, IntervalRegressionAccuracyRowSplit)
 
 TEST_F (FederatedMetricTest, IntervalRegressionAccuracyColumnSplit)
 

Variables

constexpr std::size_t dynamic_extent = std::numeric_limits<std::size_t>::max()
 
constexpr double kPI = 3.14159265358979323846
 Constant PI.
 
constexpr double kEulerMascheroni = 0.57721566490153286060651209008240243104215933593992
 The Euler-Mascheroni_constant.
 
constexpr size_t kBlockThreads = 256
 

Detailed Description

Copyright 2017-2023, XGBoost Contributors.

Copyright 2018-2023, XGBoost contributors.

Copyright 2020-2023 by XGBoost contributors.

Copyright 2019-2023, XGBoost Contributors.

Copyright 2018-2023 by XGBoost Contributors.

Copyright 2023, XGBoost Contributors.

Copyright 2022-2023 by XGBoost Contributors.

Copyright 2022-2023 by XGBoost contributors.

Copyright 2019-2023, by XGBoost Contributors.

Copyright 2015-2023 by Contributors.

Utility for fast column-wise access

Function Documentation

◆ Decision()

XGBOOST_DEVICE bool xgboost::common::Decision ( common::Span< CatBitField::value_type const >  cats,
float  cat 
)
inline

Whether should it traverse to left branch of a tree.

Go to left if it's NOT the matching category, which matches one-hot encoding.

◆ FileExtension()

std::string xgboost::common::FileExtension ( std::string  fname,
bool  lower = true 
)

Get file extension from file name.

Parameters
lowerReturn in lower case.
Returns
File extension without the .

◆ FindMaxIndex()

template<typename Iterator >
XGBOOST_DEVICE Iterator xgboost::common::FindMaxIndex ( Iterator  begin,
Iterator  end 
)
inline

Find the maximum iterator within the iterators.

Parameters
beginThe beginning iterator.
endThe end iterator.
Returns
the iterator point to the maximum value.
Template Parameters
IteratorThe type of the iterator.

◆ GetCfsCPUCount()

std::int32_t xgboost::common::GetCfsCPUCount ( )
noexcept

Get thread limit from CFS.

This function has non-trivial overhead and should not be called repeatly.

◆ GetCGroupV1Count()

std::int32_t xgboost::common::GetCGroupV1Count ( std::filesystem::path const &  quota_path,
std::filesystem::path const &  peroid_path 
)

Modified from github.com/psiha/sweater/blob/master/include/boost/sweater/hardware_concurrency.hpp.

MIT License: Copyright (c) 2016 Domagoj Šarić

◆ IOAlignment()

constexpr std::size_t xgboost::common::IOAlignment ( )
constexpr
Parameters
Alignmentfor resource read stream and aligned write stream.

◆ LoadSequentialFile()

std::string xgboost::common::LoadSequentialFile ( std::string  uri,
bool  stream = false 
)

Helper function for loading consecutive file to avoid dmlc Stream when possible.

Parameters
uriURI or file name to file.
streamUse dmlc Stream unconditionally if set to true. Used for running test without remote filesystem.
Returns
File content.

◆ LogSum() [1/2]

float xgboost::common::LogSum ( float  x,
float  y 
)
inline

perform numerically safe logsum

Parameters
xleft input operand
yright input operand
Returns
log(exp(x) + exp(y))

◆ LogSum() [2/2]

template<typename Iterator >
float xgboost::common::LogSum ( Iterator  begin,
Iterator  end 
)
inline

perform numerically safe logsum

Parameters
beginThe beginning iterator.
endThe end iterator.
Returns
the iterator point to the maximum value.
Template Parameters
IteratorThe type of the iterator.

◆ PartialSum()

template<typename InIt , typename OutIt , typename T >
void xgboost::common::PartialSum ( int32_t  n_threads,
InIt  begin,
InIt  end,
init,
OutIt  out_it 
)

Varient of std::partial_sum, out_it should point to a container that has n + 1 elements.

Useful for constructing a CSR indptr.

◆ Quantile()

template<typename Iter >
float xgboost::common::Quantile ( Context const *  ctx,
double  alpha,
Iter const &  begin,
Iter const &  end 
)

Quantile using linear interpolation.

https://www.itl.nist.gov/div898/handbook/prc/section2/prc262.htm

Parameters
alphaQuantile, must be in range [0, 1].
beginIterator begin for input array.
endIterator end for input array.
Returns
The result of interpolation.

◆ ReadVec()

template<typename Vec >
bool xgboost::common::ReadVec ( common::AlignedResourceReadStream fi,
Vec *  vec 
)

Read a vector from stream.

Accepts both std::vector and RefResourceView.

If the output vector is a referenced counted view, no copying occur.

◆ Sigmoid()

XGBOOST_DEVICE float xgboost::common::Sigmoid ( float  x)
inline

calculate the sigmoid of the input.

Parameters
xinput parameter
Returns
the transformed value.

◆ SketchOnDMatrix()

HistogramCuts xgboost::common::SketchOnDMatrix ( Context const *  ctx,
DMatrix m,
bst_bin_t  max_bins,
bool  use_sorted = false,
Span< float const >  hessian = {} 
)

Run CPU sketching on DMatrix.

Parameters
use_sortedWhether should we use SortedCSC for sketching, it's more efficient but consumes more memory.

◆ Softmax()

template<typename Iterator >
XGBOOST_DEVICE void xgboost::common::Softmax ( Iterator  start,
Iterator  end 
)
inline

Do inplace softmax transformaton on start to end.

Template Parameters
IteratorInput iterator type
Parameters
startStart iterator of input
endend iterator of input

◆ Split()

std::vector< std::string > xgboost::common::Split ( const std::string &  s,
char  delim 
)
inline

Split a string by delimiter.

Parameters
sString to be split.
delimThe delimiter.

◆ TEST() [1/2]

xgboost::common::TEST ( ProbabilityDistribution  ,
ExtremeDist   
)

Enforce known properties of the extreme distribution (also known as Gumbel distribution). The mean is the negative of the Euler-Mascheroni constant. The variance is 1/6 * pi**2. (https://mathworld.wolfram.com/GumbelDistribution.html)

◆ TEST() [2/2]

xgboost::common::TEST ( ProbabilityDistribution  ,
LogisticDist   
)

Enforce known properties of the logistic distribution. (https://en.wikipedia.org/wiki/Logistic_distribution)

◆ TestCategoricalSketch()

template<typename Fn >
void xgboost::common::TestCategoricalSketch ( size_t  n,
size_t  num_categories,
int32_t  num_bins,
bool  weighted,
Fn  sketch 
)

Test for sketching on categorical data.

Parameters
sketchSketch function, can be on device or on host.

◆ VerifyAFTNegLogLik()

void xgboost::common::VerifyAFTNegLogLik ( DataSplitMode  data_split_mode = DataSplitMode::kRow)
inline

Test aggregate output from the AFT metric over a small test data set. This is unlike AFTLoss.* tests, which verify metric values over individual data points.

◆ WeightedQuantile()

template<typename Iter , typename WeightIter >
float xgboost::common::WeightedQuantile ( Context const *  ctx,
double  alpha,
Iter  begin,
Iter  end,
WeightIter  w_begin 
)

Calculate the weighted quantile with step function.

Unlike the unweighted version, no interpolation is used.

See https://aakinshin.net/posts/weighted-quantiles/ for some discussions on computing weighted quantile with interpolation.

◆ WriteVec()

template<typename Vec >
std::size_t xgboost::common::WriteVec ( AlignedFileWriteStream fo,
Vec const &  vec 
)

Write a vector to stream.

Accepts both std::vector and RefResourceView.