dmlc::Parser< IndexType, DType > Class Template Reference

parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once More...

#include <data.h>

Inheritance diagram for dmlc::Parser< IndexType, DType >:

Public Types
typedef Parser< IndexType, DType > *(*	Factory) (const std::string &path, const std::map< std::string, std::string > &args, unsigned part_index, unsigned num_parts)
	Factory type of the parser.

Public Member Functions
virtual size_t	BytesRead (void) const =0
Parser< uint32_t, real_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Parser< uint64_t, real_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Parser< uint32_t, int32_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Parser< uint64_t, int32_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Parser< uint32_t, int64_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Parser< uint64_t, int64_t > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
Public Member Functions inherited from dmlc::DataIter< DType >
virtual	~DataIter (void) DMLC_THROW_EXCEPTION
	destructor
virtual void	BeforeFirst (void)=0
	set before first of the item
virtual bool	Next (void)=0
	move to next item
virtual const DType &	Value (void) const =0
	get current data

Static Public Member Functions
static Parser< IndexType, DType > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
	create a new instance of parser based on the "type"

Detailed Description

template<typename IndexType, typename DType = real_t>
class dmlc::Parser< IndexType, DType >

parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once

See also

RowBlockIter

Template Parameters

IndexType	type of index in RowBlock
DType	type of label and value in RowBlock Create function was only implemented for IndexType uint64_t and uint32_t and DType real_t and int

Member Function Documentation

BytesRead()

template<typename IndexType , typename DType = real_t>

virtual size_t dmlc::Parser< IndexType, DType >::BytesRead ( void  ) const

pure virtual

Returns

size of bytes read so far

Implemented in dmlc::data::DensifyParser< IndexType >, dmlc::data::ParquetParser< IndexType, DType >, dmlc::data::TextParserBase< IndexType, DType >, dmlc::data::TextParserBase< IndexType, real_t >, dmlc::data::ParserImpl< IndexType, DType >, and dmlc::data::ParserImpl< IndexType, real_t >.

Create()

template<typename IndexType , typename DType = real_t>

static Parser< IndexType, DType > * dmlc::Parser< IndexType, DType >::Create ( const char *  uri_, unsigned  part_index, unsigned  num_parts, const char *  type  )

static

create a new instance of parser based on the "type"

Parameters

uri_	the uri of the input, can contain hdfs prefix
part_index	the part id of current input
num_parts	total number of splits
type	type of dataset can be: "libsvm", "auto", ...

When "auto" is passed, the type is decided by format argument string in URI.

Returns

the created parser

---

The documentation for this class was generated from the following file:

• External/xgboost/dmlc-core/include/dmlc/data.h