dmlc::RecordIOWriter Class Reference

writer of binary recordio binary format for recordio recordio format: magic lrecord data pad More...

#include <recordio.h>

Public Member Functions
	RecordIOWriter (Stream *stream)
	constructor
void	WriteRecord (const void *buf, size_t size)
	write record to the stream
void	WriteRecord (const std::string &data)
	write record to the stream
size_t	except_counter (void) const
size_t	Tell (void)
	tell the current position of the input stream

Static Public Member Functions
static uint32_t	EncodeLRec (uint32_t cflag, uint32_t length)
	encode the lrecord
static uint32_t	DecodeFlag (uint32_t rec)
	decode the flag part of lrecord
static uint32_t	DecodeLength (uint32_t rec)
	decode the length part of lrecord

Static Public Attributes
static const uint32_t	kMagic = 0xced7230a
	magic number of recordio note: (kMagic >> 29U) & 7 > 3 this ensures lrec will not be kMagic

Detailed Description

writer of binary recordio binary format for recordio recordio format: magic lrecord data pad

• magic is magic number
• pad is simply a padding space to make record align to 4 bytes
• lrecord encodes length and continue bit
  • data.length() = (lrecord & (1U<<29U - 1));
  • cflag == (lrecord >> 29U) & 7;

cflag was used to handle (rare) special case when magic number occured in the data sequence.

In such case, the data is splitted into multiple records by the cells of magic number

(1) cflag == 0: this is a complete record; (2) cflag == 1: start of a multiple-rec; cflag == 2: middle of multiple-rec; cflag == 3: end of multiple-rec

Constructor & Destructor Documentation

RecordIOWriter()

dmlc::RecordIOWriter::RecordIOWriter ( Stream *  stream )

inlineexplicit

constructor

Parameters

stream

the stream to be constructed

Member Function Documentation

DecodeFlag()

static uint32_t dmlc::RecordIOWriter::DecodeFlag ( uint32_t  rec )

inlinestatic

decode the flag part of lrecord

Parameters

rec	the lrecord

Returns

the flag

DecodeLength()

static uint32_t dmlc::RecordIOWriter::DecodeLength ( uint32_t  rec )

inlinestatic

decode the length part of lrecord

Parameters

rec	the lrecord

Returns

the length

EncodeLRec()

static uint32_t dmlc::RecordIOWriter::EncodeLRec ( uint32_t  cflag, uint32_t  length  )

inlinestatic

encode the lrecord

Parameters

cflag	cflag part of the lrecord
length	length part of lrecord

Returns

the encoded data

except_counter()

size_t dmlc::RecordIOWriter::except_counter ( void  ) const

inline

Returns

number of exceptions(occurance of magic number) during the writing process

WriteRecord() [1/2]

void dmlc::RecordIOWriter::WriteRecord ( const std::string &  data )

inline

write record to the stream

Parameters

data	the data to write out

WriteRecord() [2/2]

void dmlc::RecordIOWriter::WriteRecord	(	const void *	buf,
		size_t	size
	)

write record to the stream

Parameters

buf	the buffer of memory region
size	the size of record to write out

---

The documentation for this class was generated from the following files:

• External/xgboost/dmlc-core/include/dmlc/recordio.h
• External/xgboost/dmlc-core/src/recordio.cc