StreamConversionHelper< SRCTYPE, DSTTYPE > Class Template Reference

#include <streamconversion_helper.h>

Detailed Description

template<typename SRCTYPE, typename DSTTYPE>
class maxon::StreamConversionHelper< SRCTYPE, DSTTYPE >

This class implements a helper function for sequential access when using stream conversions Usually with stream conversions you get an arbitrary number of bytes within the ConvertImpl callback. This means you cannot longer read data in a classic linear fashion since you might not get enough bytes to finish your reading.

For example something simple like

ReadBytes(buf, 4) 

is not possible whithout checking for the remaning bytes left. In case of insufficient number of available bytes you have to store your current variable heap and continue reading in subsequent calls of ConvertImpl.

While this might be possible for simple algorithms it can get quite complex or even imposible when trying to read more elaborate data files which are for example arranged in hierarchies with chunks or subchunks. Or think about using an external complex library for accessing a data format like Alembic or TIFF. In this case you can not change the code structure - only replacing a ReadBytes access is possible.

StreamConversionHelper frees you from dealing with stream bytes deliverd in arbitrary chunks. First you simply overlad StreamConversionHelper template by specifying your source and dest datatype (here both Char) and fill the DoIt routine with your normal code e.g. reading the file (in this example decompressing a LZ4 stream)

class Lz4StreamConversionHelper : public StreamConversionHelper<Char, Char>
{
public:
  virtual Result<void> DoIt()
  {
    // ... read your data here
    Lz4Decompress();
  }
};

In this example if you want to read 4 bytes (aka Chars) within the DoIt routine you simply write

Char buf[4];
Read(buf, 4) iferr_return;

Second you put the helper object somewhere on the stack - ideally as a static variable in your main StreamConversion class and initialize it everytime ConvertImpl is called by simply calling AppendStream - boom you are done.

class Lz4BaseImpl : public Component<Lz4BaseImpl, StreamConversionInterface>
{
  MAXON_COMPONENT();
 
public:
  Result<Int> ConvertImpl(const Block<const Generic>& xsrc, WritableArrayInterface<Generic>& xdst, Int dstLimitHint, Bool inputFinished, Bool& outputFinished)
  {
    iferr_scope;
 
    const Block<const Char>&      src = reinterpret_cast<const Block<const Char>&>     (xsrc);
    WritableArrayInterface<Char>& dst = reinterpret_cast<WritableArrayInterface<Char>&>(xdst);
 
    helper.AppendStream(src, dst, inputFinished) iferr_return;
 
    if (inputFinished)
      outputFinished = true;
 
    return helper.GetCurrentPosition();
  }
 
protected:
  Lz4StreamConversionHelper helper;
};

Public Member Functions
Result< void >	AppendStream (const Block< const SRCTYPE > &src, WritableArrayInterface< DSTTYPE > &dst, Bool inputFinished)
Result< void >	Write (DSTTYPE *buf, Int count)
Result< void >	WriteWithOffset (DSTTYPE *buf, Int count, Int offset)
Result< void >	Read (SRCTYPE *buf, Int count)
Result< Int >	ReadEOS (SRCTYPE *buf, Int maxCount)
Result< void >	Skip (Int count)
MAXON_WARN_UNUSED Int	GetCurrentPosition ()
MAXON_WARN_UNUSED WritableArrayInterface< DSTTYPE > *	GetDst ()
MAXON_WARN_UNUSED Bool	EndOfStreamReached ()

Protected Member Functions
virtual Result< void >	DoIt ()=0
virtual	~StreamConversionHelper ()

Private Member Functions
Result< void >	SignalEmitterThreadToContinue ()
Result< void >	SignalCollectorThreadToContinue ()

Private Attributes
ThreadRef	_thread
ConditionVariableRef	_inputAvailable
ConditionVariableRef	_outputAvailable
Error	_returnErrorVar
Bool	_DoItDone
const Block< const SRCTYPE > *	_src
WritableArrayInterface< DSTTYPE > *	_dst
Int	_seg_pos
Int	_seg_count
Bool	_inputFinished

Friends
template<typename SRCTYPE2 , typename DSTTYPE2 >
class	CollectorThread

Constructor & Destructor Documentation

~StreamConversionHelper()

virtual ~StreamConversionHelper ( )

protectedvirtual

Member Function Documentation

AppendStream()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > AppendStream	(	const Block< const SRCTYPE > &	src,
		WritableArrayInterface< DSTTYPE > &	dst,
		Bool	inputFinished
	)

Adds a new part of the stream to the existing one. Has to be called each time ConvertImpl is executed

Parameters

[in]	src	input stream
[in]	dst	output stream
[in]	inputFinished	signal from ConvertImpl if this is the last part of the stream

Example: You put the helper object somewhere on the stack - ideally as a static variable in your main StreamConversion class and initialize it everytime ConvertImpl is called by simply calling AppendStream. Here a LZ4 stream of Chars as input is decoded.

class Lz4BaseImpl : public Component<Lz4BaseImpl, StreamConversionInterface>
{
  MAXON_COMPONENT();
 
public:
  Result<Int> ConvertImpl(const Block<const Generic>& xsrc, WritableArrayInterface<Generic>& xdst, Int dstLimitHint, Bool inputFinished, Bool& outputFinished)
  {
    iferr_scope;
 
    const Block<const Char>&      src = reinterpret_cast<const Block<const Char>&>     (xsrc);
    WritableArrayInterface<Char>& dst = reinterpret_cast<WritableArrayInterface<Char>&>(xdst);
 
    helper.AppendStream(src, dst, inputFinished) iferr_return;
 
    if (inputFinished)
      outputFinished = true;
 
    return helper.GetCurrentPosition();
  }
 
protected:
  Lz4StreamConversionHelper helper;
};

Write()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Write	(	DSTTYPE *	buf,
		Int	count
	)

Append elements to the internal destination dst buffer specified in AppendStream.

Parameters

[in]	buf	source buffer
[in]	count	number of elements to be copied from buf to dst

WriteWithOffset()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > WriteWithOffset	(	DSTTYPE *	buf,
		Int	count,
		Int	offset
	)

Simply copy elements to the internal destination dst buffer specified in AppendStream. This function is especially useful for speeding things up when the final size oft dst ist know. Instead of enlarging dst with each Write call this i just a plain Memcopy Note: you to have to garantue that the dest buffer is sufficiently large enough

Parameters

[in]	buf	source buffer
[in]	count	number of elements to be copied from buf to dst
[in]	offset	number of elements to be copied from buf to dst

Read()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Read	(	SRCTYPE *	buf,
		Int	count
	)

Read elements from internal src buffer specified in AppendStream to buf.

Parameters

[in]	buf	destination buffer
[in]	count	number of elements to be copied from src to buf

ReadEOS()

MAXON_ATTRIBUTE_FORCE_INLINE Result< Int > ReadEOS	(	SRCTYPE *	buf,
		Int	maxCount
	)

Read elements from internal src buffer specified in AppendStream to buf. Contrary to Read this routine does not read a specific number of elements but reads untile either the end of the stream is reached of the maxCount number.

Parameters

[in]	buf	destination buffer
[in]	maxCount	max number of elements to be read

Returns

number of elements actually read

Skip()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Skip

(

Int

count

)

Skip elements

Parameters

[in]

count

number of elements

GetCurrentPosition()

MAXON_ATTRIBUTE_FORCE_INLINE Int GetCurrentPosition

Current reading position. Note that this is a relative position to the part of the stream that was initialized with AppendStream.

Returns

Current reading position

GetDst()

MAXON_WARN_UNUSED WritableArrayInterface< DSTTYPE > * GetDst

Direct access to the destination array

Returns

pointer to destination array

EndOfStreamReached()

MAXON_ATTRIBUTE_FORCE_INLINE Bool EndOfStreamReached

Check for end of stream condition.

Returns

True if end is reached.

DoIt()

virtual Result<void> DoIt ( )

protectedpure virtual

you have to overload this function here you fill in your code to process the data of the stream

SignalEmitterThreadToContinue()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > SignalEmitterThreadToContinue

private

SignalCollectorThreadToContinue()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > SignalCollectorThreadToContinue

private

Friends And Related Function Documentation

CollectorThread

friend class CollectorThread

friend

Member Data Documentation

_thread

ThreadRef _thread

private

_inputAvailable

ConditionVariableRef _inputAvailable

private

_outputAvailable

ConditionVariableRef _outputAvailable

private

_returnErrorVar

Error _returnErrorVar

private

_DoItDone

Bool _DoItDone

private

_src

const Block<const SRCTYPE>* _src

private

_dst

WritableArrayInterface<DSTTYPE>* _dst

private

_seg_pos

Int _seg_pos

private

_seg_count

Int _seg_count

private

_inputFinished

Bool _inputFinished

private