StreamConversionHelper< SRCTYPE, DSTTYPE > Class Template Referenceabstract

#include <streamconversion_helper.h>

Detailed Description

template<typename SRCTYPE, typename DSTTYPE>
class maxon::StreamConversionHelper< SRCTYPE, DSTTYPE >

This class implements a helper function for sequential access when using stream conversions Usually with stream conversions you get an arbitrary number of bytes within the ConvertImpl callback. This means you cannot longer read data in a classic linear fashion since you might not get enough bytes to finish your reading.

For example something simple like

ReadBytes(buf, 4)

is not possible whithout checking for the remaning bytes left. In case of insufficient number of available bytes you have to store your current variable heap and continue reading in subsequent calls of ConvertImpl.

While this might be possible for simple algorithms it can get quite complex or even imposible when trying to read more elaborate data files which are for example arranged in hierarchies with chunks or subchunks. Or think about using an external complex library for accessing a data format like Alembic or TIFF. In this case you can not change the code structure - only replacing a ReadBytes access is possible.

StreamConversionHelper frees you from dealing with stream bytes deliverd in arbitrary chunks. First you simply overlad StreamConversionHelper template by specifying your source and dest datatype (here both Char) and fill the DoIt routine with your normal code e.g. reading the file (in this example decompressing a LZ4 stream)

class Lz4StreamConversionHelper : public StreamConversionHelper<Char, Char>
{
public:
virtual Result<void> DoIt()
{
// ... read your data here
Lz4Decompress();
}
};

In this example if you want to read 4 bytes (aka Chars) within the DoIt routine you simply write

Char buf[4];

Second you put the helper object somewhere on the stack - ideally as a static variable in your main StreamConversion class and initialize it everytime ConvertImpl is called by simply calling AppendStream - boom you are done.

class Lz4BaseImpl : public Component<Lz4BaseImpl, StreamConversionInterface>
{
public:
Result<Int> ConvertImpl(const Block<const Generic>& xsrc, WritableArrayInterface<Generic>& xdst, Int dstLimitHint, Bool inputFinished, Bool& outputFinished)
{
const Block<const Char>& src = reinterpret_cast<const Block<const Char>&> (xsrc);
WritableArrayInterface<Char>& dst = reinterpret_cast<WritableArrayInterface<Char>&>(xdst);
helper.AppendStream(src, dst, inputFinished) iferr_return;
if (inputFinished)
outputFinished = true;
return helper.GetCurrentPosition();
}
protected:
Lz4StreamConversionHelper helper;
};

Public Member Functions

Result< void > AppendStream (const Block< const SRCTYPE > &src, WritableArrayInterface< DSTTYPE > &dst, Bool inputFinished)
 
Result< void > Write (DSTTYPE *buf, Int count)
 
Result< void > WriteWithOffset (DSTTYPE *buf, Int count, Int offset)
 
Result< void > Read (SRCTYPE *buf, Int count)
 
Result< IntReadEOS (SRCTYPE *buf, Int maxCount)
 
Result< void > Skip (Int count)
 
MAXON_WARN_UNUSED Int GetCurrentPosition ()
 
MAXON_WARN_UNUSED WritableArrayInterface< DSTTYPE > * GetDst ()
 
MAXON_WARN_UNUSED Bool EndOfStreamReached ()
 

Protected Member Functions

virtual Result< void > DoIt ()=0
 
virtual ~StreamConversionHelper ()
 

Private Member Functions

Result< void > SignalEmitterThreadToContinue ()
 
Result< void > SignalCollectorThreadToContinue ()
 

Private Attributes

ThreadRef _thread
 
ConditionVariableRef _inputAvailable
 
ConditionVariableRef _outputAvailable
 
Error _returnErrorVar
 
Bool _DoItDone
 
const Block< const SRCTYPE > * _src
 
WritableArrayInterface< DSTTYPE > * _dst
 
Int _seg_pos
 
Int _seg_count
 
Bool _inputFinished
 

Friends

template<typename SRCTYPE2 , typename DSTTYPE2 >
class CollectorThread
 

Constructor & Destructor Documentation

◆ ~StreamConversionHelper()

virtual ~StreamConversionHelper ( )
protectedvirtual

Member Function Documentation

◆ AppendStream()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > AppendStream ( const Block< const SRCTYPE > &  src,
WritableArrayInterface< DSTTYPE > &  dst,
Bool  inputFinished 
)

Adds a new part of the stream to the existing one. Has to be called each time ConvertImpl is executed

Parameters
[in]srcinput stream
[in]dstoutput stream
[in]inputFinishedsignal from ConvertImpl if this is the last part of the stream

Example: You put the helper object somewhere on the stack - ideally as a static variable in your main StreamConversion class and initialize it everytime ConvertImpl is called by simply calling AppendStream. Here a LZ4 stream of Chars as input is decoded.

class Lz4BaseImpl : public Component<Lz4BaseImpl, StreamConversionInterface>
{
public:
Result<Int> ConvertImpl(const Block<const Generic>& xsrc, WritableArrayInterface<Generic>& xdst, Int dstLimitHint, Bool inputFinished, Bool& outputFinished)
{
const Block<const Char>& src = reinterpret_cast<const Block<const Char>&> (xsrc);
WritableArrayInterface<Char>& dst = reinterpret_cast<WritableArrayInterface<Char>&>(xdst);
helper.AppendStream(src, dst, inputFinished) iferr_return;
if (inputFinished)
outputFinished = true;
return helper.GetCurrentPosition();
}
protected:
Lz4StreamConversionHelper helper;
};

◆ Write()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Write ( DSTTYPE *  buf,
Int  count 
)

Append elements to the internal destination dst buffer specified in AppendStream.

Parameters
[in]bufsource buffer
[in]countnumber of elements to be copied from buf to dst

◆ WriteWithOffset()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > WriteWithOffset ( DSTTYPE *  buf,
Int  count,
Int  offset 
)

Simply copy elements to the internal destination dst buffer specified in AppendStream. This function is especially useful for speeding things up when the final size oft dst ist know. Instead of enlarging dst with each Write call this i just a plain Memcopy Note: you to have to garantue that the dest buffer is sufficiently large enough

Parameters
[in]bufsource buffer
[in]countnumber of elements to be copied from buf to dst
[in]offsetnumber of elements to be copied from buf to dst

◆ Read()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Read ( SRCTYPE *  buf,
Int  count 
)

Read elements from internal src buffer specified in AppendStream to buf.

Parameters
[in]bufdestination buffer
[in]countnumber of elements to be copied from src to buf

◆ ReadEOS()

MAXON_ATTRIBUTE_FORCE_INLINE Result< Int > ReadEOS ( SRCTYPE *  buf,
Int  maxCount 
)

Read elements from internal src buffer specified in AppendStream to buf. Contrary to Read this routine does not read a specific number of elements but reads untile either the end of the stream is reached of the maxCount number.

Parameters
[in]bufdestination buffer
[in]maxCountmax number of elements to be read
Returns
number of elements actually read

◆ Skip()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > Skip ( Int  count)

Skip elements

Parameters
[in]countnumber of elements

◆ GetCurrentPosition()

MAXON_ATTRIBUTE_FORCE_INLINE Int GetCurrentPosition

Current reading position. Note that this is a relative position to the part of the stream that was initialized with AppendStream.

Returns
Current reading position

◆ GetDst()

Direct access to the destination array

Returns
pointer to destination array

◆ EndOfStreamReached()

MAXON_ATTRIBUTE_FORCE_INLINE Bool EndOfStreamReached

Check for end of stream condition.

Returns
True if end is reached.

◆ DoIt()

virtual Result<void> DoIt ( )
protectedpure virtual

you have to overload this function here you fill in your code to process the data of the stream

◆ SignalEmitterThreadToContinue()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > SignalEmitterThreadToContinue
private

◆ SignalCollectorThreadToContinue()

MAXON_ATTRIBUTE_FORCE_INLINE Result< void > SignalCollectorThreadToContinue
private

Friends And Related Function Documentation

◆ CollectorThread

friend class CollectorThread
friend

Member Data Documentation

◆ _thread

ThreadRef _thread
private

◆ _inputAvailable

ConditionVariableRef _inputAvailable
private

◆ _outputAvailable

ConditionVariableRef _outputAvailable
private

◆ _returnErrorVar

Error _returnErrorVar
private

◆ _DoItDone

Bool _DoItDone
private

◆ _src

const Block<const SRCTYPE>* _src
private

◆ _dst

WritableArrayInterface<DSTTYPE>* _dst
private

◆ _seg_pos

Int _seg_pos
private

◆ _seg_count

Int _seg_count
private

◆ _inputFinished

Bool _inputFinished
private
Int
maxon::Int Int
Definition: ge_sys_math.h:62
MAXON_COMPONENT
#define MAXON_COMPONENT(KIND,...)
Definition: objectbase.h:2036
iferr_return
#define iferr_return
Definition: resultbase.h:1434
maxon::StreamConversionHelper::DoIt
virtual Result< void > DoIt()=0
maxon::src
const T & src
Definition: apibase.h:2525
iferr_scope
#define iferr_scope
Definition: resultbase.h:1343
Bool
maxon::Bool Bool
Definition: ge_sys_math.h:53
Char
maxon::Char Char
Definition: ge_sys_math.h:54
maxon::Component
ComponentWithBase< C, ComponentRoot, INTERFACES... > Component
Definition: objectbase.h:2604
maxon::StreamConversionHelper::Read
Result< void > Read(SRCTYPE *buf, Int count)
Definition: streamconversion_helper.h:339