Stream Conversions Manual

Table of Contents

• About
• StreamConversionInterface
  • Conversion
  • Streams
  • Properties
• Conversions
  • Hex
  • UTF
  • Base64
  • Cryptography
  • Hashes
  • Compression
• Further Reading

About

The maxon::StreamConversionInterface provides a generic interface for any kind of data conversion. This includes string encoding, compression, encryption, hashing etc.

StreamConversionInterface

Conversion

maxon::StreamConversionInterface is used to convert the given source data. The result of this conversion is typically stored in a given array.

• maxon::StreamConversionInterface::Convert(): Converts (part of) the source data.
• maxon::StreamConversionInterface::ConvertAll(): Converts the complete source data.
• maxon::StreamConversionInterface::SupportInplaceConversion(): Returns true if the conversion supports in-place conversion. Typically only supported by encryption algorithms.
• maxon::StreamConversionInterface::ConvertAllInplace(): Converts the complete source data and overwrites it.

Note

A maxon::StreamConversionInterface object can be used only once.

 // This example converts the given string data into Base64 using Convert().
 
 // get data
 
 const maxon::String sourceText("Hello World");
 const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;
 
 // encode
 
 // Base64 data is c. 1.4 times bigger than the input data
 const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>((maxon::Float32)source.GetCount() * 1.4);
 
 maxon::BaseArray<maxon::Char> destination;
 maxon::Bool lastPart;
 const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
 base64encoder.Convert(source, destination, targetSizeEstimation, true, lastPart) iferr_return;
 
 // print result
 const maxon::String base64string(destination);
 DiagnosticOutput("Base64: @", base64string);

 // This example converts the given string data into Base64 using ConvertAll().
 
 // get data
 
 const maxon::String sourceText("Hello World");
 const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;
 
 // encode
 
 // Base64 data is c. 1.4 times bigger than the input data
 const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>((maxon::Float32)source.GetCount() * 1.4);
 
 maxon::BaseArray<maxon::Char> destination;
 const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
 base64encoder.ConvertAll(source, destination, targetSizeEstimation) iferr_return;
 
 // print result
 const maxon::String base64string(destination);
 DiagnosticOutput("Base64: @", base64string);

Streams

A given conversion instance can be converted to an input stream. See InputStream Manual.

• maxon::StreamConversionInterface::ConvertToStream(): Returns an input stream.
• maxon::StreamConversionInterface::ConvertToReader(): Returns a DataFormatBaseReaderRef.
• maxon::StreamConversionInterface::ConvertToWriter(): Returns a DataFormatBaseWriterRef.

 // This example converts the given text to Base64 using input streams.
 
 // source stream reads from memory block
 const auto memblock = maxon::CharToBlock("Hello World");
 maxon::InputStreamRef inputStream = maxon::InputStreamInterface::FromBlock().Create(memblock, false) iferr_return;
 
 // create stream conversion
 const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
 inputStream = base64encoder.ConvertToStream(inputStream) iferr_return;
 
 // prepare buffer to read from stream
 maxon::BaseArray<maxon::Char> buffer;
 const maxon::Int bufferSize = 1000;
 buffer.Resize(bufferSize) iferr_return;
 
 // read from stream
 const maxon::Int count = inputStream.ReadEOS(buffer) iferr_return;
 
 // convert to string
 const maxon::String string(buffer.GetFirst(), count);
 DiagnosticOutput("Base64: @", string);

Properties

Various properties of the given conversion type can be accessed with:

• maxon::StreamConversionInterface::GetBatchSize(): Returns the recommended working size.
• maxon::StreamConversionInterface::GetBlockSize(): Returns the block size. The encoded/decoded data needs to be a multiple of this value.
• maxon::StreamConversionInterface::GetCounterpart(): Returns the corresponding counter part converter id.
• maxon::StreamConversionInterface::GetSourceType(): Returns the accepted source data type.
• maxon::StreamConversionInterface::GetDestinationType(): Returns the accepted destination data type.

// This example function automatically decodes the given data using the "counterpart" of the given encoder.
 
static maxon::Result<maxon::String> DecodeData(const maxon::StreamConversionRef encoder, maxon::BaseArray<maxon::Char>& data)
{
 iferr_scope;
 
 // check input data
 if (data.GetCount() == 0)
 return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
 // get counterpart (decoder)
 const maxon::Id decoderID = encoder.GetCounterpart();
 const maxon::StreamConversionRef decoder = maxon::StreamConversions::Get(decoderID).Create() iferr_return;
 
 // check if decoder can handle maxon::Char data
 const maxon::DataType charDataType = maxon::GetDataType<maxon::Char>();
 
 if (decoder.GetSourceType() != charDataType)
 return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
 if (decoder.GetDestinationType() != charDataType)
 return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
 // decode data
 maxon::BaseArray<maxon::Char> destination;
 decoder.ConvertAll(data, destination) iferr_return;
 
 // convert to maxon::String
 return maxon::String(destination);
}

Conversions

Existing conversions are registered at the registry maxon::StreamConversions or presented as published objects. Some conversions can be configured by defining some settings in a maxon::DataDictionary that is used with the Create() function (maxon::StreamConversionFactory).

Hex

Arbitrary data can be converted to a hexadecimal representation:

• maxon::StreamConversions::HexEncoder: Hexadecimal encoder.
• maxon::StreamConversions::HexDecoder: Hexadecimal decoder.

See also maxon::GetHexadecimalValue().

 // This example converts the given number to a hexadecimal representation.
 
 // the original number
 const maxon::Int number = 123;
 
 maxon::BaseArray<maxon::Char> source;
 source.Append((maxon::Char)number) iferr_return;
 
 // get encoder
 const maxon::StreamConversionRef encoder = maxon::StreamConversions::HexEncoder().Create() iferr_return;
 
 // convert to HEX
 maxon::BaseArray<maxon::Char> destination;
 encoder.ConvertAll(source, destination) iferr_return;
 
 // print result
 const maxon::String hexString(destination);
 DiagnosticOutput("Hex: @", hexString);

UTF

Text can be converted to different Unicode encodings. See maxon::UTFTEXT_OPTIONS.

• maxon::StreamConversions::UtfTextEncoder: Unicode encoder.
• maxon::StreamConversions::UtfTextDecoder: Unicode decoder.

Additional string conversions are registered at maxon::StringEncodings and maxon::StringDecodings, see stringencoding.h. For string conversions there are also maxon::StringEncodingInterface and maxon::StringDecodingInterface.

Base64

Arbitrary data can be converted to a Base64 representation. See maxon::BASE64_OPTIONS.

• maxon::StreamConversions::Base64Encoder: Base64 encoder.
• maxon::StreamConversions::Base64Decoder: Base64 decoder.
• maxon::StreamConversions::Base64UrlEncoder: Base64 URL encoder.
• maxon::StreamConversions::Base64UrlDecoder: Base64 URL decoder.

 // This example converts some text to Base64 and back.
 
 // prepare data
 maxon::BaseArray<maxon::Char> data;
 
 // encode
 MAXON_SCOPE
 {
 // original text
 const maxon::String message { "Hello World" };
 const maxon::BaseArray<maxon::Char> source = message.GetCString() iferr_return;
 
 // convert to Base64
 const maxon::StreamConversionRef encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
 encoder.ConvertAll(source, data) iferr_return;
 }
 
 // decode
 MAXON_SCOPE
 {
 maxon::BaseArray<maxon::Char> destination;
 
 // convert to plain text
 const maxon::StreamConversionRef decoder = maxon::StreamConversions::Base64Decoder().Create() iferr_return;
 decoder.ConvertAll(data, destination) iferr_return;
 
 // print result
 const maxon::String message(destination);
 DiagnosticOutput("Message: @", message);
 }

Cryptography

Arbitrary data can be encrypted using these algorithms:

• maxon::StreamConversions::AesEncoder: AES encryption.
• maxon::StreamConversions::AesDecoder: AES decryption.
• maxon::StreamConversions::BlowfishEncoder: Blowfish encryption.
• maxon::StreamConversions::BlowfishDecoder: Blowfish decryption.
• maxon::StreamConversions::BlowfishLegacyEncoder: Legacy Blowfish encryption.
• maxon::StreamConversions::BlowfishLegacyDecoder: Legacy Blowfish decryption.
• maxon::StreamConversions::BlowfishLegacyEnDecoder: Legacy Blowfish en/decryption. See maxon::BLOWFISHLEGACYENDECODER_OPTIONS.

The cryptographic key is set as maxon::CryptographyOptions::CRYPTOKEY using maxon::CryptoKeyInterface. To pad a data block with arbitrary data use maxon::SecureRandom.

Encryption algorithms can also be used with the specialized maxon::CryptographyStreamConversionInterface.

 // This example encrypts a text with AES and the given encryption key and decrypts it later.
 
 // prepare encryption settings including the key
 
 const maxon::Id encoderID = maxon::StreamConversions::AesEncoder.GetId();
 const maxon::Int blockSize = 128;
 const maxon::Char* key = "9USGxEPo0Tx6d8ZRRLbpEc4D88xdU2bb";
 const maxon::Int keySize = 128;
 const maxon::CryptoKey cryptoKey(encoderID, blockSize, key, keySize);
 
 maxon::DataDictionary cryptoSettings;
 cryptoSettings.Set(maxon::CryptographyOptions::CRYPTOKEY, cryptoKey) iferr_return;
 
 // data
 maxon::BaseArray<maxon::Char> data;
 
 // encode
 MAXON_SCOPE
 {
 // get data
 const maxon::String message { "Hello World" };
 data = message.GetCString() iferr_return;
 
 // add null for null-terminated string
 data.Append(0) iferr_return;
 
 // encode
 const maxon::StreamConversionRef aesEncoder = maxon::StreamConversions::AesEncoder().Create(cryptoSettings) iferr_return;
 
 // prepare data: resize buffer and fill with random bits
 
 const maxon::Int initialSize = data.GetCount();
 
 const maxon::Int aesBlockSize = aesEncoder.GetBlockSize();
 const maxon::Int targetSize = ((initialSize / blockSize) + 1) * aesBlockSize;
 const maxon::Int diff = targetSize - data.GetCount();
 if (diff > 0)
 {
 data.Resize(targetSize) iferr_return; // resize data
 
 // fill rest of the data with random bits
 maxon::UChar* randomDataStart = (maxon::UChar*)(data.GetFirst()) + initialSize;
 const maxon::Int randomDataSize = diff;
 const auto randomDataBlock = maxon::ToBlock<maxon::UChar>(randomDataStart, randomDataSize);
 const maxon::SecureRandomProvider provider = maxon::SecureRandom::GetDefaultProvider();
 maxon::SecureRandom::GetRandomNumber(provider, randomDataBlock);
 }
 
 // check if encoder supports in-place conversion
 if (!aesEncoder.SupportInplaceConversion())
 return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);
 
 aesEncoder.ConvertAllInplace(data) iferr_return;
 }
 
 // decode
 MAXON_SCOPE
 {
 const maxon::StreamConversionRef aesDecoder = maxon::StreamConversions::AesDecoder().Create(cryptoSettings) iferr_return;
 
 // check if encoder supports in-place conversion
 if (!aesDecoder.SupportInplaceConversion())
  return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);
 
 aesDecoder.ConvertAllInplace(data) iferr_return;
 
 // data contains a null-terminated string
 const maxon::String message(data.GetFirst());
 DiagnosticOutput("Message: \"@\"", message);
 }

Hashes

Hash values for arbitrary data are calculated with these algorithms:

• maxon::StreamConversions::HashMD5: MD5 hash algorithm.
• maxon::StreamConversions::HashSHA1: SHA1 hash algorithm.
• maxon::StreamConversions::HashSHA256: SHA256 hash algorithm.
• maxon::StreamConversions::HashSHA512: SHA512 hash algorithm.
• maxon::StreamConversions::HashCrc32c: Crc32c hash algorithm. For convenience see also maxon::Crc32C.
• maxon::StreamConversions::HashCrc32zip: Crc32zip hash algorithm.
• maxon::StreamConversions::HashHmac: Hash-based message authentication code. See maxon::HASH_HMAC.

 // This example calculates various hashes of the given text.
 
 // source text
 const maxon::String text { "Hello World" };
 const maxon::BaseArray<maxon::Char> source = text.GetCString() iferr_return;
 
 // prepare target buffer
 maxon::BaseArray<maxon::UChar> hash;
 
 // MD5
 const maxon::StreamConversionRef md5 = maxon::StreamConversions::HashMD5().Create() iferr_return;
 md5.ConvertAll(source, hash) iferr_return;
 const maxon::String md5Hash = maxon::GetHashString(hash) iferr_return;
 DiagnosticOutput("MD5 Hash: @", md5Hash);
 
 hash.Reset();
 
 // SHA256
 const maxon::StreamConversionRef sha256 = maxon::StreamConversions::HashSHA256().Create() iferr_return;
 sha256.ConvertAll(source, hash) iferr_return;
 const maxon::String sha256Hash = maxon::GetHashString(hash) iferr_return;
 DiagnosticOutput("SHA256 Hash: @", sha256Hash);
 
 // hashing using utility function
 const maxon::String sha256PasswordHash = maxon::GetPasswordHash(text, maxon::StreamConversions::HashSHA256()) iferr_return;
 DiagnosticOutput("SHA256 Password Hash: @", sha256PasswordHash);

For easily handling passwords and hash strings see

• maxon::GetPasswordHash()
• maxon::HashPasswordWithSalt()
• maxon::GetHashString()

 // This example creates a new salt value and the resulting has for the given password string.
 
 // create hash and salt
 auto factory = maxon::StreamConversions::HashSHA256();
 const maxon::Tuple<maxon::String, maxon::String> res = maxon::HashPasswordWithSalt(password, factory) iferr_return;
 
 // print results
 const maxon::String salt = res.first;
 const maxon::String passwordHash = res.second;
 
 DiagnosticOutput("Salt: @, Password Hash: @", salt, passwordHash);

Compression

Arbitrary data can be compressed using these algorithms. See datacompression.h:

• maxon::StreamConversions::ZipEncoder: ZIP compression. See maxon::STREAMCONVERSION::ZIP::ENCODER.
• maxon::StreamConversions::ZipDecoder: ZIP decompression.
• maxon::StreamConversions::GZipEncoder: GZIP compression. See maxon::STREAMCONVERSION::GZIP::ENCODER.
• maxon::StreamConversions::GZipDecoder: GZIP decompression.
• maxon::StreamConversions::RleEncoder: RLE compression.
• maxon::StreamConversions::RleDecoder: RLE decompression.
• maxon::StreamConversions::Lz4Encoder: LZ4 compression. See maxon::STREAMCONVERSION::LZ4::ENCODER.
• maxon::StreamConversions::Lz4Decoder: LZ4 decompression. See maxon::STREAMCONVERSION::LZ4::DECODER.
• maxon::StreamConversions::Lc4dEncoder: LC4D compression.
• maxon::StreamConversions::Lc4dDecoder: LC4D decompression.

Note

For creating ZIP archives see Archives Manual.

 // This example compresses a given text using ZIP compression and decompresses it at the end.
 
 // get source text
 const maxon::String text = GetText() iferr_return;
 const maxon::BaseArray<maxon::Char> source = text.GetCString() iferr_return;
 
 // prepare target buffer
 maxon::BaseArray<maxon::Char> compressed;
 
 // compression settings
 maxon::DataDictionary settings;
 settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::COMPRESSION, maxon::Int(9)) iferr_return;
 settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::WRITESIZE, true) iferr_return;
 
 // compress
 const maxon::StreamConversionRef zipEncoder = maxon::StreamConversions::ZipEncoder().Create(settings) iferr_return;
 zipEncoder.ConvertAll(source, compressed) iferr_return;
 
 const maxon::Float compressionRation = maxon::Float(compressed.GetCount()) / maxon::Float(source.GetCount());
 DiagnosticOutput("Compression Ration: @", compressionRation);
 
 // decompress
 maxon::BaseArray<maxon::Char> decompressed;
 const maxon::StreamConversionRef zipDecoder = maxon::StreamConversions::ZipDecoder().Create(settings) iferr_return;
 zipDecoder.ConvertAll(compressed, decompressed) iferr_return;
 
 const maxon::String message(decompressed);
 DiagnosticOutput("Uncompressed Message: @", message);

Further Reading

• Algorithms
• Data & Algorithms
• BaseArray Manual
• String Manual
• InputStream Manual
• AES Manual
• AESFile Manual
• GeCipher256 Manual