Stream Conversions Manual

Table of Contents

• About
• StreamConversionInterface
  • Conversion
  • Streams
  • Properties
• Conversions
  • Hex
  • UTF
  • Base64
  • Cryptography
  • Hashes
  • Compression
• Further Reading

About

The maxon::StreamConversionInterface provides a generic interface for any kind of data conversion. This includes string encoding, compression, encryption, hashing etc.

StreamConversionInterface

Conversion

maxon::StreamConversionInterface is used to convert the given source data. The result of this conversion is typically stored in a given array.

• maxon::StreamConversionInterface::Convert(): Converts (part of) the source data.
• maxon::StreamConversionInterface::ConvertAll(): Converts the complete source data.
• maxon::StreamConversionInterface::SupportInplaceConversion(): Returns true if the conversion supports in-place conversion. Typically only supported by encryption algorithms.
• maxon::StreamConversionInterface::ConvertAllInplace(): Converts the complete source data and overwrites it.

Note

A maxon::StreamConversionInterface object can be used only once.

  // This example converts the given string data into Base64 using Convert().

  // get data

  const maxon::String sourceText("Hello World");
  const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;

  // encode

  // Base64 data is c. 1.4 times bigger than the input data
  const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>(source.GetCount() * 1.4);

  maxon::BaseArray<maxon::Char> destination;
  maxon::Bool lastPart;
  const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
  base64encoder.Convert(source, destination, targetSizeEstimation, true, lastPart) iferr_return;

  // print result
  const maxon::String base64string(destination);
  DiagnosticOutput("Base64: @", base64string);

  // This example converts the given string data into Base64 using ConvertAll().

  // get data

  const maxon::String sourceText("Hello World");
  const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;

  // encode

  // Base64 data is c. 1.4 times bigger than the input data
  const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>(source.GetCount() * 1.4);

  maxon::BaseArray<maxon::Char>    destination;
  const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
  base64encoder.ConvertAll(source, destination, targetSizeEstimation) iferr_return;

  // print result
  const maxon::String base64string(destination);
  DiagnosticOutput("Base64: @", base64string);

Streams

A given conversion instance can be converted to an input stream. See InputStream Manual.

• maxon::StreamConversionInterface::ConvertToStream(): Returns an input stream.
• maxon::StreamConversionInterface::ConvertToReader(): Returns a DataFormatBaseReaderRef.
• maxon::StreamConversionInterface::ConvertToWriter(): Returns a DataFormatBaseWriterRef.

  // This example converts the given text to Base64 using input streams.

  // source stream reads from memory block
  const auto memblock = maxon::CharToBlock("Hello World");
  maxon::InputStreamRef inputStream = maxon::InputStreamInterface::FromBlock().Create(memblock, false) iferr_return;

  // create stream conversion
  const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
  inputStream = base64encoder.ConvertToStream(inputStream) iferr_return;

  // prepare buffer to read from stream
  maxon::BaseArray<maxon::Char> buffer;
  const maxon::Int bufferSize = 1000;
  buffer.Resize(bufferSize) iferr_return;

  // read from stream
  const maxon::Int count = inputStream.ReadEOS(buffer) iferr_return;

  // convert to string
  const maxon::String string(buffer.GetFirst(), count);
  DiagnosticOutput("Base64: @", string);

Properties

Various properties of the given conversion type can be accessed with:

• maxon::StreamConversionInterface::GetBatchSize(): Returns the recommended working size.
• maxon::StreamConversionInterface::GetBlockSize(): Returns the block size. The encoded/decoded data needs to be a multiple of this value.
• maxon::StreamConversionInterface::GetCounterpart(): Returns the corresponding counter part converter id.
• maxon::StreamConversionInterface::GetSourceType(): Returns the accepted source data type.
• maxon::StreamConversionInterface::GetDestinationType(): Returns the accepted destination data type.

// This example function automatically decodes the given data using the "counterpart" of the given encoder.

static maxon::Result<maxon::String> DecodeData(const maxon::StreamConversionRef encoder, maxon::BaseArray<maxon::Char>& data)
{
  iferr_scope;

  // check input data
  if (data.GetCount() == 0)
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);

  // get counterpart (decoder)
  const maxon::Id decoderID = encoder.GetCounterpart();
  const maxon::StreamConversionRef decoder = maxon::StreamConversions::Get(decoderID).Create() iferr_return;

  // check if decoder can handle maxon::Char data
  const maxon::DataType charDataType = maxon::GetDataType<maxon::Char>();

  if (decoder.GetSourceType() != charDataType)
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);

  if (decoder.GetDestinationType() != charDataType)
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);

  // decode data
  maxon::BaseArray<maxon::Char> destination;
  decoder.ConvertAll(data, destination) iferr_return;

  // convert to maxon::String
  return maxon::String(destination);
}

Conversions

Existing conversions are registered at the registry maxon::StreamConversions or presented as published objects. Some conversions can be configured by defining some settings in a maxon::DataDictionary that is used with the Create() function (maxon::StreamConversionFactory).

Hex

Arbitrary data can be converted to a hexadecimal representation:

• maxon::StreamConversions::HexEncoder: Hexadecimal encoder.
• maxon::StreamConversions::HexDecoder: Hexadecimal decoder.

See also maxon::GetHexadecimalValue().

  // This example converts the given number to a hexadecimal representation.

  // the original number
  const maxon::Int number = 123;

  maxon::BaseArray<maxon::Char> source;
  source.Append((maxon::Char)number) iferr_return;

  // get encoder
  const maxon::StreamConversionRef encoder = maxon::StreamConversions::HexEncoder().Create() iferr_return;

  // convert to HEX
  maxon::BaseArray<maxon::Char> destination;
  encoder.ConvertAll(source, destination) iferr_return;

  // print result
  const maxon::String hexString(destination);
  DiagnosticOutput("Hex: @", hexString);

UTF

Text can be converted to different Unicode encodings. See maxon::UTFTEXT_OPTIONS.

• maxon::StreamConversions::UtfTextEncoder: Unicode encoder.
• maxon::StreamConversions::UtfTextDecoder: Unicode decoder.

Additional string conversions are registered at maxon::StringEncodings and maxon::StringDecodings, see stringencoding.h. For string conversions there are also maxon::StringEncodingInterface and maxon::StringDecodingInterface.

Base64

Arbitrary data can be converted to a Base64 representation. See maxon::BASE64_OPTIONS.

• maxon::StreamConversions::Base64Encoder: Base64 encoder.
• maxon::StreamConversions::Base64Decoder: Base64 decoder.
• maxon::StreamConversions::Base64UrlEncoder: Base64 URL encoder.
• maxon::StreamConversions::Base64UrlDecoder: Base64 URL decoder.

  // This example converts some text to Base64 and back.

  // prepare data
  maxon::BaseArray<maxon::Char> data;

  // encode
  MAXON_SCOPE
  {
    // original text
    const maxon::String message { "Hello World" };
    const maxon::BaseArray<maxon::Char> source = message.GetCString() iferr_return;

    // convert to Base64
    const maxon::StreamConversionRef encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
    encoder.ConvertAll(source, data) iferr_return;
  }

  // decode
  MAXON_SCOPE
  {
    maxon::BaseArray<maxon::Char> destination;

    // convert to plain text
    const maxon::StreamConversionRef decoder = maxon::StreamConversions::Base64Decoder().Create() iferr_return;
    decoder.ConvertAll(data, destination) iferr_return;

    // print result
    const maxon::String message(destination);
    DiagnosticOutput("Message: @", message);
  }

Cryptography

Arbitrary data can be encrypted using these algorithms:

• maxon::StreamConversions::AesEncoder: AES encryption.
• maxon::StreamConversions::AesDecoder: AES decryption.
• maxon::StreamConversions::BlowfishEncoder: Blowfish encryption.
• maxon::StreamConversions::BlowfishDecoder: Blowfish decryption.
• maxon::StreamConversions::BlowfishLegacyEncoder: Legacy Blowfish encryption.
• maxon::StreamConversions::BlowfishLegacyDecoder: Legacy Blowfish decryption.
• maxon::StreamConversions::BlowfishLegacyEnDecoder: Legacy Blowfish en/decryption. See maxon::BLOWFISHLEGACYENDECODER_OPTIONS.

The cryptographic key is set as maxon::CryptographyOptions::CRYPTOKEY using maxon::CryptoKeyInterface. To pad a data block with arbitrary data use maxon::SecureRandom.

Encryption algorithms can also be used with the specialized maxon::CryptographyStreamConversionInterface.

  // This example encrypts a text with AES and the given encryption key and decrypts it later.

  // prepare encryption settings including the key

  const maxon::Id        encoderID = maxon::StreamConversions::AesEncoder.GetId();
  const maxon::Int       blockSize = 128;
  const maxon::Char*     key = "9USGxEPo0Tx6d8ZRRLbpEc4D88xdU2bb";
  const maxon::Int       keySize = 128;
  const maxon::CryptoKey cryptoKey(encoderID, blockSize, key, keySize);

  maxon::DataDictionary cryptoSettings;
  cryptoSettings.Set(maxon::CryptographyOptions::CRYPTOKEY, cryptoKey) iferr_return;

  // data
  maxon::BaseArray<maxon::Char> data;

  // encode
  MAXON_SCOPE
  {
    // get data
    const maxon::String message { "Hello World" };
    data = message.GetCString() iferr_return;

    // add null for null-terminated string
    data.Append(0) iferr_return;

    // encode
    const maxon::StreamConversionRef aesEncoder = maxon::StreamConversions::AesEncoder().Create(cryptoSettings) iferr_return;

    // prepare data: resize buffer and fill with random bits

    const maxon::Int initialSize = data.GetCount();

    const maxon::Int aesBlockSize = aesEncoder.GetBlockSize();
    const maxon::Int targetSize = ((initialSize / blockSize) + 1) * aesBlockSize;
    const maxon::Int diff = targetSize - data.GetCount();
    if (diff > 0)
    {
      data.Resize(targetSize) iferr_return; // resize data

      // fill rest of the data with random bits
      maxon::UChar*    randomDataStart = (maxon::UChar*)(data.GetFirst()) + initialSize;
      const maxon::Int randomDataSize  = diff;
      const auto       randomDataBlock = maxon::ToBlock<maxon::UChar>(randomDataStart, randomDataSize);
      const maxon::SecureRandomProvider provider = maxon::SecureRandom::GetDefaultProvider();
      maxon::SecureRandom::GetRandomNumber(provider, randomDataBlock);
    }

    // check if encoder supports in-place conversion
    if (!aesEncoder.SupportInplaceConversion())
      return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

    aesEncoder.ConvertAllInplace(data) iferr_return;
  }

  // decode
  MAXON_SCOPE
  {
    const maxon::StreamConversionRef aesDecoder = maxon::StreamConversions::AesDecoder().Create(cryptoSettings) iferr_return;

    // check if encoder supports in-place conversion
    if (!aesDecoder.SupportInplaceConversion())
      return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

    aesDecoder.ConvertAllInplace(data) iferr_return;

    // data contains a null-terminated string
    const maxon::String message(data.GetFirst());
    DiagnosticOutput("Message: \"@\"", message);
  }

Hashes

Hash values for arbitrary data are calculated with these algorithms:

• maxon::StreamConversions::HashMD5: MD5 hash algorithm.
• maxon::StreamConversions::HashSHA1: SHA1 hash algorithm.
• maxon::StreamConversions::HashSHA256: SHA256 hash algorithm.
• maxon::StreamConversions::HashSHA512: SHA512 hash algorithm.
• maxon::StreamConversions::HashCrc32c: Crc32c hash algorithm. For convenience see also maxon::Crc32C.
• maxon::StreamConversions::HashCrc32zip: Crc32zip hash algorithm.
• maxon::StreamConversions::HashHmac: Hash-based message authentication code. See maxon::HASH_HMAC.

  // This example calculates various hashes of the given text.

  // source text
  const maxon::String text { "Hello World" };
  const maxon::BaseArray<maxon::Char> source = text.GetCString() iferr_return;

  // prepare target buffer
  maxon::BaseArray<maxon::UChar> hash;

  // MD5
  const maxon::StreamConversionRef md5 = maxon::StreamConversions::HashMD5().Create() iferr_return;
  md5.ConvertAll(source, hash) iferr_return;
  const maxon::String md5Hash = maxon::GetHashString(hash) iferr_return;
  DiagnosticOutput("MD5 Hash: @", md5Hash);

  hash.Reset();

  // SHA256
  const maxon::StreamConversionRef sha256 = maxon::StreamConversions::HashSHA256().Create() iferr_return;
  sha256.ConvertAll(source, hash) iferr_return;
  const maxon::String sha256Hash = maxon::GetHashString(hash) iferr_return;
  DiagnosticOutput("SHA256 Hash: @", sha256Hash);

  // hashing using utility function
  const maxon::String sha256PasswordHash = maxon::GetPasswordHash(text, maxon::StreamConversions::HashSHA256()) iferr_return;
  DiagnosticOutput("SHA256 Password Hash: @", sha256PasswordHash);

For easily handling passwords and hash strings see

• maxon::GetPasswordHash()
• maxon::HashPasswordWithSalt()
• maxon::GetHashString()

  // This example creates a new salt value and the resulting has for the given password string.

  // create hash and salt
  auto factory = maxon::StreamConversions::HashSHA256();
  const maxon::Tuple<maxon::String, maxon::String> res = maxon::HashPasswordWithSalt(password, factory) iferr_return;

  // print results
  const maxon::String salt = res.first;
  const maxon::String passwordHash = res.second;

  DiagnosticOutput("Salt: @, Password Hash: @", salt, passwordHash);

Compression

Arbitrary data can be compressed using these algorithms. See datacompression.h:

• maxon::StreamConversions::ZipEncoder: ZIP compression. See maxon::STREAMCONVERSION::ZIP::ENCODER.
• maxon::StreamConversions::ZipDecoder: ZIP decompression.
• maxon::StreamConversions::GZipEncoder: GZIP compression. See maxon::STREAMCONVERSION::GZIP::ENCODER.
• maxon::StreamConversions::GZipDecoder: GZIP decompression.
• maxon::StreamConversions::RleEncoder: RLE compression.
• maxon::StreamConversions::RleDecoder: RLE decompression.
• maxon::StreamConversions::Lz4Encoder: LZ4 compression. See maxon::STREAMCONVERSION::LZ4::ENCODER.
• maxon::StreamConversions::Lz4Decoder: LZ4 decompression. See maxon::STREAMCONVERSION::LZ4::DECODER.
• maxon::StreamConversions::Lc4dEncoder: LC4D compression.
• maxon::StreamConversions::Lc4dDecoder: LC4D decompression.

Note

For creating ZIP archives see Archives Manual.

  // This example compresses a given text using ZIP compression and decompresses it at the end.

  // get source text
  const maxon::String text = GetText() iferr_return;
  const maxon::BaseArray<maxon::Char> source = text.GetCString() iferr_return;

  // prepare target buffer
  maxon::BaseArray<maxon::Char> compressed;

  // compression settings
  maxon::DataDictionary settings;
  settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::COMPRESSION, maxon::Int(9)) iferr_return;
  settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::WRITESIZE, true) iferr_return;

  // compress
  const maxon::StreamConversionRef zipEncoder = maxon::StreamConversions::ZipEncoder().Create(settings) iferr_return;
  zipEncoder.ConvertAll(source, compressed) iferr_return;

  const maxon::Float compressionRation = maxon::Float(compressed.GetCount()) / maxon::Float(source.GetCount());
  DiagnosticOutput("Compression Ration: @", compressionRation);

  // decompress
  maxon::BaseArray<maxon::Char>    decompressed;
  const maxon::StreamConversionRef zipDecoder = maxon::StreamConversions::ZipDecoder().Create(settings) iferr_return;
  zipDecoder.ConvertAll(compressed, decompressed) iferr_return;

  const maxon::String message(decompressed);
  DiagnosticOutput("Uncompressed Message: @", message);

Further Reading

• Algorithms
• Data & Algorithms
• BaseArray Manual
• String Manual
• InputStream Manual
• AES Manual
• AESFile Manual
• GeCipher256 Manual