Stream Conversions Manual

About

The maxon::StreamConversionInterface provides a generic interface for any kind of data conversion. This includes string encoding, compression, encryption, hashing etc.

StreamConversionInterface

Conversion

maxon::StreamConversionInterface is used to convert the given source data. The result of this conversion is typically stored in a given array.

Note
A maxon::StreamConversionInterface object can be used only once.
// This example converts the given string data into Base64 using Convert().
// get data
const maxon::String sourceText("Hello World");
const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;
// encode
// Base64 data is c. 1.4 times bigger than the input data
const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>((maxon::Float32)source.GetCount() * 1.4);
maxon::Bool lastPart;
const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
base64encoder.Convert(source, destination, targetSizeEstimation, true, lastPart) iferr_return;
// print result
const maxon::String base64string(destination);
DiagnosticOutput("Base64: @", base64string);
Definition: basearray.h:415
Definition: string.h:1237
const Py_UNICODE * source
Definition: unicodeobject.h:54
Int64 Int
signed 32/64 bit int, size depends on the platform
Definition: apibase.h:202
bool Bool
boolean type, possible values are only false/true, 8 bit
Definition: apibase.h:195
float Float32
32 bit floating point value (float)
Definition: apibase.h:196
#define DiagnosticOutput(formatString,...)
Definition: debugdiagnostics.h:170
#define iferr_return
Definition: resultbase.h:1524
// This example converts the given string data into Base64 using ConvertAll().
// get data
const maxon::String sourceText("Hello World");
const maxon::BaseArray<maxon::Char> source = sourceText.GetCString() iferr_return;
// encode
// Base64 data is c. 1.4 times bigger than the input data
const maxon::Int targetSizeEstimation = maxon::SafeConvert<maxon::Int32>((maxon::Float32)source.GetCount() * 1.4);
const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
base64encoder.ConvertAll(source, destination, targetSizeEstimation) iferr_return;
// print result
const maxon::String base64string(destination);
DiagnosticOutput("Base64: @", base64string);

Streams

A given conversion instance can be converted to an input stream. See InputStream Manual.

// This example converts the given text to Base64 using input streams.
// source stream reads from memory block
const auto memblock = maxon::CharToBlock("Hello World");
maxon::InputStreamRef inputStream = maxon::InputStreamInterface::FromBlock().Create(memblock, false) iferr_return;
// create stream conversion
const maxon::StreamConversionRef base64encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
inputStream = base64encoder.ConvertToStream(inputStream) iferr_return;
// prepare buffer to read from stream
const maxon::Int bufferSize = 1000;
buffer.Resize(bufferSize) iferr_return;
// read from stream
const maxon::Int count = inputStream.ReadEOS(buffer) iferr_return;
// convert to string
const maxon::String string(buffer.GetFirst(), count);
DiagnosticOutput("Base64: @", string);
const char ** buffer
Definition: abstract.h:327
Py_ssize_t count
Definition: abstract.h:640
PyObject * string
Definition: asdl.h:6
Block< const Char > CharToBlock(const Char *str)
Definition: block.h:1016

Properties

Various properties of the given conversion type can be accessed with:

// This example function automatically decodes the given data using the "counterpart" of the given encoder.
static maxon::Result<maxon::String> DecodeData(const maxon::StreamConversionRef encoder, maxon::BaseArray<maxon::Char>& data)
{
// check input data
if (data.GetCount() == 0)
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// get counterpart (decoder)
const maxon::Id decoderID = encoder.GetCounterpart();
const maxon::StreamConversionRef decoder = maxon::StreamConversions::Get(decoderID).Create() iferr_return;
// check if decoder can handle maxon::Char data
const maxon::DataType charDataType = maxon::GetDataType<maxon::Char>();
if (decoder.GetSourceType() != charDataType)
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
if (decoder.GetDestinationType() != charDataType)
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// decode data
decoder.ConvertAll(data, destination) iferr_return;
// convert to maxon::String
return maxon::String(destination);
}
MAXON_ATTRIBUTE_FORCE_INLINE Int GetCount() const
Definition: basearray.h:585
Definition: datatypebase.h:770
Definition: apibaseid.h:243
Definition: resultbase.h:766
#define MAXON_SOURCE_LOCATION
Definition: memoryallocationbase.h:67
const Class< R > & Get(const Id &cls)
Definition: objectbase.h:2073
#define iferr_scope
Definition: resultbase.h:1389

Conversions

Existing conversions are registered at the registry maxon::StreamConversions or presented as published objects. Some conversions can be configured by defining some settings in a maxon::DataDictionary that is used with the Create() function (maxon::StreamConversionFactory).

Hex

Arbitrary data can be converted to a hexadecimal representation:

  • maxon::StreamConversions::HexEncoder: Hexadecimal encoder.
  • maxon::StreamConversions::HexDecoder: Hexadecimal decoder.

See also maxon::GetHexadecimalValue().

// This example converts the given number to a hexadecimal representation.
// the original number
const maxon::Int number = 123;
source.Append((maxon::Char)number) iferr_return;
// get encoder
const maxon::StreamConversionRef encoder = maxon::StreamConversions::HexEncoder().Create() iferr_return;
// convert to HEX
encoder.ConvertAll(source, destination) iferr_return;
// print result
const maxon::String hexString(destination);
DiagnosticOutput("Hex: @", hexString);
char Char
signed 8 bit character
Definition: apibase.h:198

UTF

Text can be converted to different Unicode encodings. See maxon::UTFTEXT_OPTIONS.

  • maxon::StreamConversions::UtfTextEncoder: Unicode encoder.
  • maxon::StreamConversions::UtfTextDecoder: Unicode decoder.

Additional string conversions are registered at maxon::StringEncodings and maxon::StringDecodings, see stringencoding.h. For string conversions there are also maxon::StringEncodingInterface and maxon::StringDecodingInterface.

Base64

Arbitrary data can be converted to a Base64 representation. See maxon::BASE64_OPTIONS.

  • maxon::StreamConversions::Base64Encoder: Base64 encoder.
  • maxon::StreamConversions::Base64Decoder: Base64 decoder.
  • maxon::StreamConversions::Base64UrlEncoder: Base64 URL encoder.
  • maxon::StreamConversions::Base64UrlDecoder: Base64 URL decoder.
// This example converts some text to Base64 and back.
// prepare data
// encode
{
// original text
const maxon::String message { "Hello World" };
// convert to Base64
const maxon::StreamConversionRef encoder = maxon::StreamConversions::Base64Encoder().Create() iferr_return;
encoder.ConvertAll(source, data) iferr_return;
}
// decode
{
// convert to plain text
const maxon::StreamConversionRef decoder = maxon::StreamConversions::Base64Decoder().Create() iferr_return;
decoder.ConvertAll(data, destination) iferr_return;
// print result
const maxon::String message(destination);
DiagnosticOutput("Message: @", message);
}
const char * message
Definition: pyerrors.h:189
#define MAXON_SCOPE
Definition: apibase.h:2886

Cryptography

Arbitrary data can be encrypted using these algorithms:

  • maxon::StreamConversions::AesEncoder: AES encryption.
  • maxon::StreamConversions::AesDecoder: AES decryption.
  • maxon::StreamConversions::BlowfishEncoder: Blowfish encryption.
  • maxon::StreamConversions::BlowfishDecoder: Blowfish decryption.
  • maxon::StreamConversions::BlowfishLegacyEncoder: Legacy Blowfish encryption.
  • maxon::StreamConversions::BlowfishLegacyDecoder: Legacy Blowfish decryption.
  • maxon::StreamConversions::BlowfishLegacyEnDecoder: Legacy Blowfish en/decryption. See maxon::BLOWFISHLEGACYENDECODER_OPTIONS.

The cryptographic key is set as maxon::CryptographyOptions::CRYPTOKEY using maxon::CryptoKeyInterface. To pad a data block with arbitrary data use maxon::SecureRandom.

Encryption algorithms can also be used with the specialized maxon::CryptographyStreamConversionInterface.

// This example encrypts a text with AES and the given encryption key and decrypts it later.
// prepare encryption settings including the key
const maxon::Id encoderID = maxon::StreamConversions::AesEncoder.GetId();
const maxon::Int blockSize = 128;
const maxon::Char* key = "9USGxEPo0Tx6d8ZRRLbpEc4D88xdU2bb";
const maxon::Int keySize = 128;
const maxon::CryptoKey cryptoKey(encoderID, blockSize, key, keySize);
maxon::DataDictionary cryptoSettings;
cryptoSettings.Set(maxon::CryptographyOptions::CRYPTOKEY, cryptoKey) iferr_return;
// data
// encode
{
// get data
const maxon::String message { "Hello World" };
data = message.GetCString() iferr_return;
// add null for null-terminated string
// encode
const maxon::StreamConversionRef aesEncoder = maxon::StreamConversions::AesEncoder().Create(cryptoSettings) iferr_return;
// prepare data: resize buffer and fill with random bits
const maxon::Int initialSize = data.GetCount();
const maxon::Int aesBlockSize = aesEncoder.GetBlockSize() * (aesEncoder.GetSourceType().GetSize() * 8);
const maxon::Int targetSize = ((initialSize + (aesBlockSize / 8) - 1) / (blockSize / 8)) * (aesBlockSize / 8);
const maxon::Int diff = targetSize - data.GetCount();
if (diff > 0)
{
data.Resize(targetSize) iferr_return; // resize data
// fill rest of the data with random bits
maxon::UChar* randomDataStart = (maxon::UChar*)(data.GetFirst()) + initialSize;
const maxon::Int randomDataSize = diff;
const auto randomDataBlock = maxon::ToBlock<maxon::UChar>(randomDataStart, randomDataSize);
maxon::SecureRandom::GetRandomNumber(provider, randomDataBlock);
}
// check if encoder supports in-place conversion
if (!aesEncoder.SupportInplaceConversion())
return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);
aesEncoder.ConvertAllInplace(data) iferr_return;
}
// decode
{
const maxon::StreamConversionRef aesDecoder = maxon::StreamConversions::AesDecoder().Create(cryptoSettings) iferr_return;
// check if encoder supports in-place conversion
if (!aesDecoder.SupportInplaceConversion())
return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);
aesDecoder.ConvertAllInplace(data) iferr_return;
// data contains a null-terminated string
DiagnosticOutput("Message: \"@\"", message);
}
PyObject * key
Definition: abstract.h:289
MAXON_ATTRIBUTE_FORCE_INLINE ResultRef< T > Append(ARG &&x)
Appends a new element at the end of the array and constructs it using the forwarded value.
Definition: basearray.h:628
MAXON_ATTRIBUTE_FORCE_INLINE const T * GetFirst() const
Definition: basearray.h:1175
ResultMem Resize(Int newCnt, COLLECTION_RESIZE_FLAGS resizeFlags=COLLECTION_RESIZE_FLAGS::DEFAULT)
Definition: basearray.h:1218
Definition: baseref.h:62
Reference counted class of CryptoKeyInterface.
Definition: cryptography_key.h:122
static MAXON_METHOD SecureRandomProvider GetDefaultProvider()
static MAXON_METHOD Bool GetRandomNumber(SecureRandomProvider provider, const Block< Byte > &buffer)
unsigned char UChar
unsigned 8 bit character
Definition: apibase.h:199

Hashes

Hash values for arbitrary data are calculated with these algorithms:

  • maxon::StreamConversions::HashMD5: MD5 hash algorithm.
  • maxon::StreamConversions::HashSHA1: SHA1 hash algorithm.
  • maxon::StreamConversions::HashSHA256: SHA256 hash algorithm.
  • maxon::StreamConversions::HashSHA512: SHA512 hash algorithm.
  • maxon::StreamConversions::HashCrc32c: Crc32c hash algorithm. For convenience see also maxon::Crc32C.
  • maxon::StreamConversions::HashCrc32zip: Crc32zip hash algorithm.
  • maxon::StreamConversions::HashHmac: Hash-based message authentication code. See maxon::HASH_HMAC.
// This example calculates various hashes of the given text.
// source text
const maxon::String text { "Hello World" };
// prepare target buffer
// MD5
const maxon::StreamConversionRef md5 = maxon::StreamConversions::HashMD5().Create() iferr_return;
md5.ConvertAll(source, hash) iferr_return;
DiagnosticOutput("MD5 Hash: @", md5Hash);
hash.Reset();
// SHA256
const maxon::StreamConversionRef sha256 = maxon::StreamConversions::HashSHA256().Create() iferr_return;
sha256.ConvertAll(source, hash) iferr_return;
DiagnosticOutput("SHA256 Hash: @", sha256Hash);
// hashing using utility function
const maxon::String sha256PasswordHash = maxon::GetPasswordHash(text, maxon::StreamConversions::HashSHA256()) iferr_return;
DiagnosticOutput("SHA256 Password Hash: @", sha256PasswordHash);
PyObject Py_hash_t hash
Definition: dictobject.h:35
Result< String > GetHashString(const BaseArray< UChar > &hashValue)
Result< String > GetPasswordHash(const String &password, const StreamConversionFactory &hashClass, const DataDictionary &settings=DataDictionary())
PyObject * text
Definition: pycore_traceback.h:70

For easily handling passwords and hash strings see

// This example creates a new salt value and the resulting has for the given password string.
// create hash and salt
auto factory = maxon::StreamConversions::HashSHA256();
// print results
const maxon::String salt = res.first;
const maxon::String passwordHash = res.second;
DiagnosticOutput("Salt: @, Password Hash: @", salt, passwordHash);
Py_UCS4 * res
Definition: unicodeobject.h:1113
Result< Tuple< String, String > > HashPasswordWithSalt(const String &password, const StreamConversionFactory &hashClass=StreamConversions::HashSHA256())

Compression

Arbitrary data can be compressed using these algorithms. See datacompression.h:

Note
For creating ZIP archives see Archives Manual.
// This example compresses a given text using ZIP compression and decompresses it at the end.
// get source text
const maxon::String text = GetText() iferr_return;
const maxon::BaseArray<maxon::Char> source = text.GetCString() iferr_return;
// prepare target buffer
maxon::BaseArray<maxon::Char> compressed;
// compression settings
maxon::DataDictionary settings;
settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::COMPRESSION, maxon::Int(9)) iferr_return;
settings.Set(maxon::STREAMCONVERSION::ZIP::ENCODER::WRITESIZE, true) iferr_return;
// compress
const maxon::StreamConversionRef zipEncoder = maxon::StreamConversions::ZipEncoder().Create(settings) iferr_return;
zipEncoder.ConvertAll(source, compressed) iferr_return;
const maxon::Float compressionRation = maxon::Float(compressed.GetCount()) / maxon::Float(source.GetCount());
DiagnosticOutput("Compression Ration: @", compressionRation);
// decompress
maxon::BaseArray<maxon::Char> decompressed;
const maxon::StreamConversionRef zipDecoder = maxon::StreamConversions::ZipDecoder().Create(settings) iferr_return;
zipDecoder.ConvertAll(compressed, decompressed) iferr_return;
const maxon::String message(decompressed);
DiagnosticOutput("Uncompressed Message: @", message);
#define Set(a0, a1, a2, a3, a4, a5)
Definition: Python-ast.h:586
maxon::Char Char
Definition: ge_sys_math.h:52
maxon::Float Float
Definition: ge_sys_math.h:62
maxon::Int Int
Definition: ge_sys_math.h:60
static auto Create(ARGS &&... args)
Definition: apibase.h:2818
Int GetCount(const ITERABLE &iterable)
Definition: collection.h:37
The maxon namespace contains all declarations of the MAXON API.
Definition: autoweight.h:14
ZIP
Definition: network_smtpmail.h:21

Further Reading