Media Sessions Manual

About

The MAXON API handles media data using Media Sessions. Such a session is used to connect a media source with a media output. This way one can load and save image files and convert media data.

Media Converter

maxon::MediaConverterInterface is the base interface for both maxon::MediaInputInterface and maxon::MediaOutputInterface (see below). It itself is based on:

maxon::MediaConverterInterface itself gives access to data streams. Typically one does not use these functions directly but with a maxon::MediaSessionInterface.

Media Inputs

maxon::MediaInputInterface gives access to the data of a given media source. It is based on maxon::MediaConverterInterface.

Typically one creates a media input by loading a media file. maxon::FileFormatDetectionInterface can be used to get the type of such a media file. Then maxon::FileFormatHandlerInterface::CreateHandler() is used to create a handler to access that media file.

Media Outputs

maxon::MediaOutputInterface allows to write media data into a given destination. It is based on maxon::MediaConverterInterface.

The typical destination is a media file which is represented by the maxon::MediaOutputUrlInterface. An instance can be obtained from the maxon::ImageSaverClasses namespace.

The utility function maxon::MediaSessionImport() can be used to read a media file and to write the data into the given media output.

Also image data can be written into a maxon::ImageTextureInterface. Such a target image is represented as a maxon::MediaOutputTextureInterface:

MediaSessionInterface

The maxon::MediaSessionInterface is used to connect a media input and an output and to perform the desired conversion:

Technical Details

A MediaConverterRef connects media inputs and outputs. The inputs create media streams which can be subscribed by the output. This way the input must only provide the information that is actually needed by the connected output.

Within the converter the media data is handled like this:

Analzye Phase:

  • The output's Analyze() function calls all inputs to invoke their Analyze() function (see below). After that it checks all streams provided by the inputs. It searches for a suitable stream or streams and subscribes to it.
  • The input's Analyze() function checks the source (e.g. a file) and creates the available media streams (maxon::MediaStreamInterface).

Execution Phase:

  • The output's PrepareExecute() function calls the PrepareExecute() function of all inputs.
  • The input's PrepareExecute() function checks if any of the created streams are subscribed. If so, a progress job is added (maxon::MediaSessionProgressInterface).
  • The output's Execute() function calls the Execute() function of all inputs. Then it reads the media data accessible in the subscribed streams.
  • The inputs Execute() function writes all data in the subscribed streams (maxon::MediaStreamPropertiesInterface, maxon::SetPixelHandlerStruct). When done, the progress is updated.

Examples

// This example function loads the image data from the image file at the
// given URL. The image data is loaded into the given ImageTextureRef.
// For convenience one can also use maxon::ImageTextureRef::Load() or maxon::MediaSessionImport().
//----------------------------------------------------------------------------------------
// Loads the image at the given URL.
// @param[in] url The URL of the image file.
// @param[out] targetTexture The ImageTextureRef to load the image into.
// @return OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> LoadImageFromUrl(const maxon::Url& url, maxon::ImageTextureRef& targetTexture)
{
if (url.IsEmpty())
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// define source
const maxon::FileFormatHandler importFileFormat = maxon::FileFormatDetectionInterface::Detect<maxon::MediaInputRef>(url) iferr_return;
const maxon::MediaInputRef source = importFileFormat.CreateHandler<maxon::MediaInputRef>(url) iferr_return;
// define destination
const maxon::MediaOutputTextureRef destination = maxon::MediaOutputTextureClass().Create() iferr_return;
destination.SetOutputTexture(targetTexture, maxon::ImagePixelStorageClasses::Normal()) iferr_return;
// convert
const maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
session.ConnectMediaConverter(source, destination) iferr_return;
session.Close() iferr_return;
return maxon::OK;
}
// This example function saves the given maxon::ImageTextureRef as a PNG file using the given URL.
//----------------------------------------------------------------------------------------
// Saves the given image as a PNG file.
// @param[in] url The URL of target image file.
// @param[in] image The image to save.
// @return OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> SaveAsPNG(const maxon::Url& url, maxon::ImageTextureRef image)
{
if (url.IsEmpty() || image.IsEmpty())
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// define destination
const maxon::MediaOutputUrlRef destination = maxon::ImageSaverClasses::Png().Create() iferr_return;
// convert
maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
// ImageTextureInterface::Save() sets the path of the given MediaOutputUrlRef
// and creates the needed MediaInputRef internally
image.Save(url, destination, maxon::MEDIASESSIONFLAGS::RUNONLYANALYZE, &session) iferr_return;
session.Close() iferr_return;
return maxon::OK;
}
// This example function saves the images of the given maxon::BaseArray to a MP4 file at the given URL.
//----------------------------------------------------------------------------------------
// Saves the given image sequence as a MP4 video.
// @param[in] url The URL of target video file.
// @param[in] fps The framerate of the video file.
// @param[in] images The array of images to save into the video file.
// @return OK on success.
//----------------------------------------------------------------------------------------
{
const maxon::Bool invalidFps = fps <= 0;
if (url.IsEmpty() || images.IsEmpty() || invalidFps)
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// export settings
maxon::DataDictionary exportSettings;
exportSettings.Set(maxon::MEDIASESSION::EXPORT::QUALITY, 1.0) iferr_return;
exportSettings.Set(maxon::MEDIASESSION::EXPORT::DATARATE, maxon::Megabytes(1)) iferr_return;
// first image
const maxon::ImageRef firstImage = images[0];
// define source
const maxon::ImageTextureRef source = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
const maxon::ImageRef imageData = firstImage;
source.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, imageData, maxon::ImageBaseRef()) iferr_return;
source.Set(maxon::IMAGEPROPERTIES::IMAGE::FPS, maxon::Float(fps)) iferr_return;
source.Set(maxon::IMAGEPROPERTIES::IMAGE::EXPORTSETTINGS, exportSettings) iferr_return;
// define destination
const maxon::Id MP4 = maxon::MEDIASESSION::MP4::EXPORT::GetId();
const maxon::MediaOutputUrlRef destination = maxon::ImageSaverClasses::Get(MP4).Create() iferr_return;
// check dimensions
const maxon::DrawDimensionInt inputSize(firstImage.GetWidth(), firstImage.GetHeight());
const maxon::DrawDimensionInt correctedSize = destination.CheckBitmapSize(exportSettings, inputSize) iferr_return;
if (correctedSize != inputSize)
{
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Invalid resolution."_s);
}
// convert
maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
source.Save(url, destination, maxon::MEDIASESSIONFLAGS::RUNONLYANALYZE, &session) iferr_return;
// fps
maxon::TimeValue frameDuration;
frameDuration.SetSeconds(1.0 / maxon::Float(fps));
maxon::TimeValue currentTime;
// save all given images
for (auto& image : images)
{
// get image
// CopyImageData() is a custom utility function
CopyImageData(image, imageData) iferr_return;
// save image to file
session.Convert(currentTime, maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
currentTime += frameDuration;
}
return session.Close();
}
// This example function loads an image sequence found in the given folder.
// The images are loaded in the prepared ImageRef array.
//----------------------------------------------------------------------------------------
// Loads the image sequence found in the given folder.
// @param[in] folder Folder containing an image sequence in the format "sequence_00.png".
// @param[out] images Array with pre-allocated images to load the image data into.
// @return OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> LoadImageSequence(const maxon::Url& folder, maxon::BaseArray<maxon::ImageRef>& images)
{
if (folder.IsEmpty() || images.IsEmpty())
return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
// load only as many images as the ImageRef array can hold
const maxon::Int firstFrame = 0;
const maxon::Int lastFrame = images.GetCount() - 1;
const maxon::Float fps = 25.0_f;
// define source
const maxon::String filename { "sequence_" };
maxon::Url sequenceURL = (folder + maxon::Url(filename))iferr_return;
maxon::String name = sequenceURL.GetName();
// between "{" and "}" the number format is defined as used with FormatString().
// the file name format is sequence_00.png, sequence_01.png, ...
name += "@{2'0'}.png"_s;
sequenceURL.SetName(name) iferr_return;
sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_FPS, fps) iferr_return;
sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_FIRSTFRAME, firstFrame) iferr_return;
sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_LASTFRAME, lastFrame) iferr_return;
const maxon::FileFormatHandler imageSequence = maxon::FileFormatHandlers::MovieImageSequence();
const maxon::MediaInputRef source = imageSequence.CreateHandler<maxon::MediaInputRef>(sequenceURL) iferr_return;
// define destination
const maxon::ImageTextureRef texture = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
const maxon::MediaOutputTextureRef destination = maxon::MediaOutputTextureClass().Create() iferr_return;
destination.SetOutputTexture(texture, maxon::ImagePixelStorageClasses::Normal()) iferr_return;
// convert
const maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
session.ConnectMediaConverter(source, destination) iferr_return;
// fps
maxon::TimeValue frameDuration;
frameDuration.SetSeconds(1.0 / fps);
maxon::TimeValue currentTime;
// load each frame
for (maxon::Int frame = 0; frame <= lastFrame; ++frame)
{
// load frame
session.Convert(currentTime, maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
// get image
const maxon::ImageRef image = maxon::GetImageOf(texture);
// copy image to target array
images[frame] = image.Clone() iferr_return;
currentTime += frameDuration;
}
return session.Close();
}

Further Reading

maxon::GetImageOf
ImageRef GetImageOf(const ImageBaseRef &bmp)
maxon::Megabytes
Byte value in megabytes.
Definition: bytesvalue.h:402
maxon::String
Definition: string.h:1197
maxon::OK
return OK
Definition: apibase.h:2485
maxon::Bool
bool Bool
boolean type, possible values are only false/true, 8 bit
Definition: apibase.h:172
maxon::MEDIASESSIONFLAGS::RUNONLYANALYZE
@ RUNONLYANALYZE
run only the analyze but not the execute pass.
maxon::Id
Definition: apibaseid.h:273
iferr_return
#define iferr_return
Definition: resultbase.h:1434
maxon::Url::Set
Result< void > Set(KEY &&key, T &&data, Bool persistent=true)
Definition: url.h:906
MAXON_SOURCE_LOCATION
#define MAXON_SOURCE_LOCATION
Definition: memoryallocationbase.h:66
maxon::Float
Float64 Float
Definition: apibase.h:188
maxon::BaseArray
Definition: basearray.h:366
maxon::Url
Definition: url.h:819
maxon::Result< void >
maxon::BaseArray::GetCount
MAXON_ATTRIBUTE_FORCE_INLINE Int GetCount() const
Definition: basearray.h:527
maxon::Int32
int32_t Int32
32 bit signed integer datatype.
Definition: apibase.h:167
maxon::Int
Int64 Int
signed 32/64 bit int, size depends on the platform
Definition: apibase.h:179
maxon::Classes::Get
const Class< R > & Get(const Id &cls)
Definition: objectbase.h:1881
iferr_scope
#define iferr_scope
Definition: resultbase.h:1343
maxon::TimeValue
The TimeValue class encapsulates a timer value.
Definition: timevalue.h:33
maxon::TimeValue::SetSeconds
void SetSeconds(Float64 seconds)
Definition: timevalue.h:232
maxon::BaseCollection< BaseArray< T, BASEARRAY_DEFAULT_CHUNK_SIZE, BASEARRAYFLAGS::NONE, DefaultAllocator >, BaseArrayData< T, DefaultAllocator, STD_IS_REPLACEMENT(empty, DefaultAllocator)> >::IsEmpty
MAXON_ATTRIBUTE_FORCE_INLINE Bool IsEmpty() const
Definition: collection.h:339
maxon::MEDIASESSIONFLAGS::NONE
@ NONE
maxon::IMAGEHIERARCHY::IMAGE
@ IMAGE
Adds a subImage to a texture. Images are only allowed under Textures.
maxon::DrawDimension
Size of a window/drawing operation.
Definition: gfx_basictypes.h:16