Media Sessions Manual

Table of Contents

• About
• Media Converter
• Media Inputs
• Media Outputs
• MediaSessionInterface
• Technical Details
• Examples
• Further Reading

About

The MAXON API handles media data using Media Sessions. Such a session is used to connect a media source with a media output. This way one can load and save image files and convert media data.

Media Converter

maxon::MediaConverterInterface is the base interface for both maxon::MediaInputInterface and maxon::MediaOutputInterface (see below). It itself is based on:

• maxon::DataDictionaryObjectInterface: Allows to store arbitrary settings in a converter e.g. image compression settings.
• maxon::DataDescriptionObjectInterface: Stores data descriptions.
• maxon::MediaBaseInterface: Gives access to the maxon::MediaSessionInterface.

maxon::MediaConverterInterface itself gives access to data streams. Typically one does not use these functions directly but with a maxon::MediaSessionInterface.

Media Inputs

maxon::MediaInputInterface gives access to the data of a given media source. It is based on maxon::MediaConverterInterface.

Typically one creates a media input by loading a media file. maxon::FileFormatDetectionInterface can be used to get the type of such a media file. Then maxon::FileFormatHandlerInterface::CreateHandler() is used to create a handler to access that media file.

Media Outputs

maxon::MediaOutputInterface allows to write media data into a given destination. It is based on maxon::MediaConverterInterface.

The typical destination is a media file which is represented by the maxon::MediaOutputUrlInterface. An instance can be obtained from the maxon::ImageSaverClasses namespace.

• maxon::MediaOutputUrlInterface::GetOutputUrl(): Returns the stored file URL.
• maxon::MediaOutputUrlInterface::SetOutputUrl(): Sets the file URL.
• maxon::MediaOutputUrlInterface::GetFileFormat(): Returns the maxon::FileFormat.
• maxon::MediaOutputUrlInterface::EditSystemOptions(): Opens the options dialog.
• maxon::MediaOutputUrlInterface::CheckBitmapSize(): Verifies the given resolution for the save format.

The utility function maxon::MediaSessionImport() can be used to read a media file and to write the data into the given media output.

Also image data can be written into a maxon::ImageTextureInterface. Such a target image is represented as a maxon::MediaOutputTextureInterface:

• maxon::MediaOutputTextureInterface::SetOutputTexture(): Defines the target image.

MediaSessionInterface

The maxon::MediaSessionInterface is used to connect a media input and an output and to perform the desired conversion:

• maxon::MediaSessionInterface::ConnectMediaConverter(): Connects the given input and output.
• maxon::MediaSessionInterface::Convert(): Reads the given frame from the input. See maxon::MEDIASESSIONFLAGS.
• maxon::MediaSessionInterface::Close(): Closes the session.

Technical Details

A MediaConverterRef connects media inputs and outputs. The inputs create media streams which can be subscribed by the output. This way the input must only provide the information that is actually needed by the connected output.

Within the converter the media data is handled like this:

Analzye Phase:

• The output's Analyze() function calls all inputs to invoke their Analyze() function (see below). After that it checks all streams provided by the inputs. It searches for a suitable stream or streams and subscribes to it.
• The input's Analyze() function checks the source (e.g. a file) and creates the available media streams (maxon::MediaStreamInterface).

Execution Phase:

• The output's PrepareExecute() function calls the PrepareExecute() function of all inputs.
• The input's PrepareExecute() function checks if any of the created streams are subscribed. If so, a progress job is added (maxon::MediaSessionProgressInterface).
• The output's Execute() function calls the Execute() function of all inputs. Then it reads the media data accessible in the subscribed streams.
• The inputs Execute() function writes all data in the subscribed streams (maxon::MediaStreamPropertiesInterface, maxon::SetPixelHandlerStruct). When done, the progress is updated.

Examples

// This example function loads the image data from the image file at the
// given URL. The image data is loaded into the given ImageTextureRef.
// For convenience one can also use maxon::ImageTextureRef::Load() or maxon::MediaSessionImport().
 
//----------------------------------------------------------------------------------------
// Loads the image at the given URL.
// @param[in] url               The URL of the image file.
// @param[out] targetTexture    The ImageTextureRef to load the image into.
// @return                      OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> LoadImageFromUrl(const maxon::Url& url, maxon::ImageTextureRef& targetTexture)
{
  iferr_scope;
 
  if (url.IsEmpty())
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
  // define source
 
  const maxon::FileFormatHandler importFileFormat = maxon::FileFormatDetectionInterface::Detect<maxon::MediaInputRef>(url) iferr_return;
  const maxon::MediaInputRef     source = importFileFormat.CreateHandler<maxon::MediaInputRef>(url) iferr_return;
 
  // define destination
 
  const maxon::MediaOutputTextureRef destination = maxon::MediaOutputTextureClass().Create() iferr_return;
  destination.SetOutputTexture(targetTexture, maxon::ImagePixelStorageClasses::Normal()) iferr_return;
 
  // convert
 
  const maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
  session.ConnectMediaConverter(source, destination) iferr_return;
 
  session.Convert(maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
  session.Close() iferr_return;
 
  return maxon::OK;
}

// This example function saves the given maxon::ImageTextureRef as a PNG file using the given URL.
 
//----------------------------------------------------------------------------------------
// Saves the given image as a PNG file.
// @param[in] url               The URL of target image file.
// @param[in] image             The image to save.
// @return                      OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> SaveAsPNG(const maxon::Url& url, maxon::ImageTextureRef image)
{
  iferr_scope;
 
  if (url.IsEmpty() || image.IsEmpty())
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
  // define destination
  const maxon::MediaOutputUrlRef destination = maxon::ImageSaverClasses::Png().Create() iferr_return;
 
  // convert
  maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
  // ImageTextureInterface::Save() sets the path of the given MediaOutputUrlRef
  // and creates the needed MediaInputRef internally
  image.Save(url, destination, maxon::MEDIASESSIONFLAGS::RUNONLYANALYZE, &session) iferr_return;
 
  session.Convert(maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
  session.Close() iferr_return;
 
  return maxon::OK;
}

// This example function saves the images of the given maxon::BaseArray to a MP4 file at the given URL.
 
//----------------------------------------------------------------------------------------
// Saves the given image sequence as a MP4 video.
// @param[in] url               The URL of target video file.
// @param[in] fps               The framerate of the video file.
// @param[in] images            The array of images to save into the video file.
// @return                      OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> SaveMP4Video(const maxon::Url& url, maxon::Int32 fps, maxon::BaseArray<maxon::ImageRef>& images)
{
  iferr_scope;
 
  const maxon::Bool invalidFps = fps <= 0;
 
  if (url.IsEmpty() || images.IsEmpty() || invalidFps)
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
  // export settings
  maxon::DataDictionary exportSettings;
  exportSettings.Set(maxon::MEDIASESSION::EXPORT::QUALITY, 1.0) iferr_return;
  exportSettings.Set(maxon::MEDIASESSION::EXPORT::DATARATE, maxon::Megabytes(1)) iferr_return;
 
  // first image
  const maxon::ImageRef firstImage = images[0];
 
  // define source
 
  const maxon::ImageTextureRef source = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
  const maxon::ImageRef        imageData = firstImage;
  source.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, imageData, maxon::ImageBaseRef()) iferr_return;
  source.Set(maxon::IMAGEPROPERTIES::IMAGE::FPS, maxon::Float(fps)) iferr_return;
  source.Set(maxon::IMAGEPROPERTIES::IMAGE::EXPORTSETTINGS, exportSettings) iferr_return;
 
  // define destination
 
  const maxon::Id MP4 = maxon::MEDIASESSION::MP4::EXPORT::GetId();
  const maxon::MediaOutputUrlRef destination = maxon::ImageSaverClasses::Get(MP4).Create() iferr_return;
 
  // check dimensions
  const maxon::DrawDimensionInt inputSize(firstImage.GetWidth(), firstImage.GetHeight());
  const maxon::DrawDimensionInt correctedSize = destination.CheckBitmapSize(exportSettings, inputSize) iferr_return;
  if (correctedSize != inputSize)
  {
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Invalid resolution."_s);
  }
 
  // convert
 
  maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
  source.Save(url, destination, maxon::MEDIASESSIONFLAGS::RUNONLYANALYZE, &session) iferr_return;
 
  // fps
  maxon::TimeValue frameDuration;
  frameDuration.SetSeconds(1.0 / maxon::Float(fps));
  maxon::TimeValue currentTime;
 
  // save all given images
  for (auto& image : images)
  {
    // get image
    // CopyImageData() is a custom utility function
    CopyImageData(image, imageData) iferr_return;
 
    // save image to file
    session.Convert(currentTime, maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
    currentTime += frameDuration;
  }
 
  return session.Close();
}

// This example function loads an image sequence found in the given folder.
// The images are loaded in the prepared ImageRef array.
 
//----------------------------------------------------------------------------------------
// Loads the image sequence found in the given folder.
// @param[in] folder            Folder containing an image sequence in the format "sequence_00.png".
// @param[out] images           Array with pre-allocated images to load the image data into.
// @return                      OK on success.
//----------------------------------------------------------------------------------------
static maxon::Result<void> LoadImageSequence(const maxon::Url& folder, maxon::BaseArray<maxon::ImageRef>& images)
{
  iferr_scope;
 
  if (folder.IsEmpty() || images.IsEmpty())
    return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);
 
  // load only as many images as the ImageRef array can hold
  const maxon::Int   firstFrame = 0;
  const maxon::Int   lastFrame  = images.GetCount() - 1;
  const maxon::Float fps = 25.0_f;
 
  // define source
 
  const maxon::String filename { "sequence_" };
  maxon::Url sequenceURL = (folder + maxon::Url(filename))iferr_return;
 
  maxon::String name = sequenceURL.GetName();
  // between "{" and "}" the number format is defined as used with FormatString().
  // the file name format is sequence_00.png, sequence_01.png, ...
  name += "@{2'0'}.png"_s;
  sequenceURL.SetName(name) iferr_return;
 
  sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_FPS, fps) iferr_return;
  sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_FIRSTFRAME, firstFrame) iferr_return;
  sequenceURL.Set(maxon::URLFLAGS::IMAGESEQUENCE_LASTFRAME, lastFrame) iferr_return;
 
  const maxon::FileFormatHandler imageSequence = maxon::FileFormatHandlers::MovieImageSequence();
  const maxon::MediaInputRef     source = imageSequence.CreateHandler<maxon::MediaInputRef>(sequenceURL) iferr_return;
 
  // define destination
 
  const maxon::ImageTextureRef       texture = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
  const maxon::MediaOutputTextureRef destination = maxon::MediaOutputTextureClass().Create() iferr_return;
  destination.SetOutputTexture(texture, maxon::ImagePixelStorageClasses::Normal()) iferr_return;
 
  // convert
 
  const maxon::MediaSessionRef session = maxon::MediaSessionObject().Create() iferr_return;
  session.ConnectMediaConverter(source, destination) iferr_return;
 
  // fps
  maxon::TimeValue frameDuration;
  frameDuration.SetSeconds(1.0 / fps);
  maxon::TimeValue currentTime;
 
  // load each frame
  for (maxon::Int frame = 0; frame <= lastFrame; ++frame)
  {
    // load frame
    session.Convert(currentTime, maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
    // get image
    const maxon::ImageRef image = maxon::GetImageOf(texture);
    // copy image to target array
    images[frame] = image.Clone() iferr_return;
 
    currentTime += frameDuration;
  }
 
  return session.Close();
}

Further Reading

• Files & Media
• MAXON API Files & Media
• Images Manual
• Url Manual