Draw HUD-like text to viewport

fwilleke80

Hello,

I need to draw text with a semi-transparent background that look exactly like the HUD. But I can not use DrawMultipleHUDText() for that, because it's super slow (I need to draw a lot of text entries).

I'm now using a GeClipMap to draw the text, then I get the clipmap's bitmap, and use BaseDraw::DrawTexture() for the actual drawing. Still, to get the semi-transparent background of the HUD, I need a proper alpha channel. Using GeClipMap::SetPixelRGBA() does not help, since the GeClipMap is not able to return a BaseBitmap with alpha channel.

The next thing I tried is using bitmap->AddChannel(true, true) to create an alpha channel. I then iterate over all pixels in the bitmap, and set values to the alpha channel depending on the pixel values in the bitmap. The values are being set correctly, I checked that, but in the viewport there seems to be only "visible" and "invisible", no semi-transparency. Of course, I use DRAW_ALPHA::NORMAL in the DrawTexture() call.

The only thing that gives me semi-transparency is using DRAW_ALPHA::FROM_IMAGE. But then the alpha is depending on the actual pixel colors in the bitmap, and I couldn't find any color that would give me the exact HUD look. It is fast, though, much faster than DrawMultipleHUDText().

Another problem is that the text does look almost perfectly like the HUD (close enough at least), but only on computers with retina screen. On computers with low-res screens, it looks washed out and blurred. That is, if I use DRAW_TEXTUREFLAGS::INTERPOLATION_LINEAR. Using DRAW_TEXTUREFLAGS::INTERPOLATION_NEAREST makes the text look broken, as if it had been scaled down. But I don't scale it, it should be drawn exactly in the right size (see me using the bitmaps width and height for the texture vertex coordinates).

Here's the code for texture creation:

BaseBitmap* MyClass::CreateTextBitmap(const String &text, const Vector &textColor, const Vector &bgColor, Float bgOpacity, Int32 margin)
{
	// _clipmap is a member of MyClass
	if (!_clipmap)
	{
		_clipmap.Assign(GeClipMap::Alloc());
		if (!_clipmap)
			return nullptr;
	}

	// Dummy draw, just to get the text dimensions
	if (_clipmap->Init(0, 0, 32) != IMAGERESULT::OK)
		return nullptr;
	_clipmap->BeginDraw();
	Int32 width = _clipmap->GetTextWidth(text) + margin * 2;
	Int32 height = _clipmap->GetTextHeight() + margin * 2;
	_clipmap->EndDraw();
	_clipmap->Destroy();

	// Begin actual drawing
	if (_clipmap->Init(width, height, 32) != IMAGERESULT::OK)
		return nullptr;
	_clipmap->BeginDraw();
	_clipmap->EndDraw();
	
	width = _clipmap->GetBw();
	height = _clipmap->GetBh();
	_clipmap->BeginDraw();

	// Background. Fill clipmap with a rectangle, overpainting possibly existing text
	// Drawn with alpha = 255, because this alpha value just defines the transparency of the drawn color over the background.
	// The actual background alpha is set later.
	const Int32 backgroundColorR = (Int32)(bgColor.x * 255.0);
	const Int32 backgroundColorG = (Int32)(bgColor.y * 255.0);
	const Int32 backgroundColorB = (Int32)(bgColor.z * 255.0);
	_clipmap->SetColor(backgroundColorR, backgroundColorG, backgroundColorB, 255);
	_clipmap->FillRect(0, 0, width - 1, height - 1);

	const Int32 backgroundOpacity = (Int32)(bgOpacity * 255.0);

	// Text
	// Drawn with alpha = 255, because this alpha value just defines the transparency of the drawn color over the background.
	_clipmap->SetColor((Int32)(textColor.x * 255.0), (Int32)(textColor.y * 255.0), (Int32)(textColor.z * 255.0), 255);
	_clipmap->TextAt(margin, margin, text);

	// Make corner pixels transparent (looks more like HUD then)
	_clipmap->SetColor(0, 0, 0, 255);
	_clipmap->SetPixel(0, 0);
	_clipmap->SetPixel(0, height - 1);
	_clipmap->SetPixel(width - 1, 0);
	_clipmap->SetPixel(width - 1, height - 1);

	_clipmap->EndDraw();

	// Get bitmap from clipmap
	BaseBitmap *bitmap = _clipmap->GetBitmap()->GetClone();
	if (!bitmap)
		return nullptr;

	// Iterate over pixels, set the actual alpha values
	BaseBitmap *alphaBitmap = bitmap->AddChannel(true, true);
	if (!alphaBitmap)
		return nullptr;
	alphaBitmap = bitmap->GetInternalChannel();
	if (!alphaBitmap)
		return nullptr;

	for (Int32 y = 0; y < bitmap->GetBh(); ++y)
	{
		for (Int32 x = 0; x < bitmap->GetBw(); ++x)
		{
			UInt16 r = 0, g = 0, b = 0;
			bitmap->GetPixel(x, y, &r, &g, &b);

			if (r == backgroundColorR && g == backgroundColorG && b == backgroundColorB)
				bitmap->SetAlphaPixel(alphaBitmap, x, y, 255 - backgroundOpacity);
			else if (r == 0 && g == 0 && b == 0)
				bitmap->SetAlphaPixel(alphaBitmap, x, y, 0);
			else
				bitmap->SetAlphaPixel(alphaBitmap, x, y, 255);
		}
	}
	
	return bitmap;
}

And this is the code I use for drawing:

void MyClass::DrawRect(BaseDraw *bd, BaseBitmap *bmp, const Vector* padr4, const Vector &color, DRAW_ALPHA alphamode, DRAW_TEXTUREFLAGS flags)
{
	if (!bd || !bmp || !padr4)
		return;
	
	// Vertex normal array
	static const Vector vnadr[4] = {
		Vector(0.0, 1.0, 0.0),
		Vector(0.0, 1.0, 0.0),
		Vector(0.0, 1.0, 0.0),
		Vector(0.0, 1.0, 0.0)
	};
	
	// UV coordinate array
	static const Vector uvadr[4] = {
		Vector(0.0, 0.0, 0.0),
		Vector(1.0, 0.0, 0.0),
		Vector(1.0, 1.0, 0.0),
		Vector(0.0, 1.0, 0.0)
	};
	
	// Color array
	const Vector cadr[4] = {
		color,
		color,
		color,
		color
	};
	
	bd->DrawTexture(bmp, padr4, cadr, vnadr, uvadr, 4, alphamode, flags);
}

And here's the code that starts everything inside my SceneHook plugin's Draw() call:

// Set draw matrix to screen space, prevent drawn elements from being lit
bd->SetMatrix_Screen();
bd->SetLightList(BDRAW_SETLIGHTLIST_NOLIGHTS);

String valueText = FormatNumber(123.45, FORMAT_METER, 25);
BaseBitmap *valueTexture(CreateTextBitmap(valueText, Vector(0.9), Vector(0.25), 0.2, 2));

const Float valueTextureWidth = valueTexture->GetBw();
const Float valueTextureHeight = valueTexture->GetBh();

const Vector textPadr[4] = {
	textUpperLeftCornerPos,
	textUpperLeftCornerPos + Vector(valueTextureWidth, 0.0, 0.0),
	textUpperLeftCornerPos + Vector(valueTextureWidth, valueTextureHeight, 0.0),
	textUpperLeftCornerPos + Vector(0.0, valueTextureHeight, 0.0),
};

const Vector hudTextColor = bd->GetParameterData(BASEDRAW_HUD_TEXTCOLOR).GetVector();

DrawRect(bd, valueTexture, textPadr, hudTextColor, DRAW_ALPHA::NORMAL, DRAW_TEXTUREFLAGS::INTERPOLATION_LINEAR);

Any help would be very appreciated. As I said, the only goal here is just to have a function that draws text with semi-transparent background into the viewport that looks exactly like the HUD, just faster.

Cheers,
Frank

C4DS

Hi,
This is what I use to copy the alpha over and I do get transparency. But I only use filled colored areas, no text, still I think this should provide the same result.

	UInt16 r, g, b, a;
	Int32 destX, destY;
	for (Int32 y = 0; y < sourceBmp->GetBh(); ++y)
	{
		for (Int32 x = 0; x < sourceBmp->GetBw(); ++x)
		{
			sourceBmp->GetAlphaPixel(alphaSource, x, y, &a);
			if (a)
			{
				sourceBmp->GetPixel(x, y, &r, &g, &b);
				destBmp->SetPixel(destX, destY, r, g, b);
				destBmp->SetAlphaPixel(alphaDest, destX, destY, a);
			}
		}
	}

EDIT: however, I don't see any alpha values different from 255 in your clipmap, or am I missing something?

fwilleke80

Hi & thank you for the reply!

That is, I guess pretty much what I do. Only you are copying alpha values from another BaseBitmap, and I need to make up alpha values based on the color of pixels in the BaseBitmap I got from the GeClipMap. As long as my alpha values are in the range of 0 ... 255 (which they are), I should see at least some kind of semi transparency, right?

So, my question remains Maybe the SDK team knows some details?

Thanks & greetings,
Frank

ferdinand

Hi,

Maybe I am misunderstanding something, but you were talking about BaseBitmap::AddChannel, which will add an additional alpha channel as supported by the psd or tif format for example. BaseBitmap::SetAlphaPixel will write to the primary alpha channel as supported by many image formats. So there is a major difference between @C4DS's and your code.

Cheers,
zipit

fwilleke80

Oh, ok. As far as I understood the docs, BaseBitmap::AddChannel() would add an alpha channel, and passing true as the first argument would make this the "internal" alpha channel. After all, before I can set any alpha value with BaseBitmap::SetAlphaPixel() I would need an alpha channel to set the data to, right? BaseBitmap::GetInternalChannel() gives me nullptr if I don't call BaseBitmap::AddChannel() first.

And no, as I see it, BaseBitmap::SetAlphaPixel() does not always write to the primary alpha channel, but to the channel I pass to the function. If I pass nullptr, no alpha value seems to be written.

ferdinand

@fwilleke80 said in Draw HUD-like text to viewport:

As far as I understood the docs, BaseBitmap::AddChannel() would add an alpha channel, and passing true as the first argument would make this the "internal" alpha channel.

You are absolutely right, sorry, I did not read your posting carefully enough. But I also have drawn a bitmap to viewport with the approach shown by @C4DS, so it still might be worth a try.

Cheerss,
zipit

fwilleke80

But that is how I am doing it, too. Using BaseBitmap::SetAlphaPixel() to set alpha values into the bitmap's internal alpha channel. I am getting my values from another source (because I'm not copying a bitmap, but creating a new one), but they are still in the correct value range. I have even tried setting all pixels to the same alpha value (e.g. 128) and still did not get a semi transparent result in the viewport.

fwilleke80

Hm... interesting. I am getting a promising result now with this code:

BaseBitmap* MyClass::CreateTextBitmap(const String &text, const Vector &textColor, const Vector &bgColor, Float bgOpacity, Int32 margin)
{
	if (!_clipmap)
	{
		_clipmap.Assign(GeClipMap::Alloc());
		if (!_clipmap)
			return nullptr;
	}

	// Dummy draw, just to get the text dimensions
	if (_clipmap->Init(0, 0, 32) != IMAGERESULT::OK)
		return nullptr;
	_clipmap->BeginDraw();
	Int32 width = _clipmap->GetTextWidth(text) + margin * 2;
	Int32 height = _clipmap->GetTextHeight() + margin * 2;
	_clipmap->EndDraw();
	_clipmap->Destroy();

	// Begin actual drawing
	if (_clipmap->Init(width, height, 32) != IMAGERESULT::OK)
		return nullptr;
	_clipmap->BeginDraw();
	_clipmap->EndDraw();
	
	width = _clipmap->GetBw();
	height = _clipmap->GetBh();
	_clipmap->BeginDraw();

	// Background. Fill clipmap with a rectangle, overpainting possibly existing text
	// Drawn with alpha = 255, because this alpha value just defines the transparency of the drawn color over the background.
	// The actual background alpha is set later.
	const Int32 backgroundColorR = (Int32)(bgColor.x * 255.0);
	const Int32 backgroundColorG = (Int32)(bgColor.y * 255.0);
	const Int32 backgroundColorB = (Int32)(bgColor.z * 255.0);
	_clipmap->SetColor(backgroundColorR, backgroundColorG, backgroundColorB, 255);
	_clipmap->FillRect(0, 0, width - 1, height - 1);

	const Int32 backgroundOpacity = (Int32)(bgOpacity * 255.0);

	// Text
	// Drawn with alpha = 255, because this alpha value just defines the transparency of the drawn color over the background.
	_clipmap->SetColor((Int32)(textColor.x * 255.0), (Int32)(textColor.y * 255.0), (Int32)(textColor.z * 255.0), 255);
	_clipmap->TextAt(margin, margin, text);

	_clipmap->EndDraw();

	BaseBitmap *bitmap = _clipmap->GetBitmap()->GetClone();
	if (!bitmap)
		return nullptr;

	BaseBitmap *alphaBitmap = bitmap->AddChannel(true, true);
	if (!alphaBitmap)
		return nullptr;
	alphaBitmap = bitmap->GetInternalChannel();
	if (!alphaBitmap)
		return nullptr;

	for (Int32 y = 0; y < bitmap->GetBh(); ++y)
	{
		for (Int32 x = 0; x < bitmap->GetBw(); ++x)
		{
			// 100% transparent corners
			if ((x == 0 && y == 0) || (x == width - 1 && y == height - 1) ||
				(x == 0 && y == height - 1) || (x == width - 1 && y == 0))
			{
				bitmap->SetAlphaPixel(alphaBitmap, x, y, 0);
			}
			else // All other pixels: semi transparent
			{
				bitmap->SetAlphaPixel(alphaBitmap, x, y, backgroundOpacity);
			}
		}
	}
	
	return bitmap;
}

I wonder why this looks so different... I still didn't set opaque pixels for the text, but it seems to be a good start...

fwilleke80

What I also find weird is, that even if I use the DrawMultipleHUDText() function, the stuff that appears in the viewport still looks different from the standard HUD elements like "Projection" or "FPS". It's a bit darker and the margin around the text looks different.

r_gigante

Hi Frank, thanks for reaching out us.

I've played a bit with and I came to this other solution which make no use of BaseBitmap and should provide more coherent results UI-wise.

Looking forward checking how it compares with regard to performances.

maxon::Result<void> OptimizedDrawMultipleHUDText(const maxon::BaseArray<HUDTextEntry>& texts, BaseDraw* bd)
{
	if (!bd || texts.IsEmpty())
		return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION);

	AutoAlloc<GeClipMap> bgClipMap;
	if (!bgClipMap)
		return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);

	AutoAlloc<GeClipMap> clipMap;
	if (!clipMap)
		return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);
	
	// get the current view region
	Int32 cr, cl, cb, ct;
	bd->GetFrame(&cl, &ct, &cr, &cb);
	
	// calculate view sizes
	const Int32 viewW = (cr - cl + 1);
	const Int32 viewH = (cb - ct + 1);
	
	// set the BaseDraw matrix and lighting method
	bd->SetMatrix_Screen();
	bd->SetLightList(BDRAW_SETLIGHTLIST_NOLIGHTS);

	// Init HUD clipmap
	if (clipMap->Init(viewW, viewH, 32) != IMAGERESULT::OK)
		return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

	clipMap->BeginDraw();

	const Vector 	textColor = 255 * bd->GetParameterData(BASEDRAW_HUD_TEXTCOLOR).GetVector();
	const Vector 	bgColor = 255 * bd->GetParameterData(BASEDRAW_HUD_BACKCOLOR).GetVector();
	const Float 	bgTransparency = 255 * bd->GetParameterData(BASEDRAW_HUD_BACKOPACITY).GetFloat();
	
	const Int32 textH = clipMap->GetTextHeight();
	const Int32	drawH = textH + 10;
	const Int32 tX = 5;
	const Int32	tY = (drawH - textH) / 2;

	// set the complete image transparent
	clipMap->SetColor((Int32)bgColor.x, (Int32)bgColor.y, (Int32)bgColor.z, 0);
	clipMap->FillRect(cl, ct, cr, cb);

	for (const auto& it : texts)
	{
		// get text sizes
		const Int32 textW = clipMap->GetTextWidth(it._txt);
		const Int32 drawW = textW + 10;

		// init BG clip
		if (bgClipMap->Init(drawW, drawH, 32) != IMAGERESULT::OK)
			return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

		// create the BG
		bgClipMap->BeginDraw();
		bgClipMap->SetColor((Int32)bgColor.x, (Int32)bgColor.y, (Int32)bgColor.z, (Int32)bgTransparency);
		bgClipMap->FillRect(0, 0, drawW, drawH);
		bgClipMap->EndDraw();
		
		// blit BG
		clipMap->Blit((Int32)it._position.x, (Int32)it._position.y, *bgClipMap, 0, 0, drawW, drawH, GE_CM_BLIT::COL);
		
		// set text color
		clipMap->SetColor((Int32)textColor.x, (Int32)textColor.y, (Int32)textColor.z, 255);
		
		// draw Text
		clipMap->TextAt((Int32)it._position.x + tX, (Int32)it._position.y + tY, it._txt);
	}

	clipMap->EndDraw();
	
	// set the points coordinates
	const Vector 	pvText[4] = {
		Vector((Float)cl, (Float)ct, 0),
		Vector((Float)cl, (Float)ct, 0) + Vector(viewW, 0.0, 0.0),
		Vector((Float)cl, (Float)ct, 0) + Vector(viewW, viewH, 0.0),
		Vector((Float)cl, (Float)ct, 0) + Vector(0.0, viewH, 0.0)};
	
	// set the UVs coordinates
	const Vector 	pvUV[4] = {
		Vector(0, 0, 0),
		Vector(1, 0, 0),
		Vector(1, 1, 0),
		Vector(0, 1, 0)};

	// draw in Viewport
	bd->DrawTexture(clipMap->GetBitmap(), pvText, nullptr, nullptr, pvUV, 4, DRAW_ALPHA::NORMAL, DRAW_TEXTUREFLAGS::INTERPOLATION_NEAREST | DRAW_TEXTUREFLAGS::TEMPORARY);

	return maxon::OK;
}

Best, R

fwilleke80

Thank you Ricardo! This looks much simpler than the code we're currently using. I'll check it out!

fwilleke80

Hm, not bad. In deed, it has the potential to look more like the "normal" HUD, without doing hacky stunts. And it is a bit faster than DrawMultipleHUDText().

However, even if I only draw 4 texts using this code, the framerate on my iMac is down to 24 fps (tested in R21). Using my own code, I get 127 fps. I'll see if it can be optimised more (e.g. by making the clip map a member of MyClass so it does not have to be allocated for every draw call.

Cheers,
Frank

EDIT: And it gets slower in a linear way. drawing 4 texts only gives me 12 fps. So, it looked promising, and I thank you very much for the code, as it gives me some new inspiration on how to do text drawing in the viewport), but for what I need to do (drawing > 30 texts on average), it is not suitable.