[SOLVED]Know the change of bitmap

Helper

On 15/12/2016 at 01:01, xxxxxxxx wrote:

Hello I would like to know if it's possible to catch the msg when a bitmap is reloaded / created / deleted

Here is my project. We currently have huge slow down when we use the texture manager from c4d(cause it's refresh every time you move it or you do something in the UI), and since we got a lot of relative texture path wich can be on a lot of directory.
It often just take 40/50s just to show up and as said at each move we have to rewait 40/50s wich is not possible for us.

So I firstly thinked about rebuilding the texture manager witout the auto refresh... (Don't know if it's the good place for c4d feature request but it should be really good to have an option for disallow this.)
Then I thinked about another way for removing extra time. Wich will be at each time user modifiate path / reload a Xbitmap it will search for the path and then add to an array.

Technically I thinked about using messageplugin to catch the event about the change on any xbitmap (but is there a way for knowing which BaseList2d invok the event?) and then create background thread for the search like that it will not freeze the ui.

Again I'm not sticky to python... But since I never really dealing with thread (even if I understand the concept I never have to handle them. So handle them in c++ afraid me a bit ^^)

Btw can you confirm me the order for checking texture are still this one

Originally posted by xxxxxxxx

suggestedfolder + srcname

if srcname has a path, it tries to use it as is

GetDocumentPath() and GetDocumentPath()/tex

GeGetStartupWritePath() and GeGetStartupWritePath()/tex

GeGetStartupPath() and GeGetStartupPath()/tex

then all GetGlobalTexturePath() recursively

and as if this wasn't enough, it checks for a NetRenderService context and runs through all assets using ConvertRelative2Absolute()

Thanks in advance !
Gonna look to the chrismas competition for a training about threading gonna be a good idea

Helper

On 16/12/2016 at 03:06, xxxxxxxx wrote:

Anyway a litlle feature request it would be nice to have ExistPreset function avialbe in python. have https://developers.maxon.net/docs/cpp/2023_2/class_s_d_k_browser.html#a090f35abf2a3d9a6d6b73faa1e96001f

As a workaround I'm doing this but I think is way more slow than just a check.

        src = c4d.bitmaps.BaseBitmap()
        if src.InitWith(preset)[0]==c4d.IMAGERESULT_OK:

Helper

On 16/12/2016 at 04:03, xxxxxxxx wrote:

Hi,

There's no specific message when a bitmap shader is modified. You have to catch EVMSG_CHANGE and check each bitmap shader in the document.
To optimize the search for updated shaders, check shader.GetDirty(c4d.DIRTYFLAGS_DATA). The number returned is incremented every time the shader is changed.
And I think you shouldn't use multithreading in your first implementation.

You no longer have to manually build textures file paths. Since R16.50 c4d.GenerateTexturePath() is available.

About feature requests, you can make suggestions using this form.

I'll add ExistPreset() to our Python Parity list.

Helper

On 16/12/2016 at 04:43, xxxxxxxx wrote:

Thanks for the answerd.

For the moment I just rebuild the texture manager.

For my use c4d.GenerateTexturePath is not usuable. Since it's only a os.path.join like function or maybe I missunderstand it.
Just create a an standar material with a bitmap locate in a globalTexturePath

    docpath = doc.GetDocumentPath()
    srcname = doc.GetFirstMaterial().GetFirstShader()[c4d.BITMAPSHADER_FILENAME]
    suggestedfolder = None
    print c4d.GenerateTexturePath(docpath, srcname, suggestedfolder) #return None

But this is not my problem at the moment.
Moreover I don't want to use the GetAllAssets since it's very slow (took 1m15sec on my scene) and with my script event if I laod bitmap it took only 20sec. And adding like 1000 random texture just take 35sec while the GetAllAssets freeze during 3min...

I have another question.
Is it possible to know if a shader is used by a material
At the moment I just grab all bitmapshader from all material.
But I have a shader inserted into a material wich is not used (I can't see it in any link of my shader).
Is there a way for identify shuch shader like that I can delete them from my list to check.

Thanks in advance !

Helper

On 19/12/2016 at 09:09, xxxxxxxx wrote:

Hi,

When you call GenerateTexturePath() pass an empty string for suggestedfolder instead of None. This way the the expected texture filename will be returned.
And GenerateTexturePath() doesn't just concatenate path and file names. It's really more complex then that and performs the steps you quoted in your initial post.

Materials, not shaders, are assigned to objects. So you can only check if a material is used, then you can discard it and all its shaders.
To check if a material is used, retrieve the state of its BIT_MATMARK like this:

import c4d
  
# First, reset BIT_MATMARK of all materials
mat = doc.GetFirstMaterial()
while mat: 
    mat.DelBit(c4d.BIT_MATMARK)
    mat = mat.GetNext()
  
# Send MSG_MULTI_MARKMATERIALS message to document
if doc.MultiMessage(c4d.MULTIMSG_ROUTE_BROADCAST, c4d.MSG_MULTI_MARKMATERIALS) :
    mat = doc.GetFirstMaterial()
    while mat: 
        # Check BIT_MATMARK
        print "Material '" + mat.GetName() + "' Used: ", mat.GetBit(c4d.BIT_MATMARK)
        mat = mat.GetNext()

Helper

On 19/12/2016 at 09:42, xxxxxxxx wrote:

Thanks about GenerateTexturePath() gonna try on huge scene to check if it's slower than my implementation or not. (Since I'm not at work I will answerd to this only in few week but anyway thanks for the tips).
And it's seem to check if the file exist?Is it trustable or it's better to do a os.path.exists after it?
Does it also, check if presset exist? This could be a really better workaround for me to check presset file with this function than with the one I did before.

About the second point I really speak about shader not material.
It's like someone have done this.

mat = doc.GetFirstMaterial()
shd = c4d.BaseList2D(c4d.Xbitmap)
mat.InsertShader(shd)
c4d.EventAdd()

So that lead to a shader who is inserted into a material but wich is not used at anytime...

But your script would be usefull for another project then thanks you !

Helper

On 20/12/2016 at 03:55, xxxxxxxx wrote:

Originally posted by xxxxxxxx

And it's seem to check if the file exist?Is it trustable or it's better to do a os.path.exists after it?

Yes GenerateTexturePath() checks if the texture file exists.

Originally posted by xxxxxxxx

Does it also, check if presset exist? This could be a really better workaround for me to check presset file with this function than with the one I did before.

That's not what GenerateTexturePath() is meant to do. It returns the full path to a texture or another asset used in a document.

Originally posted by xxxxxxxx

About the second point I really speak about shader not material.
It's like someone have done this.
mat = doc.GetFirstMaterial()
shd = c4d.BaseList2D(c4d.Xbitmap)
mat.InsertShader(shd)
c4d.EventAdd()
So that lead to a shader who is inserted into a material but wich is not used at anytime...

Thanks for the explanation. Checking if a shader inserted into a material is really used is a complex task.
The easiest checks would be to test if the shader is used in any of the material's channel.

Helper

On 04/01/2017 at 03:28, xxxxxxxx wrote:

Thanks for the answer.
Finally I managed to get it working pretty fast and the most important not so much related to textures count using a combination of methods.
I list all textures and try to find them in all searchPath(see first post for know wich one).Like that I do only 1 os.walk(wich is the slower part of the code of course) and then for extra_checking I do GenerateTexturePath for some missings textures.
This finally give me the same output as the Texture manager from c4d.
But mine load in 18sec, where it took 1min30 with the default texture manager.

Moreover is there a way for use GenerateTexturePath not on the main thread? Because for the moment even if I use my own thread it's freeze everything
Here is some code maybe it would be more understandable, ofc this not my full code, even if I plan for a final open source project I would release it when it will be full working.

#here my threading attemps
class MyThread(c4d.threading.C4DThread) :
    utility = Utility()
  
    texture_found = list()
    texture_to_relocate = list()
    texture_not_found = list()
  
    create_ui = None
  
    def Main(self) :
        c4d.StatusSetSpin()
        self.texture_found, self.texture_to_relocate, self.texture_not_found = self.utility.get_texture_data(self.Get())
        c4d.StatusClear()
  
#the main function only a little part
class Utility(object) :
    def get_texture_data(self, thread) :
        all_tex = self.__get_all_texture()
  
        texture_found = []
        texture_to_relocate = []
        texture_not_found = []
  
        doc = c4d.documents.GetActiveDocument()
        docpath = doc.GetDocumentPath()
        suggestedfolder = str()
        for tex in all_tex:
            texture = c4d.GenerateTexturePath(docpath, tex["path"], suggestedfolder, bt=thread)
  
#And where I make the lunch of my Dialog
class Dialog(c4d.gui.GeDialog) :
    utility = Utility()
  
    texture_found = list()
    texture_to_relocate = list()
    texture_not_found = list()
  
    can_be_closed = True
    thread_get_texture = None
  
    def get_all_textures(self, is_update) :
        self.can_be_closed = False
        c4d.StatusSetSpin()
  
        self.thread_get_texture = MyThread()
        if is_update:
            self.thread_get_texture.create_ui = False
        else:
            self.thread_get_texture.create_ui = True
  
        self.thread_get_texture.Start()
  
    def CreateLayout(self) :
        ###SOME STUFF###
        self.get_all_textures(False)
        return True

Thanks in advance !

Helper

On 05/01/2017 at 04:45, xxxxxxxx wrote:

With a similar code I can't get a freeze calling c4d.GenerateTexturePath() on a custom thread.
So I don't think the issue is inside the code you posted.
How do you check the custom thread has finished its processing?

Helper

On 05/01/2017 at 06:17, xxxxxxxx wrote:

I simply not check.
When the code is over at the end of my function I do A specialEventAdd()
Then in my dialog I catch this event and fill my UI.

I uplaoded on github a bigger part of my code. It's not 100% bug free, but I hope this will help you to understand a litlle bit better how everything works.
https://github.com/gr4ph0s/C4D-TextureChecker

Anyway thanks in advance !

Helper

On 10/01/2017 at 06:06, xxxxxxxx wrote:

Ok don't know exactly what change made this happen(you know this feling when something work when normally it should not ^^) but now my code is really executed into another thread so my UI is not anymore freezing.
I still get some crash regarding threading but it's 100% an error from me, I will came back if I don't find the solution.

I got another question. In our library we got some corrompteds textures. Wich load to an incapability of C4D to export/use them. Is there an easy way for find those textures excepting with a basebitmap->InitWith? (wich is pretty slow on big file).

Is there a way for listing all assets in a library (tipicaly do a os.walk on preset?) like that I could remove the corrompted data from the library.

Thanks in advance

EDIT:
I have figure out which line make c4d crash.

bmp = c4d.bitmaps.BaseBitmap()
def __test_valid_picture(self, picture_path) :
	ascii_path = picture_path.encode('utf8')
	result = self.bmp.InitWith(ascii_path) #crash here
	self.bmp.FlushAll()
	if result[0] == c4d.IMAGERESULT_OK:
		return True
	else:
		return False

The weirdest thing is I checked more than 350 textures. And c4d crash always with the same one. So maybe is not a crash related to the thread but to the texture (wich is 100% ok, render fine when appliqued to a plan).
And it's not crashing everytime. Sometime the problematic texture load correctly and not crash c4d.

I also updated the github if you want to check
I still have to do some code cleanup.

EDIT 2:
Executing the code posted previously from the main thread don't seem to crash.

EDIT 3:
Having only the problematic texture in the scene and with my multi-threaded code it's seem to not crash
The most strange thing it's crashing while he already loaded the bitmap. And it seem to always crash on the 4th iteration of the map.
Look at my screen http://img11.hostingpics.net/pics/432714crash.png and I get this code

    def __test_valid_picture(self, picture_path) :
        bmp = c4d.bitmaps.BaseBitmap()
        print 'b'
        ascii_path = picture_path.encode('utf8')
        result = bmp.InitWith(ascii_path)
        print 'a'
        bmp.FlushAll()
        del(bmp)
        if result[0] == c4d.IMAGERESULT_OK:
            return True
        else:
            return False

Is their some threading limitation for using InitWith in a custom thread?

Helper

On 13/01/2017 at 06:56, xxxxxxxx wrote:

Sorry for the triple post.
But GenerateTexturePath on another thread when image don't exist make c4d crash.

Here is a short exemple that make c4d crash

# encoding: utf-8
import c4d
import os
from ctypes import pythonapi, c_void_p, py_object
  
PLUGIN_ID = 1038562
  
class MyThread(c4d.threading.C4DThread) :
    def __test_valid_picture(self, picture_path) :
        bmp = c4d.bitmaps.BaseBitmap()
        ascii_path = picture_path.encode('utf8')
        result = bmp.InitWith(ascii_path)
        bmp.FlushAll()
        del(bmp)
        if result[0] == c4d.IMAGERESULT_OK:
            return True
        else:
            return False
  
    def get_texture_data(self, thread) :
        doc = c4d.documents.GetActiveDocument()
        docpath = doc.GetDocumentPath()
        suggestedfolder = str()
  
        path = "qsdsqdsdqdqzz.jpg" #random name. in the case where picture don't exist.
        texture = c4d.GenerateTexturePath(docpath, path, suggestedfolder, bt=thread) #this line crash
  
        self.__test_valid_picture(path)
  
        return True
  
    def Main(self) :
        c4d.StatusSetSpin()
        self.get_texture_data(self.Get())
        c4d.StatusClear()
  
        c4d.SpecialEventAdd(PLUGIN_ID, p1=1)
  
  
class Dialog(c4d.gui.GeDialog) :
    thread_get_texture = None
  
    def CreateLayout(self) :
        c4d.StatusSetSpin()
  
        if self.thread_get_texture is None:
            self.thread_get_texture = MyThread()
            self.thread_get_texture.Start(c4d.THREADMODE_ASYNC)
        else:
            self.thread_get_texture.End(True)
            self.thread_get_texture = MyThread()
            self.thread_get_texture.Start(c4d.THREADMODE_ASYNC)
  
        return True
  
    def AskClose(self) :
        self.thread_get_texture.need_to_close = True
        self.thread_get_texture.End(True)
        return False
  
    def CoreMessage(self, id, msg) :
        if id == PLUGIN_ID:
            P1MSG_UN = msg.GetVoid(c4d.BFM_CORE_PAR1)
  
            pythonapi.PyCObject_AsVoidPtr.restype = c_void_p
            pythonapi.PyCObject_AsVoidPtr.argtypes = [py_object]
            P1MSG_EN = pythonapi.PyCObject_AsVoidPtr(P1MSG_UN)
  
            if P1MSG_EN == 1:
                if self.thread_get_texture.IsRunning() :
                    self.thread_get_texture.need_to_close = True
                    self.thread_get_texture.End()
  
                self.can_be_closed = True
        return True
  
  
class Launcher(c4d.plugins.CommandData) :
    dialog = None
  
    def Execute(self, doc) :
        if self.dialog is None:
            self.dialog = Dialog()
        return self.dialog.Open(dlgtype=c4d.DLG_TYPE_ASYNC, pluginid=PLUGIN_ID, defaultw=200, defaulth=50, xpos=-1,
                                ypos=-1)
  
    def RestoreLayout(self, sec_ref) :
        if self.dialog is None:
            self.dialog = Dialog()
        return self.dialog.Restore(pluginid=PLUGIN_ID, secret=sec_ref)
  
  
if __name__ == "__main__":
    c4d.plugins.RegisterCommandPlugin(PLUGIN_ID, "00 - Texture Manager", 0, None, None, Launcher())

Thansk in advance !

Helper

On 16/01/2017 at 02:31, xxxxxxxx wrote:

Thanks for posting the code.
I'm afraid there's an internal issue passing bt thread parameter of c4d.GenerateTexturePath().
This bug will be fixed.

Helper

On 16/01/2017 at 03:29, xxxxxxxx wrote:

Using the python threading module can be a workaround? And is it really bad to use python threading module instead of the c4d one?
Moreover this fix will be aviable in all c4d versions(all version contening this functions ofc) or just R18?

Anyway thanks for your answer

Helper

On 17/01/2017 at 03:26, xxxxxxxx wrote:

Using the python threading module can't be a workaround because GenerateTexturePath() only accepts a BaseThread.
It's recommended to use the c4d.threading classes from the Cinema Python API because these are the supported internally.

Note we don't backport fixes to past releases of Cinema 4D. So GenerateTexturePath() will be fixed in the next update of R18 and future releases.

Helper

On 17/01/2017 at 03:36, xxxxxxxx wrote:

Thanks for the infos !