R16->R17 Problem stripduplicates KeyError: '_[1]'

Helper

On 19/10/2015 at 02:02, xxxxxxxx wrote:

I get an error in my Script which worked fine in R16.
Hence this is one of the parts where I am winging it I need your help.

biglist is a list of c4d.vectors (was a bitmap)
I am stripping duplicated colors of this list.

tahnks in advance
kind regards mogh

  
def stripduplicates(biglist) :   
    newlist = [ item for item in biglist if item not in locals()['_[1]'] ]   
    return newlist

Traceback (most recent call last) :
File "'scriptmanager'", line 374, in <module>
File "'scriptmanager'", line 320, in main
File "'scriptmanager'", line 216, in getpixelcolor
File "'scriptmanager'", line 234, in stripduplicates
KeyError: '_[1]'

Helper

On 19/10/2015 at 03:07, xxxxxxxx wrote:

found something but it doesn't help me much
http://stackoverflow.com/questions/9631777/what-does-locals-1-mean-in-python

now i need to find a better solution to strip the duplicate vectors from biglist.
...

kind regards mogh

Helper

On 20/10/2015 at 01:49, xxxxxxxx wrote:

Hi,

you already found the reason yourself. With Cinema 4D R17 the internal Python implementation moved to Python 2.7 and your construct is no longer valid in Python 2.7.

An elegant way to remove duplicated from a list in Python would be:

l2 = list(set(l))

But that doesn't work in your case as c4d.Vector is not hashable.
So you could use something like this:

    l = [ ... some list content...]
    l2 = []
    for i in l:
       if i not in l2:
          l2.append(i)

or in one line:

l3 = [ii for n,ii in enumerate(l) if ii not in l[:n]]

But these may be slow with large lists. So, I leave it to the community to come up with a nifty solution.

Helper

On 20/10/2015 at 03:51, xxxxxxxx wrote:

Originally posted by xxxxxxxx

But these may be slow with large lists. So, I leave it to the community to come up with a nifty solution.

If stay in scripting environment and prepare or download a few extensions for python, like cython or custom made cuda versions of it... i made a few tests with code with cpu-cython and hybrid(cpu-gpu) cython. And found inline calculation in cython is prety fast

also i'm interesting to calculate large data sets by embree core, but not test

Helper

On 20/10/2015 at 04:17, xxxxxxxx wrote:

Originally posted by xxxxxxxx
l3 = [ii for n,ii in enumerate(l) if ii not in l[:n]]
But these may be slow with large lists. So, I leave it to the community to come up with a nifty solution.

This would indeed be very slow since the slicing on l would always create a new list. This would work
fast on large lists:

from itertools import islice
l2 = [v for n, v in enumerate(l) if v in not islice(l, n, len(l))]

PS: Should the slice be l[n:] instead, Andreas (as I did above)? That will cut off the list from the left
instead of from the right.

Helper

On 20/10/2015 at 04:33, xxxxxxxx wrote:

Hi Niklas,

hm? I'm not sure.
I got an syntax error on your code and after I changed it to:

l2 = [v for n, v in enumerate(l) if v not in islice(l, n, len(l))]

it gave me an empty list as result. Same if I change the slice in my code to l[n:] as suggested.

If I change your code instead to:

l2 = [v for n, v in enumerate(l) if v not in islice(l, 0, n)]

I'm back to the desired result.

Helper

On 20/10/2015 at 04:48, xxxxxxxx wrote:

not the fastest but works with list of lists and unhashable types:

def stripduplicates(biglist) : # ********** order preserving http://www.peterbe.com/plog/uniqifiers-benchmark   
    newlist = []   
    [newlist.append(i) for i in biglist if not newlist.count(i)]   
    return newlist

Helper

On 20/10/2015 at 04:57, xxxxxxxx wrote:

ups sorry didn't see all your solutions had an old browser window open.

Andreas Block your solution freezes my C4D ... (jewing on one core)

I am uniquifying at least 480000 (800x600px image) rgb c4d.vectors - probably more in the future and at least 6 times ... my solution takes about 1-2 seconds yours freezes.

def stripduplicates(l) :       
    l2 = [v for n, v in enumerate(l) if v not in islice(l, 0, n)]   
    return l2

Log File with my def:

Polygon Objects Counter: 1181
-----------------------------------------------------------
Rendering Camera: FS camera 9
END OF function:   0:00:01.630
Unique Color List: 524 / 480000
END OF function:   0:00:00.002
-----------------------------------------------------------
Rendering Camera: FS camera 8
END OF function:   0:00:01.283
Unique Color List: 426 / 480000
END OF function:   0:00:00.006
-----------------------------------------------------------
Rendering Camera: FS camera 7
END OF function:   0:00:01.425
Unique Color List: 480 / 480000
END OF function:   0:00:00.009
-----------------------------------------------------------
Rendering Camera: FS camera 6
END OF function:   0:00:01.683
Unique Color List: 580 / 480000
END OF function:   0:00:00.012
-----------------------------------------------------------
Rendering Camera: FS camera 5
END OF function:   0:00:01.721
Unique Color List: 579 / 480000
END OF function:   0:00:00.013
-----------------------------------------------------------
Rendering Camera: FS camera 4
END OF function:   0:00:01.614
Unique Color List: 462 / 480000
END OF function:   0:00:00.012
-----------------------------------------------------------
Rendering Camera: FS camera 3
END OF function:   0:00:01.054
Unique Color List: 433 / 480000
END OF function:   0:00:00.012
-----------------------------------------------------------
Rendering Camera: FS camera 2
END OF function:   0:00:01.674
Unique Color List: 600 / 480000
END OF function:   0:00:00.015
-----------------------------------------------------------
Rendering Camera: FS camera 1
END OF function:   0:00:01.635
Unique Color List: 496 / 480000
END OF function:   0:00:00.014
-----------------------------------------------------------
Rendering Camera: FS camera 0
END OF function:   0:00:01.720
Unique Color List: 525 / 480000
END OF function:   0:00:00.015
-----------------------------------------------------------
Colors found: 942
Object-Color List Length: 1181
--------------------------------------------------------------------
END OF SCRIPT - Duration (H:MM:SS:ms) :   0:00:36.399

Helper

On 20/10/2015 at 05:02, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Hi Niklas,

hm? I'm not sure.
I got an syntax error on your code and after I changed it to:
l2 = [v for n, v in enumerate(l) if v not in islice(l, n, len(l))]
it gave me an empty list as result. Same if I change the slice in my code to l[n:] as suggested.

If I change your code instead to:
l2 = [v for n, v in enumerate(l) if v not in islice(l, 0, n)]
I'm back to the desired result.

Hey Andras,

yes for the SyntaxError, I messed up the "not" and "in" keywords, so it would need to this instead:

l2 = [v for n, v in enumerate(l) if v **not in** islice(l, n , len(l))]

Regarding the slicing, with l[:n] you get a slice of the elements that you had already iterated about.
My suggestion works when you take n+1 instead of n. But then again, it comes to my mind that it
doesn't make any difference what side of the list you check, the result is the same. Your solution,
taking the left side of the list, is easier on the eye (less parentheses).

Best,
Niklas