python - Progress in a itertools.combinations pool() map , result passed in list -


i run multipocessing script this:

def intersterer(i):     somestuff     return x  if __name__ == '__main__':     pool = pool()      list = pool.map(intersterer, itertools.combinations(xrange(great_number), 2))   

i want know how can see progress of work. around shared counter, ugly , seems not telling truth. (perhaps wrong code in it) can me great pythonic way?

edit: exemple, want store results in list, answers @ question : show progress of python multiprocessing pool map call? doesn't me this. thks

edit2:

here soluced way. thks every helpers!

def intersterer(i):     somestuff     return x  numtot = binomial(number,2) pool = pool()    list=[] tpsb=time.time() in pool.imap_unordered(intersterer, itertools.combinations(xrange(number),2)):     list.append(i); print "{:3.2f}%, approximatly {:3.2f} seconds left       \r".format(100.*len(list)/numtot,  (time.time()-tpsb)*100./(100.*len(list)/numtot)-(time.time()-tpsb)), 

use imap_unordered instead of map:

>>> pathos.multiprocessing import pool >>> pool = pool()  >>> import itertools >>>  >>> def interstellar(i): ...   return sum(i)**2 ...  >>> result = [] >>> in pool.imap_unordered(interstellar, itertools.combinations(xrange(10),2)): ...   result.append(i); print "{:3.2f}%".format(100.*len(result)/45) ...  2.22% 4.44% 6.67% 8.89% 11.11% 13.33% 15.56% 17.78% 20.00% 22.22% 24.44% 26.67% 28.89% 31.11% 33.33% 35.56% 37.78% 40.00% 42.22% 44.44% 46.67% 48.89% 51.11% 53.33% 55.56% 57.78% 60.00% 62.22% 64.44% 66.67% 68.89% 71.11% 73.33% 75.56% 77.78% 80.00% 82.22% 84.44% 86.67% 88.89% 91.11% 93.33% 95.56% 97.78% 100.00% >>> result [1, 4, 9, 16, 25, 36, 49, 64, 81, 9, 16, 25, 36, 49, 64, 81, 100, 25, 36, 49, 64, 81, 100, 121, 49, 64, 81, 100, 121, 144, 81, 100, 121, 144, 169, 121, 144, 169, 196, 169, 196, 225, 225, 256, 289] 

i'm being lazy, , using pathos.multiprocessing can work interpreter… should work standard multiprocessing if done file , in __main__. in solution, have know there 45 combinations, otherwise can't percent w/o knowing final length.

since don't care order , care speed, i've used imap_unordered. if care order, use imap.

edit: aha… duplicate question.


Comments

Popular posts from this blog

Java 3D LWJGL collision -

spring - SubProtocolWebSocketHandler - No handlers -

methods - python can't use function in submodule -