dill
I am trying to load a whole class instance via dill rather than dump and load each class variable one at a time.
Can anybody show me how to do this:
class Object(object):
pass
class ClassA:
def __init__(self):
self.DATA = "Initial"
class ClassB:
def __init__(self, CA):
self.CA = CA
def updateValue(self):
#a = dill.load(ClassA.storage)
a = Object()
a.DATA = "new value"
self.CA = a
print self.CA.DATA
CA = ClassA()
CB = ClassB(CA)
CB.updateValue()
print CA.DATA
So that the output is:
new value
new value
Source: (StackOverflow)
Background
Robust Python function serialization is hard. Other than the stdlib Pickle there are two main choices for function serialization
Each provides serialization of lambdas, closures, interactively defined functions, etc.. Each is generally a more robust choice than Pickle when serializing functions.
Question
In what cases does the behavior of these two libraries differ? What are the contexts in which one fails while the other succeeds or that they produce different results?
My ideal answer is a comprehensive listing of contexts in which one library would perform differently than the other.
Source: (StackOverflow)
short short version:
I am having trouble parallelizing code which uses instance methods.
Longer version:
This python code produces the error:
Error
Traceback (most recent call last):
File "/Users/gilzellner/dev/git/3.2.1-build/cloudify-system-tests/cosmo_tester/test_suites/stress_test_openstack/test_file.py", line 24, in test
self.pool.map(self.f, [self, url])
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/pathos/multiprocessing.py", line 131, in map
return _pool.map(star(f), zip(*args)) # chunksize
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/multiprocess/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/multiprocess/pool.py", line 567, in get
raise self._value
AttributeError: 'Test' object has no attribute 'get_type'
This is a simplified version of a real problem I have.
import urllib2
from time import sleep
from os import getpid
import unittest
from pathos.multiprocessing import ProcessingPool as Pool
class Test(unittest.TestCase):
def f(self, x):
print urllib2.urlopen(x).read()
print getpid()
return
def g(self, y, z):
print y
print z
return
def test(self):
url = "http://nba.com"
self.pool = Pool(processes=1)
for x in range(0, 3):
self.pool.map(self.f, [self, url])
self.pool.map(self.g, [self, url, 1])
sleep(10)
I am using pathos.multiprocessing due to the recommendation here:
Multiprocessing: Pool and pickle Error -- Pickling Error: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
Before using pathos.multiprocessing, the error was:
"PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed"
Source: (StackOverflow)
Trying to render a large and (possibly very) unpicklable object to a file for later use.
No complaints on the dill.dump(file) side:
In [1]: import echonest.remix.audio as audio
In [2]: import dill
In [3]: audiofile = audio.LocalAudioFile("/Users/path/Track01.mp3")
en-ffmpeg -i "/Users/path/audio/Track01.mp3" -y -ac 2 -ar 44100 "/var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmpWbonbH.wav"
Computed MD5 of file is b3820c166a014b7fb8abe15f42bbf26e
Probing for existing analysis
In [4]: with open('audio_object_dill.pkl', 'wb') as f:
...: dill.dump(audiofile, f)
...:
In [5]:
But trying to load the .pkl file:
In [1]: import dill
In [2]: with open('audio_object_dill.pkl', 'rb') as f:
...: audio_object = dill.load(f)
...:
Returns following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-203b696a7d73> in <module>()
1 with open('audio_object_dill.pkl', 'rb') as f:
----> 2 audio_object = dill.load(f)
3
/Users/mikekilmer/Envs/GLITCH/lib/python2.7/site-packages/dill-0.2.2.dev-py2.7.egg/dill/dill.pyc in load(file)
185 pik = Unpickler(file)
186 pik._main_module = _main_module
--> 187 obj = pik.load()
188 if type(obj).__module__ == _main_module.__name__: # point obj class to main
189 try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load(self)
856 while 1:
857 key = read(1)
--> 858 dispatch[key](self)
859 except _Stop, stopinst:
860 return stopinst.value
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load_newobj(self)
1081 args = self.stack.pop()
1082 cls = self.stack[-1]
-> 1083 obj = cls.__new__(cls, *args)
1084 self.stack[-1] = obj
1085 dispatch[NEWOBJ] = load_newobj
TypeError: __new__() takes at least 2 arguments (1 given)
The AudioObject is much more complex (and large) than the class object the above calls are made on (from SO answer), and I'm unclear as to whether I need to send a second argument via dill, and if so, what that argument would be or how to tell if any approach to pickling is viable for this specific object.
Examining the object itself a bit:
In [4]: for k, v in vars(audiofile).items():
...: print k, v
...:
returns:
is_local False
defer False
numChannels 2
verbose True
endindex 13627008
analysis <echonest.remix.audio.AudioAnalysis object at 0x103c61bd0>
filename /Users/mikekilmer/Envs/GLITCH/glitcher/audio/Track01.mp3
convertedfile /var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmp9ADD_Z.wav
sampleRate 44100
data [[0 0]
[0 0]
[0 0]
...,
[0 0]
[0 0]
[0 0]]
And audiofile.analysis seems to contain an attribute called audiofile.analysis.source which contains (or apparently points back to) audiofile.analysis.source.analysis
Source: (StackOverflow)
Here I have this class definition class definition. When I run below code, it raises following errors.
sm = SaliencyMaskSlic()
operations = [('img_resize', img_resize), ('sal_mask', sm.transform)]
args_list = [{'h_size':258}, {'cropped':True}]
pre_pipeline = Pipeline(ops=operations, arg_list=args_list)
ch = ColorHist('RGB', [6,6,6], [2,2], center=True, pre_pipeline = pre_pipeline)
dill.dump(ch, open('erogol.pkl','wb'))
...
dill.loads('erogol.pkl')
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-11-c8a5937780b5> in <module>()
----> 1 dill.loads('erogol.pkl')
/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in loads(str)
158 """unpickle an object from a string"""
159 file = StringIO(str)
--> 160 return load(file)
161
162 # def dumpzs(obj, protocol=None):
/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in load(file)
148 pik = Unpickler(file)
149 pik._main_module = _main_module
--> 150 obj = pik.load()
151 if type(obj).__module__ == _main_module.__name__: # point obj class to main
152 try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)
/usr/lib/python2.7/pickle.pyc in load(self)
856 while 1:
857 key = read(1)
--> 858 dispatch[key](self)
859 except _Stop, stopinst:
860 return stopinst.value
/usr/lib/python2.7/pickle.pyc in load_appends(self)
1185 def load_appends(self):
1186 stack = self.stack
-> 1187 mark = self.marker()
1188 list = stack[mark - 1]
1189 list.extend(stack[mark + 1:])
/usr/lib/python2.7/pickle.pyc in marker(self)
872 mark = self.mark
873 k = len(stack)-1
--> 874 while stack[k] is not mark: k = k-1
875 return k
876
IndexError: list index out of range
Basically I have one class instance using another class instance inside. I also used cPickle but it raises as I dump;
TypeError: can't pickle instancemethod objects
Any idea for the solution ?
Source: (StackOverflow)
I'm trying to use the dill module to save my ipython session using dump_session() but I'm getting an error message. I'm using Ipython 1.0.0 and dill 0.2-a-dev 20120503. Does anyone out there have any insight? Thanks in advance.
Niall
Here's the enormous traceback:
--------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <ipython-input-15-9934c16a537e> in <module>()
----> 1 dill.dump_session("/data/local/nrobin/tset.sess")
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in dump_session(filename, main_module)
104 pickler._main_module = main_module
105 pickler._session = True # is best indicator of when pickling a session
--> 106 pickler.dump(main_module)
107 pickler._session = False
108 finally:
/usr/local/sci/lib/python2.7/pickle.pyc in dump(self, obj)
222 if self.proto >= 2:
223 self.write(PROTO + chr(self.proto))
--> 224 self.save(obj)
225 self.write(STOP)
226
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_module(pickler, obj)
415 if _DEBUG[0]: print "M1: %s" % obj
416 pickler.save_reduce(__import__, (obj.__name__,), obj=obj,
--> 417 state=obj.__dict__.copy())
418 else:
419 if _DEBUG[0]: print "M2: %s" % obj
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
417
418 if state is not None:
--> 419 save(state)
420 write(BUILD)
421
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_module_dict(pickler, obj)
284 else:
285 if _DEBUG[0]: print "D2: %s" % "<dict ...>" #obj
--> 286 StockPickler.save_dict(pickler, obj)
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_dict(self, obj)
647
648 self.memoize(obj)
--> 649 self._batch_setitems(obj.iteritems())
650
651 dispatch[DictionaryType] = save_dict
/usr/local/sci/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
679 for k, v in tmp:
680 save(k)
--> 681 save(v)
682 write(SETITEMS)
683 elif n:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
329
330 # Save the reduce() output and finally memoize the object
--> 331 self.save_reduce(obj=obj, *rv)
332
333 def persistent_id(self, obj):
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
417
418 if state is not None:
--> 419 save(state)
420 write(BUILD)
421
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_module_dict(pickler, obj)
284 else:
285 if _DEBUG[0]: print "D2: %s" % "<dict ...>" #obj
--> 286 StockPickler.save_dict(pickler, obj)
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_dict(self, obj)
647
648 self.memoize(obj)
--> 649 self._batch_setitems(obj.iteritems())
650
651 dispatch[DictionaryType] = save_dict
/usr/local/sci/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
684 k, v = tmp[0]
685 save(k)
--> 686 save(v)
687 write(SETITEM)
688 # else tmp is empty, and we're done
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
329
330 # Save the reduce() output and finally memoize the object
--> 331 self.save_reduce(obj=obj, *rv)
332
333 def persistent_id(self, obj):
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
417
418 if state is not None:
--> 419 save(state)
420 write(BUILD)
421
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_module_dict(pickler, obj)
284 else:
285 if _DEBUG[0]: print "D2: %s" % "<dict ...>" #obj
--> 286 StockPickler.save_dict(pickler, obj)
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_dict(self, obj)
647
648 self.memoize(obj)
--> 649 self._batch_setitems(obj.iteritems())
650
651 dispatch[DictionaryType] = save_dict
/usr/local/sci/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
679 for k, v in tmp:
680 save(k)
--> 681 save(v)
682 write(SETITEMS)
683 elif n:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_instancemethod(pickler, obj)
303 if _DEBUG[0]: print "Me: %s" % obj
304 pickler.save_reduce(MethodType, (obj.im_func, obj.im_self,
--> 305 obj.im_class), obj=obj)
306 return
307
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
399 else:
400 save(func)
--> 401 save(args)
402 write(REDUCE)
403
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_tuple(self, obj)
546 if n <= 3 and proto >= 2:
547 for element in obj:
--> 548 save(element)
549 # Subtle. Same as in the big comment below.
550 if id(obj) in memo:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_function(pickler, obj)
268 pickler.save_reduce(FunctionType, (obj.func_code, obj.func_globals,
269 obj.func_name, obj.func_defaults,
--> 270 obj.func_closure), obj=obj)
271 else:
272 if _DEBUG[0]: print "F2: %s" % obj
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
399 else:
400 save(func)
--> 401 save(args)
402 write(REDUCE)
403
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_tuple(self, obj)
560 write(MARK)
561 for element in obj:
--> 562 save(element)
563
564 if id(obj) in memo:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_module_dict(pickler, obj)
284 else:
285 if _DEBUG[0]: print "D2: %s" % "<dict ...>" #obj
--> 286 StockPickler.save_dict(pickler, obj)
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_dict(self, obj)
647
648 self.memoize(obj)
--> 649 self._batch_setitems(obj.iteritems())
650
651 dispatch[DictionaryType] = save_dict
/usr/local/sci/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
679 for k, v in tmp:
680 save(k)
--> 681 save(v)
682 write(SETITEMS)
683 elif n:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_instancemethod(pickler, obj)
303 if _DEBUG[0]: print "Me: %s" % obj
304 pickler.save_reduce(MethodType, (obj.im_func, obj.im_self,
--> 305 obj.im_class), obj=obj)
306 return
307
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
399 else:
400 save(func)
--> 401 save(args)
402 write(REDUCE)
403
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/usr/local/sci/lib/python2.7/pickle.pyc in save_tuple(self, obj)
546 if n <= 3 and proto >= 2:
547 for element in obj:
--> 548 save(element)
549 # Subtle. Same as in the big comment below.
550 if id(obj) in memo:
/usr/local/sci/lib/python2.7/pickle.pyc in save(self, obj)
284 f = self.dispatch.get(t)
285 if f:
--> 286 f(self, obj) # Call unbound method with explicit self
287 return
288
/home/h02/nrobin/.local/lib/python2.7/site-packages/dill-0.2a.dev_20120503-py2.7.egg/dill/dill.pyc in save_function(pickler, obj)
268 pickler.save_reduce(FunctionType, (obj.func_code, obj.func_globals,
269 obj.func_name, obj.func_defaults,
--> 270 obj.func_closure), obj=obj)
271 else:
272 if _DEBUG[0]: print "F2: %s" % obj
/usr/local/sci/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
403
404 if obj is not None:
--> 405 self.memoize(obj)
406
407 # More new special cases (that work with older protocols as
/usr/local/sci/lib/python2.7/pickle.pyc in memoize(self, obj)
242 if self.fast:
243 return
--> 244 assert id(obj) not in self.memo
245 memo_len = len(self.memo)
246 self.write(self.put(memo_len))
AssertionError:
Source: (StackOverflow)
In the example below, I have placed the exact same class Foo inside its own module foo. Why the external class is dumped by ref? The instance ff is not being dumped with it's source code.
I am using Python 3.4.3 and dill-0.2.4.
import dill
import foo
class Foo:
y = 1
def bar( self, x ):
return x + y
f = Foo()
ff = foo.Foo()
print( dill.dumps( f, byref=False, recurse=True ) )
print( '\n' )
print( dill.dumps( ff, byref=False, recurse=True ) )
Thanks in advance.
Well, the code above is actually wrong (should be Foo.y, instead of y). Correcting the code gives me an exception while dumping the f instance.
Source: (StackOverflow)
I'm experimenting with the Dill package, specifically it's detect module and having some trouble intuitively understanding what's is meant by referents, referers, parents and children.
A reference is a value that enables access to some data.
And referents are objects that are referred to, right?
So in the following code:
class MyClass:
"""A simple example class"""
i = 12345
def f(self):
return 'hello world'
an_instance = MyClass()
an_instance2 = MyClass()
an_instance3 = MyClass()
a_list = [an_instance, an_instance2, an_instance3]
Are an_instance, an_instance2, an_instance3 referents of a_list and would the MyClass also be a referent of a_list, but one level of depth further up the chain?
So, with -> signifying the reference, would the chain of referents look like:
a_list -> an_instance -> MyClass
Would this be viewed as:
grandchild -> child -> Parent
Conversely, is a_list a referrer of an_instance as well as an_instance2, an_instance3 and at another level of depth, MyClass?
Making the chain of referrers:
MyClass -> an_instance -> a_list
And would this also be conceived:
parent -> child -> grandchild
Can someone offer a clear explanation of where references, inheritance and containers do and don't coincide?
Source: (StackOverflow)
I have a program that creates plots - sometimes line plots, sometimes NonUniformImages - using matplotlib. I'd like to be able to pickle the plots to reopen them at a later time without going through the whole creation process again. For whatever reason, it keeps throwing a PicklingError: Can't pickle 'RendererAgg' object. I've tried using both import dill as pickle and import pickle, as well as all 4 different pickling options but no change.
The axes are defined here:
class Imaging:
def function:
ax1 = plt.subplot(2,1,1)
ax2 = plt.subplot(2,1,2)
And set here: (Imaging.figureProperties is a list and is meant to hold multiple [ax1,ax2] objects. Also in the same function as where ax1 and ax2 are defined.)
Imaging.figureProperties.append([ax1,ax2])
Finally, data is pickled here (i is chosen by the user, but it will be within the list):
class2:
with open(filename, 'wb') as f:
pickle.dump(Imaging.figureProperties[i-1],f)
I have no problem running the sample code from this question (with some slight changes such as opening in 'wb' instead of just 'w'), as long as I use import dill as pickle. If I use the standard import pickle it throws the same PicklingError. What is going on here?
Source: (StackOverflow)
Right at the outset, I tried conda install dill, and conda was not able to find it on the internet.
Then I downloaded both .tgz and .zip files in my default IPython directory from here:
https://pypi.python.org/pypi/dill
After which I tried the following commands:
conda install dill-0.2b1.zip
conda install "C:\<rest_of_the_complete_path>\dill-0.2b1.zip"
and likewise for .tgz. All four attempts yielded the error:
No packages found matching:
What is it that I am doing wrong? I am trying to repeat the examples given on the following link:
http://nbviewer.ipython.org/gist/minrk/5241793
Edit 1: I had installed dill on my system by running the .exe file from https://pypi.python.org/pypi/dill. This step installed dill on my system python (C:\Python27) but not on my Anaconda Python. I am assuming that these two pythons are separate since I can import the usual modules (say numpy for instance) on both — the python I access through cmd and the one I access through my IPython notebooks — but I can import dill only on the python I accesses through cmd and not in my IPython notebooks.
Source: (StackOverflow)
dill is a great tool for pickling most the Python objects, I use it in IPython parallel to serialize calculations. One issue I keep getting into is around dill-ing class definitions. One of the errors I get is explained below.
While trying to serialize class definitions, I keep getting AssertionError from dill. I wonder why one of these works and the other fails:
class MyClassEmpty(object):
pass
class MyClassInit(object):
def __init__(self):
super(MyClassInit).__init__()
dill.dumps(MyClassEmpty) # returns: '\x80\x02cdill.dill\n_create_type\nq\x00(cdill.dill\n_load_type\nq\x01U\x08TypeTypeq\x02\x85q\x03Rq\x04U\x0cMyClassEmptyq\x05h\x01U\nObjectTypeq\x06\x85q\x07Rq\x08\x85q\t}q\n(U\n__module__q\x0bU\x08__main__q\x0cU\x07__doc__q\rNutq\x0eRq\x0f.'
dill.dumps(MyClassInit) # AssertionError at line 244 of MyClassEmpty (assert id(obj) not in self.memo)
I'm on Python 2.7.6 using dill 0.2.2.
Source: (StackOverflow)
In the answer to Python pickle: dealing with updated class definitions, the author of the dill package writes:
"Ok, I have added this feature to dill in the latest revision on github. Implemented with far less trickery than I thought... just serialize the class definition with the pickle, and voila."
Having installed dill and tinkered with it, it's not obvious to me how to actually use this functionality in dill. Could someone provide an explicit example? I would like to pickle the class instance and also serialize the class definition.
(I am new to python and I this functionality seems extremely important, as since when pickling an object it would be great to get as close to a guarantee as possible that you could look at the object (could be the result of a simulation) in the future after the class definition may have changed and you haven't kept track of all the changes in an easily accessible way.)
Source: (StackOverflow)
Working further with the Dill package. Posted yesterday exemplifying some ignorance of referents, referrers, inheritance and containers. The answer was insightful, but I'm still having trouble coming up with examples that show a few levels of depth in referrer and referent chains.
Yesterday I was thinking that an instance would be a referrer to it's class. In the Dill docs children are the referrers, so in that case a child of depth=2 would be a grandchild, right? And would that be an object that refers (points) to another object that refers to another object?
What would be an example of an object that has a chain of referents and referrers of at least a depth of two?
Consider:
import dill
class GreatGrandparentClass(object):
"""A Great Grandparent class"""
name = "Henrietta Ancient One"
class GrandparentClass(GreatGrandparentClass):
"""A Grandparent class"""
class ParentClass(GrandparentClass):
"""A Grandparent class"""
great_grand_parent = ParentClass().name
print ("Children (depth=2):")
for element in dill.detect.children(
great_grand_parent,
list,
depth=2,
ignore=(globals())):
print(element)
print ("Parents:")
for element in dill.detect.parents(
great_grand_parent,
list,
depth=2,
ignore=(globals())):
print(element)
returns:
Children (depth=2):
['\npython pydill.py\n\n', 'dill', 'object', 'A Great Grandparent class', 'i', 'Henrietta Ancient One', 'GreatGrandparentClass', 'GreatGrandparentClass', 'A Grandparent class', 'GrandparentClass', 'GrandparentClass', 'A Grandparent class', 'ParentClass', 'great_grand_parent', 'ParentClass', 'i', 'Children (depth=2):', 'element', 'dill', 'detect', 'children', 'great_grand_parent', 'list', 'depth', 2, 'ignore', 'globals', 'element', 'Parents:', 'element', 'dill', 'detect', 'parents', 'great_grand_parent', 'list', 'depth', 2, 'ignore', 'globals', 'element']
Henrietta Ancient One
Parents:
Henrietta Ancient One
Looking specifically at list objects here, the single referrent (Parent) of great_grand_parent is the string, "Henrietta Ancient One".
And the referrers (Children) (result of gc.get_referrers(), filtered by specified object-type) contains two objects: A list that includes the string 'Henrietta Ancient One', and the string Henrietta Ancient One. (depth=2 and depth=1 return the same result.)
How can I make an object for which Dill can return:
- Two distinct depths of referrers
- Two distinct depths of referents
Source: (StackOverflow)
I'm developing a distributed application using IPython parallel. There are several tasks which are carried out one after another on the IPython cluster engines.
One of these tasks inevitably makes use of closures. Hence, I have to tell IPython to use Dill instead of Pickle by calling dv.use_dill(). Though this should be temporarily.
Is there any way to activate Pickle again once Dill is enabled? I couldn't find any function (something of the form dv.use_pickle()) which would make such an option explicit.
Source: (StackOverflow)
As i couldn't fine the setup pf cPickle i'm using dill. In documentation they said both are same, but in dill there is no option like dump and all which is being provided by cPickle or Pickle. is there any link where i can download the cPickle or Pickle ?
Thanks in advance!
Source: (StackOverflow)