Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

Here's a snippet that will succeed in serializing with dill , but fail with pickle . It is surprising that Tensor objects aren't natively pickleable. Is this a fundamental limitation of thread-aware Tensors, or is it just not implemented?

import dill
import pickle
import tensorflow as tf
dill.dumps(tf.zeros((1,1)))
print("Dill succeeded")
pickle.dumps(tf.zeros((1,1)))
print("Pickle succeeded")
Output:
$ python foo.py
Dill succeeded
Traceback (most recent call last):
  File "foo.py", line 7, in <module>
    pickle.dumps(tf.zeros((1,1)))
TypeError: can't pickle _thread.lock objects
The reason why dill can serialize these objects, but not pickle? Simple answer is that pickle cannot serialize most objects in python, the thread.lock object included.  If you want to serialize one of these objects, use an advanced serialization library like dill.  As to exactly why pickle can't, I think originally it stems from the implementation of the GIL and the frame object rendering some objects unserializable, and thus there was no drive to serialize everything in the language. There's always been talk about security issues stemming from serialization of all python objects, but I think that's a red herring.  Not having full language serialization limits the ability to operate in parallel computing, so hopefully pickle will learn from dill how to serialize more objects.
                @denfromufa: That's not what the OP was asking. What more, in your opinion, should I have shown than what the OP did in the question?
– Mike McKerns
                Sep 25, 2020 at 11:50
I find that I am unable to reproduce the output of eqzx's script and serializing fails for both dill and pickle. 
While this question is several years old, I believe the issue wasn't with pickle but rather the fact that tensors in tensorflow 1.x are not evaluated immediately. They are instead evaluated when their graph is executed. (In tensorflow 2.0 eager execution is enabled by default so you don't really have to deal with this paradigm).
Using tf.enable_eager_execution() or evaluating a tensor within a tf.Session() context 
The following script doesn't throw an error when using either Pickle or Dill:
import tensorflow as tf
tf.enable_eager_execution()
import pickle 
# import dill
n = tf.zeros((1,1))
pickle.dumps(n)
# dill.dumps(n)
print('Success')
Same script using Session.run() instead doesn't throw an error:
import tensorflow as tf
import pickle
# import dill
with tf.Session() as sess:
    n = sess.run(tf.zeros((1,1)))
pickle.dumps(n)
# dill.dumps(n)
print('Success')
                If you look at the traceback, it's obvious that the Lock object is not pickling.  Note that when you say the functions are not evaluated immediately, that means that they are put on the execution stack, which means there's a connection to the frame object as noted in my answer.  Both dill and Tensor objects have evolved significantly since the OP posted.
– Mike McKerns
                Oct 30, 2019 at 2:43
                Hi @MikeMcKerns, while I'm not entirely familar with the inner workings of tensorflow nor how the frame object is used by it, I was answering based on the docs for a tf.Tensor. A tf.Tensor in the build context doesn't actually contain the values for the tensor but rather is handle for the outputs for a tensorflow operation. As it isn't a Python object, I didn't think it made sense for it to be serializable unless it was evaluated.
– evantkchong
                Nov 5, 2019 at 3:19
                That may be true... The objects that makeup a python function are different before and after the first evaluation of the function. With lazy evaluation, a python function's skeleton is composed when it's built, but the contents of it aren't handled until it's called the first time. Before that, the guts of a function are put on the execution stack (essentially on the frame) to be retrieved when it's called.  Also, note that essentially everything in python is an object -- a list, a file handle... basically anything that has a type (i.e. derives from a python class).
– Mike McKerns
                Nov 5, 2019 at 13:20
                @MikeMcKerns I hope you don't mind me asking additional questions because you seem really well versed with this topic. In that case wouldn't it matter that tensorflow ops make calls to the tensorflow C++ api when called?  In fact frame seems to be a C++ struct.
– evantkchong
                Nov 6, 2019 at 9:29
                Python objects are generally thin classes wrapped around C-type objects -- unless you are calling directly to one of the ctypes (if I'm not mistaken).  So, I'd have to see how the calls to the tensorflow C-type api are made.
– Mike McKerns
                Nov 6, 2019 at 13:41
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.