model.fit(dataset, batch_size=1, epochs=1)
Additionally, the call method of your custom MyTransformer model is not returning the labels lable which is required for the training process. You'll need to return both the features and the labels from the call method in order for the model to be trained properly:
def call(self, inputs, training):
    feature, lable = inputs
    return feature, lable
Finally, the loss function that you are using, tf.keras.losses.BinaryCrossentropy is designed for binary classification problems, but the labels in your dataset are categorical. To handle this, you should use a different loss function, such as tf.keras.losses.CategoricalCrossentropy, which is designed for categorical classification problems.
          
Hi @DURGESH716, and thank you for the reply.
using the wrong variable was a mistake that I made when pasting the example here. Even if a not defined variable was used I would've expected "NameError: name 'train_data' is not defined" error and not one for a symbolic tensor. The link that was provided with the description of the issue points out to google colab example where the correct variable was used. So the original question still stands.

I've now modified the example according to your suggestions to return feature and label, plus using the CategoricalCrossentropy and nothing changes. Here is a link to the example in google colab:

https://colab.research.google.com/drive/1mn6iseJLnJwTmwakYa2XuxszKtR6sV9G#scrollTo=Cj9g0bGN1Fo3
Can you help me to understand what the real issue is?
          
@mihail-vladov Thanks for reporting issue.
@sushreebarsa  I was able to reproduce this issue. Please find the gist here.
The model works as expected when features and labels are given as forward pass but fail when tf.data.Dataset is passed to the model.fit.
          
Proceeding with the @pjpratik points, I think it is important to ensure that the data is correctly formatted and preprocessed before being passed to the model. Some common issues include incorrect tensor shapes, data types, and normalization. To resolve the issue, you may need to inspect and debug the data preprocessing steps, including any data augmentation or batching operations that are applied to the data before passing it to the model. Additionally, it may be helpful to compare the data and preprocessing steps used when passing features and labels directly to the model versus when using a tf.data.Dataset.
Rest everything logic and code is syntactically correct, please check the dataset properly and pre-process it carefully.
          
Hi, @mihail-vladov
Apologize for the delay and It seems like you haven't done pre-processing with your dataset, your batched_labels should be in numerical values but it was in string type so you'll have to do some pre-processing with batched_labels with one hot encoding approach so that batched_labels  will convert into numerical values so you can refer this official documentation 
When you pass dataset from tf.data.Dataset() to model.fit() it is expected that it will directly generate batches, not individual examples. You just need to batch your dataset before training as shown below and please refer tf.data.Dataset():
batch_size=32
dataset = dataset.batch(batch_size)
model.fit(x=dataset)
I have added sample code example for your reference, How to use tf.data.dataset with model.fit() so please refer this gist-file and also refer our official documentation for Customize what happens in Model.fit
I hope it will help you to resolve your issue and if issue still persists please let us know ? or Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ? Thank you!
          
Hi @gaikwadrahul8,
I'm a bit confused by your response, as the code in your gist-file appears to differ significantly from the code I provided, except for the use of TensorFlow and a dataset. Additionally, @pjpratik has confirmed that there is a bug, so I was hoping to receive an update on when the bug will be fixed.
Regarding your previous comment about the data preprocessing and batches, I understand your point and you would be typically right if there was no issue and I was actually able to use the data. However, this is not the case. Nevertheless, I've reworked the example to reflect your comments. Here is the updated example. As you can verify yourself, the exception is still there.
I would like to reiterate the issue I am encountering. According to the TensorFlow documentation, I should be able to use a dataset as the x argument for the model.fit() function and receive a tuple as inputs parameter in the model.call() function. Then, I would decide how to process and what operation to execute on the data in that function. However, I'm encountering an exception and can't obtain the data in the tuple format from the inputs parameter in the call function. This leaves me with the impression that the documentation is not accurate! Meaning there is a bug. Here is once again the exception:
File "<ipython-input-7-a0afc8cfb653>", line 8, in call   (feature, label) = inputs

OperatorNotAllowedInGraphError: Iterating over a symbolic tf.Tensor is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
If you decide to provide an example of how to overcome the exception please use the code I've provided with the specified model and a dataset containing a tuple. Thank you for your time and consideration.
          
Hi, @mihail-vladov
Apologize for the delay and It seems like you're trying to use operation which does not support by autograph because of that reason you're getting that error and even error itself is saying the same thing but there is some workaround and I found similar issue over stack-overflow. and it seems like you'll have to use tf.shape(x), please refer this official documentation, please try with tf.shape(x) and check whether your issue is resolving or not ? I tried from my end and I'm getting different error AttributeError: 'NoneType' object has no attribute 'dtype' and I have added gist-file for your reference. I hope it will help you to resolve your issue.
tf.shape(x) and x.shape should be identical in eager mode. Within tf.function, not all dimensions may be known until execution time. Hence when defining custom layers and models for graph mode, prefer the dynamic tf.shape(x) over the static x.shape.
If issue still persists please let us know ? Thank you!
          
Hi, @gaikwadrahul8,
pjpratik already confirmed that this is a bug. Why are you trying to convince me that I am doing something incorrectly?
You said that I am trying to use an operation that is not supported. Could you please point out exactly which is that operation?
Regarding your comment on the different error you are getting, you have changed the code in the model, which is why the error is different.
I will repeat myself again. According to the TensorFlow documentation, I can use the dataset as input data. This data can be packed as a tuple. Please extract the data from the tuple in the model call function in a way that won't throw an error.
          
Hi, @mihail-vladov
Apologize for the delayed response and I see @pjpratik did not say explicitly it's bug, He only replicated that issue from his end and he said code execution is failing with tf.data.Dataset is passed to the model.fit() and I was referring there are similar issues with same error message while using Autograph mode like #32546, #51472 and users found some workaround for those issues.
I'm waiting for response from concerned team for this issue whether is there any workaround to handle this issue till then could you please have look into this Autograph reference guide which may help you to solve this issue and I'll update you soon here. Thank you!
          
Hi @mihail-vladov ,
The problem not seems to related to Autograph. The problem seems the code trying to get Two values(feature, lable) from a Tensor (model.fit converts the given input to Tensor that is internal details can be checked from the model.fit API code) and Tensor can't be unpacked to Two values and Please note that Tensor is immutable in Tensorflow.
You can check the behaviour clearly when you enable Eager mode using model.compile(...,run_eagerly=True) where you can see the error below.
not enough values to unpack (expected 2, got 1)
I have added print(inputs) in call() to understand what actually is inputs. Please refer attached gist.
Hope this will clear your doubts on root cause of this error. Thanks!
          
Hi @SuryanarayanaY, Thank you for the answer.
According to the TensorFlow documentation using tuple (feature, label) as inputs argument is allowed. Here is the quote:
inputs:	Input tensor, or dict/list/tuple of input tensors.
I am using a tuple. Here are my questions:
Why the model.fit function does not call the model.call function according to the documentation?
What is the correct way to obtain the feature and label data in the model.call function when I pass a dataset to the model.fit function?
Why the model.fit function does not call the model.call function according to the documentation?
The model.fit function called the call() function. You can check by adding print() to confirm same. But here when we pass a dataset as an argument to model.fit, the API converts it into Tensors internally.Outside the model.fit() the dataset might be a tuple but within model.fit the tuple is converting into Tensors which is default behaviour.
2. What is the correct way to obtain the feature and label data in the model.call function when I pass a dataset to the model.fit function?
To get the custom behaviour as per individuals requirement you need to override the train_step . Please refer to attached tutorials for more details.
Thank you!
          
I am facing the same issue and I am confused why the example in Neural Machine Translation doesn't run into the same issue but here we run into this issue? Basically they are using the same type of code:
class Translator(tf.keras.Model):
  @classmethod
  def add_method(cls, fun):
    setattr(cls, fun.__name__, fun)
    return fun
  def __init__(self, units,
               context_text_processor,
               target_text_processor):
    super().__init__()
    # Build the encoder and decoder
    encoder = Encoder(context_text_processor, units)
    decoder = Decoder(target_text_processor, units)
    self.encoder = encoder
    self.decoder = decoder
  def call(self, inputs):
    context, x = inputs
    context = self.encoder(context)
    logits = self.decoder(context, x)
    #TODO(b/250038731): remove this
      # Delete the keras mask, so keras doesn't scale the loss+accuracy. 
      del logits._keras_mask
    except AttributeError:
    return logits
          
.call() and .fit() have different signatures. In Keras, .call(inputs) really behaves like the math model y=f(x) with f being implemented in .call(), so inputs should only match  x and not y, and y must not go into .call(). During .fit(dataset), the element (x,y) of dataset is automatically distributed such that x is passed to call() for the gradient step, and  y as well as the prediction  .call(x) are passed to the loss function loss(y_true, y_pred).
Some precautions to take when developing models outside the classic supervised learning framework, if you wish to reuse the standard Keras workflow :
dataset should have a (fake) label y. It can have the format ((x1, x2), y), and the tuple (x1, x2) treated as x by Keras can be obtained in .call(inputs) as x1, x2 = inputs without the mentioned error.
.call() should return a single tensor. For multiple outputs like return y1, y2, it seems only y1 will be passed to the loss. In this case they should be stacked like return keras.ops.stack([y1, y2]).
Custom loss must have the signature loss(y_true, y_pred) even if there is no y_true. In case of multiple tensors in y_pred, use keras.ops.take() to separate them, not y1, y2 = y_pred.
Then compile the custom model and train with .fit(). This is how I developped a self-supervised learning framework for all three backends without writing the custom .training_step() as promoted in the official doc. Hope this helps.