Hi Chris, I have a question.
I’m working on a problem and what I’m trying to do is output 2 different set of classes so lets say what I do is I pass my example through the network and in the output I attach two different layers, 1 layers outputs 10 classes and the other outputs 2 classes.In layer 1 the model output one class at a time using softmax but layer 2 has multi class prediction. So, layer 2 could have classes [0, 0], [1, 0], [0, 1] and [1, 1].Now layer 1 has 9 classes and a null class O . When layer 1 outputs the null class layer 2 will have output [0, 0]. So, it like a fixed relation between the null class and [0, 0]. Now my question is, if layer 1 outputs null class then layer 2 eventually knows that it has to output class [0, 0] so if i’m calculating the loss based on both sets of classes, will the 2nd layer automatically contribute to lower loss? Because its like, when layer 1 outputs null class, for the model it doesn’t have to think what the output for layer 2 will be.So, do you understand what I’m trying to hypothesize? And if so do you know any solutions? Like giving some set of classes more weight than the others (idk much about this though)
If I understand you correctly, you want to train a model with a loss function that works on two outputs — and ask if this will work with one loss function because the outputs have strong correlation.
Now, in TensorFlow/Keras, you can use the Functional API to define multiple output branches. You give a different name to each output layer, and then add multiple loss functions – one for each name.
The effect is that your model will be trained with both losses and hence takes into account the multiple output dimensions in updating the weights.
Example here: https://www.pyimagesearch.com/2018/06/04/keras-multiple-outputs-and-multiple-losses/
Another example (still needs to be updated to TF 2) is a VAE, which is trained with a combination between regular loss and KL divergence: https://www.machinecurve.com/index.php/2019/12/30/how-to-create-a-variational-autoencoder-with-keras/
Hope this helps!