Repetitive output using T5 seq2seq

Vishal Ahuja asked 2 months ago

Hi guys,

I am interested in building a description/caption generator using a template based approach.

I have a rule based method which converts an input sentence into a template as shown below.

Input
Shaped to a shirt silhouette, it’s easy to wear
Template
Shaped to a SILHOUETTE, it’s easy to wear


Input
The paisley print will look and make you feel amazing
Template
The PRINT will look and make you feel amazing

Now, I can fill in in the placeholders like SILHOUETTE with tags pertaining to SILHOUETTE like shift, aline, bodycon etc.

There are 215 templates and after filling in the placeholders, I have 215 * 3 = 645 total sentences. The goal is to now formulate this as a sequence to sequence problem as follows:

Input Output
shirt Shaped to a shirt silhouette, it’s easy to wear

Input Output
thin puff This dress features thin straps and puff sleeves

I trained a T5-base model using hugging face with the following model hyper-parameters:
{
"reprocess_input_data": True,
"overwrite_output_dir": True,
"max_seq_length": 128,
"train_batch_size": 8,
"num_train_epochs": 5,
"save_eval_checkpoints": True,
"save_steps": -1,
"use_multiprocessing": False,
"evaluate_during_training": True,
"evaluate_during_training_steps": 15000,
"evaluate_during_training_verbose": True,
"fp16": True,
"num_beams": None,
"do_sample": True,
"max_length": 50,
"top_k": 50,
"top_p": 0.95,
"num_return_sequences": 3,
}

The output sentences have the following problems:
1. Many a times, all the 3 returned sequences are the same.
2. There is extraneous information present in the output sentence. Here is an example:
-Input: blue mini holiday (these are tags corresponding to COLOR, LENGHT and OCCASION)
-Output: Covered in a blue color, this mini length dress will brighten up your summer season wardrobe, styled in a holiday silhouette and SHAPE with a holiday sleeve of a minimum of 10 inch

Now, my input sentences don’t have holiday silhouette or holiday sleeves. Also,SEASON was not a part of the input above, but the sentence has summer in it.

Can someone please explain how to overcome such problems. Is my dataset too small?

Your Answer

5 + 2 =