UnboundLocalError when implementing DialoGPT chatbot with HuggingFace

Ask Questions Forum: ask Machine Learning Questions to our readersCategory: Deep LearningUnboundLocalError when implementing DialoGPT chatbot with HuggingFace
Chris Staff asked 1 month ago

I am implementing DialoGPT with HuggingFace Transformers, like this:


from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

def load_tokenizer_and_model(model="microsoft/DialoGPT-small"):
"""
Load tokenizer and model instance for some specific DialoGPT model.
"""

# Initialize tokenizer and model
print("Loading model...")
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausalLM.from_pretrained(model)

# Return tokenizer and model
return tokenizer, model

def generate_response(tokenizer, model, step):
"""
Generate a response to some user input.
"""
# encode the new user input, add the eos_token and return a tensor in Pytorch
new_user_input_ids = tokenizer.encode(input(">> You:") + tokenizer.eos_token, return_tensors='pt')

# append the new user input tokens to the chat history
bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

# generated a response while limiting the total chat history to 1000 tokens,
chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)

# Print response
print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))

def chat_for_n_rounds(n=5):
"""
Chat with chatbot for n rounds (n = 5 by default)
"""

# Initialize tokenizer and model
tokenizer, model = load_tokenizer_and_model()

# Chat for n rounds
for step in range(n):
generate_response(tokenizer, model, step)

if __name__ == '__main__':
chat_for_n_rounds(5)

However, I am getting the following error:


Traceback (most recent call last):
File "dialogpt.py", line 48, in
if __name__ == '__main__':
File "dialogpt.py", line 45, in chat_for_n_rounds
for step in range(n):
File "dialogpt.py", line 26, in generate_response
# append the new user input tokens to the chat history
UnboundLocalError: local variable 'chat_history_ids' referenced before assignment

Let's pause for a second! 👩‍💻

Blogs at MachineCurve teach Machine Learning for Developers. Sign up to MachineCurve's free Machine Learning update today! You will learn new things and better understand concepts you already know.

We send emails at least every Friday. Welcome!

By signing up, you consent that any information you receive can include services and special offers by email.

What can be wrong here?

1 Answers
Best Answer
Chris Staff answered 1 month ago

The issue lies within generate_response, in the sense that you are using chat_history_ids before it is assigned. This always occurs during the first run, because a chat history is not available, but you still need it for concatenation.

To solve this, you must make available the chat_history_ids variable to the outer loop – pass it to generate_response and also return it – like this:


def generate_response(tokenizer, model, chat_round, chat_history_ids):
"""
Generate a response to some user input.
"""
# Encode user input and End-of-String (EOS) token
new_input_ids = tokenizer.encode(input(">> You:") + tokenizer.eos_token, return_tensors='pt')

# Append tokens to chat history
bot_input_ids = torch.cat([chat_history_ids, new_input_ids], dim=-1) if chat_round > 0 else new_input_ids

# Generate response given maximum chat length history of 1250 tokens
chat_history_ids = model.generate(bot_input_ids, max_length=1250, pad_token_id=tokenizer.eos_token_id)

# Print response
print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))

# Return the chat history ids
return chat_history_ids

This allows you to re-use the chat history ids in chat_for_n_rounds and assign None to it during the first round, resolving the error – like this:


def chat_for_n_rounds(n=5):
"""
Chat with chatbot for n rounds (n = 5 by default)
"""

# Initialize tokenizer and model
tokenizer, model = load_tokenizer_and_model()

# Initialize history variable
chat_history_ids = None

# Chat for n rounds
for chat_round in range(n):
chat_history_ids = generate_response(tokenizer, model, chat_round, chat_history_ids)

Your Answer

19 + 12 =