Chris Staff asked 10 months ago I am implementing DialoGPT with HuggingFace Transformers, like this:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
def load_tokenizer_and_model(model="microsoft/DialoGPT-small"):
"""
Load tokenizer and model instance for some specific DialoGPT model.
"""
# Initialize tokenizer and model
print("Loading model...")
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausalLM.from_pretrained(model)
# Return tokenizer and model
return tokenizer, model
def generate_response(tokenizer, model, step):
"""
Generate a response to some user input.
"""
# encode the new user input, add the eos_token and return a tensor in Pytorch
new_user_input_ids = tokenizer.encode(input(">> You:") + tokenizer.eos_token, return_tensors='pt')
# append the new user input tokens to the chat history
bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
# generated a response while limiting the total chat history to 1000 tokens,
chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)
# Print response
print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
def chat_for_n_rounds(n=5):
"""
Chat with chatbot for n rounds (n = 5 by default)
"""
# Initialize tokenizer and model
tokenizer, model = load_tokenizer_and_model()
# Chat for n rounds
for step in range(n):
generate_response(tokenizer, model, step)
if __name__ == '__main__':
chat_for_n_rounds(5)
However, I am getting the following error:
Traceback (most recent call last):
File "dialogpt.py", line 48, in
if __name__ == '__main__':
File "dialogpt.py", line 45, in chat_for_n_rounds
for step in range(n):
File "dialogpt.py", line 26, in generate_response
# append the new user input tokens to the chat history
UnboundLocalError: local variable 'chat_history_ids' referenced before assignment
Let's pause for a second! 👩💻
We send emails at least every Friday. Welcome!
What can be wrong here?
Chris Staff answered 10 months ago The issue lies within generate_response, in the sense that you are using chat_history_ids before it is assigned. This always occurs during the first run, because a chat history is not available, but you still need it for concatenation.
To solve this, you must make available the chat_history_ids variable to the outer loop – pass it to generate_response and also return it – like this:
def generate_response(tokenizer, model, chat_round, chat_history_ids):
"""
Generate a response to some user input.
"""
# Encode user input and End-of-String (EOS) token
new_input_ids = tokenizer.encode(input(">> You:") + tokenizer.eos_token, return_tensors='pt')
# Append tokens to chat history
bot_input_ids = torch.cat([chat_history_ids, new_input_ids], dim=-1) if chat_round > 0 else new_input_ids
# Generate response given maximum chat length history of 1250 tokens
chat_history_ids = model.generate(bot_input_ids, max_length=1250, pad_token_id=tokenizer.eos_token_id)
# Print response
print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
# Return the chat history ids
return chat_history_ids
This allows you to re-use the chat history ids in chat_for_n_rounds and assign None to it during the first round, resolving the error – like this:
def chat_for_n_rounds(n=5):
"""
Chat with chatbot for n rounds (n = 5 by default)
"""
# Initialize tokenizer and model
tokenizer, model = load_tokenizer_and_model()
# Initialize history variable
chat_history_ids = None
# Chat for n rounds
for chat_round in range(n):
chat_history_ids = generate_response(tokenizer, model, chat_round, chat_history_ids)
elvisfreb answered 3 weeks ago An UnboundLocalError is raised when a local variable is referenced before it has been assigned. In most cases this will occur when trying to modify a local variable before it is actually assigned within the local scope. Python doesn’t have variable declarations, so it has to figure out the scope of variables itself. It does so by a simple rule: If there is an assignment to a variable inside a function, that variable is considered local.
Python has lexical scoping by default, which means that although an enclosed scope can access values in its enclosing scope, it cannot modify them (unless they’re declared global with the global keyword). A closure binds values in the enclosing environment to names in the local environment. The local environment can then use the bound value, and even reassign that name to something else, but it can’t modify the binding in the enclosing environment. UnboundLocalError happend because when python sees an assignment inside a function then it considers that variable as local variable and will not fetch its value from enclosing or global scope when we execute the function. However, to modify a global variable inside a function, you must use the global keyword.
