RuntimeError: mat1 and mat2 shapes cannot be multiplied (384×32 and 1024×64) in PyTorch

Ask Questions Forum: ask Machine Learning Questions to our readersCategory: PyTorchRuntimeError: mat1 and mat2 shapes cannot be multiplied (384×32 and 1024×64) in PyTorch
Chris Staff asked 6 months ago

I am creating the following model with PyTorch:

class MLP(nn.Module):
  '''
    Multilayer Perceptron.
  '''
  def __init__(self):
    super().__init__()
    self.layers = nn.Sequential(
      nn.Linear(32 * 32 * 3, 64),
      nn.ReLU(),
      nn.Linear(64, 32),
      nn.ReLU(),
      nn.Linear(32, 10)
    )

Then I want to train it with the CIFAR-10 dataset:

  # Prepare CIFAR-10 dataset
  dataset = CIFAR10(os.getcwd(), download=True, transform=transforms.ToTensor())
  trainloader = torch.utils.data.DataLoader(dataset, batch_size=10, shuffle=True, num_workers=1)

  # Run the training loop
  for epoch in range(0, 5): # 5 epochs at maximum
    
    # Print epoch
    print(f'Starting epoch {epoch+1}')
    
    # Set current loss value
    current_loss = 0.0
    
    # Enumerate over the DataLoader for training data
    for i, data in enumerate(trainloader, 0):

I am getting this error:

Traceback (most recent call last):
  File "classic.py", line 61, in 
    outputs = mlp(inputs)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "classic.py", line 26, in forward
    return self.layers(x)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\container.py", line 117, in forward
    input = module(input)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\linear.py", line 93, in forward
    return F.linear(input, self.weight, self.bias)
  File "C:\Users\chris\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\functional.py", line 1692, in linear
    output = input.matmul(weight.t())
RuntimeError: mat1 and mat2 shapes cannot be multiplied (384x32 and 1024x64)

Why does it occur?

1 Answers
Best Answer
Chris Staff answered 6 months ago

The model is being trained with densely-connected layers (i.e. Linear layers). The ‘problem’ with these layers is that they only accept 1D data. If you’d have used Conv layers, this wouldn’t be a problem – and can be recommended especially with image data. However, if you really want to use Dense layers instead, you can add a Flatten() layer as the first layer, which essentially flattens the multidimensional data into 1D.

    self.layers = nn.Sequential(
      nn.Flatten(),
      nn.Linear(32 * 32 * 3, 64),
      nn.ReLU(),
      nn.Linear(64, 32),
      nn.ReLU(),
      nn.Linear(32, 10)
    )

Your Answer

12 + 6 =