device error in SinusoidalPositionalEmbedding
Summary:
Not sure if I'm doing something wrong elsewhere, but I had a device error in `SinusoidalPositionalEmbedding` when running on GPU > 0 because the weights were on a different device than the input.
Pull Request resolved:
@@ -65,7 +65,7 @@ class SinusoidalPositionalEmbedding(nn.Module):
self.embedding_dim,
self.padding_idx,
)
- self.weights = self.weights.type_as(self._float_tensor)
+ self.weights = self.weights.to(self._float_tensor)
if incremental_state is not None:
# positions is the same for every token when decoding a single step
