only check grad norms if not using BMUF
Summary: Since fairseq is checking if grad norms are consistent across workers. This is causing issues for BMUF since we don't sync every iteration (f161652382). Let's disable checking grad norm consistency for BMUF.
@@ -366,6 +366,7 @@ class Trainer(object):
grad_norm = self.optimizer.clip_grad_norm(self.args.clip_norm)
# check that grad norms are consistent across workers
+ if not self.args.use_bmuf:
self._check_grad_norms(grad_norm)
# take an optimization step
