Getting "view size is not compatible with input tensor's size and stride" due to missing ".contiguou

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

才高八斗的西红柿 · 手写笔记软件 MarginNote 4 ...· 2 周前 ·

玩足球的铅笔 · 魔性社区官方正版APP下载-魔性社区官方正版 ...· 1 月前 ·

坚韧的萝卜 · 行政事项-深圳市卫生健康委员会网站· 2 月前 ·

追风的花生 · 小甲鱼python论坛 - CSDN文库· 2 月前 ·

很酷的跑步机 · 人类输给了AI，这一次，事关蛋白质改造_系统 ...· 3 月前 ·

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account Getting "view size is not compatible with input tensor's size and stride" due to missing ".contiguous()" in megatron-core dependency #7055 Getting "view size is not compatible with input tensor's size and stride" due to missing ".contiguous()" in megatron-core dependency #7055 yaroslavvb opened this issue Jul 18, 2023 · 3 comments

Trying the latest stable Nemo release on wikitext by following the README instructions I run into the following error

  File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 123, in decorate_bwd
    return bwd(*args, **kwargs)
  File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/megatron/core/tensor_parallel/layers.py", line 284, in backward
    total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
The work-around is to open /disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/megatron/core/tensor_parallel/layers.py file and change line 224 from
        total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
                                       total_input.shape[2])
to instead read
        total_input = total_input.contiguous()
        total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
                                       total_input.shape[2])
This file is part of the following package:
megatron-core             0.2.0                    pypi_0    pypi
      changed the title
Getting "  File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 123, in decorate_bwd     return bwd(*args, **kwargs)   File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/megatron/core/tensor_parallel/layers.py", line 284, in backward     total_input = total_input.view(total_input.shape[0] * total_input.shape[1], RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead."
Getting "view size is not compatible with input tensor's size and stride" due to missing ".contiguous()" in megatron-core dependency
    Jul 18, 2023