Aborted (core dumped) - Reg

[Adhiyaman-Manickam]

Hi @daa233

Thanks for you great work.

I have got the following error in torch.stack after completing the 3k iterations of training model. I unable to found the error.

Kindly help me out the error.
Thanks in advance.

[daa233]

In my view, core dumped error may happen when your GPU memory is insufficient during training. Maybe you can monitor the GPU memory usage when you train it again.

Since there are no more log information, I have no more idea about it.

[Adhiyaman-Manickam]

Thanks for your reply.
I have used linux server.
I have checked. I have a enough GPU memory.
Is this problem due to stack (torch.stack and python code) or linux server (i,e gpu)
It is working on single gpu but not muliti gpu.

[daa233]

I didn't meet this problem when training on multi-GPUs so far. The code has been tested on PyTorch 1.0.1 and PyTorch 1.2.

You'd better post more information about the error trace (although I know sometimes core dump errors give little information). It will save a lot of time if you can locate which line caused the error. Maybe you can try to debug or print some log by inserting some code.

Since it is possibly caused by torch.stack, maybe you can check this visualization code under your settings.

[Adhiyaman-Manickam]

Thank you so much for you prompt reply. I will try my level best to fix the issues as per your directions.
Thanks.

[Adhiyaman-Manickam]

Hi @daa233

continue to the above issue,

I am getting the following warning, if i am starting to use multi GPU then core dumped error comes.

UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector. warnings.warn('Was asked to gather along dimension 0, but all

kindly, let me know, where i made a mistake.
Many thanks

[daa233]

@Adhiyaman-Manickam Just as the above words, you'd better debug by yourself and find the exact line of code caused the error.

Without debugging or any information with the lines of code, it is difficult to judge just from the error.