AttentionBlock with for loop exporting a large graph?

Is there any way to handle attention decoder block which including a loop? The implementation is on this Colab Link

Question

I wish to export a Attention Block that include a loop for decode. It seems like the LSTMCell can't export to ONNX due to this issue. So I used the LSTM work like LSTMCell by taking single time step to inference. How can I handle this kind of model? Is this a problem with the onnx export function in pytorch?

If i export with use_external_data_format=True option, it doesn't save to a single file but instead a lot of separated files.

I have open an issue in ONNX github issue and ask in stackoverflow because I'm not sure it is an bug/feature issue or a question.

Notes

It seems like the following implementation exporting a very large graph can't be export to ONNX and show the following errors.

RuntimeError: Exporting model exceed maximum protobuf size of 2GB. Please call torch.onnx.export with use_external_data_format=True.


Read more here: https://stackoverflow.com/questions/68463124/attentionblock-with-for-loop-exporting-a-large-graph

Content Attribution

This content was originally published by luvwinnie at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: