pytriton use onnx is slower than onnx runtime for tiny bert model

[yan123456jie]

My model path is https://huggingface.co/jesseyan/tiny_classify_bert/tree/main

onnx runtime server is https://github.com/yan123456jie/model_speed_test/blob/master/src/onnx/onnx_server.py
onnx speed test https://github.com/yan123456jie/model_speed_test/blob/master/src/onnx_web.py

pytriton server is https://github.com/yan123456jie/model_speed_test/blob/master/src/pytriton/pytriton_server.py
pytriton speed test https://github.com/yan123456jie/model_speed_test/blob/master/src/pytriton_web.py

my final test result is blow

[github-actions]

This issue is stale because it has been open 21 days with no activity. Remove stale label or comment or this will be closed in 7 days.