The rhino was shot near Bihpukhuri in Bishwanath Chariali yesterday and was being treated by a medical team of the park, but it died this morning.
It had strayed from Kaziranga National Park's northern range about two months ago.
This was the first case of a rhino being killed this year while 42 were killed last year.
Forest department personnel and local people were looking for the suspected poachers.