TRS-backed G Deviprasad conceded his defeat and congratulated Rao for his victory.
Counting was still in progress for the Khammam-Nalgonda-Warangal graduates' constituency.
The elections to the two constituencies were held on March 22.
Though the elections were not held on political lines, the main contest in the two graduates' constituencies in Telangana was between the candidates backed by the ruling Telangana Rashtra Samithi and the BJP-TDP combine.