Oli, widely regarded as pro-China, also said he wants to "update" relations with India "in keeping with the times" and favours a review of all special provisions of the Indo-Nepal relations, including the long-established practice of Nepalese soldiers serving India's armed forces.
"We have great connectivity with India and an open border. All that's fine and we'll increase connectivity even further, but we cant forget that we have two neighbours. We don't want to depend on one country or have one option," he told Hong Kong's South China Morning Post in an interview.
Oli, 65, who had also served as the country's prime minister from October 11, 2015 to August 3, 2016, was forced to resign following the political crisis in the Himalayan nation over the Madhesi issue.
Madhesis, mostly of Indian-origin had been demanding amendment of the new Constitution to address their concerns over representation in Parliament.
More From This Section
About Nepalese serving in the Indian Army, Oli said, This should be internally and mutually discussed and corrected, if necessary. We live in a new world, and Nepal is starting a new journey, we have to update whatever is considered outdated and bring it in line with the modern era."
Oli also said he would revive the China-backed USD 2.5 billion hydropower project which was scrapped by the previous government.
The dam project was scrapped by the previous government headed by Sher Bahadur Deuba citing irregularities.
China has been investing heavily in Nepal, blunting India's influence in the Himalayan country.
Oli justified his decision to build the dam with Chinese company saying that "our petroleum usage has been increasing but we import all of it. We urgently need to develop hydropower to reduce our dependence on petroleum."
He also spoke about improving rail and road network between Nepal and China through Tibet.
"Once China brings its rail network up to Shigatse and then Kyirong in Tibet, it should be easy to extend it to Nepal. Its lower altitude than Tibet, and the terrain is actually sloping all the way down from Kyirong.
"Apart from that, three roads are under construction connecting China and Nepal, which should be ready in a couple of years. If we can connect this railway network to our east-west rail project, it can revolutionise China-India trade, with Nepal in the middle," he said.