The minister was addressing a function organised by Patel Sewa Sangh, Bihar, to celebrate the 142nd birth anniversary of the Iron Man of India.
Sardar Patel was bestowed with the country's highest civilian award posthumously in 1991, 41 years after his death in 1950.
"Every eminent personality got this award. But in Patel's case, it took 41 years to honour him with the Bharat Ratna award. It was the Narsimha Rao's government who honoured him with the award," Prasad said.
Prasad said that the country's first president Dr Rajendra Prasad had said in 1959 that what India stands today, it was because of Patel.
More From This Section
"Had Patel been the prime minister for three years (1947 to 1950), the shape and direction of the country would have been different," he said, adding, it was his personal opinion.
Prasad said shortly after Independence, there were 555 princely states and Patel successfully handled 554 of them, while Jawaharlal Nehru handled just one (Jammu and Kashmir), "which he messed up by taking to the UN, against Patel's opinion on the issue".
"After 70 years of Independence, today we need to debate and discuss the issue as to why a person from only one family would become the CM, party president, the PM...How long this will continue?" he asked.