Kejriwal extended the invitation to Modi during a meeting with him which was also attended by AAP leader Manish Sisodia.
The Prime Minister informed him that he will be out of Delhi on Saturday and will not be able to attend the ceremony, Sisodia said.
Modi will attend an event organised by Krishi Vigyan Kendra in Maharashtra's Baramati on Saturday.
The AAP leader had yesterday invited Home Minister Rajnath Singh and Urban Development Minister M Venkaiah Naidu to the oath-taking ceremony in separate meetings Kejriwal had with them.