The 19-year-old was to leave for Boston so he could meet Swift and celebrate New Year, but realised the big mistake after he reached the Heathrow airport, reported Daily Mirror.
The singer later got a courier service to pick the document from his house and get it delivered to airport and took the next flight.
"Harry was due to fly out on Friday morning and his mum very kindly offered to drive him to the airport, but when he got there he realised he didn't have his passport.
"He managed to get on the evening flight but it still meant that he had to wait another eight hours for his post-Christmas reunion with Taylor," a source said.