World champions Germany also dug themselves out of trouble as a stoppage-time equaliser from Arsenal winger Serge Gnabry salvaged a thrilling 3-3 draw against South Korea in Salvador.
Hosts Brazil hope to kickstart their campaign for a maiden football Olympic gold later on Sunday against Iraq in Brasilia after stumbling to a 0-0 draw with South Africa in their opener.
Argentina's preparations for the Olympics were hugely undermined by a corruption crisis engulfing the country's football federation.
Olarticoechea's options were also slashed by a number of high-profile pullouts and European clubs not releasing their players so close to the start of the season.
More From This Section
However, it was Calleri, one of those called in at the last minute to replace Correa's former Atletico team-mate Luciano Vietto, who bundled home the winner 20 minutes from time.
Argentina captain Victor Cuesta put his side in real trouble just before half-time when picked up an unnecessary second yellow card.
- Mexico's saviour -
====================
Sofiane Bendebka levelled 14 minutes later, but Algeria were also reduced to 10 men when Ayoub Abdellaoui saw a second yellow card.
And Argentina instantly took advantage when the ball ricocheted off Calleri and into the net despite a desperate lunge from Algeria 'keeper Farid Chaal.
Argentina still need to beat Honduras in their final Group D game on Wednesday to qualify for the quarter-finals.
Germany lie in third place in Group C, two points behind
Mexico and South Korea.
They now face the prospect of having to beat Fiji by at least four goals in their final group game to progress if South Korea's final match with Mexico also ends in a draw.
Gnabry took his tally to three goals in the competition as he also cancelled out Hwang Hee-Chan's first-half opener.
Davie Selke briefly gave Germany the lead before Tottenham Hotspur's Son Heung-Min equalised and Suk Hyun-Jun looked to have sealed Korea's place in the last eight.
Earlier, 2012 gold medallists Mexico needed a second-half fightback to stave off one of the biggest shocks in Olympic football history to beat Fiji 5-1.
Fiji benefited from New Zealand's disqualification for fielding an ineligible player to qualify for the Olympics via last year's Pacific Games, and were duly thrashed 8-0 by South Korea on their debut on Thursday.
However, they led 1-0 at half-time in Salvador thanks to Roy Krishna's opener.
Erick Gutierrez was Mexico's saviour as he struck four times in the second-half to join Nigeria's Oghenekaro Etebo as top scorer in the tournament, whilst Carlos Salcedo was also on the scoresheet.