The quake struck yesterday afternoon in the central Taiwanese county of Nantou, which is mountainous and sparsely populated.
The tremor was felt all over the island, but most severely in the central and southern regions.
The NFA said the third victim died after he was hit by a falling rock in Nantou, and the fourth was buried under a landslide in the same location.
The NFA said 19 people sustained serious injuries in the quake.
Previously acknowledged fatalities were registered in Alishan, in southern Taiwan, where a man was killed by a rockslide while driving a car on a mountain road, and in Chushan, near the quake's epicenter, where a man working at a farm was killed by a falling rock.