The quake, which violently shook buildings in the capital Taipei on Sunday, sent people running into the streets and was felt in Hong Kong, more than 700 kilometres away.
After the National Fire Agency reported an initial death toll of two people, rescuers found the body of a man previously listed as missing, and a victim injured by falling rocks later died in hospital.
Three of the victims were killed in central Nantou county, the epicentre of the quake whose magnitude Taiwan's Seismology Centre measured at 6.3, while the US Geological Survey put it at 6.2.
Tremors caused widespread landslides in the mountainous county, as well as cracks in roads and along the walls of houses.
Television stations reported panic-stricken shoppers running out of a 12-storey department store in the central city of Taichung.
Nantou county was the epicentre of a 7.6 magnitude quake in September 21, 1999, that killed around 2400 people in the deadliest natural disaster in the island's recent history.
In late March this year a strong earthquake in the same vicinity as Sunday's tremor killed one person and injured 86 others.