The Osaka prefectural government's disaster management department said two people were found dead, while the Ibaraki city official confirmed a third victim.
The Fire and Disaster Management Agency said 91 were injured in Osaka and nearby prefectures of Hyogo and Kyoto.
One victim was a nine-year-old girl who was knocked down by a concrete wall at her elementary school as she walked by.
A man in his 80s died in the collapse of a concrete wall in Osaka city.
An 84-year-old man in nearby Ibaraki died after a bookshelf fell on top of him at home, according to city officials.
The magnitude 6.1 earthquake struck shortly after 8am local time (midnight UK time), north of Osaka at a depth of about 13km (eight miles), the Japan Meteorological Agency said.
The strongest shaking was north of Osaka, but the quake rattled large parts of western Japan, including Kyoto, the agency said.
The quake knocked over walls, broke windows and set off scattered building fires. It toppled book shelves in homes and scattered goods on shop floors. It also cracked roads and broke water pipes, leaving homes without water.
The morning commute was disrupted, as dozens of domestic flights in and out of Osaka were grounded, while train and subway services in the Osaka area including the bullet train were suspended to check for damage.
Passengers were seen exiting trains on the tracks between stations.