Comparing similar businesses, for example, with a rating of 4.8 and 4.9, a potential customer will choose the business with a rating of 4.9 about 80% of the time. That means you could be missing out on tens or hundreds of customers simply because of a 0.1 point difference.
The number of reviews also plays a huge role in getting closer or farther down the map, especially when combined with a business's rating. The more reviews you have, the further away your business will be seen from.