How much does distance affect ping?

Yes absolutely, the closest your server is from the user the better the ping is because the travelling distance / time is reduced.