Firstly I'm not sure how the code for this site is written so I'm not able to declare this a 100% solution. But it is a practical option in theory.
when someone clicks to vote the options form a list starting at the dreaded one. if the code was slightly different, could you set the list to start in the middle, say a 5 to make it quicker to track to scores either above or below average. This is all just theory in my head but it is at least theoretically possible to do and may make some authors a bit happier and voters more likely to at least vote average rather than one.