@Michael Loucks
The vote distribution on my stories doesn't bear this out (i.e., the number of scores below 5 is consistent despite there being fewer overall votes).
However the overall trend of the scores of the stories in your series (pl) does tend to follow the trend (although I didn't bother doing a sequence/score correlation).
Perhaps some kind of synthesized score that gave more weight to the first story, similar to how the displayed score is mean-reverted from the actual score.
I had considered amalgamating every vote for every story in a series and mean-reverting that value against its peers, but that would 'suffer' the same upward drift.
Perhaps take the story with the most votes as being representative of the series. That would insulate the score from the consequences of authors who write prequels and adjust the story sequence to represent chronology.
AJ