Home » Forum » Bug Report and Feature Requests

Forum: Bug Report and Feature Requests

A separate display of the score from the last 6 or 12 months

ChiMi 🚫

We are at the "score is meaningless" discussion again with the "Three Square Meals" thread.

The main issue is, that the score for very long series is kind of meaningless because of early scorers.

Is the story still good, or is it going downhill with repetitiveness for the last 2 years?
Is the story now better than 3 years ago?

We don't know.
Steam has a 2-part scoring system. For one, the overall Scoring and the Scoring for the last 30 days. So you can see if a game got better or worse with the latest patches etc.

30 days is a bit too narrow for SOL stories, probably not enough new scoring in 30 days. So 6 months or 12 months should be enough to see how the story resonates with viewers that finished the story recently. (Well, "finish" is a strong word, they still could score with just 1 chapter read, but eh, you can't have everything.)

Replies: Switch Blayde

Switch Blayde 🚫

@ChiMi

Is the story now better than 3 years ago?

What about finished stories? Maybe recent voters are scoring higher or lower than previous ones and someone might want to see the trend.

I'm not saying that's needed, but isn't it similar to what you're asking for?

Replies: ChiMi

ChiMi 🚫

@Switch Blayde

The overall score isn't going to be changed or influenced, there would be one overall score and one score from the last months. If a 9.x Story from 10 years ago has a lower 6-month score, than that's just a changing reader market or expectations. Something that was new and exciting story wise or sexual wise 10 years ago, maybe overblown and trope'd into oblivion today.

@Dominions Son
This wouldn't be a change to the overall score or scoring system, just a second display.

Replies: Grant

Grant 🚫

@ChiMi

This wouldn't be a change to the overall score or scoring system, just a second display.

What you described above,

The overall score isn't going to be changed or influenced, there would be one overall score and one score from the last months.

You're going from 1 single score, to 2 different scores.
Sounds like a significant change to the coring system to me.

The fact is that many of those stories that go on forever have the high sores because the people that are still reading them still enjoy it, even though most other people don't (which is why they stopped reading ages ago).
Which means the overall score, and your new month by month (or whatever period) score would still be pretty much the same. Hence no real need for that second score.

Replies: ChiMi

ChiMi 🚫

@Grant

How can it be the same if the old readers can't score again? They can just update their score.

There would be no new scoring, no new scoring "system", just a new Display that lets you see what readers, that scored in the last x months, thought about the story.

The Score from the last x months would still be applied to the overall score.

Dominions Son 🚫

It should be noted that Lazeez swore off further mods to the scoring system more than a year ago.

ChiMi 🚫

Still, that system could be flawed for stories that don't get many scores. A story's store could be a "1" if just 1 person scored the story in those months. The "1" score doesn't matter for the overall, but alone it would be not representative, even if the person who scored alone in those months thought it deserved it.

So maybe, instead of displaying a second score of the last months, Lazeez could implement a "score from the last 5%".

First display: The score from all the participants.
Second display: The score from the last 5% of participants. This doesn't matter for new stories but would be helpful for older stories.

REP 🚫

The idea sounds like a great deal of effort for little to no gain.

I see no "added value" to being able to see how today's scores, to include updated scores, compare to the scores readers gave a story X months ago.

Replies: ChiMi

ChiMi 🚫

@REP

You sound like it takes a rocket scientist to just display a second line that takes its value from the last 5% of the current value.

Replies: REP

REP 🚫
Updated:

@ChiMi

The coding is far more complex than just a statement to display a second line. The coding would have also have to address sampling each of the stories' ratings at periodic intervals and save those values to a database for each of the 41K+ stories on the site.

Ignoring the amount of time necessary to do the coding, Lazeez would also have to allocate disk space for storing the database.

The main point I was making is, "What value does Lazeez's efforts provide to the reader?" Personally, I would say none. I also doubt that there would be more than 20 or so people seriously interested in the data; most readers would say "Oh, that's interesting." and immediately forget it.

ETA: Your initial proposal was two ratings X months apart, which is what I addressed above. Your 5% proposal would be even more complex to implement. And as I said, the reader gets nothing but a couple of interesting numbers that are really meaningless since they only reflect reader appeal.

Centaur 🚫

If i were a betting man, i would assume that the score is tallied as soon as some one votes. To get the last 5% 10% or last 30 60 120 days you would have to store all votes separately for these calculations. might not be must with one or two votes but storing 100+ votes would take serious disk space and number crunching though the php to display the results of one extra field. Disk space+data base+CPU cycles+band width=$$ not worth it

Replies: REP

REP 🚫

@Centaur

you would have to store all votes separately for these calculations.

Lazeez currently does that for all member accounts. That is how your rating is displayed when you reopen a story.

Dominions Son 🚫

Right - and if a timestamp is included, many of the things proposed here and some more could be done.

But it would require additional code just to do the calculations, beyond just displaying it.

Only one person can speak authoritatively on what the level of effort would be.

Switch Blayde 🚫

many of the things proposed here and some more could be done.

I think it would be confusing. Right now it's confusing because the score readers see isn't the raw score (what they give). Add another score to the mix and many will scratch their heads.

Replies: ChiMi

ChiMi 🚫

@Switch Blayde

So the score displayed in the steam store is confusing?

i.e. http://store.steampowered.com/app/272270/Torment_Tides_of_Numenera/

Replies: Dominions Son Switch Blayde

Dominions Son 🚫
Updated:

@ChiMi

So the score displayed in the steam store is confusing?

Yes. How is it calculated from the individual reviews.

Are the recent reviews tallied differently than all reviews?

SOL scores are not a simple average. The algorithm removes outliers (5% from both high and low) which means that stories with some votes but fewer than 20 won't show a score. Even after removing outliers, the scores are weighted.

If you don't follow the same algorithm for the "recent" scores, they won't be comparable in any way to the long term score. But what if a story doesn't have enough "recent" votes to calculate a score?

In SOL a reader can re-score a story many times, but the system only keeps the most recent. Are re-votes counted for recent votes? Because of the way the scores are calculated, if re-votes are counted for recent votes, a single re-vote could have drastically different impacts on the long term and recent scores.

Because of the way the scores are calculated, a high vote that pushes the story to the next higher whole number of outlier votes could actually cause the story's score to drop.

Like wise, a low score that trips the next pair of outliers could raise a story's score.

SOL story scores are confusing enough in how they should be interpreted and how the scores relate to individual votes, without adding a whole other layer of complexity to it.

Switch Blayde 🚫

@ChiMi

As Dominions Son said, the SOL scoring is very complex (and convoluted in the sense it's not a simple average).

Do you know that a 10 score can lower your score? When I was testing the latest scoring system, I gave my story a 10. The score went down. Why? I don't really understand, but it has something to do with the weight factor changing so when the score was recalculated, more came into play than the new 10.

How would you trend that? It would have looked like the trend went down, but it actually went up.

Replies: Michael Loucks REP

Michael Loucks 🚫

@Switch Blayde

Do you know that a 10 score can lower your score? When I was testing the latest scoring system, I gave my story a 10. The score went down. Why? I don't really understand, but it has something to do with the weight factor changing so when the score was recalculated, more came into play than the new 10.

The solution to the 'score creep' adjustment which causes strange scoring behavior is to switch to a thumbs-up, thumbs-down system, and display the score as a ratio between the two. But our esteemed host has said he is making no more changes to the scoring system...

Replies: Dominions Son

Dominions Son 🚫

@Michael Loucks

But our esteemed host has said he is making no more changes to the scoring system...

Because if the authors don't bitch about any changes, the readers do. And a few bitch from both ends at the same time. :)

REP 🚫

@Switch Blayde

I gave my story a 10. The score went down. Why?

Perhaps it had something to do with the algorithm taking into account the ratings assigned to other stories in calculating a new rating for your story.

Replies: Switch Blayde

Switch Blayde 🚫

@REP

Perhaps it had something to do with the algorithm taking into account the ratings assigned to other stories in calculating a new rating for your story.

That's exactly what happened. I asked the question to answer it.

When the new scoring system went into effect, all the existing stories got rescored with the new algorithm. When I gave my story a 10, the weighting factor changed from the initial rescoring so a 10 lowered the score. Actually, it was the rescoring that lowered the score. The new 10 simply invoked the rescoring.

REP 🚫

a timestamp is included

Many things can be done. The question is will the result be worth the effort to the doer.

Reply to topic

Forum: Bug Report and Feature Requests

A separate display of the score from the last 6 or 12 months

WARNING! ADULT CONTENT...