Forum: Author Hangout

See the thread about an Australian named Hicks. Relevance to SOL or any story that might have been on SOL, if we were lucky, is very close to zero.

The thread that discussion is on is about Military tribunals, which is why it got started, as there is a relevance in regards to the relevance of a military tribunal hearing a case against a civilian. That's how it got there, and then drifted a little, and the only time I've seen a thread not drift is heavy discussions on scores

I don't bitch about the scores because I understand them, their purpose, and how they apply to all evenly, so they're fair. What I don't like is the advice to over vote to pump a score, because that then devalues all the valid scores given up to that time.

What I don't like is the advice to over vote to pump a score, because that then devalues all the valid scores given up to that time.

Not sure what you mean by 'over vote'. My impression is that submitting a 'second' vote just changed the score you entered on your first vote.

Or did you mean 'submit an excessively high score'?

Not sure what you mean by 'over vote'

at one time, well in the past, no information was kept about who voted on what story beyond a few weeks. Thus it was possible for someone to vote on a story a number of times, as long as they left a long enough period between votes - so one person could vote multiple times on the same story. That got changes some years ago, and is no longer possible.

at one time, well . . . . and is no longer possible.

Thanks for the clarification. I haven't been around SOL that long.

I also haven't tracked the basis and changes to Lazeez's scoring algorithm. I do remember prior discussions regarding what his algorithm was attempting to do. I personally disagreed with what some posters attributed to him as his goals, but it is his system and I am willing to accept what he implements. Overall, I don't concern myself with scores because as you say in your second post to me, 'All votes are subjective by the reader.' The only time scores seem important is when I allow my ego to take control of my common sense.

Lets start one now. You hear about anti-heroes but why not about uncle-heroes?

There's a very simple reason: auntie-heroes are a staple of Indian porn. There are no uncle-heroes in that genre.

bb

The scores are just damned depressing! So you live with it!

Aubie, if they depress you, just ignore them. The scores are for the readers, not the authors. That's why I ignore them, except when checking something for a discussion about them.

I suggest Lazeez into a "Scoring Bitch Box", like a "Swear Box" some people use.

What, you're going to stuff poor ol' Lazeez into a box? What's worse, a box full of scoring complaints? Sounds like a fate worse that ... supporting complaining authors!

@Ross at Play

I'd like reviews of one story each from as many authors as possible.

Personally, I'd reserve new reviews for when an author writes an entire new type of story which are unlike their other stories (ex: changing from "Stroke" to "Mystery", or "Romance" to "High Fantasy").

I would not assert you are wrong.

I assert you're wrong about not being wrong! I'm not wrong, am I?

I wouldn't mind seeing is a count of each score on the Detailed Info and Stats page

top right corner of most pages on SoL has a webmaster link, and it has an option for Feature Request - it's there to make these suggestions by.

The only change I wouldn't mind seeing is a count of each score on the Detailed Info and Stats page.

Me too. I thought I was missing something because some authors already seem to know this information, but I never got around to asking how.

AJ

I thought I was missing something because some authors already seem to know this information, but I never got around to asking how.

It's on a case by case basis. If an author agrees not to complain about the scoring system, Lazeez will provide that author the ability to see the raw scores.

It's on a case by case basis. If an author agrees not to complain about the scoring system, Lazeez will provide that author the ability to see the raw scores.

My understanding of the offer was they would be given access to all the raw scores in their full glory. While any author can see the bar graph located by going to the Stats page and clicking on the story.

My understanding of the offer was they would be given access to all the raw scores in their full glory.

Not exactly. I don't see all the raw scores; only the average of those scores. They appear in a column called "VA Scr" just to the right of "Score" (the SOL score given to the story).

The bar chart referenced before tells you how many 1s, 2s, etc.

Actually, the only number you know for certain is the vote with the highest count. Once your vote count gets up over about 100 (depending on the distribution), you have to 'guesstimate' and the bars become meaningless for the other stats.

Not anymore. There's a number at the top of each bar. Maybe that's only for those who can see the "VA Scr" on the Stats page.

Not anymore. There's a number at the top of each bar.

Nope, there's only the value of the highest on my graphs. But I never promised not to discuss the scores, saw no need to.

Nope, there's only the value of the highest on my graphs. But I never promised not to discuss the scores, saw no need to.

Then you can't see the "VA Scr" on the Stats page either. They must go hand-in-hand.

As I said, Lazeez made me happy by providing it. I don't complain about the scoring system anymore.

Then you can't see the "VA Scr" on the Stats page either.

No I don't see that. I never complained about the scores or the stats page, I only ever notice them during these discussions. So i never saw a need to worry about it.

continuing to whine about it will change his mind in a positive way seems just a tad delusional to me.

Yeah! Add in that the positive whining probably wants him to change the system in two or more incompatible ways would explain why he said No More Changes!

Why should I just live with it? I am sick and tired of receiving 5 votes less than a score of 6 out from a total of around 260 votes and having that give me a score of 6.93 instead of the 8.34 that the basic raw scores amount to. I don't give a shit about whether or not it happens to anybody else, I care that it happens to me.

In my case (I never took Lazeez's pledge never to bitch about scores, because I knew the topic would continually reappear), I frequently evaluate sites to select which are best for posting stories to. When I see the same stories getting different scores (even rated differently), I tend to apply those results to the site, rather than to the scoring adjustments. Thus I, and others like me, might very well pull stories from a site because we no longer feel the stories are a good fit on the site.

I'm also a stat follower, however I don't evaluate the raw scores, instead like most economists, I look for changes in scores. Whatever the score, if the score dips for a particular chapters, chances are I did something wrong. If I understand there's something upsetting (or particularly exciting) about a given chapter, I can account for it, but if it's a mystery, then chances are I missed something about the story which I need to account for and possibly make changes to the story. That's something you can't do with book sales where readers either buy the book or they don't (and they often don't read the ones they do buy, too).

Hope that helps frame the discussions a little more.

There are many sports where the top and bottom 10% of scores are eliminated before any averages, etc. are calculated.

That's part of the SOL scoring system, although I believe it's 5%. Lazeez put that in at the request of authors.

difficult fix for Lazeez to program into the system

It already happens, the top and bottom 5% (in whole numbers rounded down) are cut from the raw scores before the total is averaged for use in the score.

From my understanding of Lazeez weighted scoring system, it seems very subjective.

1. I understand he tries to adjust the score for readers who don't vote. I don't understand how this can be done fairly since he doesn't know the opinions of those readers. Besides, they had the opportunity to register their vote and decided to not do so.

2. I understand that readers often give a story a score that is higher than it deserves, but there are also readers who give the story a score lower than it deserves. I can't think of a fair way to adjust a story's score for inaccurate scoring.

Personally, I think the average of the raw scores is the only fair way to rate a story. At least it will be an accurate reflection of the scores given to all writers' stories. So what if a story's average come out high, it is just an indication of reader satisfaction.

1. I understand he tries to adjust the score for readers who don't vote.

I don't want to speak for him (or is it Him?), but I believe it's because when you like a story you give it a high score; but when you don't like a story you probably don't finish it so you don't give it a (low) score. That skews the scores to the higher end.

As I said, I can now see the real average and distribution of scores given. I hardly ever look at scores after the story has been out there for a week or so, but if I do that's what I look at. I like sorting my stories by the "VA Scr."

That skews the scores to the higher end.

That's not always the case. I've had people email me to tell me they voted the score down without finishing it because they didn't like either present tense or vernacular writing. But I'm sure there are many people who don't vote when they dump a story partway through - although I'll vote a story I dump as to how I rate it.

As I said, I can now see the real average and distribution of scores given. I hardly ever look at scores after the story has been out there for a week or so, but if I do that's what I look at. I like sorting my stories by the "VA Scr."

It almost becomes a game. Once a story is initially scored, it rarely changes much beyond that initial score, so an author's best bet is to start off with a bang, presenting their most exciting chapters first--which is a traditional approach for most literature--rather than waiting for the story to be scored at completion.

I've also noted that scores drop significantly when the story is done. Fans will vote a 'final' score just after a story is completed, but those who never felt compelled to read the story will read it once it finishes, and generally post lower votes, bringing the score down. It will generate up over time, but those initial scores are essential to the ultimate score.

I've also noted that scores drop significantly when the story is done.

That's odd, mine tend to go up over time, as more people read them and the numebr of votes goes up..

That's odd, mine tend to go up over time, as more people read them and the numebr of votes goes up.

It's an interesting dynamic. Initially, after the story completes, the scores jump for one to two days, then they descend as new readers discover them to be not to their liking, but over time they tend to drift back up again.

What I'm suggesting, is that stories that score low tend to remain there, rather than they might if posted at the same time. It's like in recent politics. Despite polls consisting of only a tiny fraction of any given representative: those still owning home phone who don't immediately hang up (i.e. old people eager to talk to anyone), the polls remain highly accurate. That doesn't speak well for the polls, instead it suggests that the polls influence voters more than it reflects how voters feel at any given moment. Voters and readers both want to fit in, so if everyone else thinks a story isn't their cup of tea, most others will agree, rather than sticking their neck out and saying "No, this is a 10" or "Shit, this is only a 4 at best".

That's not to say that poor scoring stories don't belong there, it's just my view that polls and scores are subject to what's known as 'observational bias', a factor that's rarely taken seriously by the press or studied enough to document it.

scoring system, it seems very subjective.

All votes are subjective by the reader. The problem was coming up with a way to relate scores given under different scoring systems to each other. It was either find a way to bring them all to the one norm or dump all the older scores and start again. he chose to normalize them via the system explained earlier.

The funny thing about this is I'm one of the few who gets consistently high end scores on most of their stories, but don't mind the system cutting them back a bit to normalize them. That's because the system affects everyone in the same way and all the scores are in the same relative position to each other, regardless of what they actually read.

1. I understand he tries to adjust the score for readers who don't vote. I don't understand how this can be done fairly since he doesn't know the opinions of those readers. Besides, they had the opportunity to register their vote and decided to not do so.

He's never claimed this. Instead, most scores tend to be overinflated (too many 10 votes for average stories) so he downgrades most 'typical' stories to produce meaningful scores.

Authors and a few readers object when they vote 10s and see a resulting score of only 7. Personally, my objection is not knowing whether a lower score on a particular site (say FS) is because the readers there don't appreciate a particular story, or the one story got graded on a curve at that particular point in time.

Shouldn't you be able to figure that out from looking at the score histogram?

That doesn't account for why a single story does better or worse. It may be that the story wasn't appreciated, or it may have been the site's normalizing at that particular time (say when Lazeez dumped a bunch of professional historic stories to boost the sites content).

The histogram appears to be the raw votes.

that's my understanding of it.

Scores from histogram in declining order 10(181), 8, 9, 7 6, 5, 1, 2, 4, 3

Thanks for publishing those figures. They corroborate my theories about readers' voting patterns.

AJ

@DS

Scores from histogram in declining order 10(181), 8, 9, 7 6, 5, 1, 2, 4, 3

@Awnlee Jawking

Thanks for publishing those figures. They corroborate my theories about readers' voting patterns.

There are a lot more people voting 76 than I'd thought! :)

It would take a lot of very-high-scroring posts to affect the curve on SOL.

Not so long on Sci-Fi, slightly longer on FS.

I would love to be able to look at a story score and know if it got low marks because it was a really poorly told story, or if it just contained terrible writing technique that made an otherwise good story unreadable.

If the story has a review it does most of that for you.

However the number of reviewed stories is even smaller than the percentage of people that score a story once they're read it.

True, but why not look for and use it when it's there?

However the number of reviewed stories is even smaller than the percentage of people that score a story once they're read it.

Especially since many reviewers have a policy of "never review the same author twice", despite how many books they've written or how good his/her writing is.

I could be wrong, but I think my job as reviewer, in addition to bringing good stories that haven't been reviewed, at least fairly recently (the last 5 or 10 years), to the readers of the review (not sure how many do, but I am reasonable certain its far less than 100% of everyone who looks at the front page that day) but also by doing so motivate a few to look at the author's page(s) where the story appears and perhaps select another or more than one.

I wasn't necessarily objecting to that position, merely stating it as a semi-official standard denoting just how few reviews there are. With so few reviewers and so many stories, you've got to pick and choose. The point was, the reviews are not necessarily apropos of what's out there.

"never review the same author twice",

I thought it was 'never review the same story twice' not author. I know a few reviewers who review a number of stories from the same author.

I thought it was 'never review the same story twice' not author. I know a few reviewers who review a number of stories from the same author.

In that case, I stand corrected. That wasn't my understanding, probably because it's not an official policy but just the decision by individual reviews to act in concert.

"Most Amazing Story"

An appropriate score for what any politician on the electoral trail tells you about what they've done during their last term in office, and what they plan to do during their next one.

That's what they translate to.

Did you happen to notice 'Good', which is normally considered the average value is rated a '7'. My personal thought is that 'Good' should be a 5 on a scale of 1 to 9 or 0 to 10.

I always preferred my method of voting. I send a thank you note to the writer when I read a story.

And thank you for that. We appreciate it. Scores are open to interpretation, but thanks are fairly universal.

The ultimate objective of grading curves is to minimize or eliminate the influence of variation between different

At that's the basis behind most of the current system, to provide a valid relativity between the different scoring systems used in the past so they can be brought together to give one valid relative score across all the scoring periods and systems.

At that's the basis behind most of the current system, to provide a valid relativity between the different scoring systems used in the past so they can be brought together to give one valid relative score across all the scoring periods and systems.

That may be an ingredient to it, but the main reason was to not give so many As (or in our case 9s and 10s).

This is from another article on grading on a curve:

That fatal flaw is found in the requirement that any area in the positive half of the curve must be duplicated in the negative. A true bell-curve grading scheme requires that instructors produce equal numbers of "A"s and "F"s, equal numbers of "D"s and "B"s, with the largest area reserved for the middle "C"s.

But it is what it is. Now that I can see the raw scores I'm satisfied.

EDITED TO ADD:

The article with the above quote was written in response to someone complaining they got a 96% on a test and received an F (everyone else in the class scored higher).

That may be an ingredient to it, but the main reason was to not give so many As (or in our case 9s and 10s).

Actually, from what I remember of all the past blog posts Lazeez had on this before they got eaten by a dying server is a little different. If I've got this wrong, I'm sure I'll get jumped on and corrected by someone. Also, I'm sure the reality is a lot more complex, but this is really a simplified explanation.

From what I remember the history of the situation is what shaped the current system and the key factors are:

1. Provide the readers with a measure of relative worth to other readers about how they liked the stories.

2. Different scoring systems in the past, due to the system evolving since the site started. Of which there has been four or five or more - I'm not sure.

3. Retian the old scores from the earlier systems and create a way to make them relevant to each other.

4. A need to eliminate most of the fanboi and hate scores.

The way this was done, from what I remember, was to select a value and normalize all the scores in each period using that value as the mean for all the scores in each period. This was applied to every score given to a story. Also, when the three aspect scoring system was killed only the Appeal score was used. I think the chosen mean was 6.

So each scoring period was processed by itself to rate all the scores given in that period relative to each other, then the mean of those scores calculated. From which a value was calculated to have a mean of 6, and that value applied to all the scores to adjust them to have a mean of 6. This was done to each the scores of each scoring period as a separate calculation so each period now had a mean of 6. This allowed all the scores given to now have a valid value relative to each other, and meant they could now all be used in a single calculation. In short - this action took a bunch of mixed fruit scores and made them all apple scores on a grade curve for a fair comparison.

The next stage was to calculate the individual score for each story. That is done by removing the top and bottom 5% of the scores (but only in whole numbers, so they get rounded down to whole numbers), and an average for that story is calculated from what's left - and that's the official score.

There are two aspects of the system that do cause some temporary anomalies.

First the rounding down means that when the votes get to a certain point they do not cause another score to be dropped until the number of votes makes it another whole number. Example: 80 votes means 4 votes are dropped at each end, this continues to happen up to 99 votes, and only when 100 votes are in does the number dropped move over to 5 votes being removed at each end. This will see a sudden change bigger than expected by one vote.

The other is a vote at either end of the spectrum results in a change in the value of the votes being dropped. Example: Most of the scores are grouped in the middle or top end. there are 87 votes so the top and bottom 4 are dropped. The story has 3 votes of 1 and the next is a vote of 5. This means the 3 x 1 and the 1 x 5 are dropped. A new vote of one means the 4 x 1 are dropped and the 1 x 5 is now counted. Thus a 1 vote can put the score up by moving the cut off further down the curve. Similar anomalies can happen at both ends.

An example of a combined application to see a score drop is: 99 votes in place that includes 7 votes of 10, 4 votes of 1, and 2 votes of 6. Thus 4 x 10 and 4 x 1 are dropped. A new vote of 10 is made for the 100th vote, now 5 x 10 are dropped along with 4 x 1 and 1 x 6. The total to be averaged goes up by 10 from the new vote, but loses another 10 from the cut, so no change to the total for that, but the drop of the low end now includes a 6 so the total drops by 6 while the average is now done by 90 and not 89. Thus the official score will drop due to a vote of 10 at an important time. Say the story raw scores are 99 votes for 580 giving a score of 5.89 (since the lower decimals aren't rounded) calculated by 536 divided by 91 due to 4 scores at each end being dropped - 4 x 10 and 4 x 1 removing 44 points. The new vote of 10 makes the raw score 100 votes for 590. Now ten votes are dropped: 5 x 10, 4 x 1, and 1 x 6 removing 60 to give 530 divided by 90 to give 5.88 (since the lower decimals aren't rounded so what comes after the 2nd decimal doesn't show.) The anomalies are always short lived and usually vanish at the next vote.

The two key things to remember are the scores are there solely for the readers and they affect all the authors in the same way, so the system is fair.

Using the curve can be unfair also.

Assume 100 students earn raw percentage grades within the range of 90-99%. The students receiving 90% would fail the test/course.

The problem is that regardless of which formulas are used. Some writers among others will be unhappy with the results. As I recall the present calculations are a result of trying to please as many people as possible with the results.

Just my own cantankerous opinion, but scores should be less about making writers happy and more about providing useful information to readers when they are trying to find something to read. Can a sci-fi story with a 6.5 score be a better read than a similar one with a 7.0? Does a story with a 5.2 score that low because the author is a terrible storyteller, or because his story is unedited and his command of basic English is atrocious...or both - making it pure crapo?

Can a sci-fi story with a 6.5 score be a better read than a similar one with a 7.0?

Absolutely. I've found a 6.5 better than a 9.0.

Does a story with a 5.2 score that low because the author is a terrible storyteller, or because his story is unedited and his command of basic English is atrocious...or both - making it pure crapo?

Or none of the above. Readers' scoring is fickle to say the least.

When I enjoy something I always say thank you to the writer (hopefully they pass that on to the editors). I would probably do the same to the editors but there are no links for them at the end of a story like there is for writers.

Lazeez, if you're listening in: I typically list editors, is it possible to include active links to my editors in my end-of-story notes? If not, could we add that as a feature to help highlight authors. Not only could readers congratulate the editors, but it would be easier for authors to contact the editors of the better edited stories for consultations. Seems like a win-win to me.

Update: Just tried adding active links to my editors on Sci-Fi. It doesn't fly. It strips out the links entirely.

Of course I have no idea of how many other readers would take advantage of that feature.

Authors only receive feedback from a small percentage (it tops out at less than 5%) of readers, so I wouldn't imagine it'd be large. Still, reader feedback is a major feature of the site, and a primary motivation to posting stories.

I don't typically pass on each response to a story to my editors (less it sound like bragging on my part), so having a button to include them in the email responses would be a nice touch (though it would involve a new mechanism to list each story's editors).

I typically list editors

So do I, and I've had some authors ask about the editors, and sent the emails on to the editors, and let them decide. I've got one editor who would like a bit more work, but he doesn't accept a story unless it's no sex or minimal sex so he ends up turning down a lot of authors.

Ah, Basic. I wrote a golf program in Basic that keeps my handicap plus all kinds of statistics. I guess when I move to a Mac I'll lose that. :(

When I programmed, it was in BAL (Assembler language on a mainframe — Basic Assembly Language). When I bought my first PC in 1980 I bought DOS's assembler. Never wrote a single program with it which is why I used Basic.

Nobody used punch cards in the 80's except this class.

I used punch cards, though it was in the 1970s. Before mainframes had TSO (Time Sharing Option) and then VM (Virtual Machine), we coded our programs on coding sheets and the data entry gals typed it to create punch cards. Then we'd put the stack of cards in a tray for the computer room guys to run it. When we got our compile listing back we were able to use keypunch machines to change the bad cards (changes to the program). Man, I can't believe we ever got anything done.

Assembler was not a high level language. It was just up from machine code. The high level language at the time was COBOL and FORTRAN, mostly COBOL for business.

we coded our programs on coding sheets and the data entry gals typed it to create punch cards. Then we'd put the stack of cards in a tray for the computer room guys to run it. When we got our compile listing back we were able to use keypunch machines to change the bad cards (changes to the program).

I had forgotten all about punch cards. I wasn't a programmer, but I wrote technical manuals on computer controlled systems. I had to be basically familiar with programming languages, so I took a few classes in DOS, Fortran, Pascal, and other programming languages and subjects. During labs, I had to create and then submit the punch cards for my programming assignment. Then I had to find evaluate the results to see why my program failed to run as expected.

I had forgotten all about punch cards. I wasn't a programmer, but I wrote technical manuals on computer controlled systems. I had to be basically familiar with programming languages, so I took a few classes in DOS, Fortran, Pascal, and other programming languages and subjects.

I came of age a little later. We'd (largely) abandoned the use of punch cards, but had only advanced to 'punch tape' (small rolls of tape which we used a keyboard which produced a series of binary dots other machines could read into the computer.

I first used that archaic device in high-school, and by the time I reached college they were moving to full magnetic tapes.

During that time, I was also involved with DOS, Fortran, BASIC and Cobol (learned Pascal and C years later).

Nobody used punch cards in the 80's except this class.

I job I was doing for an international communications group in 1979 was cutting the punch cards for the proposed routing changes, then taking them and the program cards over to be run by the computer company. Get the print outs, look for issues, cut new cards as needed, and do it all again. Those some cards and program were still being used in 1983, don't know if they were still in use after that, but I know they were still in use until then as a minimum. And that's a real world usage to work out where to lay cables on the sea floor and set up earth stations and cables for the sat comms.