“Okay,” she said, “let’s start!” She clicked the link, the submission form opened. She examined it top to bottom, biting her lip.
He glanced at her. “You’ve got everything prepared in the document. You just have to copy and paste the bits to the right places.”
“I know, I know. Don’t rush me!” she said.
He fell silent, just waited. Finally, she started working with keyboard and mouse, selecting, copying, switching, pasting, clicking: title, description, progress...
“It’s ‘Story is Finished’, right?” she asked.
That should have been clear, and was also neatly explained on the screen, so... “Right,” he simply said.
She clicked. Sex contents, story genre ... categories.
“Oh, fuck!” she said, staring at a large grid of options. “Do I really have to find and click all the tags I want?”
“Yes,” he said, “of course. What else?”
“Why can’t I just paste in the comma-separated list I’ve prepared?”
“Because you might have misspelled an item, or included something that doesn’t exist in their system?” he said.
“Yeah, sure; trust you to come up with an excuse!”
“Come on,” he said, “relax! You have the items in the same order as the options on the screen, and you don’t have that many, so this shouldn’t be too hard.”
She didn’t answer, just went to work with her lips pursed. “I guess I’ve got them all,” she finally said and clicked once more. “What’s that?”
“That’s the upload,” he said. “Got your file ready?”
“Sure, but ... wait ... Do I have to go through a file dialog? Why? What happened to drag and drop?”
“Oh, you can’t...” He stopped himself. “Just do it, okay? It’s not much of a hassle.”
She did. “And now?”
He pointed at a button and she clicked it. Activity was indicated, seconds passed...
“There, it’s done,” he said.
“That’s it?” she asked. “It’s online now?”
“Not quite,” he said. “A moderator has to check it first.”
“To check whether things are okay, at least at a first glance?”
“They read it first?”
“No. At least I don’t think so: not enough time. They will just skim over it whether it looks okay, I guess. Or take a few samples: beginning, end, random position in the middle ... Or maybe not, I don’t know. I don’t work there, you remember?”
She stared at the screen. “Do I get a message when it’s online?”
“No,” he said. “Just check every now and then.”
“Oh, fuck!” she said. “Where do I check?”
“Preferably on the home page. It will appear on the list there.”
She jumped to the home page. The story wasn’t there yet. She clicked on the reload button. A few seconds later, she clicked again.
“That might take hours,” he said.
She didn’t reply, just clicked again. And again.
He grabbed his tablet and silently retreated.
“It’s there! It’s online!” she shouted.
“Mhm,” he said, not looking up. Then his self-preservation instincts kicked in and his head yanked up. “Oh, great!” he hastily added. “Let me have a look.”
“Here it is!”
He clicked, scrolled, scanned, came to the bottom. “Looks good,” he said. “Oh, comments are not on. Do you want comments?”
“Of course!” she said.
“Wait,” he said, “we just need to go...”
After two minutes of trial, error, and verbal abuse, comments were on.
“Shall I write a comment?” he asked.
“No, no. That would be cheating. I mean ... well ... No. No, I’ll wait for impartial readers to comment.”
“All right,” he said. “So, what do we do now?”
“What do you mean?” she asked.
“I mean: how do we spend our time now? Together, maybe?”
“I don’t know what your plans are but I’ll be right here, waiting for my first readers and their comments.”
“Ah,” he said. “Um,” he added. “That might take a while...”
“Can’t take that long,” she said. “It’s a short story. There are thousands of users. Some of them are bound to stumble over it and give it a try. It’s on this news feed or how they call it, isn’t it?”
“The stream, yes,” he said, “as are all new stories, updates, et cetera, et cetera...”
“Fine,” she said and clicked reload. “I’ll just wait a while.”
He shrugged and retreated again. It was good to have another device at hand.
“Hm, no votes yet,” she said, clicking for the zillionst time. “But what are these ‘downloads’? People are downloading the story?”
He glanced over her shoulder. “Technically, yes,” he said. “But usually only by viewing it. Think of it as page hits. Um, nah, forget that, think of it as ... reads. Yes, reads.”
“Okay — then quite a few people have already seen it! But no votes and no score yet!”
“That view is not entirely reliable in this respect,” he said. “That’s the public view, the author’s page. Better switch to the stats page. Click here ... and there ... yes, that’s the one.”
“Ah,” she said. “Aaah! There’s votes! There are some votes! Nine votes! Can I see more by clicking on the link ... no...”
“I think that’s just for sorting, doesn’t do much with only one item. Click there for more details.”
“Here? Oh. Oh! That diagram — that are the votes! I’ve got votes! I’ve got a ten!”
“One ten,” he said, “and a nine, sevens, sixes, and a four.”
“Ten is ten,” she said. “Someone really likes it! Can I see who?”
“No,” he said. “Voting is anonymous.”
“Well, yeah, of course. Doesn’t matter. I’ve got a ten! But where’s the score?”
“You don’t have an official score yet,” he said.
“Why not? I’ve got votes, and votes make a score, don’t they?”
“Yes, but there is some ... threshold. You need a minimum of votes for a score. Without, you would get completely distorted scores from the first few votes.”
“What is the threshold?”
“Sixteen votes,” he said.
“That’s ... stupid,” she said.
“I just explained it,” he said. “Without, you might have started with a score of four, if this was the first vote, then...”
“Yeah, I’ve got that, thank you. I mean sixteen is a stupid threshold.”
“Ah,” he said. He needed a few moments to switch gears. “Why? Too high?”
“Too low. It should be thirty.”
“Thirty? That’s ... Why thirty?”
“Thirty is more or less a magical number in statistics. A sample size of thirty will in most cases be big enough for the arithmetic mean of all samples to have a normal distribution, which means you can assess the chances the sample is representative for the whole population — and it’s pretty high.”
He looked at her with big eyes. “Since when did you become an expert in statistics?”
“I ain’t,” she said. “That’s just beginners’ stuff I need at my job to avoid making a complete mess of things. I’d have expected you to know them too.”
“Well,” he said, “basically — yes. It’s only ... You know, there are thousands of readers on the site, but not necessarily thousands of voters. Most readers don’t vote, or do so rarely. So you might be lucky and get sixteen votes, but thirty could be a stretch...”
“We’ll see,” she said and clicked reload.
He grabbed his tablet.
“Fuck!” she shouted.
“Yeah?” he asked, half hopefully, but he quickly realized it wasn’t meant as a request. Well, not as that kind of request, more likely as a demand to come over and give moral support. He approached carefully.
“There are no more votes coming in — and the story is not on the news feed anymore!” she said and waved at the screen.
He had a look. “Yeah, many other things have come in too, other new stories, updates, blog posts, and there’s only a limited number of items shown.”
“Fuck!” she repeated. “So, that’s it?”
“Hm — ‘blog posts’ might be the cue.”
“You could make a blog post too, a short text about your story, or about you as a first-time author, whatever. It will appear on the stream, and if people give it a try, they might do another two clicks and get to your story...”
“Why didn’t you tell me before?!”
“Sorry, just didn’t think of it. If you return to your stats page, yes, this one, then you can click there ... right, and now you could write a post ... Darn, that’s stupid; I’m really sorry. You will need some time to come up with a nice post, of course. But maybe by tomorrow evening—”
“Tomorrow evening? No way!”
“Come on, you can do it. I could—”
“—be a nice guy and leave me alone for a good quarter of an hour, will you? Thanks a ton!” She thought for a moment, then started banging the keys.
He retreated without a sound.
“I’ve got a score!” she shouted.
He woke up. So he had been sleeping? What...
“But it’s ... it’s 5.85?! There’s a bug! These votes never average to a score of 5.85!”
He dragged himself up, stumbled over to her, rubbed his eyes, and tried to focus on her screen. “Ah,” he said. “Yes. That’s the weighting.”
“Yeah.” He tried to concentrate. “The score is not just a raw arithmetic mean, it’s weighted.”
He massaged his temples. “It’s ... it’s the arithmetic mean in proportion to all arithmetic means of all stories. It’s rescaled.”
He scratched the back of his head. “To account for high-voting. You know, many people don’t vote, or they do it rarely, but when someone does, they more often do it because they like the story, at least to some degree, than because they don’t. So most scores end up high, and you can’t discern meaningful differences between all those high scores.”
“Hm,” she said. “And how is it done?”
He frowned. “Well ... I think there is an explanation somewhere; let’s see...” He started tapping on his tablet. “Ah,” he said only a few taps later. “Here it is. The median of all the raw arithmetic means is determined. This median value is equated to the center value of the new scale. The lowest and highest values of both scales are the same, and all values in between are translated in proportion to the difference between lowest and center value or center and highest value respectively.”
He tore his hair. “Okay, I’ll explain. The median is—”
“—of all the values lined up minimum to maximum the middle one, or the mean of the two in the middle. The 50th percentile,” she said.
“Er,” he said. “Yes.”
“So,” she said, grabbing a notepad and a pen. “Let’s have ... let’s have a scale of one to five stars; much more common.” She scribbled five starts in a row and the numbers 1 to 5 under them. “And let’s assume most items end up with a score of four stars,” she said and circled the four and its star. “Now we make a new scale of one to five stars,” she said, scribbling them in a row below the others. “We map one star on the old scale to one star on the new scale, five stars on the old scale to also five stars on the new scale, but four stars on the old scale to three stars on the new scale.” She drew the respective arrows. “Now we just have to map the rest in between; that’s two stars on the old scale to ... about one and a half, or rather one and two-thirds stars on the new scale, three on the old scale to two and one-third on the new scale. And four and a half stars on the old scale to four on the new scale. That maps pretty well.”
“Um ... yes,” he said. “So, you get it.”
“I get how it is done, but — why?”
“Because of what I said before: to rectify the lump of high votes, so—”
“I know what it is meant to do. But why this way?”
He hesitated for a while, then shrugged. “Don’t know. Maybe it’s a common statistical method?”
“That it’s common doesn’t mean it’s good — or the right one for the case at hand. Even calculating the arithmetic mean of votes is common but questionable.”
“Oh, for ... Come on! Why should that be ‘questionable’?!”
“Because the scale of the votes is most likely ordinal, while an arithmetic mean requires at least an interval scale to be valid.”
He just stared at her.
“An ordinal scale means that the only thing sure about the voting options is that some are better than others, but not by how much, or whether every single one is by the same degree better or worse than its neighbors,” she said.
He continued staring.
“With such a scale, you should use the mode, in this case the vote option chosen most, or the median instead,” she said.
“Well,” he said, “the median is used, to find the point on the old scale to map to the center of the new scale.”
“Big deal,” she said, “using the right method on the wrong values.”
“Oh, come on,” he said. “You know everybody uses arithmetic means on ordinal scales. And you could argue that many ordinal scales are close to an interval scale, so it’s not completely off.”
“Yeah, I know,” she said, “but I guess it’s only because everybody learns how to calculate an arithmetic mean at school, but they never teach when it is validly applied and when not.”
He shrugged. “Here, it has a practical reason: If you’d use mode or median, you can have only one of the few scale values. That doesn’t help to differentiate between thousands of stories.”
“Neither does comparing the arithmetic means, weighted or not.”
“But it does, weighted! As I said, the lump of high votes...”
“ ... does not necessarily go away with this weighting.”
“Of course it does! The scores get spread evenly!”
“Don’t be...” He stopped himself. “Pray tell, how should they not get evenly spread?”
“Easily,” she said. “Just assume the votes and thus the raw scores mostly end up on a tiny subrange of the whole scale.”
He hesitated, trying to get the mental picture. “You mean, given your five-star scale, nearly all of the raw scores are, let’s say three and a half to four and a half stars?”
“Or even closer together than that; how about three and a half to four stars?”
“Okay; and the median, that would be three and three-quarters then?”
“No,” she said, “let’s say there’s an overwhelming number of raw four scores, so the median is at three point nine stars.”
“Three point nine?! That would mean ... that would mean...”
“That would mean that a lot of weighted scores would end up as a lump at three and three-quarter stars,” she said.
“With large parts of the upper half of the scale mostly unused... , “ he added.
“Right,” she said.
He stared at her for some moments. “That’s quite unlikely,” he finally said.
“It’s an extreme example,” she said, “but the tendency for such a situation to happen is in line with the tendency for high votes you gave as the main reason for the whole weighting thing.”
He stared for a few moments more, then he shrugged. “So it’s not perfect, but you can’t do any better, can you?”
“Easily,” she said and grinned.
“Oh, come on!” He took a deep breath. “Okay, tell me, Miss Know-it-all!”
“The median is the key,” she said.
He waited, but she said no more. Okay, she obviously wanted to make the most of this, and he had to play along. “How so?” he asked.
“Rescaling by the median at least tells me that any rescaled score below the center score means the raw score was in the lower half of all scores, while any rescaled score above the center score means it was in the upper half.”
“Yes, obviously,” he said.
“With the weighting, this is, however, the only certainty about a weighted score you get from looking at it.”
He shrugged. “Yeah, I guess. So what else do you propose?”
“The median is the 50th percentile,” she said.
“Yeah,” he said, “as you said before.”
“So — why not use percentiles throughout?”
“Percentiles throughout?” he said. “You mean, not only the median but also ... other percentiles mapped to other scale values? Then there would not only be two but ... nine ranges to calculate the exact position in...”
“What?” she asked. “No! Why so complicated? Just use the percentiles!”
“Yes, that’s what I’m talking about! Mapping more percen—”
“Not mapping — just use the percentiles!”
“Use the percentiles?” He gave her a confused look. “Use them for what?
“The percentiles?” he asked. “As scores?”
“But ... but percentiles are 0th to 100th!”
“Yeah, what’s the problem?” she asked.
“The scale is 1 to 10!”
“The scale for votes is 1 to 10,” she said, “or more exactly, is ten options mapped to 1 to 10 for aggregation. Who said the scores have to be on the same scale?”
“Um,” he said, “no one, but ... if the same scale is used, you can also translate it back to the same options...”
“Nonsense,” she said. “The weighting makes sure you can’t. If you do, you should be very upset, because your ‘Very Good’ gets downrated to ‘Not Bad’ — that doesn’t make sense, does it?”
“The way you say it, it doesn’t, no ... But ... but switching from a scale from 1 to 10 to one from 0 to 100, isn’t that just score inflation?”
“Give me a break!” she said. “The current score is given with two decimal places, so it’s actually a scale from 100 to 1000!”
“Okay, okay, you’ve got a point there.”
“Maybe this is part of the problem of the current system: It feels like depreciation, for both the readers and the authors. To avoid it, use a wider scale, so the votes and their assigned values get mostly aggregated to higher score values.”
“Okaaay,” he said, trying to wrap his head around this new concept.
“And percentiles as scores have the added bonus that they have an inherent meaning, too: They tell you which rank the score has among the scores of all stories.”
“Um,” he said, “but rank would be first for the best score down to eksty-eks thousand eks hundred eksty-eksth for the lowest score.”