Forum: Story Discussion and Feedback

This got me interested. There are 141 stories on here with a 3.5 or lower. 20 are longer than 15,000 words. Most of those predate any possibility of being AI.

The lowest-rated story tagged 'AI generated' is a 3.86. Three are over a 9.

The oldest AI Generated story is from 2020. That seems early.

Hard to tell who/what generates the worst slop..LLMs or humans. We may have set a record during the weekend with a 3.6 story.

I think the best we can hope for is that LLM's evolve to become what ITers call a 60% solution.

If human output is represented as a bell curve with a mean of 50%, if LLMs become able to produce stories consistently around the 60% level, they become worth having. But for good human authors, producing work above the 60% level, that is a step down. That's why it's really depressing when good, award-winning SOL authors embrace LLMs and start producing lower quality work, albeit in greater quantities because it involves less effort.

AJ

On both, I'm generally in agreement (potentially surprisingly? or not).

A 60% solution may be better than a 70% or higher solution. At some point, if LLMs become truly 'good enough,' they really become a threat to human creations. Ditto video models, image models, and so forth. For advertising, they're already enough 'good enough' to be imperiling the jobs of many graphic designers. Ditto for 'grunt coders,' as they used to be called.

And I agree: if a person is a better author than an LLM, it makes little sense to embrace them. I do understand the people who use them for 'background' or 'atmosphere,' but I also don't, in that I'm not sure how much time it really would save. Not much, the way I write, but perhaps others find those things taking up a lot of time.

AI is currently at the stage where if it's used well then we're not going to be able to tell.

In art and music possibly. But when it comes to writing, there are still plenty of tells and the finished product lacks depth.

AJ

I'll note here that you've mentioned previously you haven't read output from current-generation LLMs. That might significantly alter whether there are 'tells' and what those tells might be.

For instance, current-generation LLMs objectively produce fewer em-dashes than human writers do on average, so lack of em-dashes is now a 'tell.' But older LLMs produced more than some (but not all) human authors, so that was a 'tell'. Meanwhile, both J. K. Rowling and a number of SoL human authors use more em-dashes per unit of text than any LLM has ever used, so it was never a very definitive tell.

'Depth' is contingent on how the LLM is used. If it's used well', there should be 'depth' - that's inherent in using it well. Blaming the tool is like blaming Photoshop when a blah, uninspired visual work is produced by someone with no depth in the visual arts. Me, for instance - I'm generally hopeless at producing something creative and original in Photoshop, but reasonably decent at using it for more simple editing tasks. On the other hand, both my graphic designer friend and my artist friend can and do produce works of depth and meaning using exactly the same tool.

LLM output from poor prompting and lazy use will almost certainly have tells, and that's most of what we see. But that just says that, when not 'used well,' they will produce bad writing.

Bad writing is bad writing. It doesn't matter what produced it. It's still bad.

I'll note here that you've mentioned previously you haven't read output from current-generation LLMs.

I don't know where you got that from.

I haven't used the latest LLMs but I wouldn't be surprised if some SOL authors had. Unfortunately, many don't even acknowledge their use of AI, let alone what brand and version.

Meanwhile, both J. K. Rowling and a number of SoL human authors use more em-dashes per unit of text than any LLM has ever used

That seems like a claim plucked out of thin air, unless you've checked all the outputs from all the LLMs. And given the recent tendency for LLM nationalism, I'm sure you haven't.

JKR's published stories use lots of em-dashes in dialogue (being a Brit, they may actually be converted dashes), and her characters have lots and lots of dialogue. Her usage is specific and correct, unlike the scattergun usage by many LLMs. That's another reason to treat your claim with scepticism.

'Depth' is contingent on how the LLM is used. If it's used well', there should be 'depth' - that's inherent in using it well.

LLMs simply haven't the capability for depth. That probably requires sentience. No matter how well anyone uses a tool, it can't produce something it hasn't been built/trained to do.

LLM output from poor prompting and lazy use will almost certainly have tells, and that's most of what we see. But that just says that, when not 'used well,' they will produce bad writing.

According to that, every usage of an LLM involved poor prompting and lazy use. Although I don't necessarily accept that an AI tell is a sign of bad writing. After all, many of the authors whose works were stolen to train the AIs were pretty decent, so the original use can hardly be described as bad writing.

AJ

I don't know where you got that from.

I haven't used the latest LLMs

Could have sworn I read it, but I'm not sure where. But I was right, at least, so I'm glad about that.

That seems like a claim plucked out of thin air, unless you've checked all the outputs from all the LLMs. And given the recent tendency for LLM nationalism, I'm sure you haven't.

There were recently studies posted showing the sharp decline in em-dash usage amongst most models. Google ngram also shows a higher rate among humans in the pre-AI era than most models ever generated once there were models.

LLMs simply haven't the capability for depth. That probably requires sentience. No matter how well anyone uses a tool, it can't produce something it hasn't been built/trained to do.

Strong disagree here. Yes, the LLM by itself can't have depth, but I don't know of any self-prompting LLMs. Photoshop, not being sentient, also has no depth, but a human can produce work of depth by using Photoshop.

The question isn't whether the LLM adds depth. The question is to whether a motivated user can use an LLM to produce a work with depth. I don't see any reasonable case why that wouldn't be so. At the simplest level, compare it to sculpting. Take the 'block of granite' the LLM gives you and remove / regenerate anything lacking depth.

According to that, every usage of an LLM involved poor prompting and lazy use.

No, every usage of an LLM that was easily detected involved poor prompting and lazy use. That's a wildly different statement. There are no perfect AI detectors (either at avoiding false positives or false negatives, barring reductio ad absurdum cases such as flagging everything as AI - no false negatives! - or flagging everything as non-AI - no false positives!). Given that, there are no perfect AI tells. And that's even amongst users who aren't overly motivated to evade detection or produce works with depth.

But I was right, at least, so I'm glad about that.

You made up a lie, and now you claim you were right.
No comment.

Take the 'block of granite' the LLM gives you and remove / regenerate anything lacking depth.

That would require the LLM to understand what depth means in writing. It doesn't.

No, every usage of an LLM that was easily detected involved poor prompting and lazy use.

When it comes to fiction, that would be a huge majority. When it comes to wallpaper, which is similar to non-fiction, AI is harder to detect. But AI isn't detected by a single tell in a single sentence but multiple tells across the whole work. I agree AI detection by robot is currently dodgy, because it can't adopt a holistic approach - it doesn't understand the feel of a story, including the voice, colour and depth (that's a triple - I must be an LLM!), any more than the LLM that didn't know it's omitting them.

AJ

You made up a lie

No, I either misremembered (quite possible, and I said that) or you're not remembering having said that (also quite possible, since you didn't say you hadn't said it, just questioned where I got it from).

You're the one who said I was right, so - unless you were wrong about my being right ('I haven't used the latest LLMs') - I'll stand by that.

Is there really a reason to argue this? You agree with the meaning of what I said, and I previously said I thought I had read it but wasn't sure.

That would require the LLM to understand what depth means in writing. It doesn't.

No, it requires the human prompting the LLM and editing its output to understand what depth means in writing. You seem to keep forgetting that LLMs don't do things by themselves and that there are humans involved in the entire process of using one.

When it comes to fiction, that would be a huge majority.

Agreed, no hesitation or quibbling. In my experience, that's completely correct.

But AI isn't detected by a single tell in a single sentence but multiple tells across the whole work.

Eh. Sometimes it really is a single tell in a single sentence. Judging the whole work can be just as problematic. Maybe the author fixed all of the 'slop' except in one point. There has been at least one work actually published (and I do not have the reference) where people generally thought it wasn't LLM-written until they found an actual LLM prompt buried in the text.

I'm not sure 'understanding the feel of a story' will ever be a basis for detecting LLMs. The really-low-rated works on SoL (most written before LLMs happened) probably don't feel much like a story. I suspect it'll be looking at the probabilities in the word distribution and noting that they match those of LLMs.

Of course, there are a lot of ways to affect the probability mix of LLM output, and those will confuse detectors, in the absence of more glaring tells.

I'll note here that you've mentioned previously you haven't read output from current-generation LLMs.

I haven't generated any output from latest generation LLMs, which is what Marc Nobbs was challenging, but that doesn't mean I haven't read any. Since I became interested in AI detection articles, it's almost certain I have read some.

You were wrong.

No, it requires the human prompting the LLM and editing its output to understand what depth means in writing.

Users have minimal control over word choice via LLM prompts so it's vanishingly unlikely that an AI would be able to instantiate a prompt like 'AI, write this scene with depth and colour'.

And if a human understands depth and colour, it makes no sense for them to use a tool that doesn't. For an analogy, it's like trying to paint a scene in colour when your tool only has black and white - it's simply not capable.

Sometimes it really is a single tell in a single sentence.

By analogy, AI tells are like cliches. The training material included lots of them because they were popular for a reason. In modern writing they've become somewhat deprecated, although there are still circumstances where they might be the best option. If you eradicate triples, for example, AI wouldn't be able to generate 'red, white and blue'. If you eradicate not X but Y, AI wouldn't be able to generate 'No, it's not a Little Auk, it's a Great Auk'.

(I chose those two tells because an advert for an AI detection company listed them as examples of tells that few people know about. I wouldn't buy their product if they paid me.)

So I think it's vanishingly unlikely you could embed a single AI tell in a single sentence that would taint a whole work.

AJ

You were wrong.

Then I apologize.

Users have minimal control over word choice via LLM prompts

Um. No. That's highly incorrect. I'm not sure where in the world you got that idea.

it's vanishingly unlikely that an AI would be able to instantiate a prompt like 'AI, write this scene with depth and colour'

On the other hand, that's right. An LLM is not likely to really understand 'depth' and 'color.' Maybe. It could, if the training data happened to flag 'depth' and 'color' as tags on the data. But that has little to nothing to do with controlling word choice.

If you eradicate triples, for example, AI wouldn't be able to generate 'red, white and blue'. If you eradicate not X but Y, AI wouldn't be able to generate 'No, it's not a Little Auk, it's a Great Auk'.

You don't eradicate them, you lower the probability. If 'red, white, and blue' is right, the model will still generate that, but it'll avoid triples that aren't necessarily triples. Ditto 'not X but Y'. None of this needs to be binary choices.

Of course, by saying you can force binary choices, you're saying the user has plenty of control over word choice.

So I think it's vanishingly unlikely you could embed a single AI tell in a single sentence that would taint a whole work.

'Grok, that's too strong. Reword to make it weaker.'

Things like that appear in published (even traditionally published works. I would consider that tainting the entire work.

Um. No. That's highly incorrect. I'm not sure where in the world you got that idea.

Because, from descriptions on the forum, you can boost or decrease the likelihood of a particular word being generated with a good degree of success, but you it would be impractical to do that for a whole dictionary's worth of words.

An LLM is not likely to really understand 'depth' and 'color.' Maybe. It could, if the training data happened to flag 'depth' and 'color' as tags on the data. But that has little to nothing to do with controlling word choice.

That means the 'author' having to write the words themself. In which case they're better off using a word processor.

Sometimes it really is a single tell in a single sentence.

Do you have an example of a single tell in a single sentence that would mark a whole work as AI-generated, or is that a hypothetical situation?

AJ

Because, from descriptions on the forum, you can boost or decrease the likelihood of a particular word being generated with a good degree of success, but you it would be impractical to do that for a whole dictionary's worth of words.

There is an ocean of difference between 'only minimal control' and 'total control is difficult.' Very few people are trying to control the entire dictionary. If the LLM often emits a 'bad' word, ban it or bias against it. If it emits a 'bad' word once in a while, fix it in editing.

That means the 'author' having to write the words themself. In which case they're better off using a word processor.

No, it means the 'author' needs to edit some subset of the words themselves. Quite a few people who aren't good at writing prose are nonetheless good at recognizing lousy phrasing, poor word choice, and so forth. I get messages quite often saying 'I'm not a good author, but I think ...' And, usually, they're right.

Do you have an example of a single tell in a single sentence that would mark a whole work as AI-generated, or is that a hypothetical situation?

Repeating myself: 'Grok, that's too strong. Reword to make it weaker.'

Similar phrases (literal LLM prompts in the work) have appeared in published works. That is a single tell in a single sentence.

You're right: it doesn't mark the whole work as AI-generated. I agree, that's impossible. But that wasn't the point. The point was that there are single tells that make it nearly certain a portion of the work was AI-generated.

No, it means the 'author' needs to edit some subset of the words themselves. Quite a few people who aren't good at writing prose are nonetheless good at recognizing lousy phrasing, poor word choice, and so forth.

That's quite bizarre, telling people who are poor at writing prose to correct the AI's attempt at prose.

AJ

I know people who are lousy writers but good editors. A fair number of them, too.

I know people who are lousy writers but good editors. A fair number of them, too.

Okay, I'm being hypocritical here because I tidied up some of the prose by authors whose works I was editing, but when it comes to changing words and tidying up sentence structures, they're actually contributing to the writing. If they're doing writing, they can write.

AJ

No, it means the 'author' needs to edit some subset of the words themselves. Quite a few people who aren't good at writing prose are nonetheless good at recognizing lousy phrasing, poor word choice, and so forth. I get messages quite often saying 'I'm not a good author, but I think ...' And, usually, they're right.

As a complete divergence from the discussion, someone's Mary Sue character said

[Public] Speaking is honestly less scary than most people think, and it's a really useful skill to have

I think that about writing, and if someone lets an LLM generate their prose, their writing will never improve. But the UK's Education Minister said about her proposed curriculum change

'Speaking skills will become just as important as reading and writing in GCSE assessments'

So, like Grey Wolf, she's a fan of speaking skills. However I don't think she's realised just how many more oral examinations that's going to require.

AJ

I think that about writing, and if someone lets an LLM generate their prose, their writing will never improve.

Depends on what you mean by 'generate.' If they prompt and accept the output blindly, I agree. If they pick over the output, reprompt, and edit, I disagree.

By analogy to graphic design, if you pull out Photoshop and slop together a quick edit on things, you'll never become a better graphic designer. If you use the tools carefully and learn how they work, you will. The tool doesn't dictate your ability at creating the things the tool is designed to help you create, your ability to use the tool does.

And speaking is an interesting analogy. One can become a better speaker by presenting their own words. But one can also become a better speaker by only repeating others' words.

Yes, I agree - you'll need a lot more oral examinations. There's no substitute for that with speaking. Unless, of course, we train an AI model to evaluate oral presentations :)

And yes, that was also a joke - except that it's not entirely a joke. There are good AI models for listening to and correcting speech while learning a foreign language, and those have greatly improved people's ability to practice speaking without burdening others. Could we do that with real public speaking? Definitely not today.

You're right, and this is one of our disconnects.

You have to 'write' (in the bang the keys) sense to become better at producing a story that way.

You have to 'paint' (in the sense of pushing around a paintbrush to make a picture that way.

You don't have to 'paint' (in that sense) to create visual artworks of depth and meaning; you can use computer-based tools and work with the output.

You don't have to 'write' (in that sense) to create written works of depth and meaning; you can use computer-based tools and work with the output.

Your focus is on the mechanics of writing. You see 'an author' as a person who follows a certain path to create written works, hopefully of quality.

My focus is on the quality of the written works. I see 'an author' as a person who produces written works, hopefully of quality.

It's probably insurmountable, but that doesn't mean there haven't been a lot of interesting things found along the way.

A person prompts an AI to write a speech. That makes them a speechwriter.
That person then prompts an AI to generate a video of a human-like figure presenting that speech. They post the video on eg social media. That makes them a public speaker.

AJ

If they're doing writing, they can write.

I disagree. The ability to detect bad writing, flag it, and make small corrections is very different from the ability to write in the first place. Again, I know too many examples of that personally for there to be any doubt in my mind about it.

I do know people 'in the middle,' who write an 'absolutely wretched' first draft (their words, not mine), then painstakingly clean it up. While writing, they have to turn off the part of their brain that sees the problems, or they'll bog down rewriting the first sentence.

Given that their first draft is much worse than what an LLM would output (seriously!) and is not at all 'in their voice' (again, seriously), it's easy to see them writing with the help of an LLM, since both their voice and their prose is basically all added in editing, with only the outline and broad structure created in drafting.

Which may be why I see editing as part of writing? The real meat of the writing process is during editing, for them.

but it's not hard to knock together a simple cover.

Ever tried it? I mean, not just a title and author name in yellow before a dark red background? Considering copyrights of imagery used? It's easy only if you are willing to risk law suits, but to find non-copyrighted imagery is not as easy as you may think. I'll readily admit that my mainstay, historic adventure, is easy, because paintings, water colours and etchings of ships are a dime a dozen on the net with the copyright expired a century ago. Yet, contemporary stories require contemporary imagery, and most of those carry that pesky "alamo" watermark.

And if i suspect your artwork is genllm/genai why would i think your writing is not?

Just because someone can write doesn't mean they can draw.

Do you think James Patersen or Stephen King or JK Rowling draw their own covers? And who knows, maybe their publishers' art teams use AI to assist them.

And who knows, maybe their publishers' art teams use AI to assist them.

In these days of economising and cost-savings, you can bet your dearest body part that covers are generated with AI in the workflow. Even before AI, covers were collages cobbled together in Photoshop, with all the toolboxes involved. Hint: Fabio did not really meet all the women he supposedly held in his arms 😉

Photoshop is at least someone that's real doing the work, figuring out how to compose how to place had to edit what needs to be in there. Photoshop is fine.

Whoa. I use GIMP which is like Photoshop. Unless your cover is simply words, anyone using Photoshop or GIMP still need images to compose. I don't use AI to compose/generate my covers. I use it to create images to use on the cover.

Wasn't there a recent incident where a (former) Adobe user was so incensed at his Adobe products being moved to a subscription-only model that he had AI jail break Adobe's programming and have it create a free version of every Adobe product on the market?

Or did I dream that?

I suspect you're thinking of ArtCraft. The developer Vibe coded working replicas of the full Adobe Suite and the MS Office Suite.

BTW, given this thread, Vibe Coded = Done with AI.

Sounds about right. Though I had it wrong about it moving to a subscription, it was being charged a 'cancellation' fee that drew his ire.

It's an interesting topic, as AI is being used more and more in ways big corporations never anticipated, and I find that a strangely beautiful state of affairs.

Case in point, is the way people are using AI to recreate games in other games (Tony Hawk skater games inside the Battlefield series/ Mine-craft inside Elden Ring etc etc).

Big corporations are slow, ponderous, and I don't think they truly recognise the danger they are in, and by the time they do, it's going to be far too late for them.

And it's not just the abilities; it's the sheer speed these things are being accomplished at.

Certain companies have grown lax in their monopolies. I am looking at software companies especially, and these companies probably have no contingency plan in place for being potentially obsolete overnight. They may well survive, but only if they pivot hard and accept potentially brutal financial haircuts. If I was an investor, I would be removing myself from any financial ties to software firms. A crash is on the horizon, and that horizon is very, very close.

Not everyone who can afford the time to produce stories for little cost (or for free) can afford to pay artists. And, if you go looking for inexpensive artists, recent history shows you'll probably find someone who's using AI-generated art.

The question is whether an AI-generated cover is better than no cover, in that case. I think the jury is still out on that point, and likely always will be.

I completely understand that. But, for the author, it's what the majority of potential readers would appreciate. If 'some cover' is better than 'no cover', and the budget doesn't allow 'artist-generated cover that's really an artist and not a person using AI', then 'AI cover' has to win.

Note: I still don't have covers. But I'm also not on ZBookStore yet. When I finally get going there (maybe, eventually, I'll get around to it soon :) ), then some sort of cover needs to happen. But it might be very generic.

then some sort of cover needs to happen. But it might be very generic.

You know you made it when the majority of the cover is the author name — like STEPHEN KING in large font taking up a good portion of the cover. You don't even need an image. It's the author name attracting the reader.

Don't judge a book by its cover.

Sometimes the moderators add AI-generated cover art to stories. The authors are not always involved in that.

Don't judge a book by its cover.

A perfect example of just how bad it can be, from LONG before AI/LLMs were in use is the cover for one paperback edition of Agent of Byzantium by Harry Turtledove.

The artist clearly read a blurb about the story and drew the cover. It had images which literally contradicted the story.

See the center top image here: Agent of Byzantium

The blurb clearly talked about 'technology', but it was referring to gunpowder, vaccinations, the telescope, and printing, not the satellite phone and rifle depicted.

What's worse, a minaret is depicted, and in this alternate history, Islam never developed. Muhammad is known as the second greatest Christian preacher after Paul, and his refrain was 'There is no god but God and Jesus is His Son'.

In other words, you don't need AI to make bad covers. And if I'd not known what this was about from a friend, I've passed on it seeing that cover.