[go: up one dir, main page]

This is a post from Robin Sloan’s lab blog & notebook. You can visit the blog’s homepage, or learn more about me.

Claude revision report, May 2025

May 26, 2025

I mostly burned out on writing with the machine circa 2016-2018, but I am still com­mitted to checking in, every so often, on the state of the art.

Pol­ishing up my recent microfiction for pub­li­ca­tion here, I noted the release of some new Claude models, so I thought, okay, let’s give it a go.

I wrote a quick script to loop a draft through two prompts: one requesting a plan for revision, the other exe­cuting those revisions. My idea was, I’d run the text through this edit/write loop a few times and see where it got me.

Now LISTEN: the fact that this strange, gloppy com­puter pro­gram can pro­pose rea­son­able changes at all — in the same moment that it is tran­scribing some other user’s tax forms, and explaining to yet another user, in Japanese, how they might treat their eczema — is wild. I mean the flu­ency and flex­i­bility here is just incredible. But, we are being asked to judge these sys­tems as poten­tial full-spectrum superintelligences, so: we will.

None of Claude’s edits met my standards, which are very high. The model’s sug­ges­tions made the story duller rather than sharper; they clob­bered pre­cisely the bits of lan­guage (often weird-angled) that were the most fun; they pushed the story’s shape and tone pow­er­fully towards conventionality.

Just a few exam­ples from var­ious runs of the script:

Opening line adjustment: Change “Another idiot with a tril­lion souls in his back pocket” to some­thing more spe­cific that estab­lishes the narrator’s world-weariness, like “Third hyper­hostage sit­u­a­tion this month, and this kid’s claiming a tril­lion souls.”

It’s dif­fi­cult for me to explain exactly why these edits are bad … 

Tighten the researcher description: “Some weirdo researcher develops an AI model with min­imal rea­soning but max­imal experiencing.” → “Some researcher fig­ured out how to strip rea­soning from consciousness — pure expe­ri­ence, no thought.” (More precise, more chilling)

 … which is interesting: artistic taste oper­ates right up at the limits of lan­guage and explainability. However, I have absolutely 100% con­fi­dence in my correctness. Claude’s edits … 

Strengthen the ship­wreck metaphor: After the quote, specify who said it (Vir­ilio) and tighten the parallel: “Paul Vir­ilio said that. Well, the inven­tion of the expe­ri­ence model was the inven­tion of the hyper­hostage.”

 … read like sug­ges­tions from someone who has read a hun­dred books about How To Write, rather than someone … 

Stronger ending: “Guess I’ll get another coffee.” → “I get another coffee and wonder if model #265,358,979,323 drinks coffee in there, or just dreams about it.” (Ties back to the cen­tral dilemma, main­tains narrator’s detached tone while adding depth)

 … who knows how to write.

Anyway! We’ll try again in another couple of model gen­er­a­tions and see where we stand.

To the blog home page