RE: https://tldr.nettime.org/@tante/116102639569199871
I think this is worth talking about, but I do think both miss the point a bit...
Discussion
Loading...
Cory Doctorow, in the quoted parts in this article you posted, makes this argument:
"When I write stuff, I take endangered birds and I shove them up my ass, and their twitching while they suffocate inspires me. Now, you may ask: do I *have* to use *endangered* birds?"
"And to that, I say 'fuck you purity police'. So what if I fuck myself with almost-extinct little fellas while I write? Do you know how many *other people* have fucked themselves with birds to achieve greatness?"
Wait, someone thinks he's not making that argument?
Then maybe someone doesn't actually understand LLMs and their impact on the planet very well.
Maybe someone doesn't understand how arbitrary it is to use LLMs for *spellchecking*, something MS Word has been doing since 1994.
Maybe someone fails to realize that using LLMs is a literal absurdity, even more absurd in this context, and that's why I acted like he was putting little birdies in the butthole.
Absurdity deserves absurdity.
@johnzajac Except that "spell checking" and "finding typos" are radically different tasks. Also, LLM and SLM are pretty radically different.
There are orders of magnitude difference here in both directions.
@johnzajac Like, is Cory Doctorow making a bad argument? Yeah. Is using a small language model, run locally, for language manipulation technologically appropriate... yeah, actually it is. Finding proper typos is hard. It's not just comparing to a word list because you can have a properly spelled word in the wrong place. Statistical models are actually *extremely* useful for this task.
Do you need a huge LLM that requires gigs of ram to query? No. Do you need to burn the planet to train a spell checker? No, absolutely not.
They're burning the planet to "get to AGI," which is impossible. It's a different class of problem.
Should you use a spell checker to make important financial decisions or learn things? No. That's absurd. That use of an LLM is a really bad one.
LLMs probably don't need to be more complex than they currently are to achieve... like... anything they're actually useful for. The extra complexity is just cultists.
Cory writes he's using an LLM. Is this the "celebrated technology and culture writer" (who, btw, I normally enjoy reading and deserves a place right next to David Sedaris' "santorum" for his creation of the word "enshittification") completely misunderstanding technology, then writing something that completely misunderstands culture?
Could be. I based my post on what you had linked to, in which CD says he uses an "LLM".
Also, I'm in a bad mood, so
sorry if my tone wasn't great
🤷♂️
@johnzajac I think his arguments sucked and were probably destructive.
I also think that's separate from "should you use a SLM for spellchecking." That's my point.
1 more replies (not shown)
I was being sarcastic because Cory Doctorow is making the argument that publishing something with a typo is worse than depriving millions of clean water.
Perhaps Cory could publish, I don't know, 150 10,000-word essays a week instead of 200? Use that extra time to read them before he hits "post".
None of this makes any sense, that's why none of the arguments are sensical or effective.
"Only one Palestinian child has to die each week for me to have flawless grammar in my copious output" is not a sensical argument because it has an invalid premise.
"Only one town has to go without water because I'm writing 10 books this week and just don't have time to proofread them" is equally nonsensical.
"I put birds up my butt bc it helps me think" is nonsense.
It's all nonsense. There's no sensical argument for using LLMs.
@johnzajac The problem here is that we have two technologies. Like... you don't need a Ford F250 lightning or whatever to get eggs. But batteries are also in e-bikes. One is extremely wasteful, one is actually appropriate. We're talking about both as though they were the exact same thing and they aren't.
1 more replies (not shown)
@Hex
where do you see the article missing the point?
@johnbrowntypeface I added a bit more in comments.
It seems as though Doctorow is trying to preemptively deflect criticism. That's understandable, but the arguments are not good...
Meanwhile, tante points out these flaws but raises some questionable arguments of their own.
"The second aspect is often illustrated by how ships are organized: Because ships are sometimes in dangerous situations and sometimes critical decisions need to be made, the existence of ships implies the existence of a hierarchy of power relationships with a captain having the final say. Because democracy would be too slow at times."
OK, so are ships inherently authoritarian? There's a whole history of pirates who would argue otherwise...
Is building an LLM inherently problematic? Not necessarily, but there's no good way to do it under capitalism. Is using an local LLM funding these evil companies? No. It's not.
Spelling and grammar checking is one of the few uses of LLMs that is not based on fundamentally failing to understand what an LLM actually is. A statistical model is gonna be *really good* at flagging things that are probably typos (low probability areas). There will be false positives, which is fine if you're actually paying attention...
I agree with the initial premise: if you're going to do something that isn't popular, that may be unethical, explain yourself. Don't try to deflect.
@Hex hmmmmm....
You're right that using an LLM for spelling/grammar does make some kind of sense, but I'd argue it's still bad. By doing so, you normalize others using them ("Well if Doctorow thinks it's okay to use it must be fine" is a thought I'm sure many are having right now), and that normalization extends beyond your narrow use case even if you're explicit about the circumstances under which you think it's acceptable, which Doctorow wasn't. Spelling & grammar checking is also something that can be done quite well by non-LLM technology, including well-established and widely-available stuff, so the marginal gains of doing it with an LLM are small, if any.
I think the most pernicious aspect (among many) of LLM use is the need to plunder & then saturate the digital commons. Inherent in their production (and thus relevant to their use) is the non-consensual harvesting of digital commons knowledge, and similarly to the extent that they're available, free, and seen as acceptable, they promise a churn of regurgitated inaccurate spam content via all available methods of publication that buries legitimate sources of knowledge on the internet under piles of lies. The ability to find useful knowledge on the internet is literally already greatly eroded thanks to SEO-seeking LLM spam, which has also destroyed the main business and social-reward models of everyone producing and posting such useful knowledge. Doctorow is essentially saying that using an LLM for spell checking is worth that systemic cost, and I think Tante is exactly correct to say it's not actually.
I also disagree with the point about ships in specific, but I don't think pirates are a great counterexample, and I do think that the point about technologies having politics is a valid one, with the nuclear reactors example being the better demonstration of that.
To me, an even more insidious point about using LLMs to do grammar checking is that you're going to run into cases where what you've written is grammatically dubious but not clearly wrong, and the LLM is going to advise you to hammer down that flair into conformity. You'll have to ignore a steady stream of those suggestions to retain a certain dimension of your own creative spark, and a simple rules-based grammar checker would probably catch 99+% of the actual unintentional errors without ever making these kinds of conformity suggestions (and a dictionary-based checker is completely fine for spelling issues). In fact, I wonder what the workflow is for Doctorow to add spelling exceptions to the LLM checker...
Used by many people over time, LLM-based grammar checking would tend to homogenize language and stifle natural linguistic innovation in a way that a rules-based checker would not. It would also tend towards reinforcing linguistic hegemony and squeezing dialects even further outside of a more-tightly-constrained "mainstream." For that reason I think I'm opposed to using LLMs for grammar checking despite the fact that they're actually somewhat suited to the task.
@tiotasram I really just want something that gives me a statistical heat map without giving me any specific guidance. *That* would be useful.
The people building LLMs are trying to kill all humans. Literally. They are exterminationists who want to replace human life with AI and upload their brains to the cloud. They're AI death cultists.
They are the enemy. Anything that empowers them, at all, in any way, is unethical and suicidal.
I don't think admitting that there can be good use cases for LLMs does that, because the use cases are not being served by all this training. The *reason* they keep training is that they're trying to make LLMs do something that isn't possible.... because they're cultists.
That's the point, IMHO.
@Hex
People who refuse to consider whether there may be value in the LLM, at least would want to know what an LLM is, I would think. But, am I wrong?
@Hex
i can't really disagree with that point, although i feel compelled not to boost it
cause historically even small apologetics for new technology with great potential harms ends up helping to make it ascendant, then it becomes unimpeachable and nearly impossible to reign in
so i refuse to admit that point. it's not incorrect, i just don't think we should be talking about any use of LLMs in our current situation
sorta a reverse version of why i don't support banning guns in our current era
1 more replies (not shown)