Genre Grapevine: Book Club Scams Are a Warning of Emerging AI Super-Scams [Jason Sanford - nota bene, I've been the target of such scams but have not fact-checked Sanford's specific details]
I'm sad that people are stuck in positions so desperate that they fall for this. I hope people get warned about this. I've gotten a couple of these and gotten asked about one that involved a scammer that cited that I was working with them (I was not, lol).
That said, I'm almost positive I've seen accounts of similarly structured scams from a time before modern mass telecommunications, when now you can fake up a bunch of "people" to convince greedy/hopeful/desperate marks that they've stumbled on some Good Thing and the marks can't (easily) verify those "people." You can do this in print with ~testimonials, but not at scale and not in realtime in this manner.
I'm not saying AI isn't a problem; I'm saying that if people weren't forced to desperation (or straight-up greedy), the incentive structure that enables the AI deployment to be profitable (so to speak) with this target ~audience would not be as successful. Which is perhaps splitting hairs and is the point at which I expect to be flamed off my own DW.
Very simplified but: Anytime you create an incentive A, you create a secondary incentive A' for bad actors to exploit the system to access A.
Hilarious terribad example of this: I was contacted for a blurb/etc for what sounded like an extremely unoriginal sexploitation "trans woman" sci-fi book (you know, sexbot cyberpunk sleazy noir but with a trans angle). That's not all that surprising and it's theoretically possible the book exists and was written by some human, or it exists but was written by some LLM, whatever. That's not the incentive. (For that matter, I'm not in a position to criticize a sci-fi book artistically on sleaziness grounds, please! I have published books full of genocide, rape, incest and other objectionable material. I'm a trash panda aesthetically.)
No: what was interesting from a scammer vs. mark arms race evolution perspective was that this author claimed to be (approximately, I'm writing this from memory) a trans woman in ~South Asia who was inspired by having done ~sex work. This is a clever way to appeal both to "woke" crowds and A Certain Sleazy Crowd! For ~privacy/safety reasons she could not accept interview/live call requests. This was accompanied by a SUPER fake-looking (likely AI-generated or badly Photoshopped, take your pick) Hot Asian Chick headshot.
So yes, absolutely as a trans person I know that safety/privacy are hideously important. But once incentive A exists, someone has incentive A' to piggyback on A, which is what looked like was happening here. I just blocked the email address and moved on. At this point, I've set up my email to auto-delete any email that mentions "Goodreads" or "Amazon", unless they're on a SMALL whitelist, among other countermeasures. Life is too short and I have ramie to spin!
I said cynically to![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
telophase that I suspected that the "actual" "author" was some middle-aged white dude scammer sitting in North Dakota or, more tragically and pessimistically, some human trafficking scam farm outside the US.
I assume this is also where the fake-looking-ness is partly to screen out people who are moderately suspicious/vigilant/smart enough to avoid weird, scammy emails and/or ask around for more information, and to screen for people who are sufficiently desperate, greedy, or naive (cf. shitty obvious "tells" in phishing scams). But I'm out of field so I could be wrong.
Regardless: it's not that legislative or technological protections aren't important or necessary or desirable, it's that the underlying human problem of the incentives vs. secondary incentives is inherently intractable. :(
NOTE: I'm screening comments from non-[access] and may be scarce/slow because I'm recovering from a health thing. Thanks.
I'm sad that people are stuck in positions so desperate that they fall for this. I hope people get warned about this. I've gotten a couple of these and gotten asked about one that involved a scammer that cited that I was working with them (I was not, lol).
That said, I'm almost positive I've seen accounts of similarly structured scams from a time before modern mass telecommunications, when now you can fake up a bunch of "people" to convince greedy/hopeful/desperate marks that they've stumbled on some Good Thing and the marks can't (easily) verify those "people." You can do this in print with ~testimonials, but not at scale and not in realtime in this manner.
I'm not saying AI isn't a problem; I'm saying that if people weren't forced to desperation (or straight-up greedy), the incentive structure that enables the AI deployment to be profitable (so to speak) with this target ~audience would not be as successful. Which is perhaps splitting hairs and is the point at which I expect to be flamed off my own DW.
Very simplified but: Anytime you create an incentive A, you create a secondary incentive A' for bad actors to exploit the system to access A.
Hilarious terribad example of this: I was contacted for a blurb/etc for what sounded like an extremely unoriginal sexploitation "trans woman" sci-fi book (you know, sexbot cyberpunk sleazy noir but with a trans angle). That's not all that surprising and it's theoretically possible the book exists and was written by some human, or it exists but was written by some LLM, whatever. That's not the incentive. (For that matter, I'm not in a position to criticize a sci-fi book artistically on sleaziness grounds, please! I have published books full of genocide, rape, incest and other objectionable material. I'm a trash panda aesthetically.)
No: what was interesting from a scammer vs. mark arms race evolution perspective was that this author claimed to be (approximately, I'm writing this from memory) a trans woman in ~South Asia who was inspired by having done ~sex work. This is a clever way to appeal both to "woke" crowds and A Certain Sleazy Crowd! For ~privacy/safety reasons she could not accept interview/live call requests. This was accompanied by a SUPER fake-looking (likely AI-generated or badly Photoshopped, take your pick) Hot Asian Chick headshot.
So yes, absolutely as a trans person I know that safety/privacy are hideously important. But once incentive A exists, someone has incentive A' to piggyback on A, which is what looked like was happening here. I just blocked the email address and moved on. At this point, I've set up my email to auto-delete any email that mentions "Goodreads" or "Amazon", unless they're on a SMALL whitelist, among other countermeasures. Life is too short and I have ramie to spin!
I said cynically to
telophase that I suspected that the "actual" "author" was some middle-aged white dude scammer sitting in North Dakota or, more tragically and pessimistically, some human trafficking scam farm outside the US.I assume this is also where the fake-looking-ness is partly to screen out people who are moderately suspicious/vigilant/smart enough to avoid weird, scammy emails and/or ask around for more information, and to screen for people who are sufficiently desperate, greedy, or naive (cf. shitty obvious "tells" in phishing scams). But I'm out of field so I could be wrong.
Regardless: it's not that legislative or technological protections aren't important or necessary or desirable, it's that the underlying human problem of the incentives vs. secondary incentives is inherently intractable. :(
NOTE: I'm screening comments from non-[access] and may be scarce/slow because I'm recovering from a health thing. Thanks.
![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
