413. ScamGPT – How AI Supercharges Fraud (ft. Alice Marwick)
We chat with Alice Marwick — director of research at Data & Society — about a new report she co-authored on how generative AI is now unleashing a new world of scams and fraud. AI is supercharging the business of scamming by making it easier and cheaper than ever to deploy sophisticated scams at scale. Scams are now automated, ubiquitous, and dynamic. The victims of AI-fuelled fraud are not just traditional demographics like older people. We get into types of fraud like pig butchering and harpoon whaling, and what we can do to defend against the automation of fraud. Then we discuss how these technical infrastructures are compounded by social conditions (and societal crises) that are making more people vulnerable to scams—but also to high risk behaviors like gambling, forms of financial exploitation like multi-level marketing, and misinformation like conspiracy theories.
••• Data & Society | Research https://datasociety.net/research/
••• Scam GPT: GenAI and the Automation of Fraud https://datasociety.net/library/scam-gpt/
••• The Future of Conspiracy Theory Scholarship https://www.tandfonline.com/doi/abs/10.1080/19331681.2025.2491687
••• Mountains of Evidence: Processual “Redpilling” as a Socio-Technical Effect of Disinformation https://ijoc.org/index.php/ijoc/article/view/21934
Standing Plugs:
••• Order Jathan’s new book: https://www.ucpress.edu/book/9780520398078/the-mechanic-and-the-luddite
••• Subscribe to Ed’s substack: https://substack.com/@thetechbubble
••• Subscribe to TMK on patreon for premium episodes: https://www.patreon.com/thismachinekills
Hosted by Jathan Sadowski (bsky.app/profile/jathansadowski.com) and Edward Ongweso Jr. (www.x.com/bigblackjacobin). Production / Music by Jereme Brown (bsky.app/profile/jebr.bsky.social)
--------
1:24:21
--------
1:24:21
Patreon Preview – 412. Sweat Your Genetic Assets
We dive into Nucleus Genomics, a startup backed by the Founders Fund, that wants to help you evaluate, rank, and name your embryos in a pre-IVF dashboard designed for “genetic optimization.” We talk about the junk science and wild ideologies that drive these desires to treat your genes and children like assets to be managed – with the ultimate goal of creating and enforcing a society ordered by genetic hierarchies. We then wrap up by tying these biotech visions to their AI counterparts in the “Gentle Singularity.”
••• This 25-Year-Old Biotech Founder Says His Startup Can ‘Optimize’ Embryos for Intelligence https://www.inc.com/ben-sherry/kian-sadeghi-says-nucleus-genomics-can-optimize-embryos-for-intelligence/91198815
••• Controversial genetics testing startup Nucleus Genomics raises $14M Series A https://techcrunch.com/2025/01/30/controversial-genetics-testing-startup-nucleus-genomics-raises-14m-series-a/
••• Genetics testing startup Nucleus Genomics criticized for its embryo product: ‘Makes me so nauseous’ https://techcrunch.com/2025/06/06/genetics-testing-startup-nucleus-genomics-criticized-for-its-embryo-product-makes-me-so-nauseous/
••• OpenAI wins $200m contract with US military for ‘warfighting’ https://www.theguardian.com/technology/2025/jun/17/openai-military-contract-warfighting
••• This is the gentle singularity? https://www.bloodinthemachine.com/p/this-is-the-gentle-singularity
••• China shuts down AI tools during nationwide college exams https://www.theverge.com/news/682737/china-shuts-down-ai-chatbots-exam-season
Standing Plugs:
••• Order Jathan’s new book: https://www.ucpress.edu/book/9780520398078/the-mechanic-and-the-luddite
••• Subscribe to Ed’s substack: https://substack.com/@thetechbubble
••• Subscribe to TMK on patreon for premium episodes: https://www.patreon.com/thismachinekills
Hosted by Jathan Sadowski (bsky.app/profile/jathansadowski.com) and Edward Ongweso Jr. (www.x.com/bigblackjacobin). Production / Music by Jereme Brown (bsky.app/profile/jebr.bsky.social)
--------
5:09
--------
5:09
411. Big Easy’s Big Brother
We get into a secret partnership between the New Orleans Police Department and Project NOLA, a private nonprofit organisation that owns and operates an extensive network of cameras blanketing New Orleans. For years, Project NOLA has been running live facial recognition through their cameras and sending automated notifications to the police when a match is made using Project NOLA’s privately maintained list of “wanted people.” By going through an unofficial private partner, police have been able to sidestep and undermine legal prohibition on their use of AI technologies like facial recognition. We get into the history of using New Orleans as a testbed for policing technology, the dangerous precedent being set by this public-private relationship, and how this surveillance nightmare is on track to become even more expansive and unleashed thanks to potential policy changes.
••• Police secretly monitored New Orleans with facial recognition cameras https://www.washingtonpost.com/business/2025/05/19/live-facial-recognition-police-new-orleans/
••• A bad facial recognition match costs Jefferson Parish Sheriff Joe Lopinto's office. See how much https://www.nola.com/news/jefferson_parish/facial-recognition-sheriff/article_3ed262bc-e360-4383-82fb-728b3783cb93.html
••• New Orleans City Council proposed ordinance https://cityofno.granicus.com/MetaViewer.php?view_id=42&clip_id=5115&meta_id=741682
Standing Plugs:
••• Order Jathan’s new book: https://www.ucpress.edu/book/9780520398078/the-mechanic-and-the-luddite
••• Subscribe to Ed’s substack: https://substack.com/@thetechbubble
••• Subscribe to TMK on patreon for premium episodes: https://www.patreon.com/thismachinekills
Hosted by Jathan Sadowski (bsky.app/profile/jathansadowski.com) and Edward Ongweso Jr. (www.x.com/bigblackjacobin). Production / Music by Jereme Brown (bsky.app/profile/jebr.bsky.social)
--------
1:25:52
--------
1:25:52
Patreon Preview – 410. Palantir’s Psychohistory
We chat about Palantir and their role in enacting Trump’s executive order for a unified database across federal agencies and a master profile for every individual — and their larger role in bringing about the glorious new age of theocratic techno-nationalism for the West.
••• Trump Taps Palantir to Compile Data on Americans https://www.nytimes.com/2025/05/30/technology/trump-palantir-data-americans.html
••• ICE Just Paid Palantir Tens of Millions for ‘Complete Target Analysis of Known Populations’ https://www.404media.co/ice-just-paid-palantir-tens-of-millions-for-complete-target-analysis-of-known-populations/
Standing Plugs:
••• Order Jathan’s new book: https://www.ucpress.edu/book/9780520398078/the-mechanic-and-the-luddite
••• Subscribe to Ed’s substack: https://substack.com/@thetechbubble
••• Subscribe to TMK on patreon for premium episodes: https://www.patreon.com/thismachinekills
Hosted by Jathan Sadowski (bsky.app/profile/jathansadowski.com) and Edward Ongweso Jr. (www.x.com/bigblackjacobin). Production / Music by Jereme Brown (bsky.app/profile/jebr.bsky.social)
--------
6:50
--------
6:50
409. The Breakup Meltdown
It’s a real friends and enemies episode of TMK. We get into the messy breakup between Trump and Musk, then talk about the rekindling relationship between Palmer Luckey and Mark Zuckerberg, before ending with OpenAI trying to forge a new tier lists of countries who support “democratic AI” (aka friends of America), countries who aren’t yet doing enough to support “democratic AI” but could upgrade to friend status, and countries who are “CCP-led China” (aka enemies of America), plus we learn more about a man who Sam Altman describes as a “dear personal friend.”
••• Live Updates: Trump Threatens to Cut Musk’s Government Contracts as Online Feud Escalates https://www.nytimes.com/live/2025/06/05/us/trump-elon-musk
••• Builder.ai Faked Business With Indian Firm VerSe to Inflate Sales, Sources Say https://www.bloomberg.com/news/articles/2025-05-30/builder-ai-faked-business-with-indian-firm-verse-to-inflate-sales-sources-say
••• Builder.ai collapses after revelation that its "AI" was hundreds of engineers techspot.com/news/108173-builderai-collapses-after-revelation-ai-hundreds-engineers.html
••• Meta Fired Palmer Luckey. Now, They’re Teaming Up on a Defense Contract. https://www.wsj.com/tech/meta-army-vr-headsets-anduril-palmer-luckey-142ab72a
••• Whose National Security? OpenAI’s Vision for American Techno-Dominance https://theintercept.com/2025/06/03/openai-sam-altman-trump-china/
Standing Plugs:
••• Order Jathan’s new book: https://www.ucpress.edu/book/9780520398078/the-mechanic-and-the-luddite
••• Subscribe to Ed’s substack: https://substack.com/@thetechbubble
••• Subscribe to TMK on patreon for premium episodes: https://www.patreon.com/thismachinekills
Hosted by Jathan Sadowski (bsky.app/profile/jathansadowski.com) and Edward Ongweso Jr. (www.x.com/bigblackjacobin). Production / Music by Jereme Brown (bsky.app/profile/jebr.bsky.social)
A podcast about technology and political economy /// Agitprop against innovation and capital /// Hosted by Jathan Sadowski and Edward Ongweso Jr., Produced by Jereme Brown /// Hello friends and enemies
Listen anywhere that fine podcasts are distributed. Subscribe at patreon.com/thismachinekills to get premium episodes every week.
USA