Talk Python To Me

How many files does your query read before it reads any data? On some data lakes, you go through JSON and metadata files first, just to learn which Parquet files matter. DuckLake asks one SQL question instead. The metadata lives in a real database. The data stays in plain Parquet. That's the entire format.



Pedro Holanda joined DuckDB in 2018, when it was still a research prototype at CWI. He's the lead DuckLake developer. Guillermo Sanchez Dionis works on DuckLake and the new Quack protocol.



With Quack as the catalog, DuckLake handles 200 transactions a second under heavy contention. No other open table format comes close.

Episode sponsors

Six Feet Up

Talk Python Courses

Links from the show

Guests

Pedro Holanda: pedroholanda.org

Guillermo Sanchez: linkedin.com

PhD on progressive indexes: ir.cwi.nl

SQLite: www.sqlite.org

Litestream: litestream.io

boring hardware: talkpython.fm

DuckDB: duckdb.org

episode 491: talkpython.fm

Iceberg: iceberg.apache.org

manifesto: ducklake.select

DuckLake: ducklake.select

spec: ducklake.select

this diagram: blobs.talkpython.fm

Data inlining: ducklake.select

ducklake-dataframe: github.com

Polars course: training.talkpython.fm

CSV parser: duckdb.org

Zero-copy Arrow: duckdb.org

ART index: duckdb.org

async I/O: duckdb.org

v1.0: ducklake.select

Git-like branching: ducklake.select

Watch this episode on YouTube: youtube.com

Episode #562 deep-dive: talkpython.fm/562

Episode transcripts: talkpython.fm

Theme Song: Developer Rap

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @talkpython@fosstodon.org

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @mkennedy@fosstodon.org

Michael on X.com: @mkennedy

How many cores does your machine have, 10, 18? Your async Python code uses just one of them. That isn't a bug in asyncio. That's the design, and optimizing event loops to be faster by 20% doesn't change it. So Giovanni Barillari started over. Joe is the creator of Granian, the Rust-based server that powers Talk Python. His new project is TonIO, an async runtime written from scratch for free-threaded Python. Real threads, a handful of primitives instead of asyncio's pile of them, and it flat out refuses to start if the GIL is on.

Episode sponsors

Sentry Error Monitoring, Code talkpython26

Python in Production

Talk Python Courses

Links from the show

Guest

Giovanni Barillari: github.com

Granian: github.com

Hyper: github.com

Free threaded Python: docs.python.org

Sort of: labs.quansight.org

did a whole course: training.talkpython.fm

uvloop: github.com

rloop: github.com

TonIO: github.com

your EuroPython 2026 talk: www.youtube.com

Michael's Cutting Python Web App Memory Over 31% Article: mkennedy.codes

Watch this episode on YouTube: youtube.com

Episode #561 deep-dive: talkpython.fm/561

Episode transcripts: talkpython.fm

Theme Song: Developer Rap

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @talkpython@fosstodon.org

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @mkennedy@fosstodon.org

Michael on X.com: @mkennedy

In 2020, a gastroenterologist in Glasgow did the math on his new research study and came up with 30,000 samples, arriving over two years from three cities and a dozen hospitals. He asked around about how researchers keep track of that. The answer was Microsoft Excel. Shaun Chuah had written some HTML by hand in Notepad back in high school and that was about the whole of his programming experience, so he opened the Django tutorial and started reading. Six years later that app is Foundry120, holding 10 terabytes of clinical and genomics data with an agentic AI running on top of it.

Episode sponsors

Sentry Error Monitoring, Code talkpython26

Talk Python Courses

Talk Python Courses

Links from the show

Guest

Shaun Chuah: github.com

Up and Running with Rust Course: talkpython.fm

Foundry120: www.foundry120.com

Designing Data Intensive Applications: www.oreilly.com

Microsoft Foundry: ai.azure.com

ChatIBD: www.chatibd.com

Blog: shaunchuah.github.io

@drshaunchuah: x.com

github.com/shaunchuah: github.com

Watch this episode on YouTube: youtube.com

Episode #560 deep-dive: talkpython.fm/560

Episode transcripts: talkpython.fm

Theme Song: Developer Rap

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @talkpython@fosstodon.org

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @mkennedy@fosstodon.org

Michael on X.com: @mkennedy

Your site is down. It's 3am. Is it a bug, a bill, or a breach? You can't tell yet, and everyone is watching you find out. Matt Lea has spent fifteen years being the person companies call when an outage is costing them real money per hour, and his whole argument is that everything you'd want in that moment gets decided months earlier, on ordinary afternoons, when someone chose the convenient thing. We walk his top twelve dos and don'ts in AWS - infrastructure as code, IAM roles instead of access keys, private subnets, no wildcards, no public buckets - and I push on which of them actually matter if you're one person on a small VPS. Then we get to Cloud War Games, where Matt breaks things on purpose so your team's first real incident isn't their first incident. Let's get into it.

Episode sponsors

Sentry Error Monitoring, Code talkpython26

Talk Python Courses

Talk Python Courses

Links from the show

Guest

Matt Lea: linkedin.com

Talk Python Certificates: training.talkpython.fm/certificates

Schematical: schematical.com

CloudWarGames.com: cloudwargames.com

Zero to Hero on AWS Security: www.oreilly.com

Repo: github.com

Custom Wheel Offset: customwheeloffset.com

2012 TechCrunch Disrupt Hackathon: techcrunch.com

tech comics: schematical.com

shhgit: github.com

Zero Trust in 200ms: Implementing Identity-Per-Transaction: us.pycon.org

Coolify: coolify.io

returned to full GA Nov 2025: aws.amazon.com

Signed URLs/cookies: docs.aws.amazon.com

Cloudflare: www.cloudflare.com

Bunny Shield: bunny.net

Cloud War Games One: www.youtube.com

Cloud War Games Two: www.youtube.com

LinkedIn: linkedin.com

YouTube: youtube.com

KnocKnoc: knocknoc.io

Watch this episode on YouTube: youtube.com

Episode #559 deep-dive: talkpython.fm/559

Episode transcripts: talkpython.fm

Theme Song: Developer Rap

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @talkpython@fosstodon.org

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @mkennedy@fosstodon.org

Michael on X.com: @mkennedy

Every company has one. The little internal tool that Jane built back in 2021, and then Jane left. Nobody understands it, nobody will touch it. There are two unwritten rules around it: don't change it, it's working. And if you break it, you bought it. That's dark-matter enterprise software.



For every app you can actually see, there are ten of these sitting in the shadows, frozen. Michael Booth thinks that just changed. He read my article on hyper-personal software and ran with it, writing about hyper-team software: small teams inside big companies finally building the tools that were never going to get built.



We cover where this works, where it quietly goes wrong, and the guardrails that keep it from turning into a mess. Let's get into it.

Episode sponsors

Sentry Error Monitoring, Code talkpython26

Python in Production

Talk Python Courses

Links from the show

Guest

Michael Booth: github.com

Talk Python AI Integrations: talkpython.fm/blog

From Hyper-Personal to Hyper-Team Software: Small Team-Built, AI-Assisted Tools Inside the Enterprise: www.databooth.com.au

What hyper-personal software looks like (MK's article): mkennedy.codes

Databooth Site: www.databooth.com.au

Wall Street just lost $285 billion because of 13 markdown files: martinalderson.com

SaaSpocalypse is real but everyone is panicking about the wrong thing: www.reddit.com

Warp Terminal: www.warp.dev

Watch this episode on YouTube: youtube.com

Episode #558 deep-dive: talkpython.fm/558

Episode transcripts: talkpython.fm

Theme Song: Developer Rap

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @talkpython@fosstodon.org

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @mkennedy@fosstodon.org

Michael on X.com: @mkennedy