Posit PBC

The clearest framing of AI I have heard this year comes down to middle school arithmetic.

"There was somebody who said something like, hey, like LLMs, you know, sort of an exponent on on who you are and what you are. So if you're if you're already a greater than one, then you will that num that ex you know that exponential is going to grow you dramatically. But if you're just a little bit under one for whatever reason. then that you know the greater power is gonna make you a much smaller impact."

That's Posit CEO Tareef Kawaf on The Test Set, and the math holds up. Raise 1.2 to the tenth power and you land somewhere near 6. Raise 0.9 to the same power and you end up around 0.35. The exponent is identical in both cases, and the two people come out of it in completely different places.

That puts the variable on your side of the equation rather than the tool's. Full episode out now.

Listen now on Spotify (https://open.spotify.com/episode/7AbutF3Xvsf7ScaglRfiFo?si=23aea660fef34a04) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000790947311).

Leland McInnes’s gripe about data science is that we have amazing tools for tabular data, and basically nothing good for the rest of it. On The Test Set, the mathematician behind UMAP explains the problem that's driven his entire body of work. If you've got a giant pile of documents, images, videos, or some mixture of all three, how do you even figure out what's in there, let alone explore it in a way that makes sense? His answer isn't "let an AI summarize it for you." It's that humans still need to be the ones looking, because understanding a dataset takes taste, and taste is something AI doesn't have … yet. Every tool he's built, UMAP, HDBSCAN, EVōC, exists to condense a messy pile of unstructured data down to something a person can actually look at and learn from.

Listen now on Spotify (https://open.spotify.com/episode/5KhfqQxJCgPeDoYyndj0Oz?si=15d68ee7a5744b2f) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000788474295).

"You guys always have way too much to say in the time allotted. It's just the nature of your business."

Ruth Milligan said this to Michael, Wes and Hadley on this week's The Test Set, and nobody argued. Because she's right.

Her question, the one she's put to hundreds of conference speakers, is brutal and useful. How much information do I have to share for you to believe something is true? Do I have to show you 20 charts, or can I show you two?

The uncomfortable part is that you spent six weeks on the analysis and your audience is going to walk out remembering one thing. So pick the thing.

Ruth curates TEDx Columbus, wrote The Motivated Speaker, and has coached hundreds of data folks through exactly this problem.

Listen now on Spotify https://open.spotify.com/episode/1AxvHyeKsPPU3E5PfW8gGT?si=3aa9bf9006144332 and Apple Podcasts https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000785578039.

Hot take from someone who actually shipped it —> most MCP integrations have too many tools. marimo pair had a whole drawer of them. List my variables. List my data frames. Tell me what's in cell three. Trevor Manz and the marimo team deleted all of it, and what’s left is one tool to run Python code, in the kernel, where the notebook actually lives. Feedback comes in the same turn as the action. Redefine a variable, break marimo's rules, hallucinate a value that isn't there — the error comes straight back, without a follow-up query. This means no more agent babysitting (agentsitting?). One sharp tool beats a pile of dull ones, and the model already speaks Python. Trevor's on latest episode of The Test Set with Michael Chow and Hadley Wickham.

Listen now on Spotify (https://open.spotify.com/episode/2gmO1zQFv5PRQ0HandRj2N?si=9da3245c0a3d4c05) and Apple Podcasts (https://podcasts.apple.com/us/podcast/let-the-agent-cook-with-trevor-manz/id1823736938?i=1000782024767).

Being reasonable and being ethical are not the same thing, and Leilani Battle thinks data science low-key mixes the two up all the time. On the new episode of The Test Set, the UW professor hits on why so much harm in this field doesn't come from bad actors cutting corners, it comes from people acting completely rationally inside a definition of rational that was never built to account for ethics. "It's not that people aren't rational or people aren't reasonable," she says. "It's that they're rational and reasonable. Rational behavior is not necessarily, in the current context, the definition of rationality might not be ethical. So if you define rational as responsible, you end up in a different frame." A useful reframe for anyone who assumes good outcomes just follow from good intentions.

Listen now on Spotify (https://open.spotify.com/episode/6fn22DYxSoKX9OFBx61N41?si=06a884a1103a44a0) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000778566461).

Joe Cheng, CTO of Posit, was genuinely unsure about whether he should keep building AI tools at all. On this episode of The Test Set, he explains how he landed on an answer. His logic: The train keeps moving with or without him. If he personally stepped away from AI, or even got his whole company to abstain, it wouldn't slow the technology's development anywhere else. Joe points to Posit’s decades of perspectives about reproducible, correct data science and the leverage to help people avoid getting confidently wrong answers from these tools. He argues that's exactly why the company has an obligation to stay in the room.

Listen on Spotify (https://open.spotify.com/episode/5pD6aH5Y2vPuecLpEeJEMW?si=6b645cc9a77c4d74) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000776617646).

I feel like when we started The Test Set, I didn't totally know what we’d be doing.

The data science world has no shortage of podcasts, talks, and hot takes. But there’s something neat about digging into what makes people tick. I really appreciate Hadley Wickham and Wes McKinney for wanting to spotlight contributors in their communities. And a big thanks to all our guests for letting us in.

We released roughly 25 episodes this year. There were so many thoughtful discussions. Some “spicy takes” for sure!

Here’s a post looking back on this year: https://posit.co/blog/test-set-1-year

-Your host, Michael Chow

Caitlin Colgrove's management philosophy, from The Test Set: Give people permission not to be miserable.

Hex has intentionally structured their org so that senior engineer and engineering manager are the same level — same pay, different title. Trying a new role and deciding it's not for you isn't failure. It's information.

She told the story of an engineer who moved into a management role, hated it, and came back to IC. The team's reaction? Basically nothing, because it had happened before. Once there's a precedent, it stops being scary. That matters a lot for people who've spent their whole career being the one who’s figuring it out.

Listen now on Spotify (https://open.spotify.com/episode/75UlWhiPCTGIRUpjYwyp2k?si=ad0ad7522100411b) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000774726490).

Here's a reframe on selling. Persuasion isn't getting someone to do something they weren't going to do. It's helping them make a decision that's already in their best interest. Alex Hillman lays it out in about 40 seconds on the latest episode of The Test Set. And somehow he makes it sound obvious. Alex is co-founder of Indy Hall and creator of Stacking the Bricks, a program that's helped thousands of engineers, artists, and independent workers get better at the part of their job they were always told wasn't their job.

Listen now on Spotify (https://open.spotify.com/episode/0JsGT6em8qX0Qok6wmBFPb?si=17d4140f10904cd1) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-test-set-by-posit/id1823736938?i=1000772836232).

In 1998, Mike Bostock was a Netscape intern who wanted to make charts in a browser. There was no Canvas. No SVG. Just HTML tables and a good bit of determination.

He had three one-by-one pixel GIFs. A transparent one, a blue one, and a red one. Arranged inside table cells in increasingly cursed ways. The result was a bar chart! Kind of!

Twenty-five years later, he's the creator of D3 (once the third-most-starred repo on GitHub), a former NYT graphics editor, and the founder of Observable. The three-GIF era is behind him.

The full story — including why D3's success came from betting on the browser when JavaScript was still a punchline — is on The Test Set.

Listen now on Spotify (https://open.spotify.com/episode/01BJXUJl2yoRZMO5yeijcP?si=4eRiAGLTSseI_ZcYrWKWHQ&nd=1&dlsi=1e3d8ab46ff3470a) and Apple Podcasts (https://podcasts.apple.com/us/podcast/the-code-doesnt-lie-with-mike-bostock/id1823736938?i=1000770586868).