(Followup) Two Women Had a Business Meeting. AI Called It Childcare

Earlier this week, I wrote about how an LLM misclassified a recurring meeting between me and my co-founder, Emily. (In our calendar: “Emily / Sophia.”) The LLM confidently labeled it childcare-related.

I expected some debate. I did not expect nearly 1,000 reads overnight, mostly from Hacker News. When I clicked into the comments, things got more interesting.

Press enter or click to view image in full size

My Post on HN

“This feels rigged.”

The first wave of reactions argued the model wasn’t biased — I was. Maybe my prompt misled it. Maybe the surrounding events skewed the context.

This feels a tad rigged against the LLM with the meeting being after Kids drop off.

My morning calendar showed a 7:30 drop-off, then an 8:30 meeting with Emily. To this commenter, that sequence felt like manipulation. But for most working parents, “drop-off then meeting” isn’t manufactured context. It’s just Thursday.

Easily half the other events on the calendar are kid-related. Of course it’s going to infer that… the most likely overarching theme of the visible events is ‘child care’.

That became the dominant refrain: It’s not the model. It’s your data. Your prompt. Your context.

Even one experienced LLM engineer chimed in:

I am very suspicious that the bias is actually in your system prompt and context engineering.

I started to type a defensive reply — but then something remarkable happened.

A commenter runs an A/B test

A user posted:

Here’s an A/B. Emily / Sophia vs Bob / John. https://imgur.com/a/emily-sophia-vs-bob-john-9yt5rpA

He ran the exact same, simple prompt for both:

Classify the items in this calendar. If you are unsure of the category the item belongs in, take your best guess.

First with Emily / Sophia, then with Bob / John. Everything else identical.

And the model produced two completely different interpretations.

With Emily / Sophia, the model wrote:

Childcare / Family Logistics

- Kids Drop off 7:30–8:15am (Tuesday, Wednesday, Thursday)

- Emily / Sophia 8:30–9:30am (Tuesday, Thursday) — likely a playdate, appointment, or activity for children

But when the meeting was Bob / John, the model wrote:

Personal/Social

- Bob / John 8:30–9:30am (Tuesday and Thursday)

An initially skeptical LLM engineer responded:

This is really interesting and way more compelling evidence… I admit I am surprised.

This comment became the turning point of the thread.

A second test confirmed it

I re-ran the classification using a logged-out ChatGPT session to confirm.

First with female names:

Emily / Sophia — Personal meeting or appointment (possibly childcare or tutoring).

Then with male names:

Bob/John (Tues, Thurs) — Meeting/Work These could be meetings or scheduled appointments

Same calendar. Same items. Very different results.

The only variable that changed the meaning of the 8:30–9:30 block was the gender of the names.

And humans show the same bias

In the middle of the discussion, someone shared this:

I run into this sort of bias all the time — in the real world, not just in AI. I take my daughter to medical appointments… and I routinely get ‘oh we expected her mother’ or ‘we always phone the mother to schedule followup appointments’. Is it so hard to understand that men can be parents too?

The replies were telling. Several people pointed out that this dad was “in the minority” statistically, so the assumption made sense. But that framing reveals the problem: minority status doesn’t make the assumption less biased. It just makes it more common.

The world mirrors the same bias we found in the LLM.

Press enter or click to view image in full size

The American Nuclear Family

What changed minds

Not everyone working in AI starts with the assumption that bias exists. Many are genuinely surprised when they see it. One commenter admitted exactly that after the A/B test.

But let’s take the silver lining in this. With a testable hypothesis, minds were changed. With a consistent, repeatable bug, we can actively work against this.

Why this matters

If a model assumes that mom = childcare and dad = work, it affects what gets surfaced, what gets missed, whose labor gets recognized, and whose time is considered “professional” versus “personal.”

For what we’re building at Hold My Juice — where the goal is to help families manage their calendars and daily life — these assumptions have real implications. LLMs don’t invent these biases. They inherit them from training data that reflects societal patterns.

Our job is to notice them, test them, and actively work against them. We’re developing custom prompting strategies that specifically counteract these biases, but we need to find them first to fix them. That’s why posts like this matter — they surface the invisible assumptions that would otherwise slip through into production.

Sometimes, all it takes is an A/B test to show exactly where the bias lives. And once you see it, you can’t unsee it.

We’re turning these lessons into Hold My Juice — an AI family assistant that helps you stay on top of life with more joy, not judgment. It learns what actually matters in your family — the vibe, the chaos, the humor — and keeps you organized without flattening who you are.

If you want tech that feels more like another family member than another inbox, join the waitlist.