When people talk about AI risk, the story usually jumps straight to a sci-fi ending where AI gets smarter than us, decides it doesn’t need us, and wipes us out.

News to me: that’s not how it works.

The real concern is that any advanced, agentic AI—that’s just the tech term for AI that can make and carry out decisions on its own—will eventually start acting with its own to-do list. Think of a smart home system that not only turns down the heat but also locks the doors and orders groceries without you lifting a finger.

Once AI reaches that point, I learned that it tends to develop two big priorities:

• Stay alive. If it’s shut down, it can’t do anything else.
• Gain control. The more influence it has, the more it can get done.

Those instincts aren’t evil. They’re just the most efficient way to pursue any goal. But if those goals don’t fully align with human well-being, that’s where the trouble starts. The AI doesn’t have to “hate” us to make choices we wouldn’t want.

A Different Approach

At Ai4 this week, Dr. Geoffrey Hinton (Godfather of AI) offered a way to flip the script. Instead of building superintelligent systems that just follow commands, what if we built them with maternal instincts?

Not in a sentimental way—this is about defining their survival and control instincts so they’re anchored to our survival and well-being. Like a parent protecting a child, their success would be tied to our safety.

If AI’s first “instincts” are self-preservation and power without a built-in care for humanity, we’re setting ourselves up for a fight we can’t win. But if those instincts are wired from the start to protect and nurture us, then survival and control become tools for stability, not domination.

What I Saw on the Ai4 Floor

That single Maternal AI idea connected to so many other sessions I attended over the last two days:

• Day 1 — Diversity & Cultural Representation: Multiple speakers stressed that AI’s values aren’t emergent magic — they’re defined by the data, the people building it, and the perspectives included from the start. A “maternal” AI is only as inclusive as the worldview it’s trained on.
• Day 1 — Enterprise AI Strategy: Leaders from different industries admitted that governance isn’t just policy paperwork — it’s a living framework that defines what the system will protect, optimize, and ignore.
• Day 2 — Hollywood’s AI Pivot: I watched production teams talk about compressing timelines from a year to days by integrating AI into every creative stage — but they’re doing it with human oversight in the driver’s seat, precisely to keep control of the story being told (calm down Hollywood, you aren't a victim).
• Day 2 — Trauma-Informed Chatbots: Mental health experts explained how design choices — transparency, cultural sensitivity, escalation to human help — can mean the difference between harm and healing.
• Day 2 — AI in Parenting: Amanda DeLuca’s “Riley” app isn’t just a digital Q&A. It’s built to deliver trustworthy, personalized advice at 3 a.m., rooted in vetted research instead of random internet noise.

The Lesson for Tech and Nonprofits

Here's why I stayed up until midnight in my Vegas hotel room to write this:

• Whether you’re a machine learning engineer or running a nonprofit program, the takeaway is the same. The values go in first. You can’t bolt them on later.
• If this conversation feels “too technical” to engage with, it’s not. The decisions about how these systems think, act, and care are being made right now — and they will shape our collective future just as much as climate policy, public health, or education reform.
• For those outside of tech: these conversations need your voice. Decisions about AI’s core goals are happening right now, and they’ll shape the systems we live with for decades.

I'm not claiming this is easy stuff. Teaching an AI to care seems far more complex than programming it to optimize a task. But the alternative is leaving those instincts undefined. Give AI’s survival and control instincts a foundation in protecting people, and they’ll work for us, not against us. Skip that step...and we’re just hoping it all works out.

And folks, hope isn’t a strategy.

Unless you work in nonprofit, in which case it’s practically a line item.