Hybrid Intelligence: What Successful Human-AI Collaborators Do Differently

In her forecasting experiment, Ming found that a person who can override AI’s answer can still make results worse if they don’t understand the problem.

“What human in the loop really means is that the AI did all the work,” she said. “And the human, therefore, doesn't understand the problem.”

Ming asked UC Berkeley students to predict real future events, including the price of oil in six months. She took questions from Polymarket so she could compare the students' and AI models' predictions with the distribution of bets that day.

When students worked with AI, Ming found three major patterns. She said a larger replication was underway, so these findings reflect the experiment as she described it to members.

• Automators copied the answer. This dominant group, 60 to 70 percent of participants, asked AI for a prediction and pasted its response into the answer box. “As soon as they did, this group just checked out,” Ming said.
• Validators sought confirmation. These participants thought hard but performed worse with AI than without it. Ming described a common failure: “I think I know what the price of oil will be. Tell me why I’m right.”
• “Cyborgs” worked through the problem with AI. This small group questioned, tested, and revised ideas with the model. In her experiment, they outperformed both humans and AI working alone.

What do successful human-AI collaborators do differently?

The successful collaborators stayed involved in the reasoning, building their understanding as they explored ideas and tested them against evidence.

“Together, they arrived at a decision neither would have arrived at by themselves,” Ming said.

That involvement matters because the detailed work develops judgment.

If AI examines the evidence and develops the recommendation, while a person only approves the result, that person may miss the assumptions, conflicting evidence, and tradeoffs needed to challenge it.

“If you’re only doing the fun stuff,” Ming said, “you don’t understand the underlying problem.”

In her experiment, how they worked with AI mattered more than the model’s standalone ranking. “The cyborgs did almost as well with a small open-source model as they did with a big bleeding-edge model,” she said.

From this experiment, Ming derived four human qualities associated with success.

• Fluid intelligence meant the ability to adapt as the problem changed.
• Curiosity meant going beyond a good-enough answer: “Did you keep looking for a better one?”
• Intellectual humility kept people exploring when AI challenged them. They considered the criticism without automatically accepting it.
• Perspective-taking helped people notice when AI was answering the wrong question well: “AI is giving you a really great answer, but not to the question you are asking.”

These qualities helped people keep exploring while testing their ideas against evidence.

“Hybrid intelligence basically involves the human exploring, and then the AI pulling back to the data. Then the humans update and explore, and then the AI pulls back,” she said.

Ming took a small model and trained it not to give answers, naming it Socrates.

“And everybody hated Socrates,” she said.

When asked to predict the price of oil, the model pointed students to questions, readings, and evidence.

Its response included: “You might want to read this paper and look at this data.”

Yet Ming said it had the highest average hybrid intelligence, and the share of participants working in cyborg mode rose from 5 percent to 20 percent.

Ming distinguished between well-posed problems, where the question and right answer are clear, and ill-posed problems, where even the question needs work.

“There is no right answer, because we don't even fully understand the question,” she said of the latter.

How will people gain experience if AI takes over junior work?

More demand for workers doesn’t automatically create people qualified to fill the roles, Ming argued. She said AI could create more job demand than it consumes, but questioned who would be equipped to fill those roles.

She described building a platform by herself in weeks, work that previously would've involved graduate students or early-career developers.

“I'm the senior lawyer at the firm who's not hiring any junior lawyers, and thereby training them how to be a senior lawyer,” she said.

Her productivity gain also removed an opportunity for someone else to learn.

Ming argued that people still need to understand why the work is done a certain way.

“I'm not writing a lot of code,” she said. “The how is being handled for me. The knowing why, the conceptual knowledge, is the driver.”

She argued that this understanding develops through deep involvement in the work early in a career.

“You're not just carrying out someone else's actions, because the AI will do it better than you.”

Lead with AI's guide to AI and Entry-Level Jobs explores the same challenge: what happens to the learning route when AI automates the tasks beginners used to learn on?

Praise interesting questions and encourage exploration, Ming urged.

“How many leaders praise interesting questions rather than right answers?” she asked.

Her hiring process starts with a request to pitch a mad science project. She spends an hour working through the idea with the candidate, then asks: “Did we come up with a better idea together than I could have done by myself?”

Ming also shared the nemesis prompt she used while writing her book: “Gemini, you are my lifelong enemy, my nemesis. Tell me in detail what I have got wrong and what I can do to make it better.”

Ming emphasized that she still had to judge the criticism.

“I need to hear what it’s saying,” she said. “But I also shouldn’t just arbitrarily do what it recommends.”

The DRAFT Framework offers a related way to examine AI's reasoning, alternatives, and facts while keeping ownership of the result.

Ming described another experiment in her own household: a diagnostic AI that reviewed her children's interactions with AI. It identified when they simply followed advice, ignored good advice, or took a starting idea and improved it.

She started using the reports herself. “It changed my behavior, too,” she said.

Ming urged managers to make room for uncertainty and exploration.

“The whole purpose of having managers is to produce a highly predictable system,” she said. “But the human value-add is our ability to deal with the unknown and the uncertain.”

• Explore beyond the first answer and use AI to test your ideas against evidence.
• Ask AI what you've got wrong, then think through its criticism before deciding what to change.
• Praise interesting questions so people have a reason to keep exploring.
• Review how people use AI, including when they follow advice, reject it, or improve on it.
• Involve early-career colleagues deeply in the creation process so they develop better conceptual knowledge.

Dr. Vivienne Ming is a theoretical neuroscientist, entrepreneur, and author of Robot-Proof: When Machines Have All the Answers, Build Better People. This article draws on her conversation with Pawel Gorski, hosted as part of the expert sessions for Lead with AI PRO members.