Agency is the core of the human experience. We prize our ability to act, to make decisions and to choose our futures—individually and collectively. The act of making choices brings meaning to our lives. We live authentically by making choices of our own, not following the choices of others. And through making choices, we understand the meaning of our existence and we create our future selves.

AI relies on data and data is, by definition, the documented past of those who were documented. So its predictions for your future are based on your past or the past of others like you. But will you be who you were? Will you be who others were? Most likely not as we all make different choices today than we made in the past. The challenge for AI designers is to create an AI that helps us make these authentic choices.

As more of our world becomes datafied and subject to computation, our interactions with machines become identity-altering. Machines that learn and engage with us can change what we know about ourselves, affecting our sense of who we are.

Choices are complicated enough when we make decisions based on what others know and do. We are social creatures who evolved to be excellent at reading other people’s minds. What is he thinking about? How does she feel? How will they respond to what I do next? What are they hiding from me? Does she know that I know that she knows? How can I win this argument? Do they believe what I believe?

Just as we have an intuition about trusting another person, we form intuitions about trust in AI. We call this machine intuition—how good our intuition is for the output of a machine. If the machine was a person, could we, would we, trust it?

Humans are flawed but our weaknesses can be compensated by machines’ strengths. Most importantly, humans maintain the upper hand in unpredictable situations. But we are easily distracted and perform poorly when bored while machines never get bored and have infinite patience. Machines are fast and learn at scale. Human learning is individual, context-dependent and takes a lifetime.

Machines can only learn using data from the past while it’s us humans who make the decisions that create our futures. Fundamentally, machines and humans tend to make different kinds of mistakes. How can we make use of this complementarity and get the best of both worlds?