Take a moment to reflect on a desire you have at the moment. How does it feel to want this thing? Why do you think you want it? Where does this desire come from? Is it a desire for a thing, such as a cupcake or a graduate degree, or is it a desire about a desire? Do you want to be the sort of person who doesn’t want the cupcake? It’s easier to not want the cupcake in the first place than resist its temptation. Do you want to want a graduate degree? Could a graduate degree be the best way to advance your career but you can’t seem to conjure the desire to go back to school?
The American philosopher Harry Frankfurt thought that desires existed in layers. A first order desire is a desire for a thing. A second order desire is a desire for the first order desire. True autonomy comes when first and second order desires line up. We are closer to our true selves, when our in-the-moment choices align with our long term goals. Alignment provides satisfaction and a sense of who we are or want to be.
But there’s a nuance here that matters. What we want comes to us through our community. We don’t always consciously choose our desires.
What we want is, in part, a function of what we see others wanting. We are tantalized by desires that seem a little out of reach, things we see others enjoying, and when the obstacle to getting something is big. What we want depends on what others want. If our choices are intertwined with what choices people around us make, how can we be more aware of what we want? Are we more in control of our desires if we are more self-aware?
Stephen Fleming is a Professor of Cognitive Neuroscience at University College London. He wants to answer the question of what supports the remarkable capacity for human self-awareness. The technical term for self-awareness is metacognition and it’s our ability to think about our thinking. Metacognition underpins learning because we can be aware of the state of our knowledge. We self-monitor and understand our thinking. Metacognition drives sensemaking because we have an innate, biological urge to make sense of our lives.
Fleming studies when, why, and how we reflect on ourselves. Self-reflection can be sparked by a sense we are about to do something stupid or when someone points out that we seem to be doing something contrary to what makes us happy. In a discussion on the Artificiality podcast, Steve said that his wife reminds him how much better he feels when he makes a plan to see friends on the weekend rather than do nothing. Her observation is a type of metacognition. She knows him, at that moment, better than he knows himself.
Self-awareness is important for learning and in the pursuit of meaning. But we also need to be able to turn it off. We have to automate our learning so that we can make room in our minds for something new.
Do you remember learning to drive? You had to focus hard on every move and you were conscious of your skill level and mistakes. But as you became more skilled, you stopped thinking about driving. Now you regularly drive across town, navigating the complexity of stop signs and traffic circles, and arrive safely without even remembering the trip. There is a tension here: while we need metacognition to make us aware of what we know, we also have to transfer what we learn to non-conscious brain circuits or we will never acquire new skills.
Stephen Fleming suggests that our feeling of being in charge of our lives is a “construction—a narrative that is built up at a metacognitive level from multiple sources.” Our experience of agency can be manipulated by explanations we provide to ourselves. This is a phenomenon called “choice blindness.”
In an experiment conducted by researchers from Lund University in Sweden, shoppers in a local supermarket were asked to taste test different jams and asked which ones they preferred. The scientists then sneakily reversed the participants favorite jams. Less than a third of customers noticed that their preferences had been switched. Customers enthusiastically explained their preferences, blind to the fact that their choice blindness was in charge.
We can be completely ignorant of the fallibility of our choices. We can regularly construct narratives that provide us with the illusion of control and agency over our own actions. If you don’t know why you want something it may be that you only want it because everyone else does too. Desires can inhabit our minds without us being aware of why.
Great Human Strength: We have agency in our choices and have the capacity for self-reflection. We have the ability to set long term goals and align our short term decisions to create our future selves.
Great Human Weakness: Our short term preferences can overwhelm our decision making, especially when our modern environment creates an evolutionary mismatch.
Machine Opportunity: Designs which encourage self-awareness, self-reflection and self-education.
Machine Threat: Designs which cripple metacognition.
