Being a parent of young children, I’ve happily discovered, means getting a second chance to do things I missed in my own childhood—at least the things that won’t leave my body in splints and braces. I’ve had particular fun spending hours reading children’s classics aloud to my kids, stories I’d skipped when I was young or that hadn’t been written yet—L. M. Montgomery’s Anne of Green Gables, Kenneth Oppel’s Airborn series, Philip Pullman’s His Dark Materials trilogy, and, of course, all seven volumes of the original Harry Potter series—twice.

One tome that I’d never mastered was J. R. R. Tolkien’s The Lord of the Rings—science fiction interested me far more than epic fantasy and myth. So, when I opened volume one, The Fellowship of the Ring, on an icy December evening in 2013, cuddled with then eight-year-old Ben in an armchair, I wasn’t exactly a study in enthusiasm.

Tolkien isn’t for everyone, that’s for sure. The Lord of the Rings is in many ways a prototypical hero’s journey, full of daring adventures, bloody battles, and much valour to protect honour, friends, tribe, and the truth. With only a couple of notable exceptions, powerful, courageous, or clever women don’t cross the pages. And, even taking into consideration the genre’s prevailing style, Tolkien’s prose is—how to be polite?—a tad laboured.

Yet I was soon enthralled. The saga captivated me in some primal way, as it has so many others. And, by somewhere in the middle of the first volume, I realized why I was hooked. The Lord of the Rings is an extended meditation on what one should do when things appear utterly hopeless. It is, in fact, an account of how to survive by creating and living through hope. Such hope is informed by an attempt to understand the minds of people we encounter—friends, allies, and enemies—as we strive to reach our vision of a positive future. And it employs that knowledge in ways that are strategically smart.

If you’re not familiar with the story, here’s the gist (and bear with the recap—it is relevant!): the Dark Lord Sauron has established himself in Mordor, a forbidding, barren, and mountainous domain in the eastern regions of Middle-earth. From there, he’s sent forth his brutal servants to find and bring to him the One Ring, which will give him unassailable power over the land. But that ring is currently in the possession of Frodo, a middle-aged hobbit—a short, peaceable, human-like creature with very hairy feet (to my son’s delight). The wizard Gandalf tells Frodo that the only way to keep the ring from Sauron forever is to take it into the heart of Mordor itself and throw it into the volcanic fires of Mount Doom.

The plan seems absurd. Not only does the ring corrupt nearly everyone who touches it (although it seems to have less effect on hobbits), Mordor seethes with teeming masses of vicious creatures called Orcs, spectral beings on flying beasts, and various other soldiers, all overseen by Sauron’s panoptical eye atop a black tower that reaches far into the sky.

To make matters worse, the Free Peoples that populate Middle-earth outside of Mordor—which include Elves, Dwarves, and Men—have squabbled among themselves for centuries. Even if they could somehow manage to pull themselves together, their collective armies aren’t strong enough to penetrate Mordor’s mountainous defences to reach Mount Doom, while the idea of sneaking past Sauron’s vast forces and under his watchful eye seems preposterous. The situation is surely hopeless. But it’s not, we learn. Getting the ring to Mount Doom and defeating Sauron involves a healthy portion of luck and a good quantity of magic (the tale is, after all, a fantasy) but also an enormous amount of emotional, interpersonal, intercommunal, and strategic smarts.

Early on, a team is formed—the Fellowship—to solve the problem. It’s a cantankerous bunch that often comes close to breaking apart. But it incorporates at least one representative from every major people Sauron threatens. Over time, and under extreme duress, they learn to respect and even love one another. Equally important, though, is the mix of talents the team possesses: each member contributes something vital to the enterprise’s ultimate success.

At first, none of them has a clue how to get the ring to Mount Doom: the further they peer into the future, the worse the uncertainties become. As Elrond, the Half-elven Lord of Rivendell, advises: “None can foretell what will come to pass, if we take this road or that.” So they keep their initial plan simple and start by exploring the adjacent possible—that zone of uncertainty just beyond the edge of the known—by taking one step toward Mordor at a time. They recognize that they must figure things out as they go, constantly recalibrate their plans as they learn more, and manage their doubts and fear. Along the way, they wonder if they should turn back; divert themselves to lesser, more achievable goals; or split up and return to their homes to defend their respective groups against the coming storm. What little hope they have is under incessant siege.

The Fellowship is soon sundered by duplicity from within and attacks from without. Later, some members reunite and then split apart again, while Frodo and Sam seek to reach their destination alone. Still, every other member of the team repeatedly deliberates on how to help the quest and—even in the absence of communication among them—acts on those deliberations. The story’s climax arrives when most of the team, together again, takes what remains of their armies to Mordor’s front door to distract Sauron so Frodo and Sam can climb to the top of Mount Doom unobserved. Beforehand, the team debates at length the ridiculously low odds of their plan’s success. They know it’s a huge gamble—the equivalent of a Hail Mary pass—that depends on their estimates of Frodo’s physical and mental state and of his location on the slopes of Mount Doom being exactly right.

No surprise, their estimates are correct, the One Ring is destroyed (albeit somewhat by accident after Frodo yields to its powers), and Middle-earth is more or less saved, although profoundly and irreversibly changed. Good wins, sort of, and evil is vanquished, at least for the time being, as I learned when I turned the third volume’s last pages, eight months later, snug in a tent with my family in the pouring rain during an August camping trip.

A deep dive into Tolkien’s classic fantasy might seem a little out of place in a sober investigation of hope. But his story is not timeless without reason. It shows that, for our hope to be powerful, we need a clear, motivating vision of where we want to go, a way of identifying which paths to take, and a thoughtful understanding of the worldviews and motives of the people who can help us along the way—and of those who might try to stop us. It also reminds us that we’ll be far more effective if we’re ready to recalibrate our vision, our judgment of the best paths forward, and our assessments of others’ views as we go along.

Hope is unquestionably one of Tolkien’s central concerns in The Lord of the Rings; the word itself appears, by one count, about 300 times in the book’s three volumes. Tolkien, a scholar with a vast knowledge of history and myth as well as a keen observer of human nature and our moral struggles, had been deeply marked by the First World War, having fought as a young officer in the Battle of the Somme and lost almost all his male friends to the war. Also, he wrote The Lord of the Rings in a time and place—England around the time of the Second World War—when hope was under siege and eventually rescued through almost unimaginable courage, toil, and sacrifice, and with the help of many allies. We tend to forget how bleak the situation must have looked to Tolkien and his compatriots in the early days of the war, when England, isolated on the edge of Europe, stood almost alone against Hitler’s staggering forces.

Tolkien later dismissed attempts to draw allegorical parallels between those events and his story. Yet they may have helped him arrive at his sophisticated and subtle understanding of hope, which was revealed in unpublished writings edited and released posthumously by his son. In a conversation about hope that Tolkien crafted between an Elf king and a wise woman, he distinguished two kinds: “Amdir,” a hope that involves “an expectation of good, which though uncertain has some foundation in what is known”; and “Estel,” a deep hope born of trust or faith that things will turn out well. When Amdir succumbs because we see no escape, Estel can remain steadfast.

Many Christian commentators and scholars say Tolkien espoused a Christian hope based on faith in redemption and God’s ultimate intervention. (He was a devout Roman Catholic.) By this view, hope, which in this case would be Estel, can remain secure because we know God will take care of us in the end. Other Tolkien aficionados have argued that he eschewed hope entirely: his protagonists keep going because of nothing more than their ardent commitment to courage and cheer regardless of what the future seems to hold.

Neither argument convinces me. I see little hint of Christian eschatology in the pages of The Lord of the Rings, and the book’s life philosophy is deeply informed by Norse, Germanic, and Celtic myth. Indeed, to my mind, Tolkien’s heroes possess the Finnish virtue sisu, which translates roughly as “fierce tenacity” or “toughness” and indicates inner strength in the face of daunting odds. Tolkien’s protagonists regularly express, as well, something akin to Amdir, a hope grounded in evidence and reason. They might think that the chance of a good outcome is terribly slim, but they pursue that chance all the same, with eyes fully open to the risks such a choice entails.

But mostly I’m not convinced because, in the book, Tolkien argues time and time again for another source of hope. He identifies this source most clearly in Gandalf ’s remarkable words to the Council of Elrond when the Fellowship is formed. Pressed on whether the effort to take the ring to Mount Doom is a path of despair or folly, the wizard replies, “It is not despair, for despair is only for those who see the end beyond all doubt. We do not. It is wisdom to recognize necessity, when all other courses have been weighed, though as folly it may appear to those who cling to false hope.”

Honest hope, Tolkien implies here, emerges from the wisdom that recognizes necessity. Critically, though, within the boundaries imposed by that necessity, we can’t know for sure what will transpire—we can’t “see the end beyond all doubt.” And it’s there, within that deep uncertainty, that we can find Estel, a hope that trusts in the world’s complexities to produce as yet unseen possibilities, some of which might be good.

Tolkien seems to suggest that people keep going in dreadful circumstances through a combination of, or sometimes an alternation between, Amdir and Estel. Amdir is hope with an object—“an expectation of good” arising from a vision of a positive future. But whether this vision will ultimately be realized is highly uncertain given “what is known.” Estel, in contrast, doesn’t have an object: the solace it provides arises instead from its confidence in uncertainty’s promise, all the while acknowledging that this promise may not, ultimately, be realized in something good.

Throughout The Lord of the Rings, Tolkien also implicitly asserts that hope (in this case, Amdir) should be smart. For our hope to be smart, we need a strategy for quickly and effectively searching the adjacent possible to identify—and provide rough-and-ready evaluations of—possible paths forward. When the range of paths is immense, as it was for the Fellowship, this search will often work best when undertaken by a highly diverse group of agents (like the Fellowship’s members) linked together in a loose network (so that each agent has a chance to discover new possibilities and then share that news with everyone else), guided by rough goals, and open to constant reformulation of its plans. This approach is, in fact, how complex systems as diverse as competitive economic markets and the immune systems of mammals solve their problems—it’s usually the most effective way to find good possibilities amid a welter of noise, distraction, and false leads.

Tolkien seems to have grasped this reality. But he then gave a special twist to the idea: we should often look for good possibilities, he advises, in the opposite direction of what seems to make most sense at the time. So, rather than fleeing from Sauron’s massed forces in Mordor, which common sense would seem to counsel, perhaps the Fellowship should go straight toward them. And, rather than attacking Sauron directly, with all the armies the Fellowship can muster, perhaps the Fellowship should send in only one or two of the least formidable beings in the land. In other words, seek solutions in unexpected places.

Finally, Tolkien shows that hope needs to be emotionally and psychologically smart too. The story’s protagonists discuss at length each other’s characters, perspectives, and intentions, just as they discuss those of their potential allies and likely enemies. They anchor their decisions in a keen awareness of what’s going on in their own heads—of their own beliefs, values, and deep motivations—and, as much as possible, of what they think is going on in the heads of the people and groups who might align with or oppose them.

Like tolkien’s Fellowship, today’s children and adults face a landscape of possible paths into the future that is, frankly, terrifying in its complexities and risks. If we’re going to choose a good path—good not just for ourselves narrowly but for all of humanity—we need discriminating criteria for selecting among the multitude of options we face.

Here are two criteria: first, any path we choose should take us toward a future that’s likely, as best we can judge, to be enough of an improvement to genuinely reduce the danger humanity faces; and, second, the path should also be feasible in the sense that it gives us a good chance of reaching that desirable future, which means surmounting or bypassing any political, economic, social, or technological roadblocks along the way. These criteria are often at odds, leaving us trapped in what I call the “enough vs. feasible” dilemma.

People who care about humanity’s future and want to do something to help generally cope with the dilemma in one of two ways. Some who are idealistic—such as activists in civic groups and many academics—focus on making sure their proposals for solving humanity’s problems would be enough. They largely ignore whether the proposals are feasible, implicitly downplaying political, economic, social, and technological obstacles to their implementation. They hope either that those obstacles will turn out to be far less severe than seems likely or that a new social phenomenon will sweep them aside. Basically, they say: We should start now to do whatever’s necessary to solve the problems, and deal with obstacles to implementing our solutions later, if it turns out that we must.

Others—politicians, policy makers, corporate leaders, and researchers in think tanks, for example, who all tend to see themselves as anchored in the practical world—focus on making their proposals feasible and largely ignore whether they’d be enough. They tend to highlight the obstacles to implementing solutions, implicitly hoping that the problems themselves will turn out to be far less severe than evidence today indicates is the case—or easier to solve sometime in the future, maybe with a technological breakthrough. Basically, they say: We should start now to do whatever’s possible to solve the problems, and deal with whatever remains of those problems later, if it turns out that we must.

Both groups, in other words, cope with the dilemma by implicitly ignoring or wishing away the side they don’t like. They either posit changes in our societies and technologies that seem far-fetched (a kind of magical thinking) or they deemphasize the seriousness of humanity’s problems (a kind of denial). Each group lives in its own la-la land while blaming the other for wasting time and resources on fantasies or for not doing enough. And, as humanity’s situation worsens, the gulfs between these la-la lands and our increasingly troubling reality are becoming positively pathological.

Neither approach is far from the “what if” imaginings of kids, which could be good, except that they both neglect the tempering realism that ideally comes with adulthood.

The protagonists in The Lord of the Rings face a version of this dilemma. The single action that would be enough to stop Sauron forever—throwing the ring into Mount Doom’s fires—doesn’t seem feasible, at least on first assessment. Yet other actions that could be feasible—trying to hide the ring in the countryside, giving it to the ancient creature Tom Bombadil, or dropping it into the deep sea—ultimately won’t stop Sauron.

The Fellowship’s dilemma seems impossibly difficult, but it’s actually more tractable than humanity’s version today. First and most obviously, Tolkien creates for his group of heroes a conventional “war problem”: an external and wilfully malicious enemy threatens the group with annihilation, which catalyzes its members to bury their differences and collaborate to defeat the enemy.

In contrast, today’s enemy isn’t “out there” but at least partly “in here,” right inside our own societies, communities, families, and even our individual selves. Our severe problems are rooted in factors like bad technologies, poorly designed institutions, greedy corporate elites, self-aggrandizing states, self-interested consumers, and ingrained patriarchy and racism. As in the sinking lifeboat, nearly all of us have contributed somehow—if to greatly unequal degrees—to the dangers we’re facing. To quote the wonderful mid-twentieth-century American cartoonist Walt Kelly, “We have met the enemy and he is us.”

But, in The Lord of the Rings, Sauron isn’t just an external, personified enemy; he also creates for the Fellowship what social psychologists call a “superordinate” goal—one that everyone shares, that overrides all others, and that can’t be achieved without cooperation. These features encourage teamwork among the members, and this teamwork makes reaching the goal—to throw the ring into the fires—far more feasible, which ultimately helps the Fellowship solve their enough vs. feasible dilemma. (In the classic demonstration of the power of superordinate goals, the Robbers Cave Experiment, antagonism between two groups of boys at a summer camp was significantly reduced when, among other collaborative tasks, they had to pull on the same rope, together, to get their food truck restarted.)

Today, though, humanity has a profusion of big problems. Worse, because of both deep uncertainty about the future and bitter ideological differences among us, we heatedly disagree about what our problems are and their severities, about whose responsibility they are and to what extent, about what their solutions should be and whether those solutions would work even if we could implement them. Alas, compared to the Fellowship’s situation, ours is a massive muddle. Instead of drawing us together, this muddle often just splits us further apart, which only further erodes our ability to find effective and feasible solutions and, in the end, erodes our hope too. Tolkien, for all his understanding and smarts, can’t take us all the way.

Thomas Homer-Dixon
Thomas Homer-Dixon is a political scientist and a professor. His latest book, Commanding Hope, from which this has been excerpted and adapted by arrangement with Alfred A. Knopf Canada, a division of Penguin Random House Canada Limited, is out in September.
Alex MacAskill
Alex MacAskill is a Halifax-based printmaker, graphic designer, and illustrator.