Paul Krugman: Asimov's Foundation novels grounded my economics

There are certain novels that can shape a teenage boy's life. For some, it's Ayn Rand's Atlas Shrugged; for others it's Tolkien's The Lord of the Rings. As a widely quoted internet meme says, the unrealistic fantasy world portrayed in one of those books can warp a young man's character forever; the other book is about orcs. But for me, of course, it was neither. My Book – the one that has stayed with me for four-and-a-half decades – is Isaac Asimov's Foundation Trilogy, written when Asimov was barely out of his teens himself. I didn't grow up wanting to be a square-jawed individualist or join a heroic quest; I grew up wanting to be Hari Seldon, using my understanding of the mathematics of human behaviour to save civilisation.

OK, economics is a pretty poor substitute; I don't expect to be making recorded appearances in the Time Vault a century or two from now. But I tried.

So how do the Foundation novels look to me now that I have, as my immigrant grandmother used to say, grown to mature adultery? Better than ever. The trilogy really is a unique masterpiece; there has never been anything quite like it. By the way, spoilers follow, so stop reading if you want to encounter the whole thing fresh.

Maybe the first thing to say about Foundation is that it's not exactly science fiction – not really. Yes, it's set in the future, there's interstellar travel, people shoot each other with blasters instead of pistols and so on. But these are superficial details, playing a fairly minor part in the story. The Foundation novels are about society, not gadgets – and unlike, say, William Gibson's cyberpunk novels, which are excellent in a very different way, they're about societies that don't seem much affected by technological progress. Asimov's Galactic Empire sounds an awful lot like the Roman Empire. Trantor, the empire's capital, comes across as a sort of hyper-version of Manhattan in the 1940s. The Foundation itself seems to recapitulate a fair bit of American history, passing through Boss Tweed politics and Robber Baron-style plutocracy; by the end of the trilogy it has evolved into something resembling mid 20th-century America – although Asimov makes it clear that this is by no means its final state.

Let me be clear, however: in pointing out the familiarity of the various societies we see in Foundation, I'm not being critical. On the contrary, this familiarity, the way Asimov's invented societies recapitulate historical models, goes right along with his underlying conceit: the possibility of a rigorous, mathematical social science that understands society, can predict how it changes, and can be used to shape those changes.

That conceit underlies the whole story arc. In Foundation, we learn that a small group of mathematicians have developed "psychohistory", the aforementioned rigorous science of society. Applying that science to the all-powerful Galactic Empire in which they live, they discover that it is in fact in terminal decline, and that a 30,000-year era of barbarism will follow its fall. But they also discover that a carefully designed nudge can change that path. The empire can't be saved, but the length of the coming dark age can be reduced to a mere millennium.

The novels follow the unfolding of that plan. For the first book and a half – Foundation and the first half of Foundation and Empire – all goes well. Then the plot takes a swerve, as the plan goes off course, only to be put back on track by the mysterious Second Foundation in the eponymous third novel.

Described that way, the story can sound arid and didactic. And the truth is that if you're looking for richly nuanced character development, you should go read Anna Karenina. Asimov was actually better than many science-fiction authors at creating interesting individuals – as a teenager I had a crush on Arkady Darell, the firecracker teenaged sort-of heroine of the trilogy's conclusion – but that's not saying much.

For that matter, you'll also be disappointed if you're looking for shoot-em-up action scenes, in which Han Solo and Luke Skywalker destroy the Death Star in the nick of time. There's only one brief description of a space battle – and the true purpose of the battle, we learn, is not the defeat of an ultimately trivial enemy but the creation of a state of mind that serves the Plan. There is, to be fair, one scene in which the fate of the galaxy hinges on the quick action of a hero (or actually heroine – Bayta Darell, at the end of Foundation and Empire). But even then it's not conventional action writing: Bayta saves the day at the very last minute by shooting one of the good guys.

Yet despite their lack of conventional cliffhangers and, for the most part, either heroes or villains, the Foundation novels are deeply thrilling – suspenseful, engrossing, and, if I may say, bracingly cynical. For the absence of conventional cliffhangers doesn't mean an absence of unconventional cliffhangers.

In the first book-and-a-half there are a series of moments in which the fate of the galaxy seems to hang in the balance, as the Foundation faces the apparent threat of extinction at the hands of barbarian kings, regional warlords, and eventually the decaying but still powerful empire itself. Each of these crises is met by the men of the hour, whose bravery and cunning seem to offer the only hope. Each time, the Foundation triumphs. But here's the trick: after the fact, it becomes clear that bravery and cunning had nothing to do with it, because the Foundation was fated to win thanks to the laws of psychohistory. Each time, just to drive the point home, the image of Hari Seldon, recorded centuries before, appears in the Time Vault to explain to everyone what just happened. The barbarians were never going to prevail, because the Foundation's superior technology, packaged as religion, gave it the ability to play them off against each other. The warlord's weapons were no match for the Foundation's economic clout. And so on.

This unique plot structure creates an ironic resonance between the Foundation novels and a seemingly unrelated genre, what I'd call prophetic fantasy. These are novels – Robert Jordan's Wheel of Time cycle comes to mind – in which the protagonists have a mystical destiny, foreshadowed in visions and ancient writings, and the unfolding of the plot tells of their march toward that destiny. Actually, I'm a sucker for that kind of fiction, which makes for great escapism precisely because real life is nothing like that. The first half of the Foundation series manages, however, to have the structure of prophecy and destiny without the mysticism; it's all about the laws of psychohistory, you see, and Hari Seldon's prescience comes from his mathematics.

Yet if the Foundation books are a tale of prophecy fulfilled, it's a very bourgeois version of prophecy. This is no tale of the secret heir coming into his heritage, of the invincible swordsman winning the day with his prowess. Asimov clearly despises both aristocracy and militarism; his heroes, such as they are, are unpretentious and a bit uncouth, with nothing martial about them. "Violence is the last refuge of the incompetent," declares Mayor Salvor Hardin.

But wait: Foundation isn't about the triumph of the middle class, either. We never get to see the promised Second Empire, which may be just as well, because it probably wouldn't be very likeable. Clearly, it's not going to be a democracy – it's going to be a mathematicized version of Plato's Republic, in which the Guardians derive their virtue from the axioms of psychohistory. What this means for the books is that while a relatively bourgeois society may be the winner in each of the duels, Asimov is neither endorsing that society nor giving it a special long-run destiny. What this means for the storytelling is that the struggles don't have to be and aren't structured as a conventional tale of good guys versus villains, and the novels have that unexpected cynicism. The Foundation may start out a lot nicer than its barbarous neighbours, but it evolves over time into a corrupt oligarchy – and that's all part of the plan. And because the story arc is about the fulfilment of the Seldon Plan, not the triumph of the men in white hats, Asimov is also free to make some of his villains not especially villainous. Bel Riose, the imperial general who menaces the Foundation, is more appealing than the plutocrats running the place at the time. Even the Mule, who endangers the whole plan, is a surprisingly sympathetic character.

Which brings us to the Mule, the deus ex mutagen who drives the swerve in the plot halfway through the series. When I first read Foundation all those years ago, I resented the Mule's appearance, which interrupts the smooth tale of psychohistorical inevitability. On a reread, however, I see that Asimov knew what he was doing – and not just because another book and a half of Seldon Crises would have gotten very stale.

The Mule is a mutant whose ability to control others' emotions lets him conquer the Foundation and threaten the whole Seldon Plan. To contain the menace, the Second Foundation – a hidden group of psychohistorians, the secret keepers of the Plan – must emerge from hiding. So far, this sounds like any of a hundred tales of the struggle between good and evil. But Foundation isn't that kind of series. The problem, you see, isn't how to defeat the Mule and ensure the triumph of truth, justice, and the Foundation way. It is, instead, to get the Plan back on track – and that requires making sure that nobody understands the Plan!

So the Mule (who, as I said, isn't an entirely unsympathetic character) must be defeated, but the defeat must be subtle – no dramatic space battles, no victory parade, in fact no obvious defeat at all. Characteristically for the whole series, the accomplishment of the Mule's quiet defeat itself depends crucially on his not understanding the need for subtlety: he must believe that the Second Foundation is planning the very kind of shoot-em-up denouement that it must in fact avoid.

Even so, the Second Foundation has shown a bit of its hand – so the final episode concerns the confrontation between the First and Second Foundations, a confrontation that the Second Foundation must win by appearing to lose. For the restoration of the Seldon Plan requires the cultivation of a proper state of ignorance; the First Foundation must unlearn its dangerous knowledge of the Second Foundation's influence, and this can only be achieved through the Second Foundation's apparent destruction.

Oh, and the surprise in the very last line of the whole series still brings a smile to my face.

Are there flaws in the Foundation novels? Of course there are. The characters are, by and large, two-dimensional cardboard cutouts. There's also a notable lack of physical description of the characters or, well, anything. As I said, Tolstoy this isn't. A nerdier gripe – indeed, a very, very nerdy gripe – is that, in imposing his historical templates on the galactic civilisation, Asimov clearly had a problem with scale. Tazenda, in Second Foundation, is supposed to be a more or less barbarian kingdom, a flyspeck polity that only rules 20 planets. Um, 20 planets? Then there's Trantor, the world completely covered in metal because its 75m square miles of land surface area must bear 40 billion people. Do the math, and you realize that Trantor as described has only half the population density of New Jersey, which wasn't covered in metal the last time I looked out my window.

But these are, as I said, nerdy concerns. After all, the Foundation novels aren't really about the galaxy, or even about space travel. They're about the true final frontier – understanding ourselves, and the societies we make.

A non-nerdy concern – or anyway, a less nerdy concern – would be this: Now that I'm a social scientist myself, or at least as close to being one as we manage to get in these early days of human civilisation, what do I think of Asimov's belief that we can, indeed, conquer that final frontier – that we can develop a social science that gives its acolytes a unique ability to understand and perhaps shape human destiny?

Well, on good days I do feel as if we're making progress in that direction. And as an economist I've been having a fair number of such good days lately.

I know that sounds like a strange claim to make when the actual management of the economy has been a total disaster. But hey, Hari Seldon didn't do his work by convincing the emperor to change his policies – he had to conceal his project under a false front and wait a thousand years for results. Now, there isn't, to my knowledge, a secret cabal of economists with a thousand-year plan to save our current civilisation (but then I wouldn't tell you if there was, would I?). But I've been struck these past several years by just how much power good economics has to make correct predictions that are very much at odds with popular prejudices and "common sense".

To take a not at all arbitrary example, a standard macroeconomic approach, the IS-LM model (don't ask) told us that under depression-type conditions like those we're experiencing, some of the usual rules would cease to apply: trillion-dollar budget deficits wouldn't drive up interest rates, huge increases in the money supply wouldn't cause runaway inflation. Economists who took that model seriously back in, say, early 2009 were ridiculed and lambasted for making such counterintuitive assertions. But their predictions came true. So yes, it's possible to have social science with the power to predict events and, maybe, to lead to a better future.

That said, it's a long way from getting the medium-term path of interest rates and inflation more or less right to predicting the overall course of civilisation centuries in advance. Asimov's psychohistory evidently integrates economics with political science and sociology, which are much harder subjects than economics – economics is, after all, largely about greed, while other social sciences have to deal with more complex emotions. There are wonderful, insightful political scientists and sociologists working today, but their fields have yet to develop even the (very limited) degree of intellectual integration that makes doing economics sometimes feel like we're living in at least the very early dawn of Hari Seldon's psychohistory.

But maybe those fields will come along too. Will we then be ready to start making recordings for the Time Vault? Actually, no – and I think never. If there eventually is a true, integrated social science, it will still be a science of complex, nonlinear systems – systems that are chaotic in the technical sense, and hence not susceptible to detailed long-run forecasts. Think of weather forecasting: no matter how good the models get, we're never going to be able to predict that a particular storm will hit Philadelphia in a particular week 20 years from now. I'm willing to believe in faster-than-light travel; I'm not willing to believe that Hari Seldon can time his recorded appearance to coincide precisely with the latest crisis between Terminus and its neighbours.

But like the cardboard characters, this little implausibility in the Foundation novels matters not at all. They remain, uniquely, a thrilling tale about how self-knowledge – an understanding of how our own society works – can change history for the better. And they're every bit as inspirational now as they were when I first read them, three-quarters of my life ago.