If there are rules, it isn't magic, it's counterfactual physics. And this is not a trivial or technical distinction but goes to the heart of the structure and meaning of stories, and particularly the nature and role of fairytales in our literature.
Evan Þ recently published an interesting essay on the vanishing distinction between fantasy and science fiction, The Thin Line Between Science Fiction and Fantasy. One of the key issues he raises is that magic in fantasy is increasingly being treated as a system with elaborate and complex rules. But if magic obeys rules, how do you make a distinction between counterfactual physics in science fiction and systematized magic in fantasy? They are both essentially the same thing: counterfactual natural laws. How then do you decide which counterfactual laws fall on the side of fantasy and which on the side of science fiction, particularly when some, such as mind reading, frequently occur in both?
Counterfactual natural laws are not what magic has traditionally been in our literary culture. Magic, traditionally, is the manifestation of chaos in its eternal battle with order. Once you endow it with rules, it becomes an aspect of order, not chaos, and therefore ceases to be magic and becomes counterfactual physics. Arthur C. Clark said that any sufficiently advanced technology will seem like magic. By the same token, in an orderly universe, that which seems like magic must, in fact, be an advanced technology explainable by laws of physics not yet known to us. If magic has laws, it isn’t magic, it is counterfactual physics.
Now you might object that I don’t own the language and that if you want to call the counterfactual physics in your story “magic” you have a perfect right to do so. And you do. It is common in language that we don’t maintain separate terms for everything, but reuse common words to refer to the main things we are interested in. This is the source of much economy in language and also of many lengthy and fruitless disputes about what words “really mean.” I’m not trying to claim the exclusive right to define the word magic, but to make clear the distinction between what, if we want to use the same term for both, we might call ruly and unruly magic.
The distinction between ruly and unruly magic matters because their literary uses are profoundly different, and I think it is important to recognize and appreciate the role that unruly magic has played in the history of our culture, and the role I think it should continue to play today.
In the fairy tale tradition, magic is a manifestation of chaos. Fantasy, in the modern sense of the word, is a very recent genre. It borrows many props from fairytales but is actually quite different. In fairy tales, you will very seldom find a protagonist endowed with any kind of magical powers. The protagonists are, for the most part, ordinary folk: milkmaids and woodcutters and tinkers and cobblers imperiled by the chaos of magic or magical creatures. Even when princes and princesses appear, usually as symbolic representations of virtue, they are not magical and are imperiled in the same way. Magic and all manner of magical folk, from witches to elves to fairies to dragons, are all manifestations of chaos.
After all, these tales originate from the stories of ordinary people living on the edges of a wild world, stories they told to express their fears in a dangerous world that they did not fully understand. In Yeats's great poem, The Stolen Child, fairies are cradle robbers, expressing the fears of every parent of waking to find the cradle empty or the child cold, fears expressed in countless fairytales. Fairytales are tales of the things that go bump in the night, the wild things, the chaotic things, the jaws that bite, the claws that catch.
This is different from much mainstream fantasy and science fiction. Science fiction and fantasy have long been joined at the hip, chiefly because they seem to attract the same audience. The association is so close that they are commonly referred to by the abbreviation SFF, and there are countless SFF writer’s groups, conferences, and expositions. So it is little wonder that the border between them is somewhat porous, as Evan Þ notes. What links them, fundamentally, is that they are both concerned with power, with the acquisition and manipulation of power, or, to put it another way, with mastery.
But chaos, by its nature, cannot be mastered. A literature concerned with mastery cannot have unruly magic. If there is magic, it must be magic that can be mastered, because the story is about mastery. It must have ruly magic. You can keep calling it magic if you want, but ruly magic is a counterfactual physics, which is exactly the problem that Evan Þ notes as the difficulty in separating science fiction from fantasy since there is lots of counterfactual physics in science fiction. But whether the physics is factual or counterfactual, the point of the story is to master it.
In the past, I have made a distinction between what I called Promethean and lapsarian fantasy, the distinction being that in the former, power was a good to be acquired and manipulated and in the latter power was something to be feared and shunned. In a Promethean fantasy, power is stolen from the gods, as in the myth of Prometheus. In a lapsarian fantasy, power is a temptation from the devil, as in the serpent in the garden in the Eden story. As one commenter on that essay shrewdly pointed out, science fiction, which is also about power, tells a different story. Power is neither a temptation of the devil, nor is it stolen from the gods. Power is acquired by the ingenuity of man without the assistance of either gods or devils. But this leaves science fiction and Promethean fantasy in the same camp when it comes to mastery. Whether the power they are concerned with is discovered by human ingenuity or stolen from the gods, it is still ruly power and triumph comes from mastering it. The attempt to master the unruly magic of a Lapsarian fantasy or fairytale can only lead to disaster.
The Lord of the Rings, the supposed progenitor of modern fantasy, is, as I argued in that essay, an example of lapsarian fantasy. The ring is an instrument of chaos. It is unruly magic. It does not conform to the law. It cannot be mastered. It corrupts all who hold it. This is why both Gandalf and Galadriel refuse to take it. They know it will corrupt them. The ring is finally defeated not by any power or cunning or mastery by any of the fellowship of the ring, but by its own contradictions, in a biting scratching fight between two creatures it has corrupted.
In this respect, it would be more accurate to call The Lord of the Rings a fairytale rather than a fantasy, since unruly magic is the stuff of fairytales. And it is worthy of note that when Tolkien wrote his essay On Fairy Stories in defense of the kinds of books that he and his friends were writing, he did not call them fantasies1. He called them fairy stories. And his concern was not to draw a distinction between science fiction and fantasy, but between fairy stories and what we might call the modern novel — the kind of novels read and written by the people who looked down their noses at Lewis and Tolkien for writing fairy stories.
Tolkien was not the first to draw this distinction, nor the first to express a preference for fairy stories. G. K. Chesterton wrote:
Folk-lore means that the soul is sane, but that the universe is wild and full of marvels. Realism means that the world is dull and full of routine, but that the soul is sick and screaming. The problem of the fairy tale is—what will a healthy man do with a fantastic world? The problem of the modern novel is—what will a madman do with a dull world? In the fairy tales the cosmos goes mad; but the hero does not go mad. In the modern novels the hero is mad before the book begins, and suffers from the harsh steadiness and cruel sanity of the cosmos.
This distinction, between the literature of a sane man in a mad world and that of a mad man in a sane world, seems to me a far more important and consequential one than any that we might draw between science fiction and fantasy, and I will return to this idea in later essays. For now, let’s focus on the sane man in a mad world. (By the way, it is interesting that the AI-generated image for “unruly magic” above also fits the idea of a sane man in a mad world quite eerily well.)
The key point here is that a sane man in a mad world cannot succeed by mastery, since a mad world refuses to be mastered. If they are to succeed, it must be by some means other than mastery, and those means are either virtue or an appeal to a higher order (though an appeal to higher order is unlikely to be heard if one does not also show virtue). The appeal to a higher order often results in what Tolkien dubbed a eucatastrophe — a good catastrophe.
In Snow White, Snow White, having bitten the poisoned apple given to her by the wicked queen, is revived when one of the prince’s men carrying her glass coffin stumbles, and the shock causes the poisoned apple to dislodge from her throat, reviving her. This is a eucatastrophe. To trip while carrying a glass coffin is normally catastrophic, but here it has a virtuous effect.
The Disney version rather spoils this, as Disney versions of things usually do, by having her revived by the prince’s kiss, a piece of simple romanticism. But even with this we still have a case of triumph by virtue and an appeal to a higher order, the prince being a symbol of virtue (as is Snow White’s name) and the kiss the symbol of the higher order of love. The prince does not save Snow White either by prowess in battle or by the mastery of greater magic than that of the magic mirror or the queen, nor does he know that his kiss will revive her. He triumphs because he is a prince (a symbol of virtue) and because he loves.
The appeal to higher order is the key to Aslan’s victory over the White Witch in The Lion, the Witch, and the Wardrobe. “Though the witch knew the Deep Magic, there is a magic deeper still which she did not know.” The witch here represents not only chaos, but mastery as well, and both are overcome not by greater mastery but by an appeal to a higher order. The death of Aslan thus becomes again a case of eucatastrophe.
In Lady Isabel and the Elf Knight, I deliberately set out to write a story of unruly magic. Indeed, to emphasize the point, one character is mocked and scolded for his attempt to understand how the magic works. The inspiration for this approach, and my realization of the importance of this distinction, goes back to the genesis of the novel. I have occasionally written short stories based on folk songs (like this one) but the one I wrote based on the ballad Lady Isabel and the Elf Knight didn’t work. It is an interesting ballad, in that it demonstrates that there is nothing new about the trope of the princess who saves herself without the help of a prince, but it lacks a real character arc for its heroine. One reader suggested that since the ballad leaves Isable alone in the greenwood with the Elf Knight’s Horse, Horn, and Sword, I could write a story about how Isabel takes them and uses them to right wrongs like a knight errant. That idea did not click for me, but its inverse did. What if the Elf Knight’s Horse, Horn, and Sword, like Tolkien’s ring, are unruly magic that corrupt all who hold them? Then I had a story. And the ending… but no, no spoilers. You can get a copy here.
While magic is generally bad and protagonists are almost never magical themselves, there are also good magical characters in fairytales. The fairy godmother in Cinderella, for example. And while we should not expect every story to perfectly fit any analytical schema, I think it is reasonable to say that such characters are generally representatives of a higher order. They appear to reward virtue and punish vice, and when they perform magic on behalf of the protagonist, it generally comes with limits and rules. Thus Cinderella must leave the ball before midnight before her coach turns into a pumpkin.
This does not mean that there is no effort or ingenuity involved in the success of fairytale characters. Hansel and Gretel is resolved by an act of peasant shrewdness when Gretel, tricks the witch into demonstrating how she wants her to lean into the oven, at which point Gretel pushes the witch herself into the oven. But there is no mastery of magic involved here.
Admiration of cunning runs through the fairytale tradition, as exemplified, for instance, in the Brer Rabbit stories, which I think we might fairly regard as a particular strain of the appeal to a higher order, at least in the sense that there is a pridefulness in unruly magic that may be defeated by peasant cunning or even simple persistent love, as when the queen in Rumplestiltskin saves her child by searching the forest for Rumplestiltskin’s house and there overhears him gleefully singing that she will never guess that his name is Rumplestiltskin. Here again, the pridefulness of unruly magic is its undoing in the face of simple love.
To be sure, wizards and witches are often portrayed as learned in magic, though their magic is almost never explained to the reader. But we should notice that it is also a frequent trope that the supposed master often loses control, with the magic that they thought they controlled proving to be chaotic after all. Examples include A Wizard of Earthsea and Fantasia. In many cases, such stories contain not only a warning against unleashing chaos but also a warning against the vanity of mastery. Darth Vader, seeking mastery, becomes corrupted by the dark side of the Force.
We might see in this a kind of literary cosmology in which the ordinary world of ruly physics lies between a miraculous world of higher order above and a magical world of chaos below. This is, of course, a reflection of Christian cosmology, and of the medieval worldview that C.S. Lewis describes in The Discarded Image. But I would suggest that as a literary cosmology, as the cosmology of fairytales, it has an independent existence and an independent purpose.
It is, for instance, the cosmology of Star Wars in which Luke, in his attempt to destroy the Death Star, abandons the attempt at mastery, and the machines that go with it, and surrenders to the higher order which is the Force. The question of whether Star Wars is science fiction or fantasy may be a vexing one, but the question of whether it is science fiction or fairytale is simple to answer: it is a fairytale.
Authors can play with these elements in all kinds of ways, of course. For example, the mad world is not always a servant of chaos. Through a literary process that we might call trope inversion, the mad world can become a servant of virtue, as in the Coyote and Road Runner cartoons, where what we might call unruly physics constantly works to frustrate the Coyote’s plans.
No matter how these elements are mixed and matched, though, what really matters is the different ways in which victory is won in a story. This is what really determines what kind of story it is, regardless of its accidental features. Is victory won by competence, by virtue, by force, by courage, or by a eucatastrophe? It is the chosen ending that determines the roles played by every element of the story that comes before. It also raises interesting questions about what constitutes a satisfying ending.
Endings are an odd thing in themselves. Unless you have an ending like in Hamlet, where everybody dies, the lives of the protagonists will go on. They will live happily ever after, or not. What then makes this point in their lives the ending of a story? What makes it satisfying to end the tale here as opposed to sooner or later? Is it a foe defeated? A task accomplished? A love won? A grail found? Not really, I would suggest. If the foe are defeated because they forget to bring their muskets to the battle, if the task is accomplished by reading the manual, if love is won because the rival fell into a crevasse on that romantic skiing trip to the Alps, if the grail was found by the maid at the back of the pantry, none of these endings will be satisfactory.
What makes an ending satisfactory, I submit, is that it feels deserved. The foe are defeated because our hero found his courage, or devised a cunning plan, or correctly executed the maneuver he had struggled to master. The task is completed because of the engineering acumen of the protagonist. The love is won because the protagonist has proven himself the more worthy suitor. The grail is found because the knight demonstrated his purity and courage and piety.
Not every story ends happily, of course. But even in tragedy, an ending feels satisfactory if it feels deserved. And while we might sometimes feel that the protagonist of a tragedy deserved their ill fate because of their conduct, more often we feel that they deserved more because of their heroic sacrifice. Thus the whisky priest in Graham Greene’s The Power and the Glory returns to Tobasco to attend a dying man who has asked for him, knowing it is a trap, and knowing that it will lead to his death. But it is satisfying because this is a sign, an undeniable sign, that he has triumphed at last in the struggle within himself and has embraced his priestly responsibilities, even in the face of death.
In some sense, it is the triumph over self that makes every ending satisfactory in one form or another. A soldier finding his courage is a triumph over self. A lover proving himself worthy of his beloved is a triumph over self. Even an engineer devising a way to get his starship out of an alien trap is usually, in some minor way, configured as a triumph over self. This is why a eucatastrophic ending can be satisfactory, much as it smacks of deus ex machina. Because a eucatastrophic ending, though the triumph itself is a matter of chance, is deserved by the displayed merits of the protagonist.
And this is why the fairytale structure of a sane man in a mad world, a world of unruly magic, can provide a satisfying ending. It is the kind of ending that is satisfying based on virtue alone, without any necessary appeal to competence or mastery. It is a tale that places moral victory, the ultimate victory over the self, at the center of the story.
This is not at all to valorize simple moralizing tales or naive hagiographies. They are unsatisfying precisely because they fail to take the struggle seriously. The sane man in a mad world is still a sinner, and the mad world is full of temptations. The whisky priest is a sane man on the edge of madness whose triumph is not that he preserves his life but that he preserves his sanity, and thus acts as he knows he must.
In our cynical age, we are shy about moral merit in stories. We are more comfortable admiring, and offering for admiration, technical merit. Cleverness, good engineering, and even low cunning are more comfortable kinds of merit with which to weave a deserved and therefore satisfying ending. A gloss of moral merit is often applied over this core of technical merit, generally through the insertion of a save-the-cat moment. But eucatastrophic endings merited by virtue alone are hardly in favor. Thus in stories that involve magic, we prefer a ruly magic, a magic that can be overcome through competence and mastery rather than virtue or an appeal to a higher order.
Sometimes, I suppose, an author devises a magic system for the sheer pleasure of invention, and a reader may enjoy reading about it for the sheer pleasure of uncovering its intricate mechanisms, something akin to the pleasure of discovering the inner workings of a watch. But it seems to me worth thinking about for writer and reader alike whether ruly or unruly magic is really what the story requires, and whether an ending that feels deserved for moral rather than technical merit is what is really wanted.
In the essay, Tolkien uses the word fantasy to describe an artistic device rather than a genre, and as one of several things that fairy stories can achieve.
Mastery of a subject is not without some intrinsic virtue, or at least not to those with master's degrees. Having never gone to college, I'm not sure if I personally would be a fan of virtuous power through academics.
As a young convert, I once set out to write a blog post on (pardon the terms) Catholic magic and Protestant magic. The matter is more complicated than that, of course, and now that Catholicism is no longer shiny and new I can see more subtle distinctions.
But the original distinction is that Protestantism reduces things to systems, whereas Catholicism functions on symbols. So in the Protestant fantasy, even from those who are no longer officially Protestant, the magic can be reduced to its component parts. Compare to Tolkien, where, say, the One Ring is a symbol, recalling Plato's anarchic fear of an invisible man.
I don't think this loss was intentional, and by the efforts of many writers to somehow bring the magic back, not desired even today. Jack Vance wrote a world where magic was powerful, arbitrary, and very dangerous; his heirs in D&D reduced all that to logistics. Such is engineering.
I think there is room for those writing the more mythic magics to make a comeback, since the absence of myth is so keenly felt. In my C&D series, a LitRPG, I tried adding back some of the uncontrolled symbolic aspects of the myths that had lost the original power, and my audience loved it.
But in defense of Promethean fantasy, it is not intrisically wrong to steal fire from the gods if the gods have been unjustly keeping it from the mortals. Indeed, the question is the same as in comic books: if you have the power, can you use it justly, as compared to the villains, who do not? Literature demands the villian is more powerful than the hero, although whether this state of affairs lasts is up to the author.
But to return to my original distinction, there are more things in Heaven and Earth than are dreamt of in such a division. Brandon Sanderson, a Mormon, writes the most systematic systems, but frequently it is the weaklings who win. Japanese fiction is still pagan, at least for the moment--the Death Note follows an elaborate system of rules, yet it is as much a cautionary tale about the pursuit of power as any pagan myth. And I often find myself straddling the lines.
Interesting discussion. I admit I'm not very familiar with "fantasy" literature beyond Tolkein, so I can't really comment on those magical systems. But I think, despite our best efforts, rules are hard to apply to the supernatural generally, which is probably where much of the anxiety around religious behavior comes from: how should we conduct ourselves in order to not provoke these unpredictable and chaotic unseen forces (or maybe gain their favor)? Fairy stories highlight the mysteriousness and danger associated with capricious supernatural powers, which humans aren't meant to meddle with (the occult version of learning not to touch hot stoves.) Once a vague understanding of those forces and that behavior has been achieved, though, there have always been attempts to gain some control over the chaos with magic (cooking on hot stoves), whether to make the crops grow, cure illness, or give a neighbor the evil eye. Supernatural means to supernatural ends...
Though it's not quite the same kind of magic you're referring to here (story magic is often a different animal than what people practiced in real life), I always liked this perspective on magic vs science from an anthropological perspective:
"The fatal flaw of magic lies not in its general assumption of a sequence of events determined by law, but in its total misconception of the nature of the particular laws which govern that sequence. If we analyse the various cases of sympathetic magic which have been passed in review in the preceding pages, and which may be taken as fair samples of the bulk, we shall find, as I have already indicated, that they are all mistaken applications of one or other of two great fundamental laws of thought, namely, the association of ideas by similarity and the association of ideas by contiguity in space or time. A mistaken association of similar ideas produces homoeopathic or imitative magic: a mistaken association of contiguous ideas produces contagious magic. The principles of association are excellent in themselves, and indeed absolutely essential to the working of the human mind. Legitimately applied they yield science; illegitimately applied they yield magic, the bastard sister of science. It is therefore a truism, almost a tautology, to say that all magic is necessarily false and barren; for were it ever to become true and fruitful, it would no longer be magic but science. From the earliest times man has been engaged in a search for general rules whereby to turn the order of natural phenomena to his own advantage, and in the long search he has scraped together a great hoard of such maxims, some of them golden and some of them mere dross. The true or golden rules constitute the body of applied science which we call the arts; the false are magic."
-James George Frasier, The Golden Bough