The perfect essayist: Joan Didion in Golden Gate Park, San Francisco, in 1967 

Why do novelists write essays? Most publishers would rather have a novel. Bookshops don’t know where to put them. It’s a rare reader who seeks them out with any sense of urgency. Still, in recent months Jonathan Safran Foer, Margaret Drabble, Chinua Achebe and Michael Chabon, among others, have published essays, and so this month will I. And though I think I know why I wrote mine, I wonder why they wrote theirs, and whether we all mean the same thing by the word “essay”, and what an essay is, exactly, these days. The noun has an unstable history, shape-shifting over the centuries in its little corner of the OED.

For Samuel Johnson in 1755 it is: “A loose sally of the mind; an irregular undigested piece; not a regularly and orderly composition.” And if this looks to us like one of Johnson’s lexical eccentricities, we’re chastened to find Joseph Addison, of all people, in agreement (“The wildness of these compositions that go by the name of essays”) and behind them both three centuries of vaguely negative connotation. Beginning in the 1500s an essay is: the action or process of trying or testing; a sample, an example; a rehearsal; an attempt or endeavour; a trying to do something; a rough copy; a first draft. Not until the mid 19th century does it take on its familiar, neutral ring: “a composition more or less elaborate in style, though limited in range.” Which is it, though, that attracts novelists – the comforts of limit or the freedom of irregularity?

A new book by the American novelist-essayist David Shields (to be published here by Hamish Hamilton early next year) makes the case for irregularity. In Reality Hunger: A Manifesto Shields argues passionately for the superiority of the messy real – of what we might call “truthiness” – over the careful creations of novelists, and other artists, who work with artificial and imagined narratives. For Shields it is exactly what is tentative, unmade and unpolished in the essay form that is important. He finds the crafted novel, with its neat design and completist attitude, to be a dull and generic thing, too artificial to deal effectively with what is already an “unbearably artificial world”. He recommends instead that artists break “ever larger chunks of ‘reality’ into their work”, via quotation, appropriation, prose poems, the collage novel … in short, the revenge of the real, by any means necessary. And conventional structure be damned. To make the point, Reality Hunger is itself without obvious authorial structure, piecing its arguments together by way of scattered aphorisms and quotation, an engaging form of bricolage. It’s a tribute to Shields’s skill that we remain unsure whether the entire manifesto is not in effect “built” rather than written, the sum of many broken pieces of the real simply shored up and left to vibrate against each other in significant arrangement. The result is thrilling to read, even if you disagree with much of it, as I do.

A deliberate polemic, it sets what one could be forgiven for thinking were two perfectly companionable instincts – the fictional and non-fictional – at war with each other. Shields likes to say such things as “Story seems to say everything happens for a reason, and I want to say No, it doesn’t”; to which I want to say, “Bad story does that, yes, but surely good story exists, too”. Anyway, there’s a pleasure to be had reading and internally fighting with Shields’s provocations, especially if you happen to be a novelist who writes essays (or a reader who enjoys both). The pages are filled with anti-fiction fighting talk: “The creators of characters, in the traditional sense, no longer manage to offer us anything more than puppets in which they themselves have ceased to believe.” And: “All the best stories are true.” And: “The world exists. Why recreate it?” It’s tempting to chalk this up to one author’s personal disappointments with the novel as a form (Shields hasn’t written a novel since the early 90s), but in expressing his novel-nausea so frankly he hopes to show that he is not alone in having such feelings – and my sense is that he’s right.

An excited American writing student gave me a proof copy of the book, and during a recent semester spent teaching I met many students equally enthused by Shields’s ideas. Of course, it’s easy to be cynical about this kind of student enthusiasm. Generally speaking, there are few things more exciting to a certain kind of writing student than the news that the imaginative novel is dead (with all its vulgar, sentimental, “bourgeois” – and hard to think up – plots, characters and dialogue). When your imagination fails you it’s a relief to hear that it need no longer be part of a novelist’s job description. But if “cui bono?” is a reasonable question to ask of writing students who may fear fiction is beyond them, who benefits when it is the novelists themselves who are grave-dancing?

I ask because Reality Hunger comes with “advance praise” from an impressive clutch of imaginative writers – Jonathan Lethem, Geoff Dyer, Tim Parks, Charles D’Ambrosio and Rick Moody, among others – all apparently eager to commit literary hara-kiri. Most striking is the response of John Coetzee, worth quoting in full: “A manifesto on behalf of a rising generation of writers and artists, a ‘Make It New’ for a new century, an all-out assault on tired generic conventions, particularly those that define the well-made novel. Drawing upon a wide range of sources both familiar and unfamiliar, David Shields takes us on an engaging and exhilarating intellectual journey. I enjoyed Reality Hunger immensely and found myself cheering Shields on. I, too, am sick of the well-made novel with its plot and its characters and its settings. I, too, am drawn to literature as (as Shields puts it) ‘a form of thinking, consciousness, wisdom-seeking’. I, too, like novels that don’t look like novels.”

Coetzee is one of our finest novelists, and one whose nausea with the novel’s form grows more evident with each publication. First-person journals, the wholesale importation of the autobiographical, philosophical allegory and the novel disguised as public lecture – he has used all these to circumvent the “well-made novel”, that rather low form of literary activity that even as relatively un-neurotic a novelist as EM Forster found himself defining with a sigh: “Yes – oh dear, yes – the novel tells a story.” But while aesthetic and ethical objections to the “well-made novel” are not difficult to understand, we should be careful not to let old literary pieties be replaced with new ones. This easy dismissal of well-made novels deserves a second look. In the first place, “well-made novel” seems to me to be a kind of Platonic bogeyman, existing everywhere in an ideal realm but in few spots on this earth. Reality Hunger wants us to believe that this taste for “novels that don’t look like novels” is in some way unusual, the mark of a refined literary palate.

But even the most conventional account of our literary “canon” reveals the history of the novel to be simultaneously a history of nonconformity. For as readers we have loved and celebrated not some hazy general idea of the novel but rather the peculiar works of individual imaginations. Even in those familiar lists of “great novels”, classics of the genre, and so on, it’s hard to find a single “well-made” novel among them, if by well-made we mean something like “evenly shaped, regular, predictable and elegantly designed”. Is War and Peace, with its huge tracts of undigested essay, absurd plotting and obscene length, a well-made novel? Is The Trial? And those neat Victorian novels we’re now expected casually to revile – is it not only from a distance, and in the memory, that they look as neat as they do? Which of them is truly “well made”? Jane Eyre seemed hysterical and lopsided to its earliest readers; we now think ofMiddlemarch as the ultimate “proper” novel, forgetting how eccentric and strange it looked on publication, with its unwieldy and unfeminine scientific preoccupations and moral structure borrowed from Spinoza. In our classic novels there always remains something odd, unruly, as distinctly weird as Hardy’s Little Father Time. Novels that don’t look like novels? When it comes to the canon – to steal a line from Lorrie Moore – novels like that are the only novels here. And though it may well be the case that the pale copies of such books to be found in bookshops today are generic and conventional and make the delicate reader nauseous, is the fault really to be found with imagined narrative itself? Will the “lyrical essay”, as Shields calls it, be the answer to the novel’s problems? Is the very idea of plot, character and setting in the novel to be abandoned, no longer fit for our new purposes, and all ground ceded to the coolly superior, aphoristic essay?

In these arguments the new received wisdom is that all plots are “conventional” and all characters sentimental and bourgeois, and all settings bad theatrical backdrops, wooden and painted. Such objections are, I think, sincere responses to the experience of reading bad novels, and I don’t doubt the sincerity of Shields or Coetzee or any writer who responds strongly to Reality Hunger as a manifesto. A bad novel is both an aesthetic and ethical affront to its readers, because it traduces reality, and does indeed make you hunger for a kind of writing that seems to speak truth directly. But I also feel, as someone who just finished a book of more or less lyrical essays, that underneath some of these high-minded objections, and complementary to them, there is another, deeper, psychological motivation, about which it is more difficult to be honest. In “The Modern Essay” Virginia Woolf is more astute on the subject, and far more frank. “There is no room for the impurities of literature in an essay,” she writes. “The essay must be pure – pure like water or pure like wine, but pure from dullness, deadness, and deposits of extraneous matter.” Well, yes, that’s just it. An essay, she writes, “can be polished till every atom of its surface shines” – yes, that’s it, again. There is a certain kind of writer – quite often male but by no means exclusively so – who has a fundamental hunger for purity, and for perfection, and this type will always hold the essay form in high esteem. Because essays hold out the possibility of something like perfection.

Novels, by contrast, are idiosyncratic, uneven, embarrassing, and quite frequently nausea-inducing – especially if you happen to have written one yourself. Within the confines of an essay or – even better! – an aphorism, you can be the writer you dream of being. No word out of place, no tell-tale weak spots (dialogue, the convincing representation of other people, plot), no absences, no lack. I think it’s the limits of the essay, and of the real, that truly attract fiction writers. In the confined space of an essay you have the possibility of being wise, of making your case, of appearing to see deeply into things – although the thing you’re generally looking into is the self. “Other people”, that mainstay of what Shields calls the “moribund conventional novel”, have a habit of receding to a point of non-existence in the “lyrical essay”.

These are all satisfactions the practice of writing novels is most unlikely to provide for you. Perfect essays abound in this world – almost every one of Joan Didion’s fits the category. Perfect novels, as we all know, are rarer than Halley’s comet. And so, for a writer, composing an essay instead of a novel is like turning from staring into a filthy, unfathomable puddle to looking through a clear glass windowpane. How perfectly it fits the frame! How little draught passes through! And naturally writers who feel a strong sense of nausea towards their own fiction are even more likely to feel it when reading the fiction of their peers. It’s hard to read a novel with any pleasure when you can see all the phoney cogs turning. I’m willing to bet that the great majority of proofs sent to novelists by other novelists barely get read beyond the first two pages. (“The Corrections” writes Shields, in aphorism no 560, “I couldn’t read that book if my life depended on it. It might be a ‘good’ novel or it might be a ‘bad’ novel, but something has happened to my imagination, which can no longer yield to the earnest embrace of novelistic form.”) Tired of the rusty workings of one’s own imagination, it’s easy to tire of the wearisome vibrancy of other people’s, and from there it’s a short skip and a jump to giving up on the novel entirely.

Except, except. Then something remarkable comes into your hands. Not very often – no more or less often now than in the 1930s, or the 1890s or the 1750s – but every now and then, you read something wonderful. (Despite all the dull talk of the death of literature, the rate of great novels has always been and will always be roughly the same. By my reckoning, about 10 per decade. Although behind them are dozens of very good novels, for which this reader, at least, is grateful.) Every now and then a writer renews your faith. I’m looking around my desk at this moment for books that have had this effect on me in the not-too-distant past: Bathroom and Television by Jean-Philippe Toussaint, Asterios Polyp by David Mazzucchelli, Number9Dream by David Mitchell, Hilary Mantel’s An Experiment in Love, Dennis Cooper’s My Loose Thread,The Piano Teacher by Elfriede Jelinek, the collected short stories of JG Ballard.

Some people are not condemned to the generic by their use of plot and setting and character. Some people are in fact freed by precisely these things. Whether what they write is disappointingly “well made” I can’t say; certainly there is something a little queer about them all, though that queerness comes not from an excess of the real but from the abundance of their own imaginative gifts. “There is no excellent beauty that hath not some strangeness in the proportion,” wrote Francis Bacon in his essay “Of Beauty”. Well said. This year Ballard’s stories in particular have been a revelation to me, being at once well made, full of the supposedly contemptible components – plot, setting, character – and yet irreducibly strange in proportion. It’s a marvel how implacably and consistently weird he managed to be despite appearing to use all the normal tools at the disposal of any English short-story writer. All in all there is something a little shaming in reading Ballard: you have to face the fact that there exist writers with such fresh imaginations they can’t write five pages without stumbling on an alternate world.

When our own imaginations dry up – when, like Coetzee, we seem to have retreated, however spectacularly, to a cannibalisation of the autobiographical – it’s easy to cease believing in the existence of another kind of writing. But it does exist. And there’s no need to give up on the imaginative novel; we just need to hope for better examples. (In Coetzee’s oeuvre, of course, we have better examples. The fully imagined artistry of novels such as The Life and Times of Michael K and Disgrace offer their readers distinct pleasures, not easily dismissed, and not easily found in those impressive but rather anaemic later works, the essayistic and self-referential Diary of a Bad Year andSummertime.) It may be that this idea of the importation of “more reality” is exactly the call to arms a young writer somewhere at her desk needs at this moment, but for this writer at this desk, the argument feels ontologically dubious. When I turned from my own dirty pond to a clear window, I can’t say that I felt myself, in essence, being more “truthy” in essay than I am in fiction. Writing is always a highly stylised and artificial act, and there is something distinctly American and puritan about expecting it to be otherwise. I call on Woolf again as witness for the defence. “Literal truth-telling,” she writes, “is out of place in an essay.” Yes, that’s it again. The literal truth is something you expect, or hope for, in a news article. But an essay is an act of imagination, even if it is a piece of memoir. It is, or should be, “a form of thinking, consciousness, wisdom-seeking”, but it still takes quite as much art as fiction. Good non-fiction is as designed and artificial as any fairy story. Oddly, this is a thesis Reality Hunger readily agrees with: in its winding way it ends up defining the essay as imaginative at its core, and Shields wants to encourage its imaginative qualities – it seems to be only in the novel that the imagination must be condemned. It’s a strange argument, but I guess the conventional form so many imaginative novels take has been enough to give fictional imagination itself a bad name.

For myself, I know, now that I’ve finished them, that I wrote my own essays out of exactly the kind of novel-nausea Shields describes. I was oppressed by a run-of-the-mill version of that narrative scepticism Kafka expresses so well in one line in “Description of a Struggle”: “But then? No then.” Simply put, my imagination had run dry, and I couldn’t seem to bring myself to write the necessary “and then, and then” which sits at the heart of all imagined narratives. When you’re in this state – commonly called “writer’s block” – the very idea of fiction turns sour. But in a strange circular effect, it has been the experience of writing essays that has renewed my enthusiasm for the things fiction does that nothing else can. Writing essays on Kafka, on Nabokov, on George Eliot, on Zora Neale Hurston, I was newly humbled and excited by the artificial and the fully imagined. The title of the book, Changing My Mind, is meant to refer to the effect great fiction like this always seems to have on me. I once thought, for example, that I didn’t want ever to read another lengthy novel about family life – and then I read The Corrections. That book gave something to me I could never get from an aphoristic personal essay about the nature of art (I think that “something” might be “a convincing imitation of multiple consciousnesses”, otherwise known as “other people”). And vice versa. I don’t think I’m alone in that feeling. As general readers, who thankfully do not have to live within the strict terms of manifestos, we are fortunate not to have to choose once and for all between two forms that offer us quite different, and equally valuable, experiences of writing.

The last essay in my book considers the work of David Foster Wallace, a writer as gifted in fiction as in essay. I can’t offer a better example of a writer whose novel-nausea was acutely developed, whose philosophical objections to the form were serious and sustained, and yet who had the cojones and the sheer talent to write them anyway. Like all great fiction writers he is hard for other writers to read because his natural ability is so evident it makes you nauseous by turn. But that’s fiction for you: it taunts you with the spectre of what you cannot do yourself. Meanwhile, the essay teases you with the possibility of perfection, of a known and comprehensible task that can be contained and polished till it shines. For the reader who cares above all for perfection, there are many sophisticated, beautiful and aphoristic side roads in literature that will lead you safely away from the vulgarity of novels with their plots and characters and settings. Off the top of my head: David Markson’s Reader’s Block, Peter Handke’s The Weight of the World, Raymond Queneau’s Exercises in Style, Georges Perec’s Species of Spaces and Other Pieces and Kafka’s own Blue Octavo Notebooks …

But after you have raged at the impossible artificiality of storytelling, once you have shouted, with Kafka, “But then? No then”, well, maybe you will find yourself returning to the crossroads of “And then, and then”, if only to see what’s going on down there. Because there is a still a little magic left in that ancient formula, a little of what Werner Herzog, talking recently of the value of fiction, described as “ecstatic truth”. And every now and again some very imaginative writer is sure to make that “And then” worth your while.