In Charlie Kaufman’s classic screenplay Adaptation, the main character (also named Charlie Kaufman) is charged with adapting Susan Orleans’ colorful albeit densely layered book The Orchid Thief into film. Unable to find a through line (or spine), the sine qua non of narrative film, the vexed screenwriter moans to his agent, “The book has no story. There’s no story.” Replies the agent blithely, “Make one up.” As anyone who has seen the movie knows, it’s a response that can only deepen Charlie’s already high anxiety. Adaptation is a process that requires an improbable balancing act of fidelity to the author’s original text with sufficient creative freedom for the filmmaker. Torn between his responsibility to the author—and her many readers–and the agent’s and moviegoing public’s expectation of “story”, Charlie spends the rest of the film desperately attempting to pay proper obeisance to others while somehow also asserting himself and his own artistic vision.

Listening to the panel of novelists/screenwriters on stage at the Jack Skirball Center at New York University for the “Adaptation” panel at the PEN World Voices festival on Thursday, I was reminded of this delicious exchange—the bewilderment of the screenwriter, the callousness of the Hollywood agent, and mostly, the utter agony and joy that adaptation entails. The process of adapting books into film, it seems, can be a hellish vortex from which few emerge unscathed.

The authors on stage before us, Philippe Djian, Jean-Philippe Toussaint, Richard Price, Barry Gifford, and moderator Francine Prose, have had varying experiences with the movie jungle. Richard Price begins his comments with, in his own words, a rather “dyspeptic rant” against Hollywood. It quickly becomes clear that this very successful novelist (Clockers, Lush Life) and screenwriter (Sea of Love, The Color of Money) has a Charlie Kaufmanesque love-hate relationship with Tinseltown. Barry Gifford takes a sunnier point of view. A veteran of two films, Wild at Heart and Lost Highway, by the revered, brilliant filmmaker David Lynch–an experience that Gifford acknowledges as uniquely best-case scenario for an author–Gifford is quick to defend as highly rational the parting of Hollywood and novelistic ways. Writing screenplays, he says, is to work in a completely different language.

“The writer,” Gifford explains, “is the pinsetter in the movies.” Gifford’s implicit advice to writers: act accordingly. Screenwriters are set up men, they should not expect to have the bowler’s full control (also implied: authors shouldn’t claim the victim status of the pins.) Philippe Djian, a popular French author whose novel 37°2 le Matin was adapted for film by Jean-Jacques Beineix as Betty Blue (a film that garnered a great deal of attention in indie circles in America) falls closer to Price on the dissatisfaction spectrum. When asked what was lost and what was gained by the cinematic translation of his work, Djian responds, “All was lost.” Jean-Philippe Toussaint, a prize-winning Belgian writer (select titles are available in the U.S. from Dalkey Archive Press) writes screenplays–original and adapted–and has directed his own works. Toussaint possesses perhaps the evening’s most temperate perspective. As a writer-director, Toussaint observes, an unfaithful adaptation would be unfaithful only to him. Infidelity, in his case, is out of the question.

Unsurprisingly, there’s little agreement on the value of the Hollywood experience for novelists. “Stay away,” advises Richard Price. “Take the money and run.” This is a bit ironic coming from Price, clearly the one person on stage who has clearly made a comfortable living from Hollywood. To his credit, however, it’s a highly self-aware statement and Price generates a great deal of laughter by playing on the discordance between “doing what he says” and “doing what he does.” Granting that his stance might be an easier one in France than in America, Philippe Djian chooses to maintain his distance from film. Djian prefers the interior space novels afford. “Things are not comprehensible when they are first experienced,” he says. “They must be ruminated over to be understood … My world [unlike the cinematic world] is one of reflection.” Barry Gifford takes the opposite tack. Hollywood, which Gifford embraces, has meant the welcome relief of a social life. And watching movies all night in hotel rooms when young, he explains, taught him crucial lessons about narration. But plot, moderator Francine Prose interjects, is often the least interesting aspect of novel writing. In response, Gifford pointedly quotes Somerset Maugham, “You have to have a story.” Nuff said.

PEN panels feature writers from different nations and sparks often result. Occasionally cultural divides threaten to overwhelm individual differences. In extreme cases, compatriots withdraw from the multilateral conversation to bilateral exchanges where they feel better understood. This panel is no exception. At times the gulf between the French and the Americans seems impossibly vast. Philippe Djian and Jean-Philippe Toussaint speak of the “abstraction” involved in writing versus the concrete fleshiness of the cinema. For Toussaint, literature is like mathematics–solitary, abstract and cheap–while cinema is like biology–team-based, concrete, and expensive. In stark contrast to Gifford, stories—of which there are only a set number–don’t interest Philippe Djian. Djian prefers to ponder new ideas for communicating the world around him, perhaps through language, perhaps through style. Stories, the basis of film, are merely the skeleton, while literature is “a tool to dig under the surface.” It’s a statement that most literary as opposed to genre writers would probably agree with. And though there are exceptions, writers in France generally take style and ideas as their starting point.

For the Americans, this all seems a bit beside the point. It’s not that Price and Gifford are unsympathetic to the gap that Djian articulates between writing for novels and writing for film, but it’s clear that the theoretical clouds that hang over the French like smoke from Gauloises are alien to the more self-consciously straightforward Americans. “I don’t have a philosophical bone in my body,” says Price. Instead he offers screenwriters practical advice. “Take all the richness that the novel affords you and translate that for the eye and the ear.” It’s a brutal process, he warns. One must strip away all that one knows and relearn the world. Or as Gifford says, “you have to find a different way to tell story. Tell that story in images.” It’s unclear whether Gifford and Price are unconcerned with style (unlikely) or simply see style as something that must inevitably be retranslated for the cinema. Having worked with some of the best filmmakers in America, Gifford and Price acknowledge how dependent they are on the director for an effective translation of their work.

And, of course, as Price points out, it can also depend on “which director shows up that day”—the one that made that wonderful movie five years ago or the one that’s going to make a hash of yours today. It’s also not entirely clear why some of the panelists believe that language must be irretrievably lost in a cinematic translation. Some of the best dialogue written today is in the movies–or plays. Interiority and exposition are clearly casualties of adaptation, but language? Certainly dialogue is only one way that language is employed in a novel. But a great line—and Price himself has created many of them—can also have a power in the movies or theater that it rarely attains in more padded formats such as the book.

Despite their very different personal styles of expression, the Americans and the French soon jettison nation-based commonalities for the distinctiveness of individual experience. Francine Prose mourns the tradeoff between a novelistic focus on consciousness, psychology and language for a filmic focus on plot (a focus on story, she amusingly complains, that reminds her of how the two paragraphs of summary that begin every book review always seem to reduce the novelist’s art to an unrecognizable plot.) Jean-Philippe Toussaint embraces both film and book formats. In a poignant appreciation for the singularity of film, he also observes that as a filmmaker he “feels” more than he does as a novelist. Richard Price similarly talks about being “knocked on his ass” when filmmakers go to his ‘hood and film school walls painted that “St Joseph’s aspirin-colored orange,” an image that at once powerfully recalls and recreates his childhood. Price also pays tribute to the powerful performances of the actors who “sometimes give you a gift from the stars.” Toussaint, Price, and Gifford, all who have been intimately involved in the filmmaking process, make a strong case—in Price’s case, even despite himself–for the power of a fully-realized adapted work.

“Movies owe nothing to the books that they are based on except to be true to the spirit of that book,” concludes Price. On one point at least, Charlie Kaufman can relax. There is plenty of room for both the original and the adaptation in our complex, highly literate, and equally highly visual world. To which the movie and book lovers among us can only add, “Amen.”

What this spirited panel has also clarified, however, is that Kaufman’s anxiety is not entirely misplaced. To do justice to the spirit of a work is no small feat. Much as each author on the stage has a unique take on their art (takes informed by cultural, national, and linguistic context, by complex individual psychologies, by experiences, and more), the screenwriter must find the exact right set of images and narrative lines that render a work “faithful” to these unique authorial alchemies. Watching and listening to the very diverse and powerful personalities on stage, it begins to hit home just how daunting a task adaptation must be. We walk out into the night granting all kudos to the Charlie Kaufmans of the world. The screenwriters and filmmakers who dare to take on the “bastardizing” of a good book have something else. Like all great artists, they have balls. Adapters might succeed beyond an author’s dream or they might fail miserably. Either way, without the adapter’s courage in intimately approaching others even as she simultaneously asserts herself, we audiences, and art itself, would be much poorer as a result.