Fact-check Yourself Before You Fact-wreck Yourself

The Facts

Not too long ago Nancy Updike stopped by my desk while I was thumbtacking fan mail to This American Life back onto a cork board. “The whole thing fell off the wall,” I explained.

“Everything falls on you,” she said. “Cork boards, ceilings, the facts.”

It’s true about the ceiling. I was fact-checking a script when some panels that were water-logged from a leaking air conditioner crashed on my head. I screamed — loudly. Everyone ran to my desk. It was pretty embarrassing. However, what Nancy said about the facts is only partially true. She and the other producers here are an exacting bunch, and stories receive a lot of scrutiny during the edits, but despite all the vetting, I still tend to be skeptical and approach the scripts as if everything is wrong and my job is to fix everything they screwed up. It’s a mindset, a way of reading aggressively. It keeps me dogged (and unpopular. . . though at least I’m right!), but fact-checking is more than perusing logs, Googling stats, and calling it a day. It’s back-end reporting. I pick pieces apart, scrutinize logic, examine assumptions, and make sure stories don’t use stats or tape out of context. I do this by talking to as many people as possible — a producer’s sources but also my own — and reading through reporting and research or academic papers. It’s all in the hope of catching embarrassing mistakes before we broadcast. I realize not every radio producer can have someone like me go through scripts and speak to sources (alas, I really can be such a pain), but there are basic fact-checking practices that can help your reporting be more sound.

Here, then, a few tips to keep the facts from falling on your head.

Get Your Oranges From O.J.

Nancy once did a story about this tasteless prank show, Juiced, which stars O.J. Simpson. In one skit, Simpson, dressed as a homeless man, stood on a street corner with a bag of oranges, saying, “Get your oranges from O.J.! O.J right here!” It’s a bizarre scene, though it actually cracked me up (hey, no judging), but even more oddly, I think it says something about fact-checking. Here’s what I mean: Nancy’s story, in a way, is an examination of Simpson — he’s the subject — and she sets out to come to some sort of understanding about him. She spoke to a number of people who worked on the show, and they told her things like Simpson often showed up drunk and told crude jokes while in the dressing room (since I was her fact-checker, they told me, too), but despite her numerous requests, she never got an interview with him or even a response from his lawyer. This made me (and Nancy) nervous. When fact-checking, you want to run facts past anyone appearing in your story. You want to get your oranges from O.J. You want their reactions. You want to say to them, “This is what we’re saying.” You’re not reading them tape. You’re not reading them the script — but you’re going through the story fact by fact, letting them know what is being reported. In this story, Simpson wasn’t a source — he was Nancy’s subject and didn’t talk to us — but when fact-checking you try to get in touch with everyone. This isn’t a chance for someone to edit your story or redo the interview — you have logs and know what they said — but people in your stories should have a chance to fix inaccuracies. You don’t want to surprise anyone with your broadcast. And you don’t want any surprises post-broadcast. You want to work through problems before the story reaches an audience.

This doesn’t always go smoothly. I started fact-checking in 2003, and over the years, I’ve been told I’m irresponsible by a spokesperson for George W. Bush and an undersecretary in the Commerce Department once screamed at me while he changed planes in San Francisco. People have cried and others have threatened legal action. I’ve been hung up on. Some people refuse to return my calls. In those cases, I’ve hired stringers to knock on doors, but other times, I’ve relied on court documents and logs. These cases are rare, though — like I said, I started fact-checking in 2003 and those are pretty much the highlights — and more often than not, people are grateful for the call, happy to talk, and thank me. All of these are good things to happen before broadcasting.

Ira, Nancy, and I exchanged some emails about what to ask O.J.’s lawyer during fact-checking. Here’s what Nancy sent:

Her whole story is worth a listen, but here’s where she works in how the lawyer never responded but also deftly handles the conditions under which the show’s producer did agree to speak:

Download

I’m not perfect, by the way. Here’s a correction we issued for a mistake I like to think I would have caught, if only I’d taken the time to reach out to someone from the podcast Ricochet:

Insulate Your Reporting

During a story meeting, one of my colleagues pitched an idea about a Serbian who could supposedly conduct enough electricity through his body to cook a hotdog pronged between forks he held in his hands. My reaction? “No way I’m fact-checking that,” I said, and while Ira had his doubts, too, he thought the documentary Battery Man beautifully captured Biba Struja’s life and the sadness of his world. (Here’s a bleak quote from Ira’s interview: “All of us on this planet are in some kind of jail, doing time here. It depends on the punishment you get, how long you’re on this planet”). He told Ira that he felt no pain when sticking electrodes into an electrical outlet and sure enough in the film there’s a scene where he does this while hooked up to a device measuring . . . well, who knows what? While the film is beautiful and sad, it’s not all that great at explaining the science — even when Biba visits a group of engineers in Belgrade for tests. Their conclusions just aren’t that clear. Their names are listed in the credits, though, so I emailed each one of them, asking about the tests. I didn’t get a single response. The film’s director had told us the story of Biba was true, but when I got in touch he wasn’t able to answer questions I had about things like amperes. I was uncomfortable not having a more scientific understanding of the subject, so I read an academic paper by a biomedical engineer who studies the effects of electricity on the human body. Without getting too technical, I learned our skin is our natural insulation to electricity — and the drier the skin, the more insulated you are. I got in touch with the author to learn more, and he ended up watching a few of the online videos of Biba cooking hotdogs. He pointed out things I hadn’t noticed. Like how the forks are connected to a power source, which means the current isn’t flowing through him but rather down the forks and across the hotdog. In at least one instance, the engineer thought he saw rubber where Biba held the forks and also speculated that maybe he has a higher than usual tolerance for pain . . . but also given that Biba has a genetic condition where he doesn’t sweat, it did seem reasonable to think his skin offers unusually high insulation. That said, it was impossible to say all that much from watching videos but he did say that anything involving electrical sockets was probably trickery.

All of which is to say: When fact-checking you want to insulate your reporting. You want to run what you know past experts. It can be hit or miss finding someone to help but once you do, you’ll learn a lot. Ira and I talked about all my concerns about the film during an edit of his story. Listen to what he did report about Biba’s condition and the hotdogs:

Download

You can listen to the whole piece here.

Tick, Tick, Tick

To go about fact-checking yourself, change the script’s font, print it, and read it over to get a feel for all the facts in the story. Changing the font is a little mind trick that’ll keep you from glossing over facts you might otherwise miss because it’ll help you read in a fresh way. Then highlight all the facts — and a fact is any number, word, phrase, or sequence of narration carrying an iota of claim meant to convey actual detail or truth. Next check the facts against source documents and on the phone with people (or via email, if need be). When you find a mistake, write in the correction with a red pencil. When you’re confident a fact is accurate, that what is written is a fair representation of what someone said or that the numbers are correct, tick each word of the sentence with a blue slash. (I like colored pencils, by the way — they’re erasable and that helps keep things cleaner). In the margins also make a note about source material, using red for mistakes, blue for what’s correct. Keep it simple, like “Notes from biomedical engineer,” or “10/7/95 Gallup poll: 78% of blacks say Simpson not guilty was the right decision vs. 42% for whites.” It’ll probably take a number of reads through the script to make sure each fact has been checked, and while fact-checking can be time consuming, the tick, tick, tick is you marking each word because it’s correct. Once confident every fact is right, change the font again for a final read. On this read do not say, “Oh, I got this detail from Simpson’s criminal complaint, I got this fact from his prison record,” etc. Read slowly, asking: “What is still wrong?”

So, correct the mistakes, and know there will still be something else . . . because there’s always something that’s not quite right.

I marked up a page from journalist Samantha Broun’s story (produced with Jay Allison) 20 Years Later to give you a sense of what a fact-checked script might look like:

Listen to how she and Ira (along with Jay Allison and Robyn Semien) addressed my concern:

Download

I feel like I should end by saying something grandiose about how facts matter more than ever now, but I’m not sure that’s actually true — and I don’t want to fact-check it! — but I guess I honestly think fact-checking is just good journalists trying to do the best possible journalism by doing things like being smart about using as many primary sources as possible (do NOT use Wikipedia), giving people chances to correct inaccuracies, running information by experts, and being meticulous and organized when checking their scripts.

I hope this helps you become a more aggressive and skeptical reader. And oh yeah: I also hope you don’t have to sit under any wet ceiling panels.