YouTube recently unveiled its Reporters’ Center, a library of videos offering advice about a variety of aspects of journalism. "The YouTube Reporters’ Center is a new resource to help you learn more about how to report the news," according to the site. "It features some of the nation’s top journalists and news organizations sharing instructional videos with tips and advice for better reporting."
One of the videos features the editor of PolitiFact offering some good, basic advice about fact checking.
You’d expect a magazine to exercise extra caution when publishing an article about a “vending machine for crows.” It’s a strange idea, not to mention one that was developed for a master’s thesis in a “Interactive Telecommunications Program.” Because the story doesn’t fall into the category of common knowledge, it requires particularly careful editing and fact checking.
The New York Times Magazine fact checks all of its articles, but the process broke down when it came to the vending machine story. On Sunday, the Times published a lengthy Editor’s Note to explain the reporting, editing and fact checking errors. The Note:
An article in the Year in Ideas issue on Dec. 14, 2008, reported on Josh Klein, whose master’s thesis for New York University’s Interactive Telecommunications Program proposed “a vending machine for crows” that would enable the birds to exchange coins for peanuts. The article reported that beginning in June 2008, Klein tested the machine at the Binghamton Zoo, that the crows learned how to use it and that after a month the crows were actually scouring the ground for loose change.
The Times has since learned that Klein was never at the Binghamton Zoo, and there were no crows on display there in June 2008. He performed these experiments with captive crows in a Brooklyn apartment; he told the reporter about the Brooklyn crows but implied that his work with them was preliminary to the work at the zoo. Asked to explain these discrepancies, Klein now says he and the reporter had a misunderstanding about the zoo.
The reporter never called the zoo in Binghamton to confirm. And while the fact-checker did discuss the details with Klein, he did not call the zoo, as required under The Times’s fact-checking standards. In addition, the article said that Klein was working with graduate students at Cornell University and Binghamton University to study how wild crows make use of his machine, which does exist. Klein did get a professor at Binghamton to help him try it out twice in Ithaca, with assistance from a Binghamton graduate student, and it was not a success. Corvid experts who have since been interviewed have said that Klein’s machine is unlikely to work as intended.
These discrepancies were pointed out to The Times by the Binghamton professor several weeks after the article was published; this editors’ note was delayed for additional reporting. These details should have been discovered during the reporting and editing process. Had that happened, the article would not have been published.
The original story came in at just under 250 words; the Editor’s Note is close to 100 words longer. That’s a sign something was seriously wrong with the article. In the end, as the Note concludes, the story should never have made it into print. Even if Klein and the writer did have a “misunderstanding,” the fact checker should have discovered the truth and alerted the editor, who then should have killed the piece.
Magazine-style fact checking is currently journalism’s best system to prevent factual errors. However, this example demonstrates that it’s far from perfect. Errors made by the source and writer were not caught by the editor and also escaped notice by the fact checker. The mistakes circumvented a process designed to catch and fix them.
Then, after publication, Klein chose not to bring the mistakes to the Times’ attention; it was up to a professor with a slight connection to the story to reveal the mistake.
Allan Britnell, a Canadian freelance writer and fact-checker, has written an article suggesting that magazines should make a point of telling readers about their dedication to fact checking. Writing for Masthead Online, a website that reports on the Canadian magazine industry, Britnell proposes “an industry-wide campaign to promote fact checking” to readers:
One of the issues facing our industry is how to justify our relevance with a media-saturated audience that, increasingly, has grown up never having had to pay for content (music, movies, or reading material)… We – or at least, some of us – already have a built-in credibility process. Let’s let our readers know about it. To my mind, having the time to fact-check copy (and, of course, actually making the effort to do it) could be what makes magazines worth paying for, even when everyone else is giving away their content for free.
I propose an industry-wide campaign to promote fact checking. In short, we’d be letting readers know that because it’s been cross-checked, the content of a magazine is likely more reliable than what they’ll find in most websites, newspapers, or books.
The cornerstone of the campaign would be a check-mark logo that participating magazines could put on their covers (along the lines of Magazines Canada’s “Genuine Canadian Magazine” icon). A coordinated campaign of house ads, perhaps under the tagline, “We only write what’s right,” would explain in more detail what the logo – and fact checking – means. To promote the campaign’s launch, a coinciding series of editorials could further explain the process, and mention the names of other participating publications.
Magazines that have a strict fact checking process should definitely do more to communicate the value of this layer of verification. However, any “industry-wide campaign to promote fact checking” would have to begin as a campaign aimed at magazines. The reality is that fact checkers are a dying breed.
Only a handful of Canadian magazines use them, and checkers are being culled at an alarming rate in the United States. When layoffs hit, fact checkers are among the first to go. The magazine industry won’t be able to launch a credible campaign aimed at consumers unless it does a better job of ensuring that fact checking doesn’t become a luxury that only the biggest and best-funded magazines can afford.
To be fair, Britnell acknowledges that magazines aren’t pursuing a uniform checking process:
Obviously, a set of standards would need to be established. Does double-checking the spelling of source names and spot-checking the odd factoid count? To me, no. I would argue that substantively checking all editorial content should be the benchmark…
That’s a fair benchmark, but it will take a lot of work to make a significant number of magazines meet the standard.
While researching a chapter about fact-checking for the Regret the Error book, it became clear that I would have to dedicate just as many words to the demise of checking as I would to telling its history and sharing amusing anecdotes about checkers who went far beyond the call of duty. In the end, the chapter was called “THE BIRTH (AND SLOW DEATH) OF MAGAZINE FACT CHECKING.”
Here’s one relevant passage:
Magazines have always billed themselves as an authoritative voice. These days, that kind of quality is even more important, if only to cut through the massive daily assault of information and misinformation… If magazines do manage to hang on to rigorous fact checking, despite the time and expense involved, they’ll have a fighting chance of providing that voice. Readers, who show an almost filial loyalty to their favorite mags, deserve no less.
In today’s media environment it seems almost negligent for a magazine to eliminate its fact-checking department.
And here’s another to help remind us of the importance of fact-checking:
In September 1997, Canadian Business moved from a monthly schedule to publishing every two weeks. With its increased frequency, the magazine attempted to introduce a light check: Only things such as names and dates would be checked in every piece. The light check lasted only a few weeks before the magazine went back to the old system.
“They had this really stupid idea,” Pat Ireland, an experienced copy editor and checker at Canadian Business, said after the episode. “The editors would supposedly go over what needed checking and they’d mark it for you. Well, they never did, and editors, in my experience, don’t have any idea of what should be checked. You check everything. You never know what you’re leaving out.”
The magazine’s decision to bring back a full fact-check paid dividends at the 2006 Canadian National Magazine Awards. “I want to take a moment to acknowledge some people in the business who don’t get enough credit,” said Canadian Business staffer Matthew McClearn as he stood onstage to accept his gold medal for investigative reporting. “We call them associate editors at Canadian Business. You probably know them as fact checkers.”
Interestingly, the story of Canadian Business’ brief flirtation with the “light check” first appeared in Masthead.
After releasing my free Regret the Error Accuracy Checklist earlier this week (download your copy here), I devoted my latest CJR online column to the subject of checklists. This column offers background on why checklists have proven useful in so many different industries and professions. I examine why they work for journalists, and why we don’t use them. My column from the week before is a look at one man in Illinois who spends his mornings spotting errors in his local paper. Excerpts are below.
From today’s column (click on the headline for the full text):
Checklist Charlie
In 1935, Boeing Corporation almost went bankrupt after its Model 299 long-range bomber literally crashed and burned during a U.S. Army flight competition. Major Ployer P. Hill, the pilot, and one other crew member died in the crash. As a result, the Army contract went to a competing company, causing major financial difficulties for Boeing.
As a consolation, the Army ordered a few Model 299s for further testing. The question was how to fly them safely. The New Yorker’s Atul Gawande writes that the Army eventually “came up with an ingeniously simple approach: they created a pilot’s checklist, with step-by-step checks for takeoff, flight, landing, and taxiing.”
“With the checklist in hand, the pilots went on to fly the Model 299 a total of 1.8 million miles without one accident,” according to Gawande. The Army eventually ordered thousands of the aircraft, which became known as the B-17.
Gawande’s December 2007 story is a paean to the checklist, one of the simplest and most effective error-reduction tools. Checklists have been proven to work for pilots, doctors, nurses, and even people working at a nuclear power stations. For example, the use of a World Health Organization surgical safety checklist helped reduce inpatient deaths following operations by 40 percent, according to a studyNew England Journal of Medicine. published in the
Checklists also work for journalists. We just don’t use them …
From my January 30th column:
Homegrown Errorists
The package arrived two weeks ago, a bulging manila envelope with a return address in Decatur, Illinois. Inside was a mass of paper with a polite letter placed on top.
“Dear Mr. Silverman,” it began, “you have published a book on errors found in journalism and have a website devoted to the subject.” The writer, Robert S. Reed, continued on for two pages:As a subscriber to the Herald & Review in Decatur Illinois, I have seen hundreds of errors in newspaper articles in addition to errors in the photo captions and the headlines/sub-headings. Most are misspelled words, missing words, extra words, wrong verb tenses, and, in some cases, factual inaccuracies.
Two of the articles from 2008 are attached to illustrate my point … I am also attaching 82 photo captions that appeared in the Decatur Herald & Review in 2008. All contain errors of one type or another. The corrections are indicated in ink. Also enclosed are 35 copies of headlines and sub-headings.
The more than 100 clippings were roughly an inch thick, and Reed was as good as his word. Each page correctly noted a copy editing or factual error from the paper. Red ink was everywhere, and in all the right places. The collection represented hours of work, not to mention the time spent photocopying them for delivery to me.
Some may wonder why anyone would choose to dedicate this amount of time to cataloging the errors in their local paper. But it’s no surprise to me at all. I’ve seen it before. (Plus, I’ve dedicated the last four years to reading hundreds of thousands of corrections and errors. I’m in no position to judge.) …
This week marks the release of the US paperback edition of the Regret the Error book. Order your copy here.
In addition to a lower price, the paperback includes a new introduction by me and the best corrections and apologies of 2007-08. We also corrected the errors identified in the hardcover. (Read and subscribe to my book corrections here.) Plus, the cover makes note of the fact that the book won the Arthur Rowse Award for Press Criticism from the National Press Club in Washington.
Free Accuracy Checklist
To celebrate the release of the paperback, I’m offering a free download of a special Regret the Error accuracy checklist. Reporters can use the checklist to help achieve accuracy in their work. Have a look:
Instructions
I recommend the photo paper, lamination and a dry erase marker because this process will enable you to use the same checklist over and over. Just keep it at your desk and reuse it for every story. Better yet, stick it to the corner of you monitor so it’s always in front of you. If photo paper/lamination isn’t your style, you can just print off a new checklist for each story. The laminated version is better because it saves time and paper, but it’s your choice.
This checklist is divided into four sections:
Why a checklist?
Checklists help reporters and editors increase their level of accuracy. Checklists are also used in other industries and professions, such as medicine. Train yourself to use one, and you’ll make fewer factual errors. Seriously, it’s one of the easiest things a journalist can do to prevent factual errors.
This checklist is primarily aimed at reporters, but sections three and four could just as easily be used by editors. If you have any edits or suggestions for improving the checklist, please let me know. I’d also love to receive testimonials from people who use it.
Enjoy!
In an Oct. 15 story fact-checking the presidential candidates’ debate, The Associated Press incorrectly reported that presidential candidate Barack Obama overstated the proportion of American households that would see tax cuts under his economic plan.
The Democrat specified that he was talking about “working Americans,” and the figure he cited – 95 percent – is essentially correct, according to the Tax Policy Center, which calculated the figures. Link
Previous example here.
A passage from Jesse Froehling’s story (spotted by Romenesko) in the Sept. 17 edition of Seattle Weekly brought back memories:
Mark Powell finds mistakes everywhere he looks. National monuments, scholarly texts, museums, The Washington Post, The New York Times: All have drawn the attention of Powell’s rabid, error-spotting eye. Powell will leave you seven-minute voicemails about these errors. When you call him back, he’ll tell you how good he is at finding them–in great detail. When after two and a half hours you finally manage to hang up the phone, you’ll vow never to speak with Mark Powell again. Then he’ll call, and you’ll listen. Because the thing is, Mark Powell is always right.
In 2006 and 2007, Powell and I had several long conversations about his fact-checking hobby. He had been spending his spare time spotting errors in the Washington Post and sending his findings to the paper. Powell had pages and pages of notes, some of which he sent to me. He also told me the Post had cancelled his subscription. (Powell forwarded me emails from the circulation department that appeared to confirm this.)
Powell undeniably has a talent for fact-checking. He demonstrated this time and again. We spoke on an off for a few months and also discussed the possibility of having him write a column or two for this site. I wanted him to produce some new checking and write about it here; he felt I should publish his previous work. It ended with me trying to convince him to start his own website. I also wrote about him in the Regret the Error book. Here’s an excerpt:
In October 2006, I was introduced to one dedicated independent fact-checker via e-mail. The subject line of the message seemed spam-like enough: “Perhaps the most important inquiry you could get.” I came close to deleting it before having a quick read.
“I’m just getting familiar with your site,” it read. “Bravo for highlighting the errors, egregious and/or just silly, that big papers make and admit. But at my first glance, you don’t seem to deal in a yet-more-important issue: the uncorrected errors constantly and permanently disinforming the public.”
The writer, Mark Powell, then outlined his months of work fact checking one U.S. newspaper. His conclusion was:
THIS IS THE BIGGEST UN-/UNDERREPORTED SCANDAL IN U.S. JOURNALISM. While the chattering class ever harps on so-and-so’s political bias, real and imagined, sheer topical and journalistic INCOMPETENCE is destroying print’s last hope in the electronic age, viz. acknowledged credibility/authority superiority.
Powell lives in Virginia and his paper of choice is the Washington Post. As far as he is concerned, the paper’s corrections “represent a very tiny fraction of the paper’s ‘correctable’ errors. Fact is nearly none of the thousand-odd errors I’ve cataloged—probably less than 2 percent— were ever corrected.”
… [Powell] was as focused and dedicated an external fact-checker as I’ve come across, and no doubt a thorn in the Post’s side since he regularly e-mailed editors with his findings. I told him he struck me as the kind of person who would have been well suited to the job of proofreader. Too bad it no longer exists.
The Seattle Weekly article is, as far as I’m concerned, an accurate portrayal of Powell and his work. It details his fact-checking prowess, the aggressive way he demands corrections (and jobs at newspapers), and his colourful way of speaking and writing. It’s nice to hear that Powell’s still doing his thing, even if the targets of his work are (understandably) frustrated with him. From the piece:
… Furthermore, Powell has been such a thorn in The Washington Post’s side that the paper has canceled his subscription—for life, he claims. And in late August, Powell wrote to Greg Brock, senior editor in charge of corrections at The New York Times. In his communiqué, Powell points out errors in three film reviews, and then essentially asks for a job: “Whether as fulltime editor or some type of outside associate, I want to find a place, if such exists, where facts, performance and principle outrank politics and personalities—where the best at something can be valued and rewarded for being that, which advances the outlet’s mission. I’m advised there is no such place, including the Times. Recent years’ general news indicates you’re in worse shape in important ways than the Post. But I won’t know till I probe there.”
Brock wrote back via e-mail, asking Powell to “take a deep breath and listen,” noting that Powell’s conduct was hurting his cause. “My own assessment—after hearing from you so far—is that you are undercutting yourself,” said Brock.
On Aug. 25, Powell responded: “It may not make you tremble, but with more editors in both America and Canada now asking me for diverse episodes from my trail, you can bet I’ll be showing some clip including this sorry little episode about my first glances at the New York Times. In fact, I think the three for three start, and your attitude in reply as soon as it was clear that I really did back up my words, will be a useful vignette.”
I’ll note that Greg Brock and I have met in person and also once spoke at the same event. Brock spends a good part of his day communicating with the public and tracking down corrections. He’s used to taking heat from a variety of people and is remarkably good-natured and professional about it. But, obviously, Powell’s not your average reader.
From journalism.co.uk:
The sourcing and fact checking process for stories on the BBC News website must be addressed by management, the BBC trust Editorial Standards Committee has recommended.
The committee made the suggestions as part of its response to a complaint about an article on the site, which pointed to inaccuracies in the report on Congressman Joseph P Kennedy II’s marriage to Sheila Rauch.
During the complaints procedure, the online news team conceded to oversimplifying the story and admitted that this could mislead the reader.
From the committee’s findings:
