Tag Archives: accuracy studies

Top fact checkers and news accuracy experts gather in Germany

April 9, 2010 – 11:38 am

Thanks for being a regular reader. You can check out the award-winning Regret the Error book here.

If you were to indulge in a bit of stereotyping and imagine the country most likely to host a conference about the pedantic discipline of fact checking, you’d probably arrive on one likely location: Germany.

And so it was that I spent the last weekend of March in Hamburg in the offices of the famous German weekly magazine Der Spiegel as a speaker and participant in a conference dedicated to fact checking. I was of course at a disadvantage in that I was one of only four English-speaking presenters; the rest of the conference took place in German.

My fellow North American presenters were Peter Canby, a senior editor at the New Yorker who heads up its fact checking department; Sarah Smith, managing editor of the New York Times Magazine and a former fact checker at the New Yorker; and Scott Maier, an associate professor at the University of Oregon’s School of Journalism and Communication and the leading newspaper accuracy researcher working today.

Nearly all of us who spoke in English couldn’t help but note what a pleasure it was to participate in, wonder of wonders, a conference about fact checking. God bless those crazy Germans. None of us were willing to offer the checker’s guarantee that it was the first ever such gathering on record, but it was the only one we could think of.

The real news of the conference, at least for us visitors, was the massive fact checking operation at Der Spiegel. The other bit of news was that Der Spiegel has a wonderfully outrageous cafeteria and meeting space. You can view all my photos on Flickr, but here’s a sample:

Can you imagine eating there every day? Just looking at it makes me want to buy some shirts with butterfly collars, and grow a moustache. But back to fact checking…

My colleagues from the New York Times Magazine and the New Yorker were just as amazed as me to discover the German weekly has roughly 70 full-time people in its fact checking and research department, as well as others who work part-time. By comparison, the New Yorker has 16 checkers, including Canby, making it the major checking operation one of the major checking operations by North American standards. [Update/Correction April 9: Canby emailed to say Vanity Fair has over 20 checkers, making it larger than the New Yorker's department. That's why I struck the text above.]  You can read all about Der Spiegel’s checking in my new column for Columbia Journalism Review.

This slide, which was part of a presentation by the head of the magazine’s checking and research department, illustrated that Der Spiegel’s approach is to hire checkers who have specific expertise in different areas. Here’s a list of some of their checkers (to give you an idea, their medical expert/checker is a former physician):

Though Der Spiegel’s approach is unique, there is one way in which German fact checking is similar to what we have (or used to have)  in Canada and the US: it’s on the decline. Very few publications — someone at the conference estimated there are six in all of Germany — practice it. The tough economic times have resulted in the reduction of staff checkers, and those that are left are looking for new ways to justify their existence.

During my discussion with a Der Spiegel fact checker and the deputy head of the department, they said they are trying to use their internal database of information and sources to generate topic pages for the website. You can view the Angela Merkel topic page here.

For them, one way to ensure the survival of fact checking is to offer something other than checking and research. In short, they’re trying to generate content, not just verify it. The department is also hoping to save time and resources by moving away from paper-based checking and towards a digital workflow. If you wonder what I mean by paper-based, take a look at this slide showing an article that was worked on by a checker (click for larger):

In terms of English-language content, I shot a bit of video of Scott Maier’s talk about newspaper accuracy. He shared some of his research into newspaper accuracy in the United States. (I have more about this research here and here.) Here’s an excerpt:

One quote from Maier that stood out for me: “In America, journalists are better educated than ever,  yet the rate of error is higher than ever. Something is going wrong.” Also, here are photos of some of the slides that were part of Maier’s presentation. These will give you a quick and dirty look at his new data about Italian and Swiss newspaper (which has not yet been published). Click for larger:

Finally, for any German speakers out there, here’s a lengthy TV report about fact checking and the conference:

By Craig Silverman | Posted in Behind the scenes, Magazines, Newspapers, Regret Articles | Also tagged , , , , | View Comments

Speed versus accuracy in journalism: towards a new debate

October 23, 2009 – 3:02 pm

Today’s edition of my weekly column in Columbia Journalism Review looks at the issue of speed versus accuracy in journalism. I hope you’ll take a moment and read it, as it relates to this post. Think of the column and post as branches on the same tree.

My column looks at the issue in terms of the consequences of rushing out stories in today’s media environment, and why scoops aren’t what they used to be. This post examines the value of speed, and how it should be weighed against accuracy.

I confess that I started working on this several months ago, when New York Times public editor Clark Hoyt wrote a column that struck a familiar chord: the rush to publish news had apparently led to some inadequate reporting and sourcing in the paper.

It was a good column, the kind of inside look at how a controversial story unfolded that you’d expect a public editor to provide. I found myself agreeing with many of the points he made. But then Jon Landman, then the deputy managing editor in charge of the Times’ online operations, responded to Hoyt and expressed a different view of how the paper handled the story. Most important, Landman gave one of the best expressions I’ve read of the value of speed in journalism. He didn’t argue against accuracy; he simply said that the two need not always be seen as enemies.

Here’s how Landman began his third paragraph:

Of course working fast increases the chance of error and clearly that is a danger to acknowledge seriously and address carefully. But absence of error isn’t the only value. If it was, we’d long ago have scrapped daily and weekly newspapers and magazines in favor of refereed scholarly journals. Speed is a value too.

It’s rare to see a newspaper editor deal so frankly with the issue of accuracy, to not trot out the old “accuracy is one of our most important values” line. The truth is that, while accuracy is valued in journalism, it’s often subjugated in favor of other values. As Landman writes, speed can trump accuracy in the minds of editors. It happens all the time. We shouldn’t pretend that accuracy is always the most important value when it comes to the actual practices of a newsroom. (We also shouldn’t forget, as I note in my CJR column, that the difference between making an error and getting it right is often a matter of making one or two phone calls. Accuracy is often easier and faster to achieve than we think.)

Let me be clear that I’m not suggesting Landman doesn’t care about accuracy, or that he’s advocating ignoring it as a standard operating procedure. He’s simply stating the reality of how journalism works: accuracy isn’t always the number one concern. There is ample evidence to back this up.

Landman isn’t alone in pointing this out. Philip Meyer, one of the most important journalism thinkers/academics of the last couple of decades, made a similar case in his book, The Vanishing Newspaper. He wrote:

A newspaper with a zero level of factual errors is a newspaper that is missing deadlines, taking too few risks, or both. The public, despite the alarms raised in [American Society of Newspaper Editors] studies, does not expect newspapers to be perfect. Neither do most of the sources quoted in the paper. The problem is finding the right balance between speed and accuracy, between being comprehensive and being merely interesting.

There is a balancing act when it comes to certain elements of accuracy. It’s never okay to get someone’s name wrong, or to make a mistake about an easily verifiable factual error. I’m sure both Hoyt and Landman would agree with that. But, in some cases, editors have to make a call about whether they have all the facts, not just the right ones. That’s the kind of thing Meyer is referring to, and, I suspect, so was Landman. These calls have been made for decades, if not centuries, so they aren’t new to the online world. (What is new are the consequences of an incorrect report.)

When it comes to the online environment, Landman argues that it enables a media organization to improve the accuracy process, rather than degrade it:

When the reporting process plays out in public, that’s a good thing. Readers can and do participate. Their participation has a salutary effect on quality — millions of amateur editors catch a lot that a few professional ones miss. And the process of constant checks on the unfolding story produce incentives to keep pushing. In the Kennedy-Paterson story, the never-ending news cycle ultimately contributed to a good result — a story that got to the bottom of the strange back-and-forth between the Paterson and Kennedy camps, sorting facts from rumor and accusation.

This is akin to Jeff Jarvis’ mantra of “publish and correct.” Here’s what Jarvis wrote in a 2006 Guardian column:

We need to recognise that the internet alters how media operate. Blogs – whether written by professionals or amateurs – tend to publish first and edit later, which can work because the audience will edit you. In this medium, stories are never done; rather than turning into fish-wrap, they can grow and become more factual and gather new perspectives, thanks to the power of the link and, yes, the correction.

We all make mistakes. We’re human. And the internet makes our humanity more apparent than polished print and broadcast do.

I have to admit that even though he wrote the (excellent) foreword to the Regret the Error book, this philosophy always made me a bit uncomfortable. I suspect Hoyt may feel the same way. Why not wait 15 minutes or even an hour if it means getting the entire story right, rather than just most of it? When I have this internal argument with myself, I reply to that question by noting that readers can — and often do — spot things that journalists wouldn’t realize even if they waited all day before publishing. So there’s value in getting it out there. Accuracy is not always an absolute. I hate having to write that, but it’s true. Some things are non-negotiable, but others have shades of grey.

This is why journalists need to at least take a few moments and think about why they’re publishing something, and if their news values – speed, accuracy, and otherwise — are in proper alignment.

That’s one process we should never sacrifice for the sake of speed.

By Craig Silverman | Posted in Regret Articles | Also tagged , | View Comments

Regret the links

September 18, 2009 – 8:00 am

In case you hadn’t noticed, I recently added a “What I’m Reading” sidebar to the site. It’s over there to the right. I link to relevant articles that I find interesting or of note. And now, every once in a while, I’ll post a round-up of some of those links to make sure you don’t miss out on them. They’re all worth checking out. Enjoy.

Press Accuracy Rating Hits Two-Decade Low – Pew Research Center
pewresearch.org | September 14, 2009
More bad news re: trust and accuracy.

DISPUTATIONS: Spy Games | The New Republic
The New Republic | September 16, 2009
Victor Navasky demands satisfaction from The New Republic.

FACTS, ERRORS AND THE KINDLE | More Intelligent Life
moreintelligentlife.com | September 4, 2009
I’m interviewed in this Economist story about book errors and corrections.

Accidental headline of the year | Media Monkey | Media | guardian.co.uk
Guardian | September 1, 2009

A horror story involving the correction of a published scientific article – Statistical Modeling, Causal Inference, and Social Science
stat.columbia.edu | August 25, 2009


By Craig Silverman | Posted in Regret Articles | Also tagged , , , , , | View Comments

“In a way it is surprising that we do not make more mistakes.”

April 28, 2008 – 8:01 am

That’s a line from a blog post by Guardian subeditor (copy editor) David Marsh. It’s long been a common refrain from journalists, especially editors. In fact, Mitchell V. Charnley said basically the same thing in the introduction to his 1936 study of newspaper accuracy, the first of its kind.

“As common as the layman’s superficial generalization that ‘the newspaper is always wrong’ is the newspaper man’s defense that the wonder is that so few errors get into print,” he wrote in an academic article published in 1936. Charnley also noted the “appalling opportunities for error in the smallest story.”

Marsh marshaled that maxim while explaining that today’s copy editors have to juggle laying out stories, editing copy, and writing headlines and photo captions. Other tasks are often piled on top of those. Now add the fact that many newspapers have reduced the ranks of copy editors, and you have an environment that’s ripe for error.

“One of our best subs, taken to task this morning for what I described as the unforgivable crime of putting an acute accent on the artist Edgar Degas’ surname in last week’s paper, held his hand up to the offence but pointed out that he had been working on seven different pages under severe time pressure,” Marsh writes. “Doubtless he had corrected many mistakes but the one he missed was, of course, what everyone noticed.”

Marsh chose to tackle this topic because of an old chestnut offered up by readers — the idea that the quality of newspaper writing, editing and accuracy is far worse than it was decades ago. Call it the Golden Age of Newspapers Gripe. Marsh writes that “like most golden ages, this one was entirely mythical.” His evidence is this astoundingly bad paragraph from an edition of the Guardian published 45 years ago:

The Republican National Comittee decided in the spring that its chances of the White House in 1964 would be very slim indeed if it did not capture California, the second largest state, in 1962. Nobody less than its strongest possible vote-getter would do to defeat the incumbent Governor, Edmund (Pat) Brown. When it said this, Mr Nion was looking towards Washington, but the committee was liiking at Mr Nixon. He would have to oick the candidate, and if he oicked another man, eho lost, the party would be loth to nominate for the Preidency a national leader whose influence could not carry his own state in a state election. Yet, if Mr Noxon ran himself and won, he would practiclly forsweat the presidency; for, like allaspiring governors, he has been bocal and bitter about men who use the governor’s mansion as a springboard int the White House.”

The findings of newspaper accuracy studies have been relatively consistent since Charnley’s 1936 report. Most have found an error rate of between 40 and 60 percent, meaning roughly half of all newspaper news stories in U.S. newspapers have some kind of error, be it factual or one of a more subjective nature. (These studies gather data by asking sources to fill out questionnaires about an article in which they were featured.)

But it should be noted that the most recent — and comprehensive – study found one of the highest error rates on record. Also, the recent rash of cuts and buyouts in American newsrooms, coupled with the demands of producing an online edition, have already offered at least one example of reduced quality.

Charnley was right about the many opportunities for error within any single article. That hasn’t changed. What has changed is that, inside many newspaper newsrooms, you have less fewer people doing more work. Copy editors, in particular, have become editors, proofreaders and designers. (And, as Marsh illustrates, bloggers.)

The danger today is that the Golden Age of Newspapers Gripe may soon become reality, even if Marsh’s maxim remains true.

(With thanks to Andrew Phillips for spotting Marsh’s post.)

By Craig Silverman | Posted in Behind the scenes, Newspapers, Regret Articles | Also tagged , | View Comments
Get Adobe Flash playerPlugin by wpburn.com wordpress themes