Tag Archives: error tracking

NYT public editor addresses errors made in Cronkite article; some basic advice for preventing errors

August 2, 2009 – 12:35 pm

nytbanner1New York Times public editor Clark Hoyt has weighed in on the paper’s recent, error-riddled story about Walter Cronkite. The story, written by television critic Alessandra Stanley, resulted in two corrections, one of which was for seven mistakes. I wrote about the mistakes, and Stanley’s history of error, in a recent column for Columbia Journalism Review.

Hoyt’s column offers new information, such as how five different editors reviewed her story and missed the mistakes. This is a classic example of how easy it is for mistakes to end up in print. It doesn’t matter how many people look at an article; they have to know what they’re supposed to be looking for.

The most interesting revelation in Hoyt’s piece was that, after attention was drawn to Stanley’s errors in 2005, the Times introduced a program to increase the fact checking of her work:

For all her skills as a critic, Stanley was the cause of so many corrections in 2005 that she was assigned a single copy editor responsible for checking her facts. Her error rate dropped precipitously and stayed down after the editor was promoted and the arrangement was discontinued. Until the Cronkite errors, she was not even in the top 20 among reporters and editors most responsible for corrections this year. Now, she has jumped to No. 4 and will again get special editing attention.

The extra scrutiny helped. Then things regressed, and that’s the lesson here. The gap in the plan for “special editing attention” is that it doesn’t include a training component. Stanley could, with a little bit of effort, improve her level of accuracy. Additional oversight isn’t going to train her to be more accurate. It will make her more careful, but it won’t fix the source of the problem. Eventually she will stop receiving special attention and things will go back to the way they were.

It’s kind of a variation on the old “give a man a fish” saying: Give an error-prone reporter special editing attention and you’ll publish fewer of her errors. But train her how to be more accurate and she’ll make fewer errors. That’s a big difference.

I concluded my CJR column by writing that “whatever system [Stanley] has for checking her work isn’t sufficient. The same goes for how the copy desk is handling her articles. The Times can let her twist in the wind with errors like these, or realize this situation is hurting the organization and come up with a training program that helps her stop making simple factual errors at such an alarming rate.”

This is, as they say, a teachable moment. It’s an opportunity for the paper to create a newsroom-wide program that will help all reporters. After all, you can’t give everyone special editing attention. But you can teach good habits that prevent the need for special attention. Eliminate or at least reduce the errors at the source and suddenly there are less things that can slip through the cracks.

After my CJR column appeared online, I received an email from an editor asking me for some error-prevention advice. Here’s what I sent to him:

1. Self-Diagnose: Are you making or missing the same kinds of errors. Do you misspell names? Garble numbers? Etc. Take a month and track your mistakes. Write them down. Note how they happened and any other relevant information. At the end of the month, tally up your errors. Now you know your pain points. I recommend keeping an error journal; just create an Excel doc or Google Doc spreadsheet and keep track of your errors. This is hugely valuable data. (The Times has an internal errors database, so it already keeps some of this data.)

2. Create Good Habits: If you have a tendency to misspell names, then you need to start every interview by asking the person to spell their name. If, as an editor, you tend to overlook misspelled names, then the first thing you do with a new story is check the names. The key is to create habits/actions that are mapped to your mistakes. The best way to do this is to…

3. Use A Checklist: Whether you’re writing or editing, you should use a checklist to guide your fact checking process. I have a sample checklist available as a free download here. And if you need convincing, read this column about why checklists are so powerful.

I know one thing for sure: if Alessandra Stanley started using a checklist to review her work prior to sending it for editing, her level of accuracy would improve.

UPDATE August 3: Steven A. Smith has some good thoughts about this situation over on his blog. A sample:

Reporters with fact-error issues have to work a bit harder, have to develop personal double-checks that can be time-consuming and frustrating, especially on deadline. But that is the only way reporters can work themselves out of an accuracy funk. Some take on the challenge because of professional pride and a genuine desire to do their jobs as well as they can.

Others require a bigger stick. That’s just the truth of it.

I remember one reporter who worked on my regional staff at The Pioneer Press in St. Paul. He had experienced a terrible run of corrections, all the result of careless reporting practices. Working with him, we developed a series of steps he was urged to take before moving any story to his editors. Within days his desktop computer was covered in yellow sticky notes reminding him to check phone numbers and addresses, use the city directory, and so on. He took responsibility and his hard work produced results. His correction rate dropped dramatically and the new habits stuck with him.

But the reporter knew his job was on the line. ‘Fix it or lose it” was the message.

Was such a message delivered to Stanley at the time her editors developed a personalized editing program? If so, does the latest debacle mean she will lose her job? Should she lose her job?

Does “intellectual heft” in reporting compensate for inaccurate reporting?

By Craig Silverman | Posted in Behind the scenes, Newspapers, Regret Articles | Also tagged , , | Comments

Wash Post ombud links loss of copy editors to increase in errors

July 7, 2009 – 8:00 am

washpost4Just over two years ago, the public editor of the Orlando Sentinel wrote a column alerting readers to the fact that the paper had experienced a spike in the number of corrections. He was clear about the cause of the increased errors:

When the Sentinel tightened its financial belt back in June, it lost a wealth of seasoned veterans, many of them editors. Those journalists not only wrote headlines and captions. They also scrutinized the work of reporters — correcting spelling, straightening out syntax, double-checking facts — before publication.
With fewer people to do that now, less of that important work gets done, and the result is more published errors.

Yesterday, the ombudsman of the Washington Post wrote basically the same column:

…Growing numbers of readers are contacting the ombudsman to complain about typos and small errors.

"As a virtually lifelong subscriber, I am disheartened by the increasingly poor quality of the editing of The Post," wrote Richard Murphy of Alexandria. If typos can’t be caught by a spell-checker, "then The Post should restore a couple of copy editor positions. You have cut that staff too much."

The Post’s copy editors are among the best I’ve worked with during nearly four decades in the newspaper business. But they’ve been badly depleted by staff cuts as the money-losing paper struggles to control costs. Those who remain are stretched thin while The Post expands to a 24-hour news operation in print and online.

Between early 2005 and mid-2008, the number of full-time copy editors dropped from about 75 to 43 through buyouts or voluntary departures. It has declined further since then, but Post managers won’t provide precise figures beyond saying that six took a recent buyout offer. The need is so critical that most are being hired back on contract through at least the end of the year, and part-timers are taking up some of the slack.

Copy editors are the unsung heroes of newsrooms. Unknown to the public, and often underappreciated by their colleagues, they’re the last line of defense against a correction or, worse, a libel suit…

"By definition, you’ll see more errors when there’s reduced staffing," said Bill Walsh, the A-section copy desk chief. On a typical weeknight a few years ago, Walsh said, the three copy desks handling national, foreign and business news could rely on perhaps 20 editors. Those desks have since been combined into one desk, headed by Walsh. Today, he said, "there are some shifts where I’m looking at seven or eight people total."…

These papers are by no means the only ones experiencing a spike in errors due to the loss of bodies on the copy desk. Adding to the problem is the fact that the move online means papers are churning out more content than ever before. Yet copy editors — and magazine fact checkers — are being shown the door.

Carl Sessions Stepp examined how some newsrooms are coping with this challenge is his recent article, "The Quality-Control Quandary,” It’s a must-read. I fear, though, that few organizations are rethinking their quality control process and means of verification. They’re just trying to do more with less. It’s a recipe for disaster.

I looked at this issue in a recent essay I wrote for Harvard’s Niemen Neiman Reports:

For more than 100 years, one of the most recognizable slogans in journalism has been “All the News That’s Fit to Print.” Lately, The New York Times motto is being challenged by the familiar phrase, “do more with less.” This new saying was, in fact, the theme of the World Editors Forum scheduled for March, but the event had to be cancelled “due to the impact of the global financial downturn on newspaper companies.”

News organizations are shedding employees. Those that remain are expected to pick up the slack and also push ahead with digital initiatives. Included in the exodus are valuable copyeditors—the people in whose encyclopedic brains reside a lot of what prevents errors from surfacing in stories. The few, the proud—and disappearing—magazine fact checkers are also being told to grab their World Almanacs and Book of Facts and move along.

Accuracy is a huge journalistic challenge. When reporters are asked to take on more work while the newsroom’s same fallible processes and error-prone technologies remain in place, the result will undoubtedly be a further downward slide in quality. More errors will be followed by more apologies and more corrections. And this is happening at a critical time for journalism—a time when consumers are being asked by journalists using digital media to lend support to their newsgathering mission…

 

By Craig Silverman | Posted in Behind the scenes, Newspapers, Ombudsmen | Also tagged , , , , | Comments

What IT security can teach us about accuracy

November 15, 2007 – 8:00 am

Bruce Schneier, one of the leading thinkers in IT security, recently wrote a column for Wired.com in which he uses the example of corrupt NBA referee Tim Donaghy to examine systems that suffer from single points of failure. The same concept directly relates to journalism and accuracy.

What sorts of systems — IT, financial, NBA games or whatever — are most at risk of being manipulated? The ones where the smallest change can have the greatest impact, and the ones where trusted insiders can make that change.

Donaghy used his position to try and influence the outcome of games, and he was able to because of the way the NBA games operate:

Because individual players matter so much, a single referee can affect a basketball game more than he can in any other sport. Referees call fouls. Contact occurs on nearly every play, any of which could be called as a foul. They’re called “touch fouls,” and they are mostly, but not always, ignored. The refs get to decide which ones to call.

Schneier lists other examples of jobs where people are “both trusted insiders and single points of catastrophic failure.” It could be a “dishonest computer-repair technician…a corrupt judge, police officer, customs inspector, border-control officer, food-safety inspector…”

Or a journalist. Newsrooms build in layers of auditing in an attempt to mitigate the ability of a trusted insider to subvert the system: copy editors, assignment editors etc. But each person in the chain of audit (editing process) is both a point of quality control and a potential point of failure. We then attempt to mitigate that reality by requiring reporters to take notes or record interviews, cite sources, and talk to experts.

Yet we still see people like Jayson Blair and Stephen Glass, trusted insiders who become single points of catastrophic failure. Incidents of plagiarism, fabrication, and extreme error abound. Clearly, we need to evolve our audit systems. The current reality of shrunken newsrooms — and therefore reduced audit controls — makes it even more imperative that we innovate new ways of ensuring quality. The speed of online news also requires us to find ways to do it at a faster pace.

Yes, a tough challenge. But an exciting one, too.

“All systems have trusted insiders,” according to Schneier. “All systems have catastrophic points of failure. The key is recognizing them, and building monitoring and audit systems to secure them.”

So what does the ideal newsroom monitoring and auditing system look like? Likely a combination of prevention — fact checking, plagiarism detection, training, editing etc. — mixed with post-publication/post-broadcast error tracking and analysis. These elements demand a mix of people, processes and technology. The challenge is creating the right mix and then constantly managing, evolving and improving the system.

It’s a difficult task, but the status quo is a recipe for repeated “catastrophic failures.”

Would you keep following the NBA if you knew another Tim Donaghy was inevitable?

Have an idea for newsroom auditing? Share it in the comments.

By Craig Silverman | Posted in Regret Articles | Also tagged , , , | Comments

More on the Times’ corrections database

November 1, 2007 – 8:00 am

CheckYourFacts.org has expanded on a previous report about the recently-launched New York Times corrections database. The new story offers more details about how the database works and how the paper will use it to track and improve accuracy. It also includes an interview with Greg Brock, the senior editor who oversees the database and the paper’s corrections. Reports CYF:

…The new Times database holds comprehensive information about the nature of errors made by reporters and editors. The editors fill in a number of fields, starting with where the error occurred (in a column, an article, a caption, a headline, a photo, or even a byline).
Next, they enter the field “identified by,” which names the person who called the error to their attention. The paper previously had no record of this information, which is particularly important for understanding the source of more questionable corrections.
The field “originator” points to person who came up with the story in question. “Category” lists the type of correction printed, usually “For the Record,” “A Correction,” or “Editor’s Notes.” Finally, the editor enters the person responsible, and a section for miscellaneous notes for pertinent information that does not fit anywhere else.

Sounds like the paper, which had to scrap an earlier version of the system, is gathering the appropriate information. One more thing:

Brock says the system will also allow the paper to track another common mistake made by reporters: using uncorroborated information from the Internet in their articles. “People turn to the Internet with a quick Google search, and put ‘Fact X’ in an article,” says Brock. “We know that’s happening, and we’ve asked people to please not do that.”

Nice. The subsequent interview with Brock also has him explaining how the paper will use the data it collects to make decisions for the paper. This is really key:

The purpose of this one is to pinpoint certain types of errors and how they’re happening. For instance, if we discover that “Department X” is having a lot of errors appearing in the Monday paper, it would probably tell us right off the bat that they’re lightly staffed on Sundays, and might need more help. It might be a department that, like “Culture,” for instance, would be putting out a Monday paper that has a lot of articles that are prepared in advance. Perhaps they should have come in on Friday, but maybe are coming in late on Sunday afternoon. So, we can pinpoint a problem area.

It’s essential that the database is not only used to deliver a simple numerical tally. Its data should be put to use in evaluating reporters, finding weak spots in sections, and making staffing decisions. If a section’s head count goes down and the number of corrections spikes, then managers can see the effect that resources, and a lack thereof, have on accuracy.
Another topic of conversation with Brock is how this system could possibly catch another Jayson Blair-type earlier on. He notes that Blair “had 50 some corrections, but no one person knew. This department would use him, and he made a few errors. Then, another department would use him, and he’d made some more. And then when he got handed off in National, they didn’t know, and there was really no way to check. Plus, I don’t think anybody thought to check, frankly.”
The lack of a centralized system was certainly a factor, but there’s also the issue of severity to consider. Blair may have had a lot of corrections, but early on he had two back-to-back corrections that were the result of highly unethical behavior on his part. The first correction, from Oct. 23, 2001:

AN ARTICLE IN SOME COPIES on Sunday about a benefit at Madison Square Garden for victims of the Sept. 11 terror attack misstated the price of the most expensive tickets. They were $10,000, not $1,000.
The article also quoted incorrectly from a remark by former President Bill Clinton to the audience, many of them police officers and firefighters. Mr. Clinton said he had been given the bracelet of Raymond Downey, the deputy fire chief who died in the attack—not Chief Downey’s hat.
Referring to the terrorists, he said, “I hope they saw this tonight, because they thought America was about money and power. They thought that if they took down the World Trade Center, we would collapse. But we’re not about mountains of money or towers of steel. You’re about mountains of courage and hearts of gold, and I hope they saw you here tonight.” He did not say “hearts of steel.”

It was followed by another the next day:

AN ARTICLE IN SOME LATE EDITIONS on Sunday about the benefit concert at Madison Square Garden for victims of the Sept. 11 attack referred incorrectly to scenes in a short film made for the event by Woody Allen, “Scenes from a Town I Love,” which showed New Yorkers talking on cell phones. An actor in one scene complained that his anthrax drugs had been stolen by muggers; he did not say the police took them.
Another man talked about opening Starbucks coffee shops in Afghanistan after the war; he did not say one had already opened there.
The article also included two performers erroneously among the participants. Bono and the Edge, of the band U2, were scheduled to appear but canceled before the concert.

Blair hadn’t gone to the concert at all. He watched it on TV while drinking at a bar, and then just sent in a story as if he had been there. His editor found out and gave him a “a formal reprimand,” according to Seth Mnookin’s book about the scandal. But Blair remained at the paper for more than two more years before resigning in the face of growing accusations. So, aside from treating such egregious lapses in a more serious manner, perhaps the Times database could also include a means of rating the severity of an error?
Obviously, this is a subjective criteria. But frequency alone isn’t the only issue to consider. Content also matters.

By Craig Silverman | Posted in Behind the scenes, Newspapers | Also tagged , , | Comments

NY Times corrections database goes live

October 12, 2007 – 4:01 pm

The good folks at Check Your Facts recently published an item stating that the roll out of the New York Times internal corrections database is complete. The paper is now entering all of its corrections into a central database, much like how the Boston Globe, Rocky Mountain News and a few other US papers have been doing. Reports CYF:

Check Your Facts has learned that the New York Times may finally be making good on one of the recommendations laid out in a 2005 report titled “Preserving Our Readers’ Trust.”
Fresh from a “training class,” a well-placed source at the Times says a computerized tracking system for corrections is being implemented. Correspondent Adam Klasfeld learned of the new system through correspondence with a New York Times editor (Check Your Facts is withholding the editor’s name in order to maintain open communications on the matter). The NYT editor said that no tracking system as described in the report currently exists but that a “training class” was held for a new system. “We are just implementing it,” wrote the editor. “We are beginning with corrections on October 1.”

We reported on the roll out of the system in December of last year after speaking with Greg Brock, the paper’s corrections editor:

A final note from the Times: Brock says the paper does have a corrections database that is being used by some departments. Each department can see its own corrections tally, and Brock has access to the total data. He says they are working to roll it out within all departments. This database was one of the recommendations of the Siegal Committee. It’s good to see the paper following up on this project. We hope it’s fully operational ASAP.

In an email response this week, Brock said the system was fully operational as of September 17. “So it is in place, though we are continuing to train editors who will use it only occasionally because they work on sections or special issues of magazines that publish just a few times a year,” he wrote. “The main news desks are already using it.”

We asked if the paper plans to release its tallies the way some other paper do (see these links: 1, 2, 3, 4), and Brock said the issue hadn’t yet been raised, but that he would look into it. As of now, he is distributing reports to the relevant editors on a monthly, quarterly, and annual basis in order to show them the most common types of errors in their sections. It’s good to hear the data is being put to use to help fuel preventative initiatives.

By Craig Silverman | Posted in Behind the scenes, Newspapers, Regret Articles | Also tagged , , | Comments Off