First-semester 2003 grades

Quality gap between newspapers and local television newscasts widens in Bay Area

 

Analysis by John McManus
Posted September 22, 2003

Focusing on the Iraq war and California's budget crisis, the Bay Area's three largest newspapers achieved top scores during the first half of 2003 on seven basic yardsticks of sound journalism. But even such compelling issues couldn't lift the five most watched local television stations from mediocrity.

From January to July, five researchers at Grade the News, a watchdog group affiliated with Stanford's Graduate Program in Journalism, matched more than 2,200 stories in newscasts and newspapers to core news standards derived from the Society of Professional Journalists' Code of Ethics.

At least on those basics, the San Francisco Chronicle, San Jose Mercury News and Contra Costa Times rated in the same category as the prestigious Washington Post, which was used as a reference standard. All four papers merited an overall grade of A.

By contrast, grades for KTVU Channel 2, KRON Channel 4, KPIX Channel 5, KGO Channel 7, and KNTV Channel 11 ranged from C+ to D+.

Compared with Grade the News' last analysis in 2000, the new study found newspapers improving, but television stagnating. Three years ago Channel 2 in Oakland competed with the best local newspapers. Not this time.

Core quality ratings of the Bay Area's most popular news media

(Click on any category in the table below for explanations, examples and scores for each news organization.)

News organization	News- worth iness*	Context	Explan- ation	Local rele- vance	Civic contrib- ution	Enter- prise	Fairness	Overall
	B+	A	A	A	A	A	A	A
	C+	C	C+	A	C+	D	D	C+
	B+	A	A	A	A	A	B	A
	C	C	C	A	C+	D	D	C
	D+	D	C+	A	C	D+	C+	C
	B+	A	B+	A	A	A	A	A
	D+	D	D+	A	C	D	B+	C
	D+	F	D	B	C	D	D	D+

*Newsworthiness index counts twice toward overall grade.

Measuring only the basics

The study did not address fundamentally important, but difficult-to-quantify, measures of news quality. So we did not consider the intelligence of writing or reporting, whether specific important stories such as a robust debate on U.S. policy in Iraq were underplayed or ignored and the quality of photos and videography.

Rather, we looked at the basic structure of news, such as the importance of the topics chosen, the level of context, the potential of a story for wide impact on local residents and fairness. We are not saying local papers match the overall excellence of the Post, only that they did as well on those basics of journalism amenable to counting.

Making the competition more equal

To level the field between print and broadcast, our research team ignored the volume advantage of newspapers. We analyzed only the top stories of the day - those on the front and local news front pages of newspapers, and the first 30 minutes of the evening newscasts, alternating between early and late. Research shows these are the most read and watched parts of the news. Grades are based on the percentage of news time or space each medium devoted to various types of stories.

Another plus for newspapers is that readers can skip over long stories if they're not interested. But viewers either have to sit through a long story or click their remote control to see what's on elsewhere. When they do they may not return, penalizing the station for boring them.

To reduce print's advantage, we capped the number of sources we counted at levels compatible with the brevity of newscast stories. We also combined closely related stories, sometimes labeled "team reporting," into one. That allowed us to add sources and award higher scores in several categories. Grading by time rather than story meant a well-reported longer story received more credit than a throw-away 30-second report.

We sampled all media on the same news-gathering cycle - evening newscasts and the next morning's paper - so all had the same raw material.

A different approach to news

Despite these adjustments, the differences between print and television were glaring:

• The Bay Area is extraordinarily diverse in all sorts of ways, giving birth to multiple perspectives on any issue. Yet stories with only one named source - or none at all - comprised 35% of the airtime during the top half-hour of news at the stations sampled. Often the single source was a police officer. In contrast, stories taking only 9% of newspaper column space on the front pages had fewer than two sources.
  
• Stories devoted to helping citizens keep track of government decisions were given nearly twice the emphasis in newspapers as in television - 47% of newspaper column inches versus 25% of television airtime. These weren't just "meeting" stories, but also those that tracked the effect of decision-making at all levels. Students and teachers' fates lies with school boards; bus riders are affected by transportation authorities; doctors, patients and university students all feel the cuts in state funding.
  
• Television newsrooms relied heavily on "spot" news, much of it generated from listening to scanner radios for mayhem, fires and collisions rather than setting their own news agendas or examining pressing public concerns. The study found that on average, only 14% of airtime was devoted to issue stories initiated by the journalists themselves. In contrast, 46% of newspaper space was devoted to these "enterprise" or investigative reports.
  
• Even elemental fairness - getting both sides of controversies - was hit and miss: No station rated above a C+ on that measure except KGO Channel 7, which earned a B+. The newspapers averaged B+.

Nevertheless, at every station there were also more than a few stories which scored at the top of our categories. When it commits the resources, local television news was excellent.

News directors respond

Several news directors challenged the validity of the analysis. Kevin Keeshan, news director at KGO Channel 7, said taking the first 30 minutes of the 6 p.m. newscast missed his station's investigative efforts and some in-depth reporting that appear in the second half of the show. "I hit some of those [quality measures] harder in the second half-hour than the first," he said. He suggested sampling the entire hour: "It would be a more realistic evaluation of what we do, and it would have to improve our grades."

Ed Chapuis, news director at KTVU Channel 2 agreed: "By eliminating the [second] half of KTVU's 10 o'clock newscast, you are excluding our award-winning 'Segment 2' reports, which are designed to provide depth and context on serious subjects."

Grade the News previously analyzed full hour-long newscasts. But news directors then complained that comparing only the top print stories with all of the television stories was unfair. While we remain open to change for future analyses, evaluating the second half might actually lower the stations' grades. The additional in-depth or investigative story would help, but the concentration of lower-scoring sports and light features in the back half of the typical newscast would hurt.

The executive of the overall lowest-scoring station rejected the whole idea of analyzing newspapers and newscasts together. "You can't compare a newspaper to a TV newscast," said Jim Sanders, vice president for news and operations at KNTV Channel 11 in San Jose. "It's like comparing the Celtics with the Red Sox."

Douglas Foster, who teaches at U.C. Berkeley's Graduate School of Journalism and has worked in both broadcast and print news, disagreed: "For so many years, local television journalists have been defending the value of their role in news. If you want to argue for the value of broadcast news you have to hold it to similar standards." Foster serves on Grade the News' journalism advisory board, which is composed of Bay Area journalists. He took no part in the study.

Venise Wagner, an assistant professor at San Francisco State University's Department of Journalism, said the basic rules of journalism are the same for both media: "The advantage of television, of course, is its immediacy, but that does not preclude it from giving viewers complete news. What I see happening in my unscientific observations is that local television news tends to devote much of its minutes to the most sensational of stories while providing little context into larger issues. We hear so many stories about crime in Oakland, but what about the stories of what people are doing about crime in Oakland? What about the reasons behind increased crime in Oakland?"

The fine print

The study sampled stories every sixth day, so it included roughly equal numbers of fat Sunday and slim Monday papers, lighter Saturday and weightier weekday newscasts. The margin of error of all our measures is approximately plus or minus 5 percentage points, about half a grade.

We also examined a random subsample of the stories to be sure two evaluators, acting independently, scored each item similarly. Agreement between analysts ranged from 100% to 76%. Only one of our 12 measures fell below 80% agreement, the accepted level for scholarly publication. It categorized specific story topic, a measure that wasn't used in grading.

Seven yardsticks of basic news quality

. Newsworthiness: Core topics, such as crime, weather, government, politics, education, economics and health, get more points than peripheral topics such as celebrity news, fender-benders and sports. Stories affecting many people score higher than those affecting just a few. Stories can score from 1 to 4 points and are weighted by the proportion of the newscast or newspaper display pages they occupy. If 88% or more of the space goes to core topics affecting more than a few (> 10,000 people in a region of 7 million), the news organization rates a grade of A.

. Context: Number of sources. Four sources, including documents and "declined comment" rate an A. Only two independent expert sources also rate an A.

. Explanation: Stories about issues or thematic treatment of events have more explanatory power and receive more credit than stories focused only on a particular event such as a fire or homicide. If 70% or more of the top stories concern issues (including meetings, policies, etc.) or patterns of events, an A is assigned.

. Local Relevance: Stories about what happens in the Bay Area, or localized for the Bay Area, and stories affecting the whole state rate better than stories from out of town. To allow for important news from Iraq and elsewhere, up to 35 percent of the news can be from afar without jeopardizing an A.

. Civic contribution: Stories about how government or politics work or affect people at any level from school board to Washington are counted. So are stories of people criticizing or protesting government actions. Forty percent or more of the space or time devoted to such stories rates an A.

. Enterprise: Stories initiated by journalists seeking answers to pressing public questions rate higher than stories initiated by press releases and conferences or listening to scanner radio reports of accidents and violent incidents. Investigative reporting merits special consideration and is weighted by a factor of 4. Forty percent or more of top story space or time devoted to enterprise or investigative reporting rates an A.

. Fairness:

This applies only to locally produced stories that involve some type of controversy or accusation of wrongdoing. If the other side is offered an opportunity to speak (even if that opportunity is rejected) that counts as fair. Eighty-five percent of content rated fair earns an A.

Overall grade: Because of its central importance, the newsworthiness index is counted twice toward the total. All others are counted once. The result is figured on a 4-point scale, like a school grade.

The GTN research team comprised Michael Stoll, associate director of the project; Magdalena Wojcieszak an intern visiting from Poland; Seeta Peña Ganghadaran and Lise Marken, Stanford doctoral students; and John McManus, the project director.

Want a printer-friendly copy?

Want to know more about how the study was conducted?

 

 

What do you think? Discuss it in The Coffeehouse.

This site is supported and maintained by the Grade the News project team.

Feel free to disseminate, but please attribute and link to us.