Demand for data

Web designer. Information architect. Developer. User experience editor. Whatever the title, the need is growing for talented people who understand the presentation and collection of data. Some journalism background helps — newsrooms have their own pacing, vocabularies and personality. We are doing
decent work on homicides and shootings and this weekend we went live with a
new analysis of bridge safety data (do you really want to drive over the
bridge rated 3 out of 100?). For 10 years we’ve been doing a school report card for the region.

As good as these are, each project could be much
better. (A significant point on the bridge data is that this is the information the State of Pennsylvania did not want to release. Officials still haven’t released it, but the Inquirer was able to obtain it and we wasted no time getting it to the public.) We are already talking about ways to improve the presentation of the bridge data, by adding 1. a way to save or share the information on a bridge or group of bridges, 2. a way to search by major route, 3. a way to search for the structurally deficient bridges that we did not include on the first list, either because they are in the parts of PA and NJ outside our
eight counties or because they carry fewer vehicles than the cutoff level we used.

We are about to redesign and
rework the entire school report card, to make it much easier for our audience and to include more information. The Loudon Extra schools
guide is one inspiration. I am thinking about using an outside designer to get us started on the conceptual work. We
have people who will build a better web experience once we give them the specs,
but we need help with the process of developing and mapping out the
specs. We’ve started with Dylan Purcell, our data editor who also handled the bridge data; Rose Ciotta, who is our education editor, and Connie Langland, a reporter now working as a free-lance writer who originated the report card more than 10 years ago. In addition to the Loudon Extra, we studied the SFUSD School Info Google Map Hack (great use of the map as an entry point to data), GreatSchools (which has a good school comparison function), and the Star-Ledger school report card (which has some advanced analysis of demographic data, too).

Is there a demand? The Inquirer school report card had about 6,000 page views in the last four weeks, and 53,000 views this year. The map of the bridge safety data had 5,000 page views in three days. Readers want access to this data and they want it clearly presented. That’s an ideal job for a newsroom.

We the people

The House hearing on Tuesday addded momentum to the campaign for a stronger right-to-know law in Pennsylvania, a law starting with the presumption that public information belongs to the public. Rep. Tim Mahoney of Uniontown is the author of the House bill. Sen. Dominic Pileggi is the author of a similar Senate bill. The good news is that the historical competitiveness between the two houses may turn this into a race to get a bill passed. Informally or formally, a conference will sort out any differences.

The Inquirer piece on A1 this morning, Bridge ratings to be released, notes that the goal of open records is "expected to be a key legislative issue this fall." Pileggi has said it is his top priority.  My prepared testimony is on the Inquirer website. I summarized the results of a 2005 open records audit:

In 2005, 50 newspapers in the state along with The Associated Press conducted a survey of 700 agencies. Of 217 police agencies, 40 percent denied access to logs or incident reports. Of 130 school districts, half denied access to the employment contract for the superintendent.

The solid attendance at the hearing showed a high level of interest in the topic. The Post-Gazette highlighted the debate over 911 call information:

The Pennsylvania Coalition Against Domestic Violence wants to prevent disclosure of 911 call recordings. But Carl Lavin, deputy managing editor of The Philadelphia Inquirer, said such access would allow residents to evaluate how well and how quickly emergency workers respond.

The Beaver County Times, which has spearheaded the effort to obtain the bridge data, emphasized Rendell’s role in the decision:

PennDOT stood firmly behind its policy of not releasing the ratings as late as Monday afternoon.

Biehler said he changed his mind again Tuesday morning after consulting with Gregory Fajt, Rendell’s chief of staff.

Rendell has been quiet in public though, making only general comments in favor of public disclosure. Despite the general sense among officials at the hearing that disclosure is a public good, the debates over particulars showed that many officials have lost track of their role. They are not in charge, they work for the public. Have they forgotten "We the people"?Take the discussion over email, for example. Anyone with a job in the private sector knows that an email account on the job is the property not of the individual, but of the employer. The boss has full access to that email account and all emails sent or received are available to the head of the company.

Government employees work for us, the public. There should not be any debate about the use of a government email account. Anyone reading a message sent from a government account realizes it is sent under the color of official authority. There should not be any question about making that email available to the employee’s boss — the public.

Advice to public sector employees: if you don’t want something disclosed, don’t do it.

Your bridge safety data

In the end, the Secretary of Transportation did show up to talk about bridge safety data. The official, Allen D. Biehler, said that although his department had a legal basis to keep the bridge information secret, he has now made a "judgment call" to release the information. He’s changed his mind. I testified afterward, and made the point that our citizens need a law that makes it clear: public information belongs to the public. One official’s judgment should not be a factor. The AP had this report:

The seeming capriciousness of PennDOT’s decision to withhold, then release the bridge safety reports was not lost on Carl Lavin, a deputy managing editor of The Philadelphia Inquirer.

Lavin, who testified after Biehler, said the Philadelphia School District had refused to release a consultant’s report on student violence against teachers – until a teacher was seriously injured by a student and the Inquirer planned a story about the district’s refusal to release the report.

"Miraculously, that report became available," Lavin told legislators. "A judgment call, just as you saw today."

Biehler made no promise about when this would be available, by the way, and none of the legislators asked him for a deadline.

It’s your right to know

I am testifying on Tuesday to a state legislative committee, headed by Rep. Babette Josephs, that is considering strengthening the Pennsylvania right-to-know legislation, widely considered the weakest in the country. An editorial in The Inquirer this morning hit the right tone:

A state House panel will review a bill that would give residents more clout
when seeking information on everything from zoning decisions to expense reports
of public officials.

Current state law has the issue backwards. It places the burden on citizens
to prove why any government record should be open for public inspection.

A bill from Rep. Tim Mahoney (D., Fayette) would shift the burden onto state
and local agencies to explain why certain records should be kept secret.

The measure contains other important features. It would cover the
legislature, which is currently exempt. It would create an independent office to
handle requests for state records and appeals of denials. It would increase the
fines for willful violations from $300 to $1,000.

I will post more from the hearing when I return. Here’s a taste of my testimony:

 

Officials say no so often because they can say no with
impunity. Substantial penalties, including legal fees, must be part of any
right-to-know law that will be taken seriously. We have seen that there is a
human tendency to keep information out of the public’s hands. A strong law will
mandate penalties when that happens.

These public records belong to the public. For a
democracy to function smoothly, citizens need access to public information – to
their information. That is why the First Amendment was written. That is why you
are working so hard to write better right-to-know legislation.

I do know that Rep. Josephs has asked for a PennDot representative to talk about that agency’s refusal to release bridge safety data. As of Monday afternoon, the agency was not going to be represented.

The Beaver County Times has been very pointed on the importance of this information:

PennDOT has denied two requests from The Times to disclose specific rankings
for structurally deficient bridges in western Pennsylvania. The most recent
denial came Thursday – just hours after a bridge spanning the Mississippi River
collapsed in Minnesota.

Releasing the bridge rankings could pose a security risk and panic the
public, PennDOT officials have said.

Kirkpatrick said Monday that the agency is reviewing its policy on disclosing
bridge ratings.

"In general, we have held that bridge inspection information is
confidential," Kirkpatrick said.

Take a look at Iowa. The Department of Transportation in that state makes all the information available. Can Pennsylvania do the same?

 

Give thanks for sugar-free Red Bull

Rob Curley talked for about six hours to two diverse groups from The Philadelphia Inquirer, The Daily News, and philly.com: executives, reporters, web people, photographers, editors, ad sales folks, producers, tech gurus, graphics experts. He inspired, he amazed, he energized. He demonstrated what can be done and he demonstrated what can be done to make money. He drank a lot of Red Bull. A Curley intern, Amanda Finnegan, who was traveling with him, said it’s a good thing that stuff is available sugar free.
Curley’s Seven Commandments (as I distilled the six hours):
1. Own breaking news, and make sure your audience knows it.
2. Go hyper-local, deeper than you ever imagined. Don’t just build a container, fill it. (Plenty of photos of youth sports? Yes!)
3. Database like crazy.
4. Multimedia overkill.
5. Evergreen content. Examples: Mark Twin (for Hannibal, Mo.), KU sports history (Lawrence, Kan.)
6. Platform independent: paper, web, sms text to cell phones, X-box, Wii, I-pod and more.
7. It’s a conversation. Leverage the wisdom of the crowd and make sure the audience is participating.

What scares him — or what inspires him? Facebook. (I had some satisfaction that our Philadelphia Sports News Feed group on Facebook, after one week, is up to about 120 members.) In the afternoon, Rob paid our own Jennifer Musser-Metz a great compliment. He said he was a new reporter when he first found Black Hawk Down on The Inquirer’s website — with source documents, bios on the people in the story, reader comments, multimedia. "This is journalism on steroids," Rob remembers thinking. That was when he decided to become an online journalist. We had a chance to applaud Jennifer again for making all that happen.

Beyond Red Bull, how is Rob’s work powered? One force is internology — the power of committed, hard-working paid interns, including Amanda. Who calls 300 restaurants to ask each of them 20 questions (children’s menu? hours you serve meals? vegetarian choices?) Who shoots dozens of digital stills? Who covers the county fair? The permanent Curley staff is five full-time editors-producers-developers. Amanda is one of about 11 interns on the Curley team at Washington Post-Newsweek Interactive. Her training, from Cabrini College in Radnor, Pa., is on display in this student project on war.

And the tech power? In Kansas, in Florida and in Washington, Curley uses Ellington CMS. Here’s the person to contact: Dean Royal toll-free at 866-454-5774 or droyal@ellingtoncms.com.

Special thanks to Adrian!

Data sets

One of the biggest opportunities we face — in my humble opinion — is with data
applications. Recipes, golf
course listings, gardening
tips, campaign finance data. We
– and every big newsroom — have all this stuff, but for the most part readers
can’t access it easily or search it in a simple, logical manner. The homicide
data and the school report card are great Inquirer examples of what is possible. Here are some tech solutions that promise to make it easy.

Curley, Loudon Extra, and HuffPo comments

Welcome to Rob Curley’s latest project: Loudon Extra, a hyper-local feature of the Washington Post. It’s hyper local only in contrast to the huge reach of the mother paper. Loudon County is not one small community with one stop light and one high school. It’s diverse and spread across a vast area. But the Extra project is meant to bring local search and local information to users and to involve users in keeping the site current. The ratio of pro content to am content is expected to start at about 99:1, but that could change. Right now data is at the core. From the welcome page:

We are currently building searchable databases of building permits, homes sales, and many other publicly available records. We also will be covering high school sports like this area has never seen.

I like the house of worship lookup and the web cams. The section’s story about the launch reviews Curley’s history increasing page views at other publication and asks a core question about the Post effort:

Curley’s revamping of the Lawrence site gained national attention because of the audience it attracted. In 2000 — before remaking its site — the Lawrence paper got 14 million page views, said Ralph Gage, chief operating officer of the World Co., which owns the paper. In 2006, that number had grown to 247 million, and the site booked about $2 million in revenue. The population of Lawrence is about 80,000, and the Journal’s circulation is 20,000.

Page views at the Naples Web site are up 17 percent in the first six months of this year compared with 2006, when the site was revamped, said editor Phil Lewis, with unique monthly users up 26 percent. The Web site is responsible for more than 10 percent revenue, Lewis said, which is above the industry average.

The question has always been whether a major metropolitan newspaper can successfully run a hyperlocal site like this, and how both local and national advertisers will react.

Those examples of page view increases can be enticing. More from the business story (on the data example 07newsroom discussed in this February post):

In December, Gannett Co.’s Asbury Park (N.J.) Press dumped three huge public-record databases onto its Web site: local property sales and ownership, and state employees’ salaries. By May, the paper had added searchable databases for local crime, school test scores, state deaths and public school employees.

In December, the paper’s Web site recorded 1.5 million page views. By May, the site was averaging more than 5 million page views per month, according to data from a Newspaper Association of America study, with a high of more than 9 million page views in April.

Curley is also known for welcoming user-generated content. More from the Loudon Extra welcome page:

We also will be adding a powerful community-publishing section to the site, allowing readers to share their stories, photos and videos.

No sign yet that the material shared — or posted — by readers will be filtered, tagged or rated by readers, too. Various models exist for this, including the Digg model (users can add one positive — or negative vote), the Plime model (users can increase their own rating, and thus their own voting power), and — starting today — the HuffingtonPost model. [update Freedom Interactive will go live next month with their own version, with SiteLife software from Pluck.] Paul Berry, the chief tech officer of HuffPo, explains all:

We’ve added a commenter profile page which includes your bio and lists all comments you’ve posted. To add a bio, simply click on the Your Account link on the upper right hand corner of every page.

We also have a new I’m A Fan Of feature which lets you vote for your favorite commenters. They can also vote for you. Your fans will be listed on your profile page. We’ll soon be able to alert you when your fans comment on stories and when your favorite bloggers post on the site.

But we’re most excited to announce a new initiative where we will choose one commenter a month to become a featured blogger at HuffPost. Yes, a blogger! Reading through the comments on our site, we realized that our readers are an underutilized resource – smart and opinionated. Our decisions will be based on how many fans a commenter has, how often their comment is selected as a favorite and our moderators’ favorites.

Even Paul’s dad is "a blogger!" Now Paul is making it possible for others to try their hand. Who wouldn’t be excited? The wisdom of the crowds, unleashed. And for the crowds in Loudon County, the power of local journalism has also been given free rein. May success blossom in many gardens.

“Public employees’ salaries are public records”

Update Continuing a chain-wide push, the Gannett paper in Lansing, Mich., is also publishing salaries of state employees. The search could be better, but the data set is clear and easy to use. The newsroom is getting thousands of page views and drawing praise and scorn. Samples:

this is such an invasion of privacy. It wouldn’t be o.k. to just list the titles
and the salary range?? You people have gone too far.

and

Way to go LSJ!! There is nothing wrong with sharing this
information with the public – it’s their legal right to know. Besides, I love
reading the State employees’ replies – priceless.

The paper published an opinion piece defending the data release. The author, Jane Briggs-Bunting, is a media law expert and herself a state employee:

It remains vitally important the citizens of this state know down to the
penny what we are paying all of our employees. We are still a government of the
people, by the people and for the people. It’s not the state of Michigan, Inc.
Every taxpayer in the state is a shareholder.

Bottom line: State employees, including me (an MSU faculty member), all work
for the people of Michigan. Any member of the public should be able to walk in
the door of any state, county, city, village, township, university or school
district office during regular business hours and ask for this very same
information and receive it.

Michigan’s Freedom of Information Act requires this information be public, as
it should be. Too bad we need a law to ensure the people get to see at least
some of the people’s business, but we do – or this information would never have
been released.

Original post The Washington Post has joined the Houston Chronicle and The Washington Examiner in publishing salary data for public school employees (see example, from Janney Elementary School). It’s all part of a school scorecard (which, aside from salaries, is similar in some ways to the Philadelphia Inquirer School Report Card). Eventually, every local news organization will make all this data public as a matter of routine. For now, it still draws some complaints. Here is how Deborah Howell, the Post ombudsman, reacts to the issue:

But some teachers are upset that their salaries were disclosed; some
said the salary figures are wrong. Dan Goldfarb, a teacher at Banneker
Academic High School, said publishing the figures is "offensive,"
invades his privacy and "is totally unnecessary."

While that
anger is understandable, there is a higher good to be served here –
accountability. There is no way to assess how each school is doing
without looking specifically at what is spent. If the names are veiled,
it is harder to hold individuals as well as the District accountable.

Public
employees’ salaries are public records. When someone is paid with
taxpayers’ money, that salary is liable to be made public whether
you’re the president of the United States
or the county dogcatcher. (It’s possible I’m inured to this from being
married to a longtime educator whose salary was frequently published.)

The
Post regularly prints the salaries of others whose records are public,
such as highly compensated executives of publicly traded companies –
obtained from the Securities and Exchange Commission — and the salaries of executives of nonprofit organizations, which must file tax records to keep that status.

The
compensation figures are what the school district budgeted for each
staffer, and many include 15 percent added for benefits; that was added
to the Web graphic Monday. Those who see inaccuracies should e-mail dc-schools@washpost.com
so the graphic can be amended. The map invites readers’ help. Readers
already have alerted The Post about issues to follow in later reporting.

The importance of data

OnlineJournalismReview interviewed Gary Kebbel, journalism program officer for the Knight Foundation, about the new Knight Grants. Here’s a highlight on the grant to EveryBlock:

OJR:Adrian Holovaty won $1.1 million for his open-source software
idea. What stood out about his project?

Kebbel: First of all, it’s an extension of his current
ChicagoCrime.org, but it’s Chicagocrime.org on steroids. It’s going to take
every possible public database that makes sense–whether that’s global or
regional or national–and combine it in a way where you type in your address and
you find out everything going on on your street or in your neighborhood. You can
find out where there’s a new school proposal, or where a restaurant is going to
be shut down, or if the city has decided to change trash pickup regulations.

Key phrase: every possible public database. That’s prime territory for newspapers and their affiliated websites. Newspapers have reporters and data experts who already know where those databases are, how to obtain them, and which ones have the best information. They know the pitfalls and they know how to move efficiently to make the most of these crime datasets (as crime maps, for example), teacher salary datasets, home sales datasets, and more. If newspapers are slow to act, others will grab that space, just as Craig Newmark grabbed the free classified space. The Knight Foundation is helping point the way, but smart newsrooms are already moving rapidly in the right direction.

Right to Know

If a news organization is gong to collect and make available data sets of information from public sources, one important step is to support a strong right-to-know law. Pennsylvania now has what is widely considered one of the worst laws and compliance with even that law is spotty. Agencies that do studies on their own problems, from the police to schools to the Department of Human Services, often fail to release those studies to the public, hiding their problems instead of helping increase public pressure to solve them.

A hearing in Harrisburg on Monday considered these questions. I talked about the babies who died in DHS care while the department and the city of Philadelphia refused to make public their own reviews that pointed out the egregious flaws in department policies and procedures. Only after The Inquirer did a series of investigatory articles did the city start to make more of this information public.

Ron Barber and Teri Henning talked about the most fundamental problem with the current law — it presumes public records are private. Ron read a powerful preamble from the West Virginia Freedom of Information law. We heard Graham Spanier of Penn State say that his university should be exempt.

When it was over, a prime sponsor of a bill to revise the law announced he agreed with us that the presumption should be in favor of a record being public. It’s an important step that can pave the way to more access to more information and better government operations.

The Pennsylvania Newspaper Association has more, including many links to more resources. And a new FOI Coallition is providing a place for more discussion.