Annotating the Wild West of Information Flow

At the White House Frontiers Conference last week, President Obama made this appeal for online fact-checking.

We’re going to have to rebuild, within this wild west of information flow, some sort of curating function that people agree to.

…

There has to be a way we can sort through information that passes some basic “truthiness” tests. That is hard to do but I think it’s going to be necessary, it’s going to be possible. The answer is obviously not censorship, but it’s creating places where people can say: ‘This information is reliable.'”

On the same day, Google News announced a new fact-checking initiative:

Today, we’re adding another new tag, “Fact check,” to help readers find fact checking in large news stories.

…

Google News determines whether an article might contain fact checks in part by looking for the schema.org ClaimReview markup.

That’s a great way to help publishers point out the fact-checking they’ve done on their own. But there’s an obvious flaw with this model. Many of the articles most in need of fact-checking will lack this metadata. And what about independent fact-checkers, like those at FactCheck.org, Politifact, or the growing team of scientists at Climate Feedback who use Hypothes.is to add independent analysis to climate stories in the mainstream media? Ideally news aggregators won’t just recognize ClaimReview markup that’s injected into news stories by publishers. They’ll also recognize claims asserted against those stories. Annotation just happens to be an excellent vehicle for laying ClaimReview metadata directly over the stories it points to.

How might that work? Here’s a Climate Feedback fact-checker in action:

The claim he’s reviewing — “We have been on an upward trend in terms of heavy rainfall events over the past two decades” — is reported in a Guardian story, Disasters like Louisiana floods will worsen as planet warms, scientists warn. Clicking that link takes you to a view of the article with the Climate Feedback annotation overlay.

How might this sort of fact-checking yield metadata that an aggregator like Google News could consume? This screenshot imagines a way:

The annotation tool knows the URL of the article that’s being annotated, the sentence that is the subject of the claim, and the identity of the fact-checker. So it need only gather a few more bits of information — the rating, the name and job title of the reporter — in order to produce a chunk of ClaimReview markup like this:

{
    "@context": "http://schema.org",
    "@type": ["Review", "ClaimReview"],
    "datePublished": "2016-10-06",
    "url": "https://www.theguardian.com/environment/2016/aug/16/louisiana-flooding-natural-disaster-weather-climate-change",
    "author": {
        "@type": "Organization",
        "url": "http://climatefeedback.org/"
    },
    "accountablePerson": {
        "@type": "Person",
        "name": "Emmanuel Vincent",
        "url": "http: //snri.ucmerced.edu/emmanuel-vincent"
    },
    "claimReviewed": "We have been on an upward trend in terms of heavy rainfall events over the past two decades.",
    "reviewRating": {
        "@type": "Rating",
        "ratingValue": 5,
        "bestRating": 5,
        "alternateName": "True"
    },
    "itemReviewed": {
        "@type": "CreativeWork",
        "author": {
            "@type": "Person",
            "name": "Oliver Milman",
            "jobTitle": "environment reporter for the Guardian US"
        },
        "datePublished": "2016-10-06",
        "name": "Disasters like Louisiana floods will worsen as planet warms, scientists warn"
    }
}

In this experimental mockup that data lives in the body of the annotation, and is made discoverable by a ClaimReview tag added to the annotation. The Climate Feedback website could then use the Hypothesis API to harvest these annotations, build a set of pages that include the ClaimReview markup, and invite Google News to crawl them.

This is purely a thought experiment, not a planned feature of Hypothesis. But we think it suggests the sort of curation that President Obama calls for. Publishers can’t be the only arbiters of truthiness. Annotation empowers us to view content through a lens of expertise. If a news aggregator is willing to trust the lens that a publisher provides directly, it should also be willing to trust the lens that an independent team like Climate Feedback provides in an overlay on that publisher’s content.