Overview of the Hypothesis System

Introduction

Hypothesis delivers a set of components and capabilities that work together to enable a wide range of annotation-powered applications. Here we provide an architectural overview. The major components are:

• An annotation viewer and editor that runs as an overlay in the browser (“the client”).
• An annotation service that stores, searches, and displays annotations, manages users and groups, and delivers the client into pages that actively embed it (“the service”).
• A Chrome extension that injects the client into web pages that don’t embed it (“the extension”).
• A proxy that injects the client into web pages routed through it (“via”).
• A PDF renderer (Mozilla’s PDF.js).
• A relay that resolves an annotation link to the extension if it’s installed, otherwise to the proxy (“bouncer”).
• An LMS app (aka LTI external tool) that injects the client into learning management systems (“LMS app”).
• An API for programmatic access to the service (“the API”).
• A bookmarklet that injects the client into web pages (“the bookmarklet”).

Terminology

An authority contains a set of users.

First-party user accounts. Most Hypothesis users who sign up directly with the service belong to the hypothes.is authority.

Third-party user accounts. These include LMS users who are provisioned by the LMS app in the lms.hypothes.is authority, and eLife users who are provisioned by the eLife/Hypothesis integration in the elifesciences.org authority.

An AuthClient is a registered credential associated with an authority.

The client

The client is written in JavaScript, uses the Angular framework, and has cooperating parts: the annotator and the sidebar.

The annotator

The annotator is injected into a host page, where it:

• Gathers page metadata. That always includes the URL and web page title. It can also include other metadata. The annotator sends host page metadata to the sidebar when the client activates in the host page.
• Reacts to selections. When a user selects text, the annotator displays buttons for making an annotation or highlight. Either, when clicked, runs code that computes the W3C-style selectors that locate the selection in the host page, and sends that information to the sidebar.
• Displays highlights. The sidebar uses the API to ask the service for annotations that belong to the current host page. When the annotator receives messages from the sidebar announcing the retrieval of annotations, it paints them on the host page as highlights.
• Tells the sidebar about actions in the host page. For example, clicking on an annotation’s highlight in the host page tells the sidebar to focus the corresponding annotation card.

The sidebar

The sidebar is injected into an iframe in the host page, where it:

• Queries the service for annotations belonging to the host page.
• Displays them in the iframed sidebar that users primarily think of as the Hypothesis app.
• Tells the annotator about actions in the sidebar. For example, hovering over an annotation card in the sidebar tells the annotator to emphasize the corresponding highlight in the host page.

The sidebar contains annotation cards that connect to highlights in the host page, display annotation metadata and content, and enable authorized users to edit the content.

When editing, users can include:

• Freeform text
• Text enriched with Markdown and/or limited HTML).
• Math expressions supported by KaTeX.
• Images, audio, and video.

The extension

If a site embeds the client in one or more of its pages, the sidebar launches for all visitors to those pages, in all browsers. Since most sites do not embed the client, users who run Chrome and have installed the Chrome extension can use it to inject the client into any pages they visit.

For users of other browsers:

Firefox: A version of the extension, based on the Chrome-inspired WebExtensions API, is in development.

Brave: The Chrome extension works in Brave, but (as of November 2018) must be manually installed.

Edge: Supports the WebExtensions API, so a version of the Hypothesis extension is possible but not yet developed.

Safari: Does not support the WebExtensions API, so the Hypothesis extension is unlikely to run there until/unless Safari supports WebExtensions.

Mobile browsers: None can currently run the extension.

The bookmarklet

The Chrome extension is the primary way users inject the client into web pages that don’t otherwise embed it. For users running other browsers, the bookmarklet affords an alternative way to inject the client.

Limitations of the bookmarklet include:

• Does not work on sites that enforce Content Security Policy (CSP), for example, GitHub.
• Does not displace the browser’s built-in PDF viewer, so PDF annotation won’t work (except in Firefox, where PDF.js is the native viewer).

The service

The service is written in Python, uses the Pyramid framework, and supports:

• The API used by the client, the LMS app, and a variety of other annotation-powered apps, to manage annotations, users, and groups.
• A companion realtime API, based on the WebSocket protocol, used by the client (or other apps) to receive notifications when there are server-side changes to annotations.
• An annotation viewer (“activity pages”) that displays sets of annotations. These include users’ own annotations (“the profile page”), those made by members of a group (“the group page”), or those matching other search criteria (“the search page”).
• A set of administrative pages used by operators of the service to create and list users and groups, mint credentials for partner services, and shadow-ban (“NIPSA”) bad actors.

The via proxy

The proxy running at https://via.hypothes.is is based on pywb, a toolkit formerly used by the Internet Archive for deep capture of web content. Hypothesis uses it to enable the formation of URLs, like https://via.hypothes.is/https://www.example.com, that inject the client into pages that don’t embed it, for users who are not running the extension.

When the proxy rewrites a page, it:

• Injects the Hypothesis client
• Rewrites URLs in the page so they also route through the proxy
• Injects JavaScript code (“wombat”) that intercepts AJAX reqests and routes them through the proxy

The proxy does not work on sites that require authentication and/or cookies.

The PDF renderer

When the client “anchors” an annotation to a selection in an HTML page, it manipulates the DOM (Document Object Model) of the page. PDF resources undergo similar treatment once rendered into the DOM using Mozilla’s PDF.js. Hypothesis deploys PDF.js in several ways:

• The extension. The Chrome browser has a built-in PDF viewer. When the Hypothesis extension launches in a Chrome window that is using that built-in viewer, the extension rerenders the PDF using PDF.js and then deploys the Hypothesis client into that rendered HTML page.
• The proxy. When via proxies a PDF resource, it injects PDF.js to render the PDF, and then injects the client to annotate the rendered HTML.
• The LMS app. It relies on via.
• Sites that embed the client. A site that embeds the client in an HTML page does so with a simple script tag. To embed the client in a PDF, it can deliver an HTML page that includes a PDF.js rendering plus that script tag.

The relay (“bouncer”)

An annotation link (“direct link“) refers to a selection within a document. It encapsulates the URL of the document, and the W3C-style selectors that locate the selection in that document. A user who clicks an annotation link expects the following: the page opens in the browser, the client loads, the annotation is focused in the host page and in the sidebar.

To capture and share an annotation link, a user clicks the Share button on an annotation card displayed in the client, and then clicks the Copy button. The captured link looks like this:

https://hyp.is/JytOjPQNEeiwxJcBcxSJTw/www.example.com/

The syntax is:

{relay}/{annotationId}/{targetUrl}

The {targetUrl}, which makes the URL legible to people, is optional.

When another user clicks that link, it can resolve in two ways. If that user isn’t running the extension, the relay produces an URL that routes the target page through the proxy like so:

https://via.hypothes.is/https://www.example.com/#annotations:JytOjPQNEeiwxJcBcxSJTw

The proxy injects the client into the page, the client loads all annotations for the page and focuses on the specified annotation.

If that user is running the extension, it instead resolves like so:

https://www.example.com/#annotations:JytOjPQNEeiwxJcBcxSJTw

In that case the client is already loaded, and only needs to focus on the specified annotation.

The relay determines whether the user who clicks a hyp.is link is or is not running the extension, and redirects to the appropriate URL.

Why not always use the proxy? Reasons include:

• Users running the extension can annotate paywalled content, proxy users cannot.
• It’s faster and more reliable to use the extension.

The LMS app

The LMS app is an LTI tool provider that enables teachers to activate the client in Canvas, Moodle, Sakai, and other learning management systems. The LMS app offers these capabilities:

• In Canvas, teachers can inject the client into web pages, PDFs hosted on Google Drive, and PDFs hosted within Canvas. In other LMSs the first two options are available.
• Hypothesis user accounts and groups are created on demand, based on LMS user accounts and courses, so students need not sign up for Hypothesis accounts.
• When the LMS app injects the client into web pages and PDFs, it automates the Hypothesis login so students need not do that either.

When the LMS app provisions users it creates third-party accounts, which differ from the first-party accounts created when users sign up for the service. Here are some key differences between the two account types:

1st party account	3rd party account
Has a password, can log in and out	No password, is automatically logged in
Can annotate any site	Can only annotate sites controlled by a 3rd-party authority registered with the service

The API

The API provides programmatic access to the service. The client uses it to:

• Authenticate to the service
• Create, update, delete, search for, and moderate annotations
• Create and manage users and groups

See https://h.readthedocs.io/en/latest/api/

Authentication

Name/password login

Access to user- and admin-facing views in the service uses name/password authentication.

OAuth login

The client served to all these users performs OAuth login that’s enabled by an AuthClient.

For first-party clients, the user opens a login window and enters name/password. The OAuth flow begins with a call to {service}/oauth/authorize to retrieve an OAuth code, and continues with a call to {service}/oauth/token to receive access and refresh tokens.

Third-party clients are delivered into the browser by way of a cooperating service that holds AuthClient credentials. That cooperating service (e.g. at elifesciences.org) uses the credentials to manage users in the AuthClient’s authority.

Token login

At the URL https://hypothes.is/account/developer, a user logged into the service can get an API token that enables other clients to use the service on that user’s behalf.

Examples of other clients include those listed here. Several of these, and some others, are helped by a library/toolkit.