Lumen — The Year in Review

September 2019-August 2020

Lumen Database Team
Oct 14, 2020 · 12 min read

By: Adam Holland, Andromeda Yelton, and Chris Bavitz

Introduction

September 2019 through the end of August 2020 marked the first year in which Lumen operated with a generous supporting grant from the Arcadia Fund. During that year, the project’s primary objectives fell within three themes: (1) technical improvements to the Lumen site and database; (2) expanding research opportunities, both internal and external; and (3) outreach, both to possible new notice-submitters and to the various constituencies of the Lumen user community. This post draws from Lumen’s first annual report to Arcadia and provides an overview of the project’s key activities during the past year.

To say the least, it was a complex and difficult year on a number of fronts — most notably, because of the COVID-19 pandemic that forced us into a remote work mode for much of 2020. That said, we were able to make significant progress on a number of key fronts:

The remainder of this overview addresses and provides more details on these main themes in the order outlined above.

(1) Technical Improvements and Progress

In addition to too many small-scale bug fixes and one-off requests to name, the Lumen developers’ activity in the first year fell into several key main categories:

Security/anti-obsolescence updates

In combination, these upgrades and improvements improved system security and system performance, making the database notably faster for users. Additionally, the various improvements keep the site effectively modernized, which in turn allows developers to take advantage of and implement further improvements without too much work. Finally, the ongoing ElasticSearch upgrades allow Lumen administrators to more quickly and effectively redact sensitive data in Lumen’s notices (in addition to making site search functionality more powerful for users).

Overall, these technical improvements make the Lumen site easier to use by and more responsive to both its internal team and the research community. They also serve to “future-proof” the site to the extent possible, making it far more likely that Lumen will be able to continue to exist and thrive indefinitely, and making continued and sustained improvements easier to accomplish.

Improvements to the Lumen administrative interface

Improvements to receiving and sharing notice data

User Interface

Lumen made a series of changes regarding how visitors to the site see the URLs that are part of each notice. The changes make it possible for Lumen to present notice URLs in a truncated form to casual Lumen visitors, while still granting access to complete URLs to Lumen accredited researchers. Casual Lumen users can view one notice’s full set of URLs by providing an email address. Researchers with credentials can be granted access to notices within a limited time frame, up to a maximum specific number of notices, and with or without use of the Lumen API, and can also be given the ability to generate “permanent” versions of Lumen notice URLs that are suitable for use in published works or for citation.

(2) Research Using the Lumen Database

Lumen granted research credentials to forty-nine different researchers during the year in question. These researchers range from college undergraduates who have recently become interested in copyright law or censorship, to international researchers from a wide range of countries, including Brazil, Turkey, Ukraine, France, India, Austria, Russia, Germany, and the UK, as well as EU-affiliated researchers and international NGOs such as the Committee to Protect Journalists, as well as law professors and journalists and others in the United States.

Many of the projects that these researchers are working on are still ongoing, such as Professor Eugene Volokh’s ongoing series of law journal articles about falsified court orders and online defamation law. Some of the completed research projects include:

There are also many shorter articles online referencing or relying on Lumen, such as this one, from the Sunday Guardian Live, or this one from TorrentFreak.

Over the summer of 2020, the Lumen team also worked closely with a Harvard Law School student research assistant to begin developing a taxonomy of takedown notices, their underlying data, and the various involved stakeholders. This draft taxonomy seeks to cast light on the range of interests and incentives that a given stakeholder in the notice and takedown (“N&TD”) ecosystem must balance with respect to whether a particular piece of information should come down and the degree to which there should be transparency regarding the request and any subsequent action taken. It is the Lumen team’s hope to soon turn this working draft into a white paper, as well as the raw material for a Lumen workshop, as well as use it to inform discussions on any statement of best practices regarding N&TD transparency.

(3) Outreach

Events

The Lumen team’s original plan had been to hold a fairly intimate in-person workshop over the course of two days, as a way of initiating conversation between the various parts of Lumen’s user and research communities, and to plant the seed for more detailed and targeted workshops to come. Unfortunately, the COVID-19 pandemic got in the way of those plans, and as a result, the June workshop was held virtually. Although the Lumen team members were of course very disappointed to not be able to have the full in-depth workshop we had planned, especially the face-to-face network building and conversations, hosting a virtual event had some positive aspects. These included lower costs and the possibility of drawing more participants. The end result was that we were able to diversify and expand the initial invitee list substantially, including a wider range of interested parties, and — critically — giving the group more international representation. On that note, it meant that some foreign human rights activists who would otherwise not have been able to attend were present — including representatives of EngelliWeb, which has published a human rights report on Turkish takedowns that relies heavily on Lumen. The most recent of EngelliWeb’s reports can be found here.

Using the lessons learned from this first virtual event, and anticipating that virtual events will be the norm for the foreseeable future, Lumen has planned a series of smaller and more topically focused events for the coming fall and winter, the first few of which will be focused on learning more from current and prospective Lumen researchers.

Outreach to New Sources of Notices and Notice Data

Encouraging recipients and senders of takedown notices to share copies of those notices with Lumen has proven to be one of the biggest challenges the team has faced. Although Lumen’s name recognition has clearly improved, due in no small part to the increased publicity from outside journalism and research publications, and although those companies with whom Lumen has existing relationships are generally positive about the benefits of sharing, some institutions are still loathe to share notices and notice data. Finding ways to be more effective at turning preliminary outreach into new data-sharing arrangements will be a top priority for the Lumen team in the coming year.

General Outreach and Media Participation

In addition to the June 2020 workshop mentioned above, and their ongoing work with Lumen researchers, members of the Lumen team participated in the following activities:

Of special note, on December 16, 2019, Lumen project manager Adam Holland and Lumen PI Chris Bavitz made comments to the Third Meeting of the Stakeholder Dialogue on Art. 17 of the Directive on Copyright in the Digital Single Market in Brussels. Article 17 references “”Use of protected content by online content-sharing service providers.”

The presentation was well-received, and also was a boost to Lumen’s broader publicity. Lumen was invited to join a multi-stakeholder mailing list regarding ongoing Article 17 discussions, in which it continues to participate, and also made several new EU contacts, including a former member to the EU Parliament, who have kept Lumen apprised of opportunities to contribute comments or thoughts to ongoing copyright and intermediary liability-related legislative and regulatory discussions within the EU.

A copy of the remarks can be found at:

Bavitz, Chris, Holland,Adam, “Lumen Presents Comments to the Third Meeting of the Stakeholder Dialogue on Art. 17 of the Directive on Copyright in the Digital Single Market in Brussels” (December 17, 2019) https://www.lumendatabase.org/blog_entries/807

A recording of the day’s proceedings is available at:

“COPYRIGHT STAKEHOLDER DIALOGUES — Streaming Service of the European Commission,” https://webcast.ec.europa.eu/copyright-stakeholder-dialogues-16-12, (accessed October 8, 2020)

Lumen’s participation begins at approximately the 4:00:00 mark.

Other outreach efforts

The Lumen team has also had productive conversations with a variety of other activists and researchers about possible cooperative efforts, including with Carrie Goldberg, an American lawyer specializing in representing victims of so-called “revenge porn”; the “Disinfodex” project emerging from the Berkman Klein Center’s 2019–2020 Assembly Program; the Digital Public Library of America, the Reporters Committee for Freedom of the Press, Harvard’s Caselaw Access Project, and the Humboldt Institute for Internet and Society in Berlin.

Social Media Statistics

Lumen maintains a Twitter account, from which it tweets or retweets about content moderation, takedowns, censorship, academic freedoms, the “right to be forgotten” and other news related to online information. During the period from September 1, 2019 to August 31, 2020:

Data and Material Produced

During this year, the Lumen database added ~2.6 million more notices, referencing many millions of URLs, involving approximately fifty-eight thousand separate entities. As mentioned above in the technical improvements sections, we put into place our planned changes for displaying URLs in a truncated form to casual Lumen visitors, while granting access to full notices with complete URLs to researchers requesting access. We were and are gratified to have received relatively few complaints from users regarding the change, and none from active researchers. Current policy is to grant a single request per email address to view a notice. Lumen has consistently averaged approximately one thousand such requests per day, but may revisit and revise the bounds of that policy in the coming year.

During the time period from September 1, 2019 to September 1, 2020, Lumen received almost six hundred thousand unique visitors, who visited Lumen close to fourteen million times, viewing over nineteen million unique Lumen website pages. These traffic numbers represent an approximately 50% increase in activity from the previous year, which the Lumen team attributes to both more research activity and greater use of the site by the public at large.

The most visited Lumen URL was http://lumendatabase.org/notices/9415, which is a Google placeholder notice for search results that contain URLs reported as illegal under German youth protection laws. There is no way to be certain as to why this notice is visited often, but it may be that this notice’s popularity is a rough proxy for the number of such removals by Google in Germany and the number of searches the internet-using German public performs for the underlying material. Or, it could be the relative novelty of the new laws is driving interest. The second most visited Lumen page, close behind first in terms of total visits, was Lumen’s own search page.

Conclusion

In the year to come, the Lumen team looks forward to continued progress on all fronts, from expanding the scope, scale and impact of research done with Lumen’s data and gathering new sources of takedown notice data, to improving the Lumen user experience and adding new members to the Lumen team, There will be more events, whether virtual or in person, more publications, and more opportunities to get involved.

Berkman Klein Center Collection

Insights from the Berkman Klein community about how…

Berkman Klein Center Collection

Insights from the Berkman Klein community about how technology affects our lives (Opinions expressed reflect the beliefs of individual authors and not the Berkman Klein Center as an institution.)

Lumen Database Team

Written by

Collecting and facilitating research on requests to remove online material. Visit lumendatabase.org and email us if you have questions.

Berkman Klein Center Collection

Insights from the Berkman Klein community about how technology affects our lives (Opinions expressed reflect the beliefs of individual authors and not the Berkman Klein Center as an institution.)

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store