Team blog: Developers

In case you haven’t been following along, the FreeNode chat network recently imploded, and hundreds of of open source projects have migrated their chat infrastructure to Libera.Chat, a community-run alternative. That includes lib.reviews, and you can now find us on #lib.reviews on irc.libera.chat.

The lib.reviews VPS is now running on Ubuntu 20.04 (Focal), the latest Long Term Support release, which receives security updates until April 2025. The upgrade was fairly painless, but the do-release-upgrade wrapper script was interrupted, and I had to manually fix some broken packages.

I also fixed a small bug which has been annoying me for a while: The Wikidata search box wasn’t working when you started a new review, until you un-focused and re-focused it.

lib.reviews is now using Greenlock Express instead of the outdated auto-sni module to provide HTTPS access. As a user of the site you should not experience any difference, but SSL Labs reports an improved security grade for the site (from A- to A).

As part of routine systems maintenance, I’ve updated the following components:

• ElasticSearch to version 7.8 (which adds improved stemming for some languages, but required significant changes to the search code)

• Node to the 12.x LTS series

• Ubuntu on the server from the 16.04 LTS series to 18.04 LTS

There should be no significant user-facing impact of these changes, but if you notice anything no longer working as it did, please don’t hesitate to file an issue.

My first patch for Lib.Reviews went live earlier this month! A small ease of use feature for users. When you follow a link to a lib.reviews page and click sign in or register from that page, you will now be redirected to the page you came from. This feature will be very nice to have as lib.reviews grows and users become more likely to enter the site from arbitrary links.

The second feature went online last week. This patch ia a hidden ease of use feature for non-english users. Users can post links with a signupLanguage query to advertise registration for lib.reviews in any supported language. Say all your friends speak esperanto, you would send them lib.reviews/register?signupLanguage=eo so the website will show up in esperanto without having to search for the language drop down. This should help new users feel welcome right away.

Mastodon is one of the most exciting developments in open source of the last couple of years: a decentralized social network that is actually taking off. If you’re unfamiliar, here is a good concise introduction, and here is a more comprehensive one.

The super-short version is that, like email, Mastodon is spread across many servers that talk to each other. You can follow users on any server, and messages spread through the network as needed. Each server gets a limited view of the total network, based on who the users on that server are interacting with.

This isn’t a new idea: GNU Social and Diaspora are following a similar model (GNU Social and Mastodon are interoperable). But Mastodon is very user-friendly, development is very active, and a rich ecosystem of apps and communities is forming around it. Exciting times!

As far as lib.reviews is concerned, there’s now a quiet little account you can follow on Mastodon. Down the road, I’d love to explore whether we can make lib.reviews actually talk to the federated social media universe. For starters, it shouldn’t be too hard to make it possible to follow lib.reviews users on Mastodon, thanks to the ActivityPub standard. Help, as always, is most welcome.

Supporting OpenStreetMap properly will take many steps (integrated maps for picking review subjects, search by location, etc.), but a first small step is now complete: lib.reviews pulls the labels for review subjects directly from OpenStreetMap for any subject with an OSM way or node URL. For example, this restaurant has an OpenStreetMap link associated with it. If you click the link to OSM, you’ll notice that the name (“Hareburger”) is a property there. We are updating these names for items with OSM URLs every 24 hours.

Next up will be support for opening hours, which are already included for many businesses in OpenStreetMap, and which are an essential property for restaurant reviews.

lib.reviews has offered database downloads almost from the beginning, but until recently, these did not include uploaded media. As of last week, we are now creating an archive of all uploaded media files every week and offering it here. Note that in order to use these files in a license-compliant manner, you will also need the file metadata from the database dumps.

Thanks to user arx, our database downloads are now also mirrored to IPFS here (may take a while to load). IPFS is a cool project to build a distributed web and very much worth checking out.

If you’d like to run a mirror of some or all lib.reviews content, please go ahead—and please let us know through a pull request against this file. One of the huge benefits of free/open licensing is that we can all work together to ensure that the stuff we create doesn’t go away if one particular website does.

I’m investigating the possibility of turning lib.reviews into a platform co-op. The underlying idea is simple: users own the platform and vote democratically on all decisions. I’m a member of social.coop, which uses this model for a social network that is part of the larger Mastodon network (a good intro if you’re unfamiliar), and my experience with it has been very positive.

What would this mean? Nothing for the day-to-day use of the site. Folks who want to participate in decision-making could sign-up and contribute to costs for hosting and development, but also vote on decisions, e.g., which features to prioritize. There would probably be a free tier for co-op membership as well, just to make sure that active contributors can join up even if they can’t contribute financially.

If this is something you’d like to see, here’s a simple thing you can do: star lib.reviews on GitHub. I’d like to use OpenCollective to manage finances for the platform, and they require a threshold of 100 GitHub stars. That seems fair—there should be active user interest before it makes sense to set up a funding/governance model.

freeyourstuff.cc is a separate project from lib.reviews, but with a related purpose. It’s a Chrome/Chromium browser extension that lets you download reviews you’ve contributed to major websites, including Amazon, Goodreads, IMDB, TripAdivsor, and Yelp. You can publish them on the site as a backup, or keep your own copy.

The most popular feature turns out to be the Quora downloader: tens of thousands of Quora answers have been downloaded and re-published under free licenses with it (see the upload directory).

Today I pushed out a week’s worth of updates to the extension. From a user perspective, the main changes are that IMDB and Amazon extraction works again (design changes had caused the plugins to break), Quora extraction should be more well-behaved, and all plugins have a clear “busy” indicator when they’re doing stuff.

Under the hood, the extension now uses async/await functions instead of callbacks to make the download flow a lot more understandable, especially for complex plugins like the Quora one which have to monitor changes to a page dynamically made with JavaScript not under the plugin’s control.

If you’ve contributed reviews to other sites, I encourage you to use this extension to keep your own copy (and please report issues you experience). In future, we’ll make it easy to migrate individual reviews to lib.reviews or other sites, as well.