Team blog: Developers

We just finished a migration away from the unmaintained Pure.css CSS framework to the minimalist and modern Pico CSS. As part of this, we’ve made a few design changes:

• The site now has dark mode. If it changed from light to dark for you, that’s because your system appearance is set to prefer dark mode. You can change your preference in the site footer.

• A few pages that didn’t work well due to tables now display a more mobile-friendly card format on smaller sizes, notably the “invite link” page.

• On mobile/tablet screen sizes, we now collapse the navigation into a small menu. This menu should work with JavaScript disabled.

• In desktop mode, reviews are given more prominence on a few pages. User pages, team pages, and review subject pages now display reviews in a wide column on the left-hand side, and metadata in a smaller sidebar.

• Text pages like the FAQ now display a table of contents, auto-generated server-side from the headlines (no JavaScript needed).

We’ve cleaned up a few smaller things across the board - padding, alignment issues, etc. Perhaps most notably, text now has a bit more line-spacing and a larger font size to aid readability.

This is a big set of changes, and it’s quite likely that there are some design regressions. If you find any, please don’t hesitate to let us know, no matter how small it may seem to you. We want to get the little details right!

Huge thanks to lib.reviews user and moderator Nortix, who built out prototypes of several of these ideas here. There are other ideas in these designs which we haven’t implemented yet, but may add in future, such as review statistics on user pages.

Next up: user preferences. As part of that, we want to revisit how content languages are handled in lib.reviews, giving users the option to write in a language we don’t have a full UI localization for.

Instead of contacting us on IRC or Mastodon to get an account, you can now simply request it through a web form. Any user with moderator permission will be able to approve such requests as they come in. If you’d like to become a moderator, please don’t hesitate to reach out via lib.reviews@permacommons.org.

As with all web forms, we may have to tighten things a bit further if form spam becomes a problem, e.g., with a simple text CAPTCHA.

It’s pretty wild that we haven’t had this until now, but you can finally reset your password if you’ve lost it, provided you’ve configured an email address on your account.

To do so, simply visit: https://lib.reviews/forgot-password

To protect user privacy, the feature won’t tell you if an account exists for a given email address. To mitigate spam risk, it’ll also only send one token every 3 hours.

The main reason this wasn’t done sooner is that email is a bit of a pain to deal with. To mitigate against spam, it has to be carefully configured to pass SPF, DKIM and DMARC checks (various techniques used to ensure the email is legitimate). This in turn should reduce the risk of email landing in your spam filter.

We’re currently using the free Mailgun plan, which should work well while we’re small. Now that we have the first email feature, it will be a lot easier to add the next one, which will likely be the account request process (sending email to moderators to review/approve requests).

Every lib.reviews user can pass along account invitations once they’ve written their first reviews. These registration addresses look like this:

https://lib.reviews/register/<invite code>

Now, if you want to invite a friend to join and immediately also sign up to join a team you participate in, you can do so. Just pass along an invite like this:

https://lib.reviews/register/<invite code>?team=<some team>

For example, if you want to recommend they join the Vegans/Vegetarian team:

https://lib.reviews/register/<invite code?team=veg(etari)an

This will result in a sign-up page for them like this:

Sign-up page with vegetarian team preselected (Own work. License: CC-BY-SA.)

You can also offer multiple teams with ?team=<first team>&team=<second team>&team=<third team>, and so on. It’s a bit of an Easter Egg feature for now; in future, I hope to integrate it more with the team management tooling.

This is the first new feature for lib.reviews since the recent modernization work. It seemed only appropriate to start with a feature that was open as a pull request since (checks notes) 2018! Thanks to Patrick Noll for the original work on the idea.

As mentioned in our last blog post, all forms (e.g., the new review creation form, or the blog post form I’m typing into right now) are now using Zod. Zod is a type-safe validation library that makes it easier for us to catch errors early in how form data is being processed.

With that said, it’s entirely possible that there are new bugs. If a form doesn’t save correctly, or something else looks weirdly broken, please don’t hesitate to let us know.

We’ve been quiet, but we’ve not been idle. As mentioned in our previous blog post, we’ve focused on the following improvements:

• Adding a code formatter and linter. We’re now using Biome, which is a joy to use: fast & powerful.

• Strengthening type passthrough from our data access layer (DAL) to the application, so the TypeScript compiler can catch when the code is trying to access data incorrectly

• Eliminating the use of raw SQL in the application by building more helpers into the DAL that handle all data lookups

TypeScript is a gift that can keep on giving forever, and the next major milestone would be to ramp up the strictness of our type checks, which will require lots of further code changes throughout the app.

For now, the immediate next focus area is an exploration of Zod, a pretty nifty validation library that may come in handy for replacing our bespoke form processing code. And then, finally, we can start adding some features again.

The lib.reviews codebase is now fully in TypeScript. But notice how the title doesn’t say “Goodbye, JavaScript!”. That’s because TypeScript is, of course, transpiled to JavaScript on the client and the server. So, lib.reviews is still fundamentally a JavaScript codebase, just one that’s statically typed.

In a nutshell, TypeScript makes it easier to avoid common mistakes in programming — e.g., situations where you’re passing a garbage value to some function that’s expected to do important work. Instead of just letting you run the code (resulting in unpredictable behavior or errors), TypeScript will yell at you and say: “Wait, that value could be garbage. Are you sure that’s what you meant to do?”

But all of that happens before you build the project. The version that the user (you) ends up running is just pure JavaScript again.

This was not quite as heavy a lift as the PostgreSQL migration, but it still had to be structured into two phases. First, we converted the codebase to the ECMAScript Module standard. Any application that’s more than a single file has to worry about how it loads the files it depends on; ESM is the modern method for doing that. It avoids many pitfalls of the older CommonJS standard, and it works both in browsers and in Node.js, which makes for some nice consistency.

While TypeScript doesn’t strictly depend on ESM, a lot of the current tooling in the JavaScript ecosystem is built around it. So it made sense to take care of this migration first.

Then came the TypeScript migration itself — which was mostly a matter of, one file at a time, spelling out exactly how each function or class is supposed to behave.

Although the code is now fully TypeScript, there are still improvements to be made for it to pass strict validation. Here’s what’s next:

• Adding a code formatter and linter (likely Biome, replacing historical eslint usage). This will help ensure the codebase is consistently formatted, and will catch other common coding issues.

• Re-architecting the data access layer. This is what connects us to PostgreSQL. While the first version of the DAL works well enough, the bootstrap logic is needlessly complicated, and it is not designed with static typing in mind.

• Auditing remaining TypeScript validation issues and ensuring we use correct types wherever possible.

Once all this modernization work is done, we’ll be in a good place to start adding features again. As always, please let us know about any new issues that may crop up.

Following up on the PostgreSQL migration, we’ve just published the first database dump as a .sql file for PostgreSQL. You can download it from the usual address.

The database dumps include all public information from lib.reviews. What’s not included:

• passwords and email addresses

• user preferences (all are normalized to be identical)

• deleted revisions

• session data

• team “join requests” (which may be regarded as DMs to a team’s moderators)

We do not remove deleted revisions from historical dumps, so in principle they are recoverable for a while — but we delete older dumps to free up diskspace, so eventually deleted revisions will disappear fully from all publicly available archives.

You can import the public database dump into a local development environment and look around (but you’ll need to create a new user account to do anything, as the existing ones are effectively locked).

Public archives are an important commitment to user freedom. They make it possible to fork websites, and to reclaim your own contributions for your own use. If you encounter any issue with the database dumps, please don’t hesitate to let us know!

lib.reviews is now powered by PostgreSQL, the veteran open source database that drives much of the open web (and a good share of the closed web, too).

When we started, we used RethinkDB, an interesting document-based database that seemed well-suited for a project like this one, where review subjects don’t necessarily conform to a fixed structure — books, movies, restaurants all need different kinds of metadata.

But RethinkDB ran out of money, and it has been in life support mode ever since. This doesn’t make it a future-proof backend for a website. PostgreSQL nowadays supports flexible column types that can store JSON documents. We use this for our multilingual text fields, review metadata, and more.

Migrating was a major technical lift, as you can see in the 97 commits that comprise the pull request. However, it is now done, and we got a lot more test coverage out of it along the way.

We also ported the database dump script over to PostgreSQL, and will soon be providing sanitized database dumps in the new format.

We’ve tested this extensively, but there are bound to be bugs. If you find one, please don’t hesitate to open an issue!

We’ve enabled the following new languages for UI and content: Arabic (experimental), Finnish, Hindi, Lithuanian, Slovak, Slovenian, Turkish, and Ukrainian. Huge thanks to all the volunteer translators who have made this possible. :)

To create a version of lib.reviews in your language, please see our FAQ.