_    _                   
  _______ _ ___| |__(_)_ _  ___ ___ _ _  
 |_ (_-< `_/ _ \  _ \ | ` \(_-</ _ \ ` \ 
 /__/__/_| \___/_,__/_|_||_/__/\___/_||_|

--------------------------------------------------------------------------------

Designing Products without Databases

Most major software projects require some sort of database to persist data for users to access. At least that’s how we usually think. The problem is that databases introduce a lot of complexity to our software and infrastructure which is often unnecessary. Put in other terms, they also just cost a lot of money to run compared to just using compute. So when designing certain products for the web, it’s useful to consider other possibilities besides spinning up a new instance of your favorite database1.

Persisting State on the Client

The web platform has progressed light-years beyond the simple HTML that Tim Berners-Lee was writing back at CERN in 1990. New technologies in the browser such as IndexedDB allow us to create full databases within the client’s browser to persist up to gigabytes of data that our programs create. For something a little less complicated, Local Storage allows us to store simple key-value pairs within the browser to store smaller amounts of data. Storing this data on the client is not only better for the user since they don’t have to wait as long for data to be transmitted, but also better for the developer since they don’t have to store this data themselves.

The major drawback of this approach is not being able to easily sync data between a user’s multiple devices. This could be a dealbreaker for many applications, but for others, it may not be as much of a problem. Not requiring users to make accounts to persist their data in any way can often improve the user experience and lower the chance that they decide it’s too much work to try out your product.

One of my favorite examples of this is Coursicle, a website that allows students to visually build and layout their course schedule during registration. Digging into the developer tools, they use Local Storage to persist all of your schedules in the browser. Since there’s no need for users to register an account before using the product, trying it out is extremely easy. Plus, there isn’t much of a need to sync state between multiple devices since most people consider creating your schedule to be more of a “laptop activity” — you don’t want to be doing all of that on a tiny little phone screen anyway.

When storing application state on the client in this way, I believe it’s best practice to offer users a choice to export and import their data as a file that they can transfer to another device or backup. This is useful when a user is migrating devices or just wants to ensure that their data doesn’t get wiped away by accident.

Persisting State within URLs

One of the most common requirements for a database are sharing or collaboration features. This often looks like generating a shareable link for users to give out so other people can view certain things. There’s a practical limit on how much information can be stored within a URL — it’s going to start looking weird if you email someone a link that’s more than a few lines long.

This is why databases are often used for these features: we can just store the ID of our data in the URL and request the full object from the database. But for relatively small amounts of data, it’s possible to just encode everything we need directly in the URL. For instance, if we were making a document editor like Google Docs, it wouldn’t be wise to encode several thousand words in the URL and send it to somebody. But going back to our course registration example, sending a link with an encoded list of a handful of classes and their sections would be pretty reasonable.

The key is distilling the information we need to store into only what is absolutely necessary. Continuing with course registration, we wouldn’t store every single piece of information associated with each class like the professor, class ratings, or even the days and times it takes place. Once we know the course and section, all of that information can be fetched dynamically from our data source (in this case, the university’s list of available courses). By shrinking down the amount of information we actually need to store, encoding this data within the URL becomes more feasible.2

Sometimes distilling this information too much can open our users up to risk. If our information can be reduced to a single API key, then the user wouldn’t want to go around sharing that key since it may give access too broadly. I just recently ran into this problem when designing a tool to create public shareable links for Todoist projects, where the only information needed to be shared was the user’s Todoist API key and the project ID they wanted to share. Instead of just encoding this data into the URL, the user first encrypts this pair of data into a token that they can then share.3 Since the server is the only device with access to the secret used to encrypt and decrypt these tokens, the user doesn’t have to worry about the people they share the link with having read/write access to their entire Todoist account. Even further, the user doesn’t have to trust me as the developer to securely store their precious API key in a database, and I don’t have to have the headache of storing and securing such important information.

Persisting state in some of the ways I’ve mentioned here is not always practical for many applications, but it does offer a way for some to reap the benefits of not having to roll an entire database for features that can be implemented in more effective ways. As someone who enjoys making little side projects to practice web development, it’s great to be able to create websites that implement more advanced features while still not requiring paying for a database.4 And as a user, it’s neat to see products that use these features in the wild, knowing that my data is safe and secure within my browser instead of some random developer’s server. It’s worth considering next time you’re implementing a new feature for a product: do you really need that database?

Footnotes

  1. Or your favorite wrapper for another database wrapper.

  2. It also helps to be careful with how you encode the data here. Using Base64 to encode data is generally preferred for URLs to make it look like some sort of random id, but certain characters have to be replaced to make it URL-safe. Additionally, Base64 encoding JSON data generally makes the URL longer than it has to be since it uses so many extra characters (all of the quotations, curly braces, etc. add up). One option is serializing your data by just concatenating your strings together, separated by some special character like a semicolon. If your data is encrypted, you could also combine the ArrayBuffers that are returned and just Base64 encode that directly.

  3. The web crypto API (crypto as in cryptography, not cryptocurrency) provides the ability to do a lot of this either on the client itself or on the server if you’re using a JavaScript backend. I’m no expert on the different algorithms and implementation details of this API, so it definitely took a bit of “help” (as much as I hate you, thanks ChatGPT) to get these details right for this project while I’m still learning the ins and outs.

  4. Or for that matter, paying for really anything to deploy these projects — the free tier on Vercel is extremely generous.

--------------------------------------------------------------------------------

Thanks for reading! If you enjoyed this post, please consider sharing it. If you have any comments, questions, or feedback, don't hesitate to reach out.

My personal projects are hosted on GitHub, and my work experience is listed on LinkedIn. I’m also able to be reached via email.

Comments

Recent Posts