hckrnws

I run an offline-first app[0] that has a sync mechanism (over Yjs). To solve server longevity we just sell you a lifetime license of the app and allow for file-based sync.

While I agree with the sentiment of the article most users do not even know what a server is, much less capable of self-hosting. Syncing a folder over Dropbox or Google Drive, though, is simple enough.

[0]: https://nestful.app

This pretty much matches the way I think it should be done ideally. I'm still pretty new to local-first though. Are there data formats or types of apps that don't lend themselves well to file-based sync?

For your app, how do notifications of changes get propagated? Does it depend on the backend (Google Drive, etc) supporting that, or does it just do polling or something?

For the desktop version just watching the directory for changes is enough to achieve that. Nestful is not aware the files are syncing, it just reads files as they are.

Sure but not all backend operations can always be mapped to flat files on Dropbox, like when you have certain real-time collaboration features, syncing on graph/tree-like data structures, or perhaps permissions in case of a company tool. That's why longevity of the "data" aspect is usually easy to solve when a copy of the data is already local, but then you still risk losing a part of what it is that makes the app when there is no way to run the app's syncing backend yourself.

It serves more use cases than you might think, especially with CRDTs. Nestful is a tree structure and uses Yjs for sync. Permissions can be done in the file system level.

Yes, I agree this is not an end-all be all solution, but the tradeoff is often worth it.

Yep love CRDTs but I think we are in agreement that it's a matter of trade-offs whether or not to use a backend (beyond a shared fs service). The idea with ejectable is that if you've decided the trade-off of a backend is worth it to enable certain features/UX, then it's a good way to ensure longevity of the entire app for users.

I think this is a subset of the fifth (of the seven) ideals of local-first software https://www.inkandswitch.com/local-first/#5-the-long-now

> Local-first software enables greater longevity because your data, and the software that is needed to read and modify your data, are all stored locally on your computer.

There are many ways to achieve this goal - open document standards, open source servers, a escrowed release of the server, or this idea of a bail out system for taking the current version and self hosting it. All are commendable.

Actually achieving all seven ideals is hard, and there isn't much modern software that does it. But in my view anyone trying to achieve even a few of the ideals is making strides towards a worthy goal.

There is a lot of exciting stuff happening in the local-first software movement at the moment, and a lot of that is related the sync engines that are being built (disclaimer: I work for ElectricSQL, we are building one). These sync engine don't inherently make your software local-first, fitting all the ideals, but they do make it a lot easer to do so. They are an important building block. But there is more needed, we need more open document standards - Automerge, Yjs, Loro and the other CRDT data structures are perfect lower level data structures to build these higher level abstractions on. Martin Kleppmann has talked quite a bit about sync engines that are disconnected from the underlying application, essentially pluggable sync engine, you choose who to use to sync you copy of a document or application - I'm excited to see where this goes.

But, we also need to free up the application distribution platform. The app stores are walled gardens that prevent some business models, native apps (while performant) are tied to a specific platform (or even version!), and the web is inherently tied to servers and browsers. The web platform though, i.e. JS, HTML, CSS, is perfect for building high longevity software that runs anywhere, on any device, the issue is the distribution. We need a middle ground, an application package that built on web standards, but isn't tied to a server. I want to download a bundled app to my machine and have a copy, email to a family member, or even open and hack on it. Thats the final missing piece.

A downloadedable app, with an open and pluggable sync engine would achieve the same goles of this ejectable idea.

The application package you're looking for is a single HTML file with no external dependencies, especially if it avoids minification, obfuscation, or the use of technologies like WASM blobs which require complex external toolchains to disassemble or modify. This is very achievable right now! See tools like TiddlyWiki or Decker, for example.

The primary barrier to an entirely server-free or server-agnostic webapp ecosystem is browser vendors choosing to seal the newest JavaScript APIs behind "secure contexts" which are only available to documents served over HTTPS.

I agree, I develop small utilities as single HTML for all the reasons you list (and fun), but having to work around browser protections for some various APIs can be a bummer.

The average internet user could be exploited fairly easily if every HTML file had immediate access to all the lower level APIs being introduced[0], and we end up looping back around to some sort of signing or alternative install method (pwa).

Curious to find the balance between distributable and "safe" enough to achieve wide adoption.

0: https://developer.mozilla.org/en-US/docs/Web/Security/Secure...

Not related to the topic, but I first came across Thymer about (I want to say) 2 years ago when I looked for a todo.txt GUI. I found the blog they used to document building it: https://80daystartup.com/

Somehow, this thing is still not available for testing yet. Not hating but isn’t this supposed to be the category where new solutions launch every single day? While I am still very much interested, I’ve lost any hope that this will become a real thing to try out any time soon.

Hah yes it took a while, thanks for following! When we started the blog we initially thought of building something very small, but along the way realized that the kind of thing we would really love using ourselves would be a much bigger project. It basically involved building a completely new editor from scratch, sync engine, adding multiplayer and end-to-end-encryption, and so on. It would definitely have been faster to take shortcuts and use many existing libs and frameworks around editors, UI and syncing, but then it wouldn't have been something meaningfully different I think, as you point out there's plenty of options in the space so why take the same approach.

We're very much still working on it though and it's finally at the point where it's just some polish, so pretty sure next Q.

It is possible that life got in the way. Thymer did seem a great idea.

If you use a local, browser-based database, and allow your app to be installed as a PWA, then you are ejectable? The only "server" is just a host for HTML+JS that your browser automatically downloads when you visit the site. The app runs locally, like a desktop app, inside your browser. No need for the items listed in the article.

In theory, sort of, but I think that in practice this is not a solid enough solution to make an app and its data future-proof. As I see it, PWAs and browser-based databases are more like an offline cache rather than a permanent way to store apps and data:

- They don't get backed up or synced between devices.

- It's very easy to delete them (e.g, you clear browsing data) or lose them (e.g., you get a new computer).

- Users barely know about them, so they don't know they should "preserve them". For users PWAs are basically just a way to get a launcher for the app, and they have no visibility over locally-stored data.

I think browsers right now are not the right platform to give a future-proof guarantee. Disclaimer: I say this while working on a product that wants to become that platform¹, so I've had the chance of thinking about this problem quite a lot in the past few months.

¹ I wrote a bit about it at https://pscanf.com/s/340/, if you're interested.

Those are good points, but, of course, until your platform takes over the world, I would argue that browsers are not going anywhere, and work now, bearing in mind the corner cases you mention. Seems to me that fixing those persistence issues is easier to do by improving, rather than replacing, browsers.

I definitely agree that we should be improving browsers in that way, though I'm not sure that it's easy and I'm pessimistic it'll ever happen. Mostly because - cynically - I think that the vast majority of users will never care about owning data and software, so I don't see the incentive for browsers vendors to implement features to make it possible.

In their current state, I personally don't find PWAs + local databases ergonomic enough. So actually what I'm hoping to get out of the platform I'm building is - somewhat egotistically - an alternative that works for me personally and possibly for a few others like-minded individuals, not a browser replacement to take over the world.

There are many customers that care about owning their own data. Despite what you might think, there are lots of companies that do not trust sending their data to the cloud. In addition, there are lots of companies and individuals that have all their data local, for a variety of reasons (often security) and primarily use desktop applications to work with that data. The "vast majority" may never care, but the compliment is still a very sizable market.

I would love to see another way to deploy local software, aside from building dedicated Windows and MacOS apps. There are efforts in the Docker container world to provide smaller deployment frameworks. There are efforts in the WASM world to replace containers with WASM deployment (which works well in browser, BTW).

> In addition, there are lots of companies and individuals that have all their data local, for a variety of reasons (often security) and primarily use desktop applications to work with that data. The "vast majority" may never care, but the compliment is still a very sizable market.

Is it still?

From what I saw personally, Azure is eating through this market. They give enough of the right guarantees that mutlinational corporations are fine with processing all their (and their customers') data in it, and from some quick research I've been involved in, it's the one (and AFAIK the only one) big cloud provider you can use in EU with sensitive data in a compliant way, up to and including medical data.

Beyond that, what's really left that most business users wouldn't just prefer to stuff into Azure and stop worrying about handling it?

> The "vast majority" may never care, but the compliment is still a very sizable market.

I agree as well here, and from that complement I hope to be able to draw a small community around my product. Still, I pessimistically feel that it's too small a share of users to make browser vendors take notice and build features for them. But of course I'd love if they'd prove me wrong. :)

Many apps have some sort of backend or server component to it though, for syncing or multiplayer features. Usually that part disappears when you don't have the option to eject and self-host.

You can think of ejectable as the opposite of being locked-in.

I'm currently building an offline-first app that has a custom sync between a local SQLite and Postgres (Supabase). The "ejectable" idea here is so good and I will definitely implement something that turns all your saved data into a spreadsheet with a few tabs.

Sound likes a very interesting project, would you like to elaborate more on the project?

> Being able to run the server executable locally. This should allow you to simply open the workspace data and continue where you left off.

The "server executable" is where basically all of the value provided by these kinds of companies exists. If it were freely downloadable and self-hostable by customers, then the company would be giving away its main value prop, and would only be able to generate revenue by, I dunno, acting as a hosting provider? Those margins are tiny and always getting smaller. It doesn't make sense.

It doesn’t have to be offered for free. If you sell a lifetime license for your software it’s actually sort of in your interest for customers to self host since then you’re off the hook for server costs.

The main thrust here is if you buy a “lifetime” license, it should be your lifetime, not the lifetime of the company that sold it to you.

I think you are underestimating how much people and businesses are willing to pay for convenience. For a business, especially, if the hosting provider costs are less (or even slightly more) than the pro-rated salary cost of the employee who'll need to keep the service up-and-running, that's a clear win.

How did the "server executable" get written in the first place, and how can it be maintained and enhanced over time, if the only revenue stream for the company is from (optional) hosting services? Especially when relevant data for a given customer is small enough that it can be "ejectable" and self-hosted in the first place?

>If it were freely downloadable and self-hostable by customers, then the company would be giving away its main value prop, and would only be able to generate revenue by, I dunno, acting as a hosting provider? Those margins are tiny and always getting smaller.

Maybe the margins aren't as high as some would like, but it's definitely viable. Example: Email.

Most people use the term self-hosting (or self-hostable) instead of ejectable.

One of the important points of an ejectable app is that you can easily move back and forth between cloud and self-hosted.

You can always start with the cloud version and then transfer to self-hosted if you change your mind and continue right where you left off. Not just that, you can also move from self-hosted back to cloud. That makes it easier to get start with a service because you're free to change your mind without lock-in or risk of painful migrations.

Bar the multiplayer aspect, I recommend TriliumNext in the same space, whose sync story is quite neat and enables patterns where replication happens over a swarm of devices, enabling strong resiliency (and it's fully open source)