hckrnws

I run an offline-first app[0] that has a sync mechanism (over Yjs). To solve server longevity we just sell you a lifetime license of the app and allow for file-based sync.

While I agree with the sentiment of the article most users do not even know what a server is, much less capable of self-hosting. Syncing a folder over Dropbox or Google Drive, though, is simple enough.

[0]: https://nestful.app

This pretty much matches the way I think it should be done ideally. I'm still pretty new to local-first though. Are there data formats or types of apps that don't lend themselves well to file-based sync?

For your app, how do notifications of changes get propagated? Does it depend on the backend (Google Drive, etc) supporting that, or does it just do polling or something?

For the desktop version just watching the directory for changes is enough to achieve that. Nestful is not aware the files are syncing, it just reads files as they are.

Ah ok so Nestful doesn't integrate with Dropbox/Drive directly, but if the user uses something like them for sync, the app picks up the changes?

Yes, exactly right. We offer our own paid sync if users would like to save the hassle

Sure but not all backend operations can always be mapped to flat files on Dropbox, like when you have certain real-time collaboration features, syncing on graph/tree-like data structures, or perhaps permissions in case of a company tool. That's why longevity of the "data" aspect is usually easy to solve when a copy of the data is already local, but then you still risk losing a part of what it is that makes the app when there is no way to run the app's syncing backend yourself.

It serves more use cases than you might think, especially with CRDTs. Nestful is a tree structure and uses Yjs for sync. Permissions can be done in the file system level.

Yes, I agree this is not an end-all be all solution, but the tradeoff is often worth it.

Yep love CRDTs but I think we are in agreement that it's a matter of trade-offs whether or not to use a backend (beyond a shared fs service). The idea with ejectable is that if you've decided the trade-off of a backend is worth it to enable certain features/UX, then it's a good way to ensure longevity of the entire app for users.

I think this is a subset of the fifth (of the seven) ideals of local-first software https://www.inkandswitch.com/local-first/#5-the-long-now

> Local-first software enables greater longevity because your data, and the software that is needed to read and modify your data, are all stored locally on your computer.

There are many ways to achieve this goal - open document standards, open source servers, a escrowed release of the server, or this idea of a bail out system for taking the current version and self hosting it. All are commendable.

Actually achieving all seven ideals is hard, and there isn't much modern software that does it. But in my view anyone trying to achieve even a few of the ideals is making strides towards a worthy goal.

There is a lot of exciting stuff happening in the local-first software movement at the moment, and a lot of that is related the sync engines that are being built (disclaimer: I work for ElectricSQL, we are building one). These sync engine don't inherently make your software local-first, fitting all the ideals, but they do make it a lot easer to do so. They are an important building block. But there is more needed, we need more open document standards - Automerge, Yjs, Loro and the other CRDT data structures are perfect lower level data structures to build these higher level abstractions on. Martin Kleppmann has talked quite a bit about sync engines that are disconnected from the underlying application, essentially pluggable sync engine, you choose who to use to sync you copy of a document or application - I'm excited to see where this goes.

But, we also need to free up the application distribution platform. The app stores are walled gardens that prevent some business models, native apps (while performant) are tied to a specific platform (or even version!), and the web is inherently tied to servers and browsers. The web platform though, i.e. JS, HTML, CSS, is perfect for building high longevity software that runs anywhere, on any device, the issue is the distribution. We need a middle ground, an application package that built on web standards, but isn't tied to a server. I want to download a bundled app to my machine and have a copy, email to a family member, or even open and hack on it. Thats the final missing piece.

A downloadedable app, with an open and pluggable sync engine would achieve the same goles of this ejectable idea.

The application package you're looking for is a single HTML file with no external dependencies, especially if it avoids minification, obfuscation, or the use of technologies like WASM blobs which require complex external toolchains to disassemble or modify. This is very achievable right now! See tools like TiddlyWiki or Decker, for example.

The primary barrier to an entirely server-free or server-agnostic webapp ecosystem is browser vendors choosing to seal the newest JavaScript APIs behind "secure contexts" which are only available to documents served over HTTPS.

I agree, I develop small utilities as single HTML for all the reasons you list (and fun), but having to work around browser protections for some various APIs can be a bummer.

The average internet user could be exploited fairly easily if every HTML file had immediate access to all the lower level APIs being introduced[0], and we end up looping back around to some sort of signing or alternative install method (pwa).

Curious to find the balance between distributable and "safe" enough to achieve wide adoption.

0: https://developer.mozilla.org/en-US/docs/Web/Security/Secure...

In practice most of those APIs are also gated behind a user's informed consent to e.g. enable access to a webcam or some other sensitive kind of I/O. I'd argue that the HTTPS delivery side of the requirements is superfluous theater pushed by "HTTPS Everywhere" ideologues and doesn't actually enhance the real security and privacy benefits already afforded by requiring manual user interaction.

Not related to the topic, but I first came across Thymer about (I want to say) 2 years ago when I looked for a todo.txt GUI. I found the blog they used to document building it: https://80daystartup.com/

Somehow, this thing is still not available for testing yet. Not hating but isn’t this supposed to be the category where new solutions launch every single day? While I am still very much interested, I’ve lost any hope that this will become a real thing to try out any time soon.

Hah yes it took a while, thanks for following! When we started the blog we initially thought of building something very small, but along the way realized that the kind of thing we would really love using ourselves would be a much bigger project. It basically involved building a completely new editor from scratch, sync engine, adding multiplayer and end-to-end-encryption, and so on. It would definitely have been faster to take shortcuts and use many existing libs and frameworks around editors, UI and syncing, but then it wouldn't have been something meaningfully different I think, as you point out there's plenty of options in the space so why take the same approach.

We're very much still working on it though and it's finally at the point where it's just some polish, so pretty sure next Q.

It is possible that life got in the way. Thymer did seem a great idea.

If you use a local, browser-based database, and allow your app to be installed as a PWA, then you are ejectable? The only "server" is just a host for HTML+JS that your browser automatically downloads when you visit the site. The app runs locally, like a desktop app, inside your browser. No need for the items listed in the article.

In theory, sort of, but I think that in practice this is not a solid enough solution to make an app and its data future-proof. As I see it, PWAs and browser-based databases are more like an offline cache rather than a permanent way to store apps and data:

- They don't get backed up or synced between devices.

- It's very easy to delete them (e.g, you clear browsing data) or lose them (e.g., you get a new computer).

- Users barely know about them, so they don't know they should "preserve them". For users PWAs are basically just a way to get a launcher for the app, and they have no visibility over locally-stored data.

I think browsers right now are not the right platform to give a future-proof guarantee. Disclaimer: I say this while working on a product that wants to become that platform¹, so I've had the chance of thinking about this problem quite a lot in the past few months.

¹ I wrote a bit about it at https://pscanf.com/s/340/, if you're interested.

Those are good points, but, of course, until your platform takes over the world, I would argue that browsers are not going anywhere, and work now, bearing in mind the corner cases you mention. Seems to me that fixing those persistence issues is easier to do by improving, rather than replacing, browsers.

I definitely agree that we should be improving browsers in that way, though I'm not sure that it's easy and I'm pessimistic it'll ever happen. Mostly because - cynically - I think that the vast majority of users will never care about owning data and software, so I don't see the incentive for browsers vendors to implement features to make it possible.

In their current state, I personally don't find PWAs + local databases ergonomic enough. So actually what I'm hoping to get out of the platform I'm building is - somewhat egotistically - an alternative that works for me personally and possibly for a few others like-minded individuals, not a browser replacement to take over the world.

There are many customers that care about owning their own data. Despite what you might think, there are lots of companies that do not trust sending their data to the cloud. In addition, there are lots of companies and individuals that have all their data local, for a variety of reasons (often security) and primarily use desktop applications to work with that data. The "vast majority" may never care, but the compliment is still a very sizable market.

I would love to see another way to deploy local software, aside from building dedicated Windows and MacOS apps. There are efforts in the Docker container world to provide smaller deployment frameworks. There are efforts in the WASM world to replace containers with WASM deployment (which works well in browser, BTW).

> In addition, there are lots of companies and individuals that have all their data local, for a variety of reasons (often security) and primarily use desktop applications to work with that data. The "vast majority" may never care, but the compliment is still a very sizable market.

Is it still?

From what I saw personally, Azure is eating through this market. They give enough of the right guarantees that mutlinational corporations are fine with processing all their (and their customers') data in it, and from some quick research I've been involved in, it's the one (and AFAIK the only one) big cloud provider you can use in EU with sensitive data in a compliant way, up to and including medical data.

Beyond that, what's really left that most business users wouldn't just prefer to stuff into Azure and stop worrying about handling it?

> The "vast majority" may never care, but the compliment is still a very sizable market.

I agree as well here, and from that complement I hope to be able to draw a small community around my product. Still, I pessimistically feel that it's too small a share of users to make browser vendors take notice and build features for them. But of course I'd love if they'd prove me wrong. :)

Many apps have some sort of backend or server component to it though, for syncing or multiplayer features. Usually that part disappears when you don't have the option to eject and self-host.

You can think of ejectable as the opposite of being locked-in.

Most people use the term self-hosting (or self-hostable) instead of ejectable.

One of the important points of an ejectable app is that you can easily move back and forth between cloud and self-hosted.

You can always start with the cloud version and then transfer to self-hosted if you change your mind and continue right where you left off. Not just that, you can also move from self-hosted back to cloud. That makes it easier to get start with a service because you're free to change your mind without lock-in or risk of painful migrations.

Most software apps that I've seen that are actually "ejectable" and use the "eject" verbiage are unidirectional. Not bidirectional. Meaning you eject once and any changes you make drift from the original implementation make it incompatible to go back. Example I know off the top of my head would be Draftbit. Visual Basic / .NET WYSIWYG would be another example. Both low code platforms, but that's usually where I see people talking about "eject"

I think that ensuring data can load back into a hosted service is a losing proposition.

As software updates, the shape of data changes. This would make a significantly harder problem to solve.

Thanks for the interesting article.

Is anyone not just me hesitant to run their own servers? I think about patching and it becomes a full on devops job to set up something safe and stay on top of upgrades etc.

One idea Ive had is that if the web facing part of the software is just a form saver. It persists the form to disk for retrieval by the local-first server that I run on my laptop after pulling all records.

Would like to talk to persons about this topic: local first and low-maintenance service design SAAS

I wrote about an imaginary cloud service that would be about hosting forms for batch processing later this is similar to the workspace.zip in this article but this is used for the app itself.

https://www.halfbakery.com/idea/Secure_20Form_20Endpoint_20S...

I'm currently building an offline-first app that has a custom sync between a local SQLite and Postgres (Supabase). The "ejectable" idea here is so good and I will definitely implement something that turns all your saved data into a spreadsheet with a few tabs.

Sound likes a very interesting project, would you like to elaborate more on the project?

> Being able to run the server executable locally. This should allow you to simply open the workspace data and continue where you left off.

The "server executable" is where basically all of the value provided by these kinds of companies exists. If it were freely downloadable and self-hostable by customers, then the company would be giving away its main value prop, and would only be able to generate revenue by, I dunno, acting as a hosting provider? Those margins are tiny and always getting smaller. It doesn't make sense.

It doesn’t have to be offered for free. If you sell a lifetime license for your software it’s actually sort of in your interest for customers to self host since then you’re off the hook for server costs.

The main thrust here is if you buy a “lifetime” license, it should be your lifetime, not the lifetime of the company that sold it to you.

I think I'm not really communicating my point effectively.

If you sell a lifetime license for your software, then the customer provides a single one-time payment (essentially CapEx) which obliges you to indefinite long-term work and support and maintenance (costs). But you can't support indefinite future costs with a single payment from a customer. The math doesn't work.

Again, if the data is small enough that it can be downloaded and self-hosted in the first place, then almost certainly hosting costs are statistically zero, compared to the R&D costs of building and maintaining and evolving the .exe file that's actually providing the business value.

But then that's an argument against lifetime licenses and not about ejectable servers.

You could run an ejectable server with other funding models. For instance, you pay yearly subscription and get updates within that year, but you get to keep your version forever if you want.

Or your server phones home for a license, but perhaps with an escrow license key if the company goes bankrupt and you have to pay subscription.

Either way, the business model is somewhat independent of the ejection mechanism.

I think you are underestimating how much people and businesses are willing to pay for convenience. For a business, especially, if the hosting provider costs are less (or even slightly more) than the pro-rated salary cost of the employee who'll need to keep the service up-and-running, that's a clear win.

How did the "server executable" get written in the first place, and how can it be maintained and enhanced over time, if the only revenue stream for the company is from (optional) hosting services? Especially when relevant data for a given customer is small enough that it can be "ejectable" and self-hosted in the first place?

The same way anything else gets created before it exists in order to create revenue - loans and/or early investors.

EDIT: whoops, I missed the second part of your question ("...how can it be maintained and enhanced over time..."). I've already implicitly answered that in my original reply, by claiming that the attractiveness of "optional hosted services" is higher than the average HN reader - who is predisposed to be a tinkerer, a self-hoster, someone who cares about DIY and learning - would assume. There are plenty of existing companies who make their base software available for free, but make a healthy income off of hosting and support. This article doesn't propose anything radically new in that regard - just extending it to "ejectability".

You give your server.exe away for free, and make all of your revenue in value-add hosting, and maybe professional services stuff to specific customers. You still have to maintain server.exe -- fix bugs, add features, etc. -- which requires engineering resources, time, money.

Someone else comes along, downloads your server.exe, and offers hosting and professional services themselves, as a competitor. To a customer there is no difference between you.com and competitor.com, they're hosting the same thing on different domains. And the competitor gets their revenue the same way you do, but they don't have to spend anything on server.exe, because you're doing all that work already.

Again, this math doesn't work out, at least not for you. The competitor will always come out ahead.

If the claim is that you have some unique ability to produce revenue that isn't available to third parties that download and use your server.exe, then whatever that is, it's not part of the server.exe, just by definition! You've got the server.exe you offer freely, and a different my-server.exe that only you can operate and which has value that only you can extract. So we're back to square one.

Ah, I see your point now. Not "how can you generate income (at all)", but "how can you generate income more effectively than a competitor who is drafting off your development, to ensure you don't get competed out of business". Makes sense, sorry for initially misunderstanding!

I still don't think this is an intractable problem, though - there are still some properties that provide a moat to allow you to survive:

* the experience of having developed the server.exe means that you have some institutional knowledge that makes you better able to offer support - you don't have to "learn it from scratch"

* you can offer upcoming features to your paid subscribers earlier than they get released in the free version

* name/brand recognition is A Thing - there is some non-zero dollar value that customers will assign to the comfort and security of buying from "the real source"

But I do recognize and agree that those are slim and not _necessarily_ decisive. In some cases, the differential revenue accrued by those things will be enough to continue development on the product that is otherwise given away for free; in others, it won't. A careful balancing act, to be sure!

>If it were freely downloadable and self-hostable by customers, then the company would be giving away its main value prop, and would only be able to generate revenue by, I dunno, acting as a hosting provider? Those margins are tiny and always getting smaller.

Maybe the margins aren't as high as some would like, but it's definitely viable. Example: Email.

This idea is very similar to what I've been working on for a while, local-first apps that synchronize data over a server that can be self-hosted. In my case, I'm using the Solid Protocol, and there are already many Open Source servers implementing this protocol.

If you're curious, check out this app: https://umai.noeldemartin.com/

Bar the multiplayer aspect, I recommend TriliumNext in the same space, whose sync story is quite neat and enables patterns where replication happens over a swarm of devices, enabling strong resiliency (and it's fully open source)