← All posts

Blog

Rethinking how Newsbin syncs

Sync is the feature nobody thinks about until it goes wrong, and then it’s the only thing they think about. It’s been the hardest part of building Newsbin by a wide margin. I want to be straight about where it’s fallen short and where I’m taking it next.

The trouble with syncing everything

Newsbin is built on SwiftData, and until now it has synced through SwiftData’s built-in CloudKit mirroring. That’s the path Apple hands you, and the appeal is obvious: flip a switch and your whole local database is mirrored to your iCloud. No servers of mine in the middle, which is exactly the privacy posture I want.

The catch is in the word “everything”. Mirroring doesn’t just sync your subscriptions and what you’ve read. It tries to sync every article too. A busy reader pulls in thousands of articles a week, and each one becomes a record to push, pull, reconcile, and version, across every device you own, forever.

That volume is behind nearly every sync problem people have reported:

  • Read articles popping back to unread, because two devices disagreed about the same record and the wrong side won.
  • Duplicates, when the same article arrived down two paths and the system couldn’t tell they were the same thing.
  • Feeds re-ordering themselves after you’d deliberately arranged them.
  • Slow, churny first syncs on a new device, as it hauled down a mountain of article records it was about to re-download from the feeds anyway.

None of it was catastrophic, and recent releases have made it much more reliable. But I kept fixing symptoms. The real problem is that the foundation is doing too much, and the answer isn’t another patch. It’s syncing less.

You don’t need to sync articles at all

The thing that reframed it for me: articles are derivable. Every device already knows how to fetch them from the feeds you follow. If two devices follow the same feeds, they’ll independently arrive at the same articles. Syncing the articles themselves means syncing something both devices can regenerate for free.

So what actually needs to travel between your devices? Only the things you decided:

  • The feeds you follow, your folders, your tags, your smart groups.
  • What you’ve done: read, saved, archived, hidden.

That’s a tiny amount of data. Instead of one record per article, it’s one record per thing you actually chose, which removes most of the conflict surface and almost all of the record volume in one move.

The plan: CKSyncEngine, and only the essentials

The next major upgrade replaces SwiftData’s automatic mirroring with CKSyncEngine, Apple’s lower-level sync API, while keeping SwiftData as the local store. It gives me direct control over what syncs and how conflicts resolve, rather than a black box that decides for me.

A few things that control unlocks:

  • Sync only your intent. Feeds, folders, tags, smart groups, and your read and saved state travel. Raw articles stay local and get re-derived per device.
  • Conflict rules written down in plain terms. Read beats unread. Saved wins. Your latest reordering wins. The outcome is explicit and predictable instead of a guess.
  • Duplicates become structurally impossible. Every record is keyed to a stable ID, so the same thing can’t land twice. That retires a whole category of bug.
  • A version stamp on every record from day one, so future changes to the data can’t silently wedge older versions of the app. New features can roll out without breaking someone who hasn’t updated.

And the part you’ll actually see:

  • A “Sync Now” button, and pull-to-refresh that means it.
  • A real sync status: last synced, what’s pending, what failed. When sync misbehaves, you should be able to look at it instead of guessing.

That last bit matters to me. Sync should be invisible when it’s working, and legible the moment it isn’t.

The one trade-off

Because articles won’t sync anymore, a brand-new device won’t restore your articles instantly. It’ll refresh them in from your feeds, the same way the app has always fetched them. In practice that’s a few seconds of the feed filling up rather than a long haul of downloading records. Saved articles keep their saved state everywhere and load their full text when you open them.

I think it’s a good trade: slightly different new-device behavior, in exchange for sync that’s dramatically smaller, faster, and less likely to fight itself. But it is a trade, and I’d rather name it than surprise anyone.

How it’ll ship

Sync is the wrong place to be bold on release day, so this won’t arrive as a flip-the-switch ultimatum. The plan is to run the old and new systems side by side for a release. Your devices stay in sync whether they’re on the old build or the new one, with no forced-update cliff and an easy rollback if anything looks off. The old system only gets switched off once the new one has soaked across real devices for a couple of update cycles, including the “leave a device dormant for a week, then wake it and check nothing’s mangled” test.

No firm date. This is the kind of foundational work that ships when it’s boring, when I’ve watched it quietly do the right thing across my own devices long enough to trust it with yours.

I’ll let you know when it lands. When it does, the goal is that you won’t notice a thing, except that the problems are gone and there’s finally a button that tells you what sync is up to.