Modern medical instruments are software. A “device” is now a React app with a camera, an AI model, some firmware, and a cloud backend. The instant you add that last piece, someone ends up holding everyone else’s data.

That was the problem I kept running into as a patient. Every time I see a new practitioner, sign up for a new portal, or get sent to yet another “provider,” I end up writing out the same set of details again - name, date of birth, address, Medicare, email, phone - onto a clipboard or into some anonymous web form. Then it disappears into someone else’s system.

  • As a patient, I have no map of where those details live, who actually runs the software, or how many copies exist.
  • As a practitioner, you are stuck between needing history to make decisions and logging into five different vendor backends to find it.
  • As a supplier, you mostly want to know if your devices are working in the field, but you end up sitting too close to identifiable data by default.

Most systems collapse those needs into a database with user tables, join tables, and permission tables. The result is familiar: a tangle of foreign keys and migrations that quietly centralises power in one place - the database owner.

I wanted something else: a backend where privacy falls out of the design, not out of a compliance document. That became the Secure Digital Instrument Data System (SDIDS).

What SDIDS Actually Is

SDIDS is a file-based system for storing instrument data. There is no database. There is a deterministic directory tree rooted in a single identifier:

THPK__assessment__run__reading::type

Every piece of data - assessment metadata, readings, charts, images - is written as a JSON (or image) file under a path derived from that composite identifier.

  • The key commands everything: IDs encode the user, test, run, reading, and type. There is no parallel ID system hiding in a database.
  • Paths enforce order: The directory structure is the schema. If a file is out of place, the system treats it as corruption.
  • Files bear witness: If a file exists on disk, the system considers that event to have happened. If it does not exist, it never did.

This is not nostalgia for flat files. It is a deliberate way of avoiding an entire class of privacy problems. When the filesystem already knows how to separate one user’s directory from another, there is no reason to smuggle that logic into SQL.

Three Keys, Three Roles

Most privacy failures in healthcare are not about “hackers.” They are about too many people being able to see too much by default. SDIDS starts from the opposite assumption: no one sees anything unless a key says so.

The system uses a three-key model:

  • PRK (Private Key): held by the patient. Never stored. Used to sign challenges.
  • PUBK (Public Key): shared with practitioners when the patient wants them to read data.
  • THPK (Truncated Hashed Public Key): derived from the public key and truncated. This is the only identifier that ever appears in file paths.

SDIDS never writes the private or public key to disk. Only the derived THPK appears in composite IDs. That means:

  • If you break into storage, you see unguessable IDs and DTOs, not raw keys.
  • If a patient rotates keys, SDIDS can reindex data from one THPK to another without exposing cryptography anywhere.

On top of that, SDIDS defines three roles:

  • Patient: full control over their own data. Can take assessments, submit forms, and rotate keys.
  • Practitioner: read-only access scoped to a patient, via their public key.
  • Supplier: access to anonymised device metrics and telemetry only.

Each role has a different way of logging in and a different set of allowed operations. But they all collapse back to the same thing on disk: files under a THPK-rooted tree.

Why Avoid a Database?

On paper, this looks perverse. Why would you avoid a perfectly good relational database and hand-build your own persistence layer?

Because databases are optimised for queries like:

  • “Show me all readings across all patients with condition X.”
  • “Join every test result to every device serial number.”

Those are exactly the queries that make privacy hard. The default shape of the data encourages cross-cutting analysis. You can build partitions and row-level security, but you are always swimming upstream against the natural behaviour of the system.

SDIDS flips the default:

  • There is no global “readings” table to query.
  • There is only a directory tree under a user.
  • If you want to aggregate, you have to explicitly traverse.

Practically, that means most day-to-day operations are very simple:

  • To save a result: build a DTO, derive its path from the composite ID, write the file.
  • To load: derive the same path, read the file.
  • To summarise: walk directories and count files, loading JSON only when you really need it.

It is not a general-purpose database. It is a purpose-built recorder for digital instruments in high-integrity domains.

Instruments as First-Class Citizens

Another design constraint: instrument applications are not allowed to freestyle against the backend. They must go through a client library.

The SDIDS client has two layers:

  • A low-level client that knows how to talk to the API and enforces capability checks.
  • A high-level client tailored to instrument workflows - login, sessions, assessments, runs, readings.

Every request carries two forms of identity:

  • Client identity via X-Client-Token (“This is the instrument app”)
  • User identity via Authorization: Bearer <jwt> (“This is that particular patient, practitioner, or supplier”)

The client library makes it almost impossible for an instrument to accidentally leak key material into logs or shove raw file paths over the wire. It knows about the three-key model, about THPK, and about which operations are even allowed for a role.

This is the opposite of “just hit the REST API.” It is a deliberately narrow surface, because every extra surface area is another place privacy can escape.

Privacy by Construction

Most privacy policies sound the same: “We care deeply about your privacy.” SDIDS tries to encode that sentiment into mechanical rules:

  • No raw keys on disk. Only THPK appears in files and paths.
  • No ad-hoc JSON. All persistence goes through DTOs with explicit schemas.
  • No silent fallbacks. If something goes wrong, the system throws loudly with enough context to debug.
  • No naked HTTP. Instruments must use the client; the server never trusts arbitrary payloads.
  • No surprise aggregates. If someone wants cross-patient analytics, they have to build explicit tooling and live with the friction.

The goal is not perfection. It is predictability. When a patient asks, “Where is my data?” the answer is: “Under your THPK directory, and nowhere else.” When a practitioner asks, “What am I allowed to see?” the answer is: “Whatever your public key and the patient’s choices say you can.”

Motivations And Incentives

There is a practical reason to structure things this way: it lines up incentives for everyone who touches the data.

Patients

For patients, SDIDS looks like a slightly unusual password system: instead of a username and password, you have one long key.

  • You store that key in your password manager like any other secret.
  • You log in by pasting the key; the client library handles challenges and signatures.
  • There is almost no new user experience to learn - except that the key is your responsibility.

If you lose the key, there is no magical “reset my password” that reconstructs your identity. The data is still on disk, but the system has no way to prove you are the person who owns it. That is intentional.

It is your data. You get the benefits of owning it, and the burden of looking after the key. If you do not care, you can simply delete the key and walk away. The system does not need to keep a secondary copy of you around “just in case.”

Practitioners

Practitioners are under increasing scrutiny for how they store and manage data. Every extra system they log into is another potential breach headline.

  • With SDIDS, a practitioner holds a patient’s public key, which gives them read-only access.
  • If they do not want the responsibility, they can simply not store the public key. No key, no access, no data to defend.
  • If a patient rotates keys, the old public key stops working. Access disappears automatically.

If the practitioner needs access again, they contact the patient through the usual channels - phone, email, letter, in-person - and ask for the new public key. SDIDS does not store contact details or “CRM data”; it stores only the medical data and the cryptographic hooks needed to prove who is allowed to read it.

Suppliers

Suppliers need data to make better tools. Most of the devices and services we rely on now are powered by large, real-world datasets - that is how they learn to spot anomalies and edge cases that matter in practice.

The tension is obvious: we want those life-improving, sometimes life-saving tools, but not at the price of handing over everything about ourselves.

  • In SDIDS, suppliers see anonymised metrics keyed by THPK, not by real-world identity.
  • They can work with the full dataset across all patients to tune algorithms and detect failures.
  • They never need to know who any particular reading belongs to.

In my view, this gives us the best of three worlds: patients keep control, practitioners shed a large part of their data-storage burden, and suppliers still get the signals they need to build better instruments - all without centralising raw identity in one forever-database.

Why I Built It

I did not set out to build a small file-based database. I set out to stop playing “clipboard roulette” with my own information.

The traditional pattern is simple:

  • Every new clinic, specialist, or service provider asks for the same details again.
  • Each one disappears into a separate system you never see.
  • No one can show you a single coherent view of where your data actually lives.

That pattern is incompatible with modern expectations of privacy, and with the reality that most “systems” are now a web app plus a random backend. I wanted a backend that forces everyone in the chain - patient, practitioner, supplier - to respect the same boundaries by design.

SDIDS is that backend. It is deliberately opinionated. It is not a general solution for every problem. But if you are building a digital instrument or service and you want:

  • Patients to own their data.
  • Practitioners to have trustworthy, read-only access.
  • Suppliers to see only the metrics they need.

Then a file-based, composite-key system with a three-key model and a strict client boundary is not a bad place to start.

At minimum, it forces you to answer the question most systems postpone until after launch:

“Exactly who is allowed to see this reading?”

← Back to home