23.107. DD 107: Beekeeper: Resumable Forms and KYC Media Collection#

Design status:

Draft

Implementation status:

untested

DD shepherd:

Christian Grothoff

Historical contributors:

Christian Grothoff

First published:

2026-09-29

Last substantive change:

2026-11-21

23.107.1. Summary#

Beekeeper is a new, standalone service that collects structured information from users through forms defined in JSON. Forms have several pages and conditional questions, and users can fill them in over several sessions. Besides ordinary fields, Beekeeper can collect KYC-grade evidence: HD selfie videos with a liveness challenge, videos of directed movements, identity documents (image, MRZ and optionally the ePassport chip), business documentation (share registers, company registration data) and phone numbers and postal addresses that are verified by SMS and by letter. A single JSON REST API serves a single-page web app and native iOS and Android apps. Administrators manage forms, invite users, review submissions and send them back for correction.

Beekeeper is a standalone product that can also act as an OAuth2 KYC provider for the Taler exchange, as the existing KYC providers do.

23.107.2. Motivation#

Taler’s KYC processes rely on external providers or on hand-built AML/KYC forms. Operators need a way to collect customer information that they can host themselves:

  • Requirements change with jurisdictions and regulators. So the questions should be data, not code, and changing them should not require an app release.

  • KYC evidence (identity documents, liveness videos, verified addresses) is highly sensitive. Sending it to a third party is a privacy and compliance risk, and many deployments would rather keep it in-house.

  • Users rarely finish a long KYC process in one sitting. They need to leave, come back from another device and go back to earlier pages, and they need to be told when a document must be submitted again.

The same machinery is useful beyond KYC, for example for customer surveys or movement assessments. So Beekeeper is designed as a general form service, with KYC as its most demanding use case.

23.107.3. Requirements#

  • Forms are JSON documents: questions, answer types, validation, a sequence of pages, and questions that are shown only under conditions. Uploading a form rejects inconsistent definitions up front.

  • Conditions and validators give the same result on the server, in the browser and in both native apps.

  • Users fill in forms incrementally and resumably. They see how many steps remain and can go back.

  • Each user has a personal link. A form can allow recovering a lost link through a verified contact e-mail address, and can require an e-mail one-time code in addition to the link.

  • Public forms are listed on the start page; if there is only one, a single “start” button is shown.

  • Administrators can upload, inspect, publish, unpublish and retire forms; create invitation links with or without an e-mail address; assign a form to a user, or replace a user’s form with another one; review, accept, reject or return submissions; and export them.

  • Answers are stored in PostgreSQL as JSON; media are stored as large objects keyed by their SHA-256 hash. Administrators can list media sorted by date or size, view them and delete them in bulk.

  • Upload types: the usual HTML fields; HD selfie videos with liveness detection; videos of directed movements; identity documents with MRZ; phone numbers verified by SMS; postal addresses verified by letter. The user is told when evidence has expired or failed and must be submitted again.

  • The personal data must be protected in line with KYC regulations: encrypted at rest, with a retention period per form and with administrator access recorded in an audit log.

23.107.4. Proposed Solution#

The detailed specifications live in the Beekeeper repository (docs/). This DD summarises the design and the reasons behind it.

23.107.4.1. Components#

Beekeeper is written in Rust, with the crates used by taler-rust (axum, hyper, sqlx, tokio, clap, tracing). It consists of these parts:

  • beekeeper-httpd serves the REST API, including resumable uploads, and the static web front ends. It is the only process that holds the master key.

  • beekeeper-analyzer workers analyse media. They take jobs from a queue in PostgreSQL (FOR UPDATE SKIP LOCKED, woken by LISTEN/NOTIFY). Because they decode untrusted video with FFmpeg, they run sandboxed: landlock and seccomp, no network access apart from the database socket, and a database role that can read only its own job. Analyzers can run on separate GPU machines.

  • beekeeper-admin is a command-line tool for database setup, the first administrator account, key rotation, garbage collection and retention.

  • Messages are delivered by helper programs for e-mail, SMS and letters. They use the same calling convention as challenger’s challenger-send-* scripts, so the existing provider integrations can be reused.

The SPA and the admin UI live in the Beekeeper repository itself (one package, two bundles), using established libraries for resumable uploads (tus-js-client), the proof of work (ALTCHA) and live camera guidance (MediaPipe tasks-vision). The native apps are written in Swift and Kotlin, mainly because reading the ePassport chip over NFC needs NFCPassportReader on iOS and JMRTD on Android. They live in the same repository (apps/), so that an API or expression change lands in one commit across server, SPA and apps. Each has a platform-independent core (evaluator, form model, API and tus client) that is tested on Linux; the Android app uses Jetpack Compose and CameraX and no Google Play services, so that it can be built for F-Droid.

The server builds on taler-common from taler-rust (configuration, database migrations, logging) and on a vendored copy of taler-api, whose changes are kept as a series of patches meant for upstream. Its error codes are registered in GANA, in a registry of their own (beekeeper-error-codes, values 100000 to 110000), and returned in the usual ErrorDetail format.

The analyzer uses Debian’s builds of ONNX Runtime, FFmpeg and Tesseract. The models are distributed in a separate beekeeper-models repository with their provenance and conversion scripts.

23.107.4.2. Form language#

A form is a JSON document with pages, and each page has a list of typed questions: info, text, number, boolean, choice, date, email, phone, postal_address, file, video and identity_document. Labels can be given per language. Each question can have required and visible_if conditions and a list of validators. Options per form cover manual or automatic acceptance, the e-mail OTP, whether users may go back, the retention period, and when evidence must be resubmitted.

Conditions use JSONLogic syntax with a small set of operators (comparison, arithmetic, string operations, a portable regular-expression subset, date arithmetic and verified). Their semantics are defined strictly rather than by “whatever JavaScript does”:

  • Equality is strict and deep.

  • Type errors are errors, not coercions.

  • String lengths and comparisons are defined over Unicode scalar values.

  • \d, \w and \s mean their ASCII classes.

Only this makes four independent implementations agree. A shared set of conformance test vectors, currently 215, keeps them in step. A form is rejected if an expression refers to a later question, so visibility is computed in one pass in document order, and the progress indicator stays stable while the user fills in later pages.

23.107.4.3. Users and authentication#

Each user has a personal link. Its secret sits in the URL fragment, so it never reaches server logs or Referer headers, and the client exchanges it for a session.

  • One-time code: a form may require an e-mail one-time code on top of the link.

  • Recovery: if the user has a verified contact address, they can request a new link by e-mail. The old link stays valid until the new one is first used, so a third party who requests a recovery cannot lock the user out.

  • Replaced forms: when a user’s form is replaced with another, answers are carried over by question id and type. A user can have several open assignments, in an order the administrator chooses.

Administrators have database accounts with the roles reviewer, manager, superuser and service, and log in with a password and TOTP. Every administrator action is written to an audit log.

23.107.4.4. Verification#

E-mail addresses and phone numbers are verified with codes while the user is on the page. A postal address is verified by a letter with a code. The form can be submitted while the letter is in transit. If the code is not entered before the deadline, or a later media analysis fails, the assignment goes back to the user in the returned state, with the affected pages marked.

23.107.4.5. Media, liveness and identity documents#

Uploads use the tus 1.0 protocol (with the creation and checksum extensions), which lets large videos survive unreliable mobile connections. A user can refer only to blobs that they uploaded themselves; knowing a SHA-256 hash is not enough.

For a liveness video, the server issues a random sequence of actions (for example “turn head left, blink, smile”) shortly before recording, and the video must show them in the right order and time windows. The analyzer runs open models through ONNX Runtime (ort):

  • YuNet for face detection;

  • the MediaPipe face and hand landmark models for eye, mouth and hand actions;

  • head pose from landmarks;

  • RTMPose or BlazePose for arm and body movements;

  • MiniFASNet for passive anti-spoofing;

  • SFace to compare the face with the document photo.

The MRZ is read with Tesseract using the OCR-B training data, and its check digits are verified. For NFC passports, the server checks the chip’s security object against the CSCA master list with RustCrypto cms and x509-cert.

Each check yields a score. Clear passes and clear failures are decided automatically; everything in between goes to a human reviewer, who sees the per-action timings and key frames. Vendor services can be added later through a trait that takes a blob and a challenge and returns the same result structure. InsightFace is not used because its models are licensed for non-commercial use only.

23.107.4.6. Encryption at rest#

Envelope encryption protects the data:

  • Keys: a master key file wraps a random data key per user and per blob.

  • Answers and addresses are encrypted with XChaCha20-Poly1305. The row’s identifiers are bound in as associated data, so a ciphertext cannot be moved to another row.

  • Blobs are encrypted in 64 KiB chunks (STREAM), so ranged reads and resumed uploads work.

  • Deletion removes the data key first (“crypto-shredding”), which makes the data unreadable in backups too.

  • Left unencrypted are identifiers, states, timestamps, sizes and hashes, so that the admin UI can list and sort. Contact addresses are found through an HMAC of the normalised address.

  • The analyzer never sees the master key. Each job carries the blob key wrapped for that analyzer’s X25519 key.

23.107.4.7. Output#

Administrators get submissions through the admin UI, as a JSON or ZIP export, and through a change feed with long polling. There are deliberately no webhooks: consumers pull, so there is no outbound delivery to retry and no receiver to authenticate.

23.107.4.8. Taler integration#

Beekeeper implements the protocol of the exchange’s oauth2 KYC logic in the same shape as challenger: POST /oauth2/setup/$CLIENT_ID (with the client secret; the measure’s initial_address pre-fills answers of the same name), GET /oauth2/authorize/$NONCE, POST /oauth2/token and GET /oauth2/info. A client (registered with beekeeper-admin oauth-client add) names the forms each process assigns; the process is one Beekeeper user, so administrators can assign further forms or return answers as usual.

As the exchange takes the answer of /oauth2/info as final, the user is handed back only once every assignment of the process is accepted, automatically or by a reviewer; until then the web app shows that the answers are being checked, and opening the exchange’s KYC link again resumes the process. A rejected or expired assignment hands the user back with error=access_denied.

/oauth2/info returns the visible answers, the verified questions and the analysis outcomes. Files are inlined as the exchange’s {FILENAME, MIME_TYPE, CONTENTS} objects for the question types the client chose and up to its size limit, and given as metadata (SHA-256, type, size) otherwise. The converter taler-exchange-kyc-beekeeper-converter maps question ids to attribute names by upper-casing them (full_name gives FULL_NAME).

23.107.5. Test Plan#

  • Conformance: the expression vectors run in the Rust test suite and, as they are written, in the TypeScript, Swift and Kotlin test suites.

  • Static checks: unit tests, and every example form in the repository must pass them.

  • Server: integration tests against a real PostgreSQL database for page saves, resumption, submission, carry-over, returns, recovery and OTP. Also: token revocation, and that users cannot access other users’ blobs.

  • Uploads: tus conformance tests, including interrupted and resumed uploads and checksum mismatches.

  • Verification: the delivery helpers are replaced by scripts that record the message, so tests can read the code.

  • Analyzer: a corpus of recorded videos (genuine; photo, replay and mask attacks; wrong action order) with expected outcomes. Thresholds are tuned on it, and regressions are measured against it.

  • Sandbox: tests that the analyzer cannot open network sockets or files outside its model directory.

23.107.6. Definition of Done#

  • Forms can be uploaded, checked, published and retired; users can be invited and can fill in, resume, go back and submit on the SPA and on both native apps.

  • E-mail, SMS and letter verification work through challenger-compatible helper programs.

  • Liveness, directed movement, MRZ and face-match analysis run in the sandboxed analyzer, with a human review queue.

  • All personal data is encrypted at rest; retention and crypto-shredding are implemented; administrator actions are audited.

  • Export and the change feed are available and documented in the API reference.

  • The exchange can use Beekeeper as an OAuth2 KYC provider; the round trip through the exchange’s oauth2 plugin (taler-exchange-kyc-tester) yields the expected attributes.

23.107.7. Alternatives#

  • A commercial KYC vendor for liveness would give certified presentation-attack detection (ISO/IEC 30107-3), but would send the most sensitive data to a third party and bind deployments to its pricing. It remains possible through the analysis trait.

  • Extending challenger would reuse its address verification, but challenger verifies one address per process, and multi-page forms, media and review do not fit its model. Beekeeper reuses its helper programs instead.

  • An existing form standard (JSON Schema with a UI schema, SurveyJS, FHIR Questionnaire) would bring existing tooling. None of them covers the media and verification types, and their condition languages lack semantics that can be reproduced exactly in four languages.

  • Plain JSONLogic has libraries for all four platforms, but these libraries disagree on coercion, equality and string handling. Conditions that behave differently on the server and in the app would be a source of bugs and of rejected submissions.

  • A TypeScript or Kotlin backend would share code with a client. Rust fits the existing taler-rust code base and its deployment, and is memory-safe near FFmpeg and the ONNX models.

  • Webhooks would push results to consumers, but they need retries, signing and a receiver. Long polling gives similar latency without these.

23.107.8. Drawbacks#

  • Own pipeline code: without MediaPipe’s runtime and OpenCV, we have to write the model pre- and post-processing ourselves. This is most of the analyzer work.

  • Four evaluators: the condition language has to be implemented four times. The conformance suite reduces this risk but does not remove it.

  • Consequences of encryption: the database cannot search inside answers, and losing the master key loses all data.

23.107.9. Discussion / Q&A#

  • Open: the expected scale (number of users and media volume), which decides whether large objects in the database remain adequate for media storage.