Billing

Developer and operator reference for ReelBolt's hosted billing (work packages B9a and B9b): the provider boundary, the Paddle implementation, how a webhook turns into subscription state and credits, the endpoints, and what an operator must configure. The credit ledger itself (grants, reservations, refunds) is in the code under ReelBolt.Shared/Billing/CreditLedger.cs; the pricing rationale is in design/saas-pricing-and-billing.md.

Switching billing on

Billing is off by default, which is the self-host behaviour. With Billing__Enabled=false (the .env variable is BILLING_ENABLED) every /api/v1/billing/* endpoint answers 404, no provider is contacted, and nothing here runs. Hosted deployments set BILLING_ENABLED=true and the Paddle values:

.env variable	Config key	Secret	Meaning
BILLING_ENABLED	Billing__Enabled	no	false (default) = billing endpoints 404.
PADDLE_API_KEY	Billing__Paddle__ApiKey	yes	Server API key, pdl_sdbx_apikey_... (sandbox) or pdl_live_apikey_....
PADDLE_CLIENT_TOKEN	Billing__Paddle__ClientToken	no	Client-side (publishable) token for the embedded checkout, test_... or live_.... The API serves it to the browser, so it is not a secret and must never be a server key. Blank = no embedded checkout (the buyer uses Paddle's hosted page).
PADDLE_WEBHOOK_SECRET	Billing__Paddle__WebhookSecret	yes	The notification destination's secret key.
PADDLE_ENVIRONMENT	Billing__Paddle__Environment	no	sandbox (default) or live. Selects the API base URL. A sandbox key only works against sandbox.
PADDLE_TOPUP_PRICE_500, _1000, _3000	Billing__Paddle__TopUpPriceIds__{credits}	no	Paddle price ids (pri_...) of the three top-up packs. A blank one means that pack is not sold.
none	Billing__Paddle__ToleranceSeconds	no	Largest accepted age of a webhook's signed timestamp. Default 5.

host.sh check-env requires both Paddle secrets on the control host when BILLING_ENABLED=true. Secret handling and rotation are in secrets.md.

The provider boundary

Everything provider specific sits behind IBillingProvider (ReelBolt.Inference.Api/Services/Billing/IBillingProvider.cs), so changing provider is one new class (decision D11). The interface has five operations: CreateCheckoutAsync (a plan or a top-up), CreatePortalSessionAsync, VerifyAndParseWebhook, ChangeSeatsAsync, and the configured top-up packs. A webhook is verified and parsed into a neutral BillingEvent with one of four kinds: SubscriptionChanged, TopUpPaid, PaymentRefunded or Ignored. BillingEventProcessor only ever sees that neutral shape, never a provider payload.

Provider price ids are data, not code. They live on the plan row in plans.provider_price_ids_json as { "paddle": { "monthly": "pri_..." } } and are environment specific (sandbox ids differ from live ids), so the plan catalog upsert never touches them. An operator sets them once per environment after creating the products in Paddle. The summary lists only the plans the active provider can actually sell, each with the priceId its checkout will use; a plan with no price id for that provider is omitted from the payload rather than listed with a blank one, and POST /checkout refuses it with 400 plan_not_purchasable.

Endpoints

All routes are on the Inference API under /api/v1/billing and 404 when billing is disabled.

Method	Path	Who	What
GET	/summary	Any member of the active organization	Current plan, status, seats, renewal date, credit balance, purchasable plans (with their priceId), available top-up packs, canManageBilling, and the paddle block the browser needs for the embedded checkout.
POST	/checkout	Owner or Billing role	Body { "plan": "creator" } or { "plan": "team", "seats": 5 } or { "topUpCredits": 1000 }. Returns { checkoutUrl, transactionId }. Grants nothing.
POST	/portal	Owner or Billing role	Returns { url }, a short-lived authenticated customer-portal link (invoices, payment method, cancel). 409 no_billing_customer before the first purchase.
POST	/seats	Owner or Billing role	Body { "seats": 7 }. Sets the seat quantity of a per-seat subscription. 202: the change takes effect locally when the provider's webhook arrives.
POST	/webhooks/{provider}	Anonymous, signature only	The provider's notification receiver.

Checkout rules: send exactly one of plan or topUpCredits (400 choose_plan_or_topup). A plan must be a purchasable subscription with a price id (400 plan_not_purchasable). The Team plan has a three-seat minimum (400 below_min_seats), and seats default to the minimum. Top-ups are for paying customers only (decision D9): without an active paid subscription the answer is 403 topup_requires_paid_plan, and the pack must be one of the configured sizes (400 unknown_topup_pack). An organization that already has a live subscription gets 409 already_subscribed instead of a second subscription; plan changes on a live subscription go through the portal. A caller in the right organization with the wrong role (Member, Admin) gets 403; a caller with no active organization gets 404.

The embedded checkout

The dashboard opens Paddle's one-page overlay — Paddle.Checkout.open({ transactionId, settings: { displayMode: 'overlay', variant: 'one-page', successUrl } }) — rather than sending the browser to Paddle's hosted page. Both are the same transaction and the same webhook; the overlay is a display choice, not a second payment path.

The transaction is always created by the server, and the browser is only ever handed its id. Paddle.Checkout.open({ items, customData }) is deliberately never used: a payment is attributed to an organization through custom_data.organization_id, which BillingEventProcessor reads to decide whose credits a payment buys (see "Subscription state machine" above), and a value supplied by the browser is one the buyer can rewrite — they could direct their credits, or someone else's, at an organization that is not theirs. PaddleBillingProvider.CreateCheckoutAsync attaches custom_data.organization_id (and user_id, kind, plan_key) from the caller's session, and the client opens the result with transactionId. The dashboard has no way to express an organization.

What the summary gives the browser, and why it is not a NEXT_PUBLIC_* value. GET /api/v1/billing/summary carries a paddle block — { environment, clientToken } — next to the prices. environment is already in Paddle.js's own vocabulary (sandbox listed as sandbox, live as production), translated once on the server, so the browser cannot be initialized against a different Paddle account than the one this API creates transactions in. The token is the client-side, publishable one (test_.../live_..., Billing__Paddle__ClientToken): Paddle.js is designed to run with it and it grants nothing on its own. It is served at runtime rather than baked into the web image because the web app is built in Docker — a build-time value could not be rotated without a rebuild, and it would be free to disagree with the server's PADDLE_ENVIRONMENT. The block is null when the configured provider is not Paddle; clientToken is omitted when the operator has not set one, which is the deployment's answer to "is there an embedded checkout?".

Prices are Paddle's. The page asks Paddle.PricePreview({ items }) for the price ids the summary served and prints the formattedTotals strings unchanged — no conversion, no rounding, no currency symbol of this platform's own, and no arithmetic on the frontend at all. When the preview is unavailable (no token, no Paddle.js, a refused call) the card falls back to the catalogue's own $ list price and says that is what it is. A number this app computed would be a second, worse price next to the one Paddle actually charges.

Prefill. The signed-in address is passed as customer: { email } — customer is a top-level member of CheckoutOpenOptions in @paddle/paddle-js 1.6.5 (there is no settings.customer). It is belt-and-braces: the server also attaches customer_id, creating the Paddle customer from the same session address when the organization has never bought (see the commit note on ResolveCustomerIdAsync).

Afterwards. settings.successUrl is the app's own /app/checkout page, absolute and built from the origin the dashboard is served on. Paddle returns the buyer there with _ptxn; that page claims nothing about the payment, because only the webhook knows. Nothing in this flow grants credits.

The client half is web/lib/paddle/ (sdk.ts for the calls, pricing.ts for the pure mappings) and web/lib/hooks/use-paddle-prices.ts for the display prices.

Webhooks are authoritative

A checkout redirect, a success page or any client call never grants credits or changes a subscription. Only a verified webhook does. The browser can send the buyer back to a "thank you" page, but the balance changes when Paddle's notification is processed.

The receiver (POST /api/v1/billing/webhooks/paddle) is [AllowAnonymous]: the signature over the exact raw body is its only credential. It reads the body as raw bytes before any JSON parsing, because the signature covers the exact bytes. Responses: 400 for a bad, missing or stale signature or a malformed body (nothing is stored); 404 when billing is disabled or the provider is unknown; 200 for an applied, duplicate, stale or unresolved event; 500 when processing throws, so the provider retries. A body over 1 MB is rejected before it is read.

nginx routes the path with its own location = /api/v1/billing/webhooks/paddle in nginx/locations.conf. That block does not use the cookie-to-Authorization translation (there is no session on a server-to-server call), blanks any client-supplied Authorization header, and caps the body at 1 MB.

Exactly-once processing

BillingEventProcessor.ProcessAsync inserts a billing_webhook_events row, applies the effect and marks the row processed inside one database transaction. The table has a unique index on (provider, event_id), so a redelivery finds the row and is a no-op (outcome Duplicate). A crash or exception rolls the whole transaction back, including the event row, so the provider's retry runs the event again from scratch. Two concurrent deliveries of the same event are resolved by the unique index: the loser rolls back and reports Duplicate.

Credit effects carry their own ledger idempotency keys as a second line of defence, so a differently numbered event that describes the same payment or period cannot grant twice:

Effect	Ledger key
Plan refill	plan:{provider}:{subscriptionId}:{periodStart UTC}
Top-up grant	topup:{provider}:{transactionId}
Refund clawback	refund:{provider}:{adjustmentId}

Subscription state machine

A subscription event carries the provider's status, billing period, items and any scheduled cancellation. The processor resolves the organization from the organization_id the platform put into the checkout's custom data, falling back to an existing subscription row and then the stored billing customer. An event that names no known organization is stored with a note and acknowledged, not retried forever.

• Ordering. Providers do not deliver in order. Every applied change stamps subscriptions.last_event_at with the event's occurred_at; an event not newer than that is recorded as Stale and not applied. A late active update therefore never resurrects a subscription that a newer event canceled.
• Status. The provider status maps to Trialing, Active, PastDue, Paused or Canceled. A scheduled cancel sets CancelAtPeriodEnd and leaves the subscription active until it ends.
• Plan and seats. The plan is found by matching the subscription item price id against plans.provider_price_ids_json. A per-seat plan takes its seat count from the item quantity and never goes below the plan minimum (Team: 3).
• New subscription. A first paid subscription replaces any other non-canceled subscription of the organization (the internal Trial row).
• Refill. An Active or Trialing subscription with a known period start grants the plan's monthly credits (times seats for a per-seat plan), expiring at the period end. Because the key includes the period start, the created, updated and renewal events for one period grant once, and a renewal (a new period start) grants again. PastDue, Paused and Canceled grant nothing, and credits already granted stay until their own expiry. A plan or seat change inside a period does not grant extra credits until the next refill.
• Top-up. A completed stand-alone payment for a configured top-up price grants the pack's credits as a TopUp bucket expiring after 12 months. The pack size comes from the configured price id that was actually paid, never from custom data.
• Refund. An approved refund adjustment of a top-up transaction claws the credits back (a negative ledger adjustment, so the balance may go negative if they were already spent). Only a full refund of a top-up is automatic. A partial refund, or a refund of a subscription payment, is recorded with a note on the webhook row and needs a human decision.

Plan priority

Pro, Team and Enterprise runs are dequeued ahead of everyone else's. There is exactly ONE source for that: entitlements.priority in plans.catalog.json (0 on Trial and Creator, 1 on Pro and Team, 2 on Enterprise). It is read through the same plan entitlement every other runtime decision uses (OrganizationEntitlementSet.Priority, mapped by ReelBolt.Shared/Billing/RunPriority.cs), so there is no second "important customer" flag anywhere and a plan change applies to the next run with no priority state to keep in sync. With Billing__Enabled=false every org is unlimited and therefore standard (0), and a run is published byte-identically to how it was before this existed.

One number orders two queues:

Queue	Field it orders by	Ordered by whom
RabbitMQ workflow-execution	the AMQP message priority	the broker, because the engine declares the queue with x-max-priority (WorkflowEngine:QueuePriority:MaxPriority, default 10)
runner_jobs (Go runner gateway)	runner_jobs.priority	the gateway's claim: priority DESC, created_at ASC, id

A standard run is published with NO priority property at all, so nothing about it depends on this feature. The engine also resolves the run's class once when the execution starts and publishes it as WorkflowExecutionContext.RunPriority, so one run keeps one class for its whole lifetime. The value is clamped into [0, 255] (AMQP priority is one byte) so no catalog edit can produce a value the publish path cannot carry.

The second queue is not fed yet. RunnerJobRequest.Priority is accepted by the engine's gateway client and written to runner_jobs.priority verbatim, but no production code constructs a RunnerJobRequest — the executors that would are D8 (render and staging) and D11 (media sessions), and neither is in the tree. WorkflowExecutionContext.RunPriority therefore has no reader, and nothing writes runner_jobs.priority today. The hop is a contract with both ends built, not a wired path. See compute-fabric.md.

Operator note — the queue argument is a one-time migration. x-max-priority is a queue argument, and RabbitMQ refuses to redeclare an existing queue with different arguments (PRECONDITION_FAILED). An engine upgrading onto a broker that already holds the argument-less workflow-execution queue fails to start its bus until that queue is deleted — drain it first, its messages are not migrated. Set WORKFLOW_QUEUE_MAX_PRIORITY=0 (WorkflowEngine__QueuePriority__MaxPriority) to keep the old declaration and turn the ordering off; it is the only value that does.

Priority is a queueing decision only: it never changes what a run costs, whether it is admitted (that is the plan's concurrent-execution cap, CONCURRENCY_LIMIT_REACHED) or what it may include.

Paddle specifics

Verified against Paddle's developer documentation on 2026-10-06:

• Signature. The header is Paddle-Signature: ts=<unix seconds>;h1=<hex>. The signed payload is "{ts}:{raw body}", HMAC-SHA256 keyed with the destination's secret key, hex encoded, compared in constant time. More than one h1 is sent while a secret is being rotated, and any match is accepted. Paddle's SDKs reject a timestamp more than 5 seconds old; this implementation does the same by default (ToleranceSeconds). Source: https://developer.paddle.com/webhooks/signature-verification.
• Envelope. event_id, event_type, occurred_at, notification_id, data. The processor keys idempotency on event_id and ordering on occurred_at. Source: https://developer.paddle.com/webhooks/overview.
• Event types used. subscription.created, .activated, .updated, .canceled, .past_due, .paused, .resumed, .trialing (all mapped from data.status), transaction.completed (top-ups) and adjustment.created / adjustment.updated (refunds). Everything else, such as customer.created, is stored and ignored. Same source.
• Response and retries. Paddle expects 200 within 5 seconds. Sandbox retries 3 times in 15 minutes; live retries 60 times over 3 days. Webhooks are not guaranteed in order and may repeat. Source: https://developer.paddle.com/webhooks/respond-to-webhooks.
• Base URLs. Live https://api.paddle.com, sandbox https://sandbox-api.paddle.com; sandbox keys work only on sandbox and live keys only on live. Requests send Authorization: Bearer <key> and Paddle-Version: 1. Source: https://developer.paddle.com/api-reference/about.
• Checkout. POST /transactions with items (price_id, quantity), custom_data and an optional customer_id. The response's data.checkout.url is the hosted checkout; it needs a default payment link set in Paddle's checkout settings, and the call fails loudly without one. The response id is the transactionId the browser opens in Paddle.js's overlay (and it is also what Paddle appends to the success URL as ?_ptxn=). Source: https://developer.paddle.com/api-reference/transactions/create-transaction.
• Portal. POST /customers/{customer_id}/portal-sessions, optionally with subscription_ids; the link is data.urls.general.overview. Sessions are temporary and must not be cached. Source: https://developer.paddle.com/api-reference/customer-portals/create-customer-portal-session.
• Seats. PATCH /subscriptions/{id} with items: [{ price_id, quantity }] and proration_billing_mode (this code uses prorated_immediately). Source: https://developer.paddle.com/build/subscriptions/add-remove-products-prices-addons.

The portal-session page itself returned 404 from the documentation fetcher on the date above, so the endpoint path and response field come from Paddle's own search-indexed documentation snippets and changelog rather than a fully rendered reference page. Re-check them in the sandbox before launch.

Testing without a Paddle account

Automated tests (ReelBolt.WorkflowEngine.Tests/Billing) cover the processor on Sqlite, the controller over a fake provider, the Paddle provider over a stubbed HttpClient, and signed payloads end to end through the real controller. The payloads are synthetic, shaped after Paddle's documentation, not recorded from a live sandbox. A real sandbox purchase has not been run: that needs a Paddle sandbox account, a notification destination pointing at a reachable /api/v1/billing/webhooks/paddle, and the price ids below.

Operator setup checklist

1. Create a Paddle account (live) and a sandbox account (separate; they share nothing).
2. In each, create a product and a monthly price for Creator, Pro and Team (Team is per seat, quantity minimum 3), and one one-time price per top-up pack (500, 1,000 and 3,000 credits).
3. Set a default payment link in Checkout settings and approve your domain.
4. Create an API key and a notification destination at https://<your domain>/api/v1/billing/webhooks/paddle subscribed to the subscription.*, transaction.completed and adjustment.* events. Copy its secret key.
5. Put the price ids into plans.provider_price_ids_json per environment, for example UPDATE plans SET provider_price_ids_json = '{"paddle":{"monthly":"pri_..."}}' WHERE key = 'creator', and set the three PADDLE_TOPUP_PRICE_* variables.
6. Set BILLING_ENABLED=true, PADDLE_API_KEY, PADDLE_WEBHOOK_SECRET, PADDLE_ENVIRONMENT and PADDLE_CLIENT_TOKEN (the client-side token of the same account; blank is allowed and leaves the hosted checkout working without the embedded one).
7. Run a sandbox purchase and watch billing_webhook_events and the credit ledger.