Keeping PHI out of your error reports

An error report is a copy of production data

When an exception is reported, the SDK packages up whatever it can see: the message, the stack trace, the request it happened in, who was signed in and what happened just before. That package leaves your servers, is stored by your error tracker for its retention period, and is read by everyone on the team, often in a chat channel or a ticket as well.

None of those fields is meant to hold health information, and all of them can. A typical crash report has these parts:

• The exception message, often built from the values that failed
• The URL or path of the request, with its query string
• Request parameters and headers, when the SDK or your own code attaches them
• The signed-in user: an id, and often an email address or a name
• Breadcrumbs: log lines, queries, clicks and page views from just before the error
• The source code around each line of the stack trace, and in some tools the values of local variables

The sections below take each one in turn: how PHI gets in, the general fix, and what ForgeOps strict privacy does about it.

Exception messages

Messages are the biggest leak, because developers write them to be helpful, and the most helpful thing to put in a message is the value that failed. "No active coverage for Jane Doe, born 1961-04-02" is a great message to debug from, and a disclosure of PHI in one line. Framework messages do it too: a uniqueness validation names the email address that was taken, and a database constraint error can quote the row it rejected.

The usual advice is to put ids rather than names in your own messages, and to scrub what the SDK sends. Scrubbing helps, but it works by pattern, and no pattern recognizes a name or a diagnosis. The only dependable rule is not to send the message at all.

With strict privacy on, ForgeOps SDKs never send the message. They send a fingerprint of it instead: a one-way hash computed in your process, with numbers and ids taken out first, so errors still group by message (the companion guide below explains how). In Rails it's two lines in the initializer you already have:

Ruby, config/initializers/forge_ops_tracker.rb

ForgeOpsTracker.configure do |config|
  config.dsn = ENV["FORGE_OPS_DSN"]
  config.strict_privacy = true                           # default false
  config.privacy_allowed_fields = %w[error_code feature] # default []
end

Ruby: what a strict SDK does with a message like this

# Without strict privacy, this sentence is the error report's title.
# With it on, ForgeOps receives the class, the stack trace and a
# 32 character fingerprint of the message. The text stays here.
raise CoverageError, "No active coverage for #{patient.full_name}, born #{patient.date_of_birth}"

Ids in messages are still the better habit under strict privacy. The fingerprint ignores numbers, so "no coverage for patient 8812" and "no coverage for patient 9004" group as one issue, while two different names hash differently and split into two.

URLs, paths and query strings

Paths name people more often than you'd expect: /patients/jane-doe/chart, /members/A12345678/claims, /search?name=doe&dob=1961-04-02. A path can appear in an error report as the request's URL, as the name of the transaction the error happened in, as the URL of every outbound API call in its trace, and as the page address of a browser error or a page speed measurement.

A route template carries none of that: /patients/:id/chart is a developer's words with a placeholder. So under strict privacy ForgeOps keeps a route template exactly as it is (:id, {id}, <int:id>, [slug] and *path all count) and never sends the raw path. When no route matched, the path keeps only its first segment: /patients/jane-doe/chart becomes /patients/:path, and a first segment holding a digit becomes :id. Query strings and fragments are always dropped.

Outbound calls are named by method and host only (GET api.payer.example), without credentials or port. A browser page address keeps its scheme and host and goes through the same path rule. Framework integrations already report the route, so a Flask app needs nothing beyond the option:

Python, Flask

import os

import forge_ops_tracker
from flask import Flask
from forge_ops_tracker.integrations.flask import init_flask

forge_ops_tracker.init(
    dsn=os.environ["FORGE_OPS_DSN"],
    strict_privacy=True,
    privacy_allowed_fields=["error_code", "feature"],
)

app = Flask(__name__)
init_flask(app)


@app.get("/patients/<int:patient_id>/refills")
def refills(patient_id):
    # Reported as GET /patients/<int:patient_id>/refills, whatever the real
    # id and query string were. The message stays in this process, and the
    # path and query string the integration adds as context are dropped,
    # since neither is an allowed field.
    raise LookupError(f"No pharmacy on file for patient {patient_id}")

Go, net/http: give the SDK the route template yourself

package main

import (
    "errors"
    "net/http"

    forgeops "github.com/Luke-Popwell/forge-ops-tracker-go"
    forgeopshttp "github.com/Luke-Popwell/forge-ops-tracker-go/integrations/nethttp"
)

func chart(w http.ResponseWriter, r *http.Request) {
    // Report the template, not /patients/8812/chart. Without this line the
    // path rule still applies, and the request is sent as GET /patients/:path.
    forgeops.SetRequestRoute(r.Context(), "/patients/{id}/chart")

    err := errors.New("chart locked for patient " + r.PathValue("id"))
    forgeops.CaptureErrorCtx(r.Context(), err, map[string]any{"error_code": "CHART_LOCKED"}, nil)
    http.Error(w, "chart unavailable", http.StatusServiceUnavailable)
}

func main() {
    forgeops.Init(func(c *forgeops.Configuration) {
        c.StrictPrivacy = true
        c.PrivacyAllowedFields = []string{"error_code", "feature"}
    })

    mux := http.NewServeMux()
    mux.HandleFunc("GET /patients/{id}/chart", chart)
    http.ListenAndServe(":8080", forgeopshttp.Middleware(forgeopshttp.Timing(mux)))
}

Request parameters and headers

Form fields and JSON bodies are where the clinical data actually lives: an intake form, a symptom checker's answers, a prescription. Headers carry cookies, tokens and sometimes a member id your own gateway added. An SDK that attaches the request body or headers to every error copies all of it into every report from that endpoint.

ForgeOps SDKs don't attach request bodies or headers to error events, and the events API has no field for them. What can carry request data is context. The Django and Flask integrations add the request's path, method and query string as context on an unhandled error, and your own code can add anything at all. Under strict privacy, context is dropped entirely unless you name a field as allowed, so none of that leaves your app.

Don't add path or query_string to the allowed fields. That would let the raw path, and whatever is in its query string, straight back through.

User context

Knowing which user hit an error is useful: it tells you how many people are affected and lets support follow up. But an email address or a username identifies someone, and in a patient portal the user is the patient.

Under strict privacy the user is sent by id only, never an email or a username. That applies to automatic detection too, such as the Rails gem's Devise support and the Django integration's user middleware, so existing code can stay as it is. Customers affected and affected user counts still work, because they only need the id.

JavaScript, Node and Express

import express from "express";
import * as forgeOpsTracker from "@forge-ops/tracker";
import { forgeOpsTrackerExpressMiddleware } from "@forge-ops/tracker/integrations/express";

forgeOpsTracker.init({
  dsn: process.env.FORGE_OPS_DSN,
  strictPrivacy: true,
  privacyAllowedFields: ["error_code", "feature"],
});

const app = express();

// Your own auth middleware has set req.user by now. Under strict privacy only
// the id goes out with an error; the email stays in this process.
app.use((req, res, next) =>
  forgeOpsTracker.runWithUser({ id: req.user?.id, email: req.user?.email }, next),
);

app.get("/appointments/:id", (req, res) => {
  // Reported as GET /appointments/:id, with no message and no query string.
  throw new Error(`No slot for ${req.user?.email} on ${req.query.date}`);
});

app.use(forgeOpsTrackerExpressMiddleware); // last, after all routes
app.listen(3000);

Breadcrumbs

Breadcrumbs are the trail of what happened before an error, and most SDKs record them automatically: SQL queries, log lines and outbound calls on the server, and clicks, console output and page navigations in the browser. Each one is a small piece of free text, and in a health app the trail reads like the patient's visit: the search they ran, the page they opened, the query that loaded their record.

Under strict privacy ForgeOps SDKs don't record breadcrumbs at all, automatic or added by hand, so there's no trail sitting in memory to leak either. In the browser, strict privacy also keeps session replay off whatever its own option says, and a page address is sent as its host and route-level path only.

TypeScript, in the browser

import * as forgeOpsTracker from "@forge-ops/tracker-web";

forgeOpsTracker.init({
  dsn: "https://<api_key>@getforgeops.net/api/v1/events",
  strictPrivacy: true,
  privacyAllowedFields: ["error_code", "feature"],
});

// No clicks, console output or navigations are recorded, nothing is recorded
// for session replay, and https://clinic.example.com/patients/jane-doe/chart?tab=2
// is sent as https://clinic.example.com/patients/:path.

Source code, local variables and the screen

Some error tools capture the values of local variables in each stack frame, which in a health app means the patient record the function was working on. ForgeOps SDKs don't capture local variables. They can capture a few lines of your source code around each frame, which is usually safe but can include test fixtures or hard-coded values, so strict privacy turns that off too: the source file is never read.

A failing database call's SQL statement isn't sent with the error either, even with statement capture on. The names of the tables and procedures it touched still are, so you know where to start looking. In traces, a database span keeps only its masked statement (values replaced) and which database ran it.

And the screen itself: session replay is a recording of what the user saw, which no scrubbing makes safe. A strict browser or React Native SDK never records one, and the server won't keep a recording for a strict project even if one arrives.

The context you choose to send

Sometimes you need a little more than the stack trace: your own error code, which feature failed, which integration returned the error. Strict privacy lets named fields through, and nothing else:

• Only the top-level context and tag keys you list, in the SDK's privacy_allowed_fields and in the project's Fields allowed under strict privacy setting
• Only plain values: text, numbers, true or false, never a nested object
• Up to 20 field names, using letters, numbers, _ or -

Ruby, a Rails controller

class EligibilityChecksController < ApplicationController
  def create
    patient = Patient.find(params[:patient_id])
    EligibilityCheck.run!(patient)
    head :no_content
  rescue EligibilityCheck::Rejected => e
    ForgeOpsTracker.capture_exception(e, context: {
      error_code: e.code,              # allowed: sent
      feature: "eligibility",          # allowed: sent
      member_id: patient.member_id,    # not allowed: stays in the app
      payer_response: e.response_body  # not allowed: stays in the app
    })
    head :unprocessable_entity
  end
end

List a field in both places. The SDK decides what leaves your app and the project setting decides what's stored, so a field named in only one of them is dropped.

Two layers, and what's still yours

Strict privacy is a per-project setting in ForgeOps (under Privacy in the project's settings), off by default, and an option in each SDK, also off by default. Turn on both. The SDK option matters most, because with it the text never leaves your process. The project setting applies the same rules again when data arrives, so an app running an older SDK, or one where someone forgot the option, still can't get text stored. Errors stored before you turned it on are kept as they were, until the project's retention period removes them.

SDKs that read their DSN from the environment read these settings from it too, which makes it easy to turn on everywhere at once. An option set in code wins over the environment:

Shell

export FORGE_OPS_STRICT_PRIVACY=true
export FORGE_OPS_PRIVACY_ALLOWED_FIELDS=error_code,feature

Strict privacy limits what an error report can contain. It doesn't settle your obligations under HIPAA or any other rule on its own: those depend on your own setup, including what else your apps send and where, your agreements with the services you use, and your own policies. Check it against your requirements, and test it: trigger an error with made-up patient data in staging and look at exactly what arrives.