What problem does the gateway solve in a clinic
A medical organization wants to speed up routine work: assemble a discharge summary from the medical record, rephrase prescriptions in plain language, and answer a patient's question, "What does my test result mean?" The strongest models for this are external cloud LLMs. But the medical record text is medical secrecy: full name, date of birth, policy and medical record numbers, address, ICD diagnosis. It cannot be sent directly to someone else's cloud service: the data will end up in the provider's logs, inference cache, and, if the vendor has an incident, in the blast radius.
The LLM & Security Gateway solves this structurally: a proxy gateway sits between the clinic application and the model. It intercepts every request, obfuscates identifiers and diagnoses before sending, sends de-identified text through the model, then restores the original values in the response before the doctor sees it. The model provider processes `[PERSON_1]` and `[MRN_7]`, not "Maria Petrovna Ivanova, medical record 4412".
How it works technically
Open reference stack - Microsoft Presidio (PII detection and anonymization, MIT license) plus a proxy gateway LiteLLM, which calls Presidio as a guardrail. The flow has three stages (LiteLLM docs):
1. Analyze. Presidio Analyzer finds entities in the text (`PERSON`, `LOCATION`, `DATE`, `PHONE_NUMBER`, medical identifiers) using NER, regular expressions, and checksums, assigning each match a confidence score.
2. Anonymize. Detected entities are replaced with placeholders (`<PERSON>`, `<MRN>`) or with tokens like `[PERSON_A3F2]`. Only the de-identified version is sent to the model.
3. Restore. When `output_parse_pii: true` is enabled, the gateway stores a "token -> original value" map for the lifetime of the request and inserts the real data back into the model response (Aptible).
Behavior is configured declaratively: for different data types, you set the action `MASK` (mask), `BLOCK` (reject the request entirely), and the confidence threshold `presidio_score_thresholds`. This gives engineering transferability: the application logic does not know about obfuscation, the same gateway is reused by all clinic services, and the model provider can be changed without rewriting code. This is the approach: keep business logic nearby, do not touch the tool core.
What matters about language and accuracy
For CIS-language medical records, the key setting is `presidio_language`: the detectors and NER model must work in the correct language, otherwise recall on CIS full names and addresses drops. Presidio can be extended with custom recognizers - you can add regex recognizers on top of the built-in core for the OMS policy number, SNILS, or the medical record format used by a specific HIS. For clinical text, the open-source community recommends strengthening general-purpose NER with specialized medical models (spaCy + clinical NER, or managed services), because generic NER catches diagnoses and medical terms less effectively (Aptible).
Honest limitations
This is not "magic anonymization," and public sources say so plainly. With NLP detection there is a share of false negatives: on benchmarks, recall is 90-99%, but on a specific clinic corpus some identifiers will slip through. Under HIPAA Safe Harbor, all 18 identifier categories must be removed, and even one missed item disqualifies the data as de-identified; in CIS, Federal Law GDPR and the medical secrecy regime play a similar role. So, legally, reversible tokenization is encoding, not de-identification: the data remains personal data, so a provider contract and a security boundary are still required (Accountable).
The token-to-value map itself becomes a sensitive asset: anyone who has both the de-identified log and the map can reconstruct the original data. The public guidance says to encrypt the map and keep a short TTL - minutes, not hours - with explicit deletion after restoration (Aptible). There is also a clinical risk: if the model returns a token in a form the map does not recognize, the doctor will see `[PERSON_A3F2]` where a name should be. That is why the gateway is added not instead of the protected environment, but as an extra layer: it sharply reduces the blast radius, sending a token into inference, cache, and vendor logs instead of medical secrecy.
Business process result
The "doctor -> discharge summary / patient reply" process changes like this: instead of manually drafting the summary, the doctor gets a draft in seconds, and the patient gets a clear explanation of the test result, while the original text with diagnoses and identifiers never physically leaves the clinic boundary. Regulatory risk shrinks from "medical secrecy was sent to an external cloud" to "de-identified text was sent, plus we keep a reversible map encrypted for minutes." Measurable impact: discharge summary preparation drops from tens of minutes to seconds for a draft that only needs editing, while the number of entities sent to the provider in plain form approaches zero with a tuned threshold and medical recognizers. Before production launch, detector recall must be measured on a real clinic corpus. That is the line between "we sped up routine work" and "we caused a leak."
