The BEST way to prevent sensitive information disclosure by large language model (LLM) chatbots is through:

Data masking is the most effective preventative control because it addresses the root cause of potential information disclosure. This technique replaces sensitive data elements (e.g., names, social security numbers, credit card details) with realistic but non-sensitive, structurally equivalent placeholders before the data is used to train or fine-tune the LLM. By masking the data at the source, the model never learns or memorizes the actual sensitive information, making it impossible for it to disclose it later. This preserves the data's format and utility for training while eliminating the privacy risk.

A. Manual monitoring: This is a detective control, not a preventative one. It is also reactive and not scalable for the vast number of interactions an LLM chatbot handles.

B. Access controls: This limits who can interact with the LLM but does not prevent the LLM from disclosing sensitive information it has already learned to an authorized user.

C. Data sanitization: While correct, this is a broad category of which data masking is a specific, and often superior, technique. Data masking is more precise as it maintains data utility for training.

1. National Institute of Standards and Technology (NIST). (2010). NIST Special Publication 800-122: Guide to Protecting the Confidentiality of Personally Identifiable Information (PII). Section 5.3

"De-identification of PII

" discusses techniques like masking to reduce the sensitivity of data

which is a foundational principle for preparing data for use in systems like LLMs.

2. Carlini

N.

Tramer

F.

Wallace

E.

Jagielski

M.

Herbert-Voss

A.

Lee

K.

... & Raffel

C. (2021). Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium (USENIX Security 21). This paper demonstrates that LLMs can memorize and regurgitate sensitive information from their training data

highlighting the critical need for preventative measures like masking to remove such data before training.

3. Stanford University

Center for Research on Foundation Models (CRFM). (2023). Report on the 2023 Workshop on Trustworthy ML: Robustness

Privacy

and Fairness. The report discusses privacy risks in foundation models

emphasizing the importance of data-centric mitigations. Section 3

"Privacy

" implicitly supports techniques like data masking by focusing on preventing the memorization of private training data.