Integrating secure inference capabilities across distributed edge nodes and Cloud VMs requires specific architectural considerations to protect…

The most effective approach to prevent model extraction from a compromised physical edge device is to use a Trusted Execution Environment (TEE). A TEE, such as ARM TrustZone which is available on NVIDIA Jetson platforms, creates a hardware-isolated, secure enclave. The model weights and the inference engine are loaded and executed within this enclave. This ensures their confidentiality and integrity, as even a privileged user or compromised operating system on the device cannot directly access the memory space of the TEE. This directly addresses the architect's primary concern of protecting the proprietary model if the physical device is stolen or compromised.

B. Homomorphic Encryption protects data during computation but is not typically used to protect the model weights at rest or in memory on an edge device from extraction.

C. Role-Based Access Control (RBAC) on a vector database manages data access for Retrieval-Augmented Generation (RAG) but does not protect the model file itself on the device's filesystem.

D. API rate limiting is a network-level control to prevent service abuse; it offers no protection against an attacker with physical access to the device.

1. Sabt, M., Achemlal, M., & Bouabdallah, A. (2015). Trusted execution environment: what it is, and what it is not. In 2015 IEEE Trustcom/BigDataSE/ISPA (Vol. 1, pp. 57-64). IEEE. This paper defines TEEs and their security guarantees, including isolating code and data from the host OS. (https://doi.org/10.1109/Trustcom.2015.357)

2. NVIDIA. (2023). Jetson Orin Series and Jetson AGX Orin Series Technical Reference Manual. Chapter 3, "Security," describes the hardware-backed security features, including the use of ARM TrustZone technology to create a secure world (TEE) for sensitive operations.

3. Hetzelt, F., & Rannenberg, K. (2021). Confidential Computing and its Application in Machine Learning. In Proceedings of the 16th International Conference on Availability, Reliability and Security. This work discusses using TEEs like Intel SGX and ARM TrustZone to protect the confidentiality of ML models during inference. (https://doi.org/10.1145/3465481.3470057)