Shipping confidential models to edge devices
You trained a model. It took months of work, and a carefully curated dataset. Now you have a functioning model, but you need to ultimately run it on a device that sits in someone else’s building, on someone else’s network, with someone else’s hands on it.
The moment it ships, the usual assumptions stop holding: there’s no perimeter, and no hypervisor you control. The device may be in a kiosk, a vehicle, a factory cell, a shop shelf.
It only takes someone to get a root shell, either a legitimate integrator with a support login, or an attacker who found a way in.
Every single device is a potential security risk for your model.
This post is about closing that gap: how to ship a model to hardware you don’t control, keep it encrypted at rest, decrypt it only inside a Trusted Execution Environment, and let only your code read the decrypted model.
The Model Is the Asset
Models are valuable IP, and your trained model is the compressed output of your data collection, your labeling, your architecture choices, and your training runs.
And unlike a service API, models on edge devices are simply there. Every unit you ship is a copy in someone else’s possession, indefinitely.
… And extraction doesn’t have to be fast or clean; an attacker has as long as they like and as many units as they can buy.
Regulation Is Coming to Supply Chains
Supply-chain security has moved from best practice to legal obligation. In the EU, the Cyber Resilience Act (CRA) puts binding requirements on products with digital elements sold into the single market.
Devices are now required to have secure-by-default configuration, documented handling of vulnerabilities, and evidence about the integrity and provenance of what you ship.
The good news is that these concerns are answered by the same mechanism: cryptographic provenance on the artifact, and enforcement on the device.
What We Built
We built a suite of tools for ARM devices running OP-TEE. The chain is short, and each link is worth stating precisely.
-
On the build host, the model is encrypted once with a fresh per-release content key (AES-256), and the resulting container is signed.
-
Per device, the content key is wrapped to that specific device’s public key. There’s no shared fleet key, and no manufacturing-time provisioning step.
-
On the device, a kernel module is the single point of enforcement. It will verify that the model is appropriately signed, and then decrypt it using the ARM TrustZone, and put the decrypted payload inside a protected memory area.
-
On the way out, finally every application accessing the model needs to also be signed. This means that even a privileged
rootuser on the device will not be able to access the decrypted model.
Any ARMv8 Device
Our solution is not tied to a specific vendor’s secure-enclave product. We built our solution on OP-TEE which means it runs on essentially any ARMv8 SoC with TrustZone — which today is most of the market.
This work has been validated on hardware from NVIDIA (Jetson Orin Nano), MediaTek (Genio 700), NXP (i.MX8M), Texas Instruments (AM68A & AM67A), and Rockchip (RK3588).
We also support an virtual QEMU target, to test the reliability of our setup. This allows us to make sure that any change will preserve the security guarantees enunciated above.
Signing
Signing is where most in-house schemes go wrong, so we deliberately didn’t invent one.
We use Sigstore’s cosign. This project from the Linux Foundation is the perfect fit for us, as it is widely popular, independently audited, and already present in most modern supply chains.
The decisive property is key custody. Thankfully, Cosign allows us to work with any cloud KMS or HSM, and the private key never exists on the build host.
For us to integrate with Cosign, we had to implement a lightweight Cosign verifier that can run in constrained environments, so we built and open-sourced cosign-verify.
Where Our Boundary Is
We want to be absolutely transparent about our security solution and the threats it addresses.
As showed above, we provide models protection against a compromised or hostile userspace, including privileged users.
To complete the full security chain for a device, we also need to address the platform from the bootloader onwards, through systems like secure boot, and certain Linux kernel features and options.
That’s also where we spend most of our time. We’ve done this bring-up across five silicon vendors, and we’re happy to advise regarding platform-specific hardening.
The next post in this series goes a level deeper into the device side: how our kernel module gates access, what the Trusted Application does, and how the per-device key wrap is put together.