SoylentNews

jelizondo [soylentnews.org] writes:

Ars Technica published an interesting article [arstechnica.com] about a new AI assistant that provides strong assurances that user data is unreadable even to the platform operator,

Moxie Marlinspike—the pseudonym of an engineer who set a new standard for private messaging with the creation of the Signal Messenger—is now aiming to revolutionize AI chatbots in a similar way.

His latest brainchild is Confer, [confer.to] an open source AI assistant that provides strong assurances that user data is unreadable to the platform operator, hackers, law enforcement, or any other party other than account holders. The service—including its large language models and back-end components—runs entirely on open source software that users can cryptographically verify is in place.

Data and conversations originating from users and the resulting responses from the LLMs are encrypted in a trusted execution environment (TEE) that prevents even server administrators from peeking at or tampering with them. Conversations are stored by Confer in the same encrypted form, which uses a key that remains securely on users’ devices.

Like Signal, the under-the-hood workings of Confer are elegant in their design and simplicity. Signal was the first end-user privacy tool that made using it a snap. Prior to that, using PGP email or other options to establish encrypted channels between two users was a cumbersome process that was easy to botch. Signal broke that mold. Key management was no longer a task users had to worry about. Signal was designed to prevent even the platform operators from peering into messages or identifying users’ real-world identities.

All major platforms are required to turn over user data to law enforcement or private parties in a lawsuit when either provides a valid subpoena. Even when users opt out of having their data stored long term, parties to a lawsuit can compel the platform to store it, as the world learned last May when a court ordered OpenAI to preserve all ChatGPT users’ logs [arstechnica.com]—including deleted chats and sensitive chats logged through its API business offering. Sam Altman, CEO of OpenAI, has said [yahoo.com] such rulings mean even psychotherapy sessions on the platform may not stay private. Another carve out to opting out: AI platforms like Google Gemini may have humans read chats [google.com].

“AI models are inherent data collectors,” Em [infosec.exchange] [she keeps her last name off the Internet] told Ars. “They rely on large data collection for training, improvements, operations, and customizations. More often than not, this data is collected without clear and informed consent (from unknowing training subjects or from platform users), and is sent to and accessed by a private company with many incentives to share and monetize this data.”

In response, Marlinspike has developed and is now trialing Confer. In much the way Signal uses encryption to make messages readable only to parties participating in a conversation, Confer protects user prompts, AI responses, and all data included in them. And just like Signal, there’s no way to tie individual users to their real-world identity through their email address, IP address, or other details.

“The character of the interaction is fundamentally different because it’s a private interaction,” Marlinspike told Ars. “It’s been really interesting and encouraging and amazing to hear stories from people who have used Confer and had life-changing conversations, in part because they haven’t felt free to include information in those conversations with sources like ChatGPT or they had insights using data that they weren’t really free to share with ChatGPT before but can using an environment like Confer.”

One of the main ingredients of Confer encryption is passkeys [arstechnica.com]. The industry-wide standard generates a 32-byte encryption keypair that’s unique to each service a user logs in to. The public key is sent to the server. The private key is stored only on the user device, inside protected storage hardware that hackers (even those with physical access) can’t access. Passkeys provide two-factor authentication and can be configured to log in to an account with a fingerprint, face scan (both of which also stay securely on a device), or a device unlock PIN or passcode.

The other main Confer ingredient is a TEE on the platform servers. TEEs encrypt all data and code flowing through the server CPU, protecting them from being read or modified by someone with administrative access to the machine. The Confer TEE also provides remote attestation. Remote attestation is a digital certificate sent by the server that cryptographically verifies that data and software are running inside the TEE and lists all software running on it.

Original Submission