Fireworks, Notion, and Zero Data Retention
Tags: AI
Here’s a funny story about a hack. I thought this would be about an engineering problem, but it turns out it’s a story about how sales/post-sales processes can be messy, and corporate comms are hard.
Notion uses an AI provider called Fireworks. Firework got hacked recently, and a bunch of Notion’s enterprise user data was exposed. Some of those users are extra upset because they thought their data was covered by a “Zero Data Retention” (ZDR) policy, which would have protected their data even in the event that Fireworks got breached.
So, what went wrong (besides Fireworks getting hacked)?
As best I can tell:
- Notion’s contract with Fireworks said that all of Notion’s usage would be handled under ZDR.
- Notion built a feature using a Fireworks endpoint that’s non-ZDR. So a bunch of Notion user data was stored in Fireworks for that endpoint/feature.
- Fireworks got hacked, and that Notion user data was exposed.
- Also, Fireworks has a setting that customers can turn on to enforce ZDR for all usage in an account. Notion must have disabled the enforce-ZDR setting (or never enabled it), because they ended up building this whole “Insights” feature on top of a Fireworks endpoint that is not compatible with ZDR. That endpoint would’ve been blocked if the enforce-ZDR switch was on.
So, one of these things must have happened.
- Fireworks’s sales or post-sales team messed up. They should’ve forced that setting on for Notion’s account, because they were contractually obligated to enforce ZDR for Notion.
- Notion messed up and somehow didn’t turn it on even though they were told to, or they turned it off later without realizing what they were doing.
- This is funniest option, and I think it’s actually what happened: Notion signed the contract, turned on ZDR, then they turned the ZDR setting off on purpose because they wanted to use this feature. But… now that Fireworks got hacked, Notion is trying to blame Fireworks for not using ZDR. Even though Notion deliberately turned off the ZDR switch.
Notion’s AI security policy
I think Notion is covered by their own policy here. Their policy says that ZDR is enforced by default for enterprise customers, except if the customer flips a switch to enable some Notion features that aren’t ZDR-compatible. As long as the Insights feature was correctly gated behind that switch, Notion is in the clear.
But their customers seem really confused about this, and I wonder whether Notion communicated this clearly to them.
How would ZDR have prevented this?
Well, it wouldn’t have protected Fireworks from getting hacked. But it would have made it very unlikely that hackers could get this customer data, because the data wouldn’t have been stored anywhere. It would only have existed for the handful of seconds that it was actively being processed by Fireworks’s servers.
Notion’s corporate communications
Despite Notion seeming to be in the clear here, I’m surprised how bad their corporate communications have been. Their communications read as thought they were written by a bad lawyer, and they also sort of contradict themselves. It actually makes me wonder if parts of the Notion org did make some mistakes or fail to understand their own policies.
At the bottom of their Notice about the Fireworks AI incident, there’s an FAQ, and one of the FAQ sections is about ZDR. This section contains the best information they’ve shared with us about the incident. Unfortunately, it’s buried in an hidden FAQ section at the bottom of a “notice,” which is not itself linked from the security bulletin.
Click here to see the full text of that section of the FAQ
I have ZDR as part of my Enterprise plan. What does it cover for Notion AI use?
Zero data retention (ZDR) is our commitment for our enterprise customers. It applies to the API requests we send to third party model vendors we use to generate responses to Notion AI requests. For those requests, we work with each third party vendor to ensure their API endpoints are ZDR compliant, and therefore have an expectation that they don’t retain any of the request and response data associated with the API request.
Notion also supports product features that require us to store the requests and responses made to those third party models, including features like chat history and Insights. Insights is a workspace tool that gives administrators information about how their workspace members use AI. Our zero data retention commitments are not intended to cover these use cases where we are required to retain the data to deliver the feature.
For the Insights feature, we sent the stored requests and responses from Notion AI use to a third party model provider, Fireworks in this case, to run the analysis that we visualize for the administrator.
We apply our general data minimization practices when providing our services, including the analysis performed for Insights. As part of these data minimization efforts, Fireworks provided us with a contractual commitment to zero data retention for our use of its platform. Unfortunately, during our investigation we learned that data was retained from the Fireworks endpoints that we used.
Looking forward, we are doing an audit of critical vendors — such as inference providers, and data processors — to verify that they abide by data minimization practices, including zero data retention policies.
First, they say “Fireworks provided us with a contractual commitment to zero data retention for our use of its platform.” That’s pretty clear, actually! Sounds like their Fireworks contract required Fireworks to follow ZDR for the entire Notion account. But, if that’s true, then someone did mess up here, right? Because clearly, some of Notion’s usage was not ZDR’ed!
Next, they seem to misunderstand their own AI policy. They say that the exposed feature was not ZDR-protected because Notion needs to store some data in order for the feature to work. This is … not right? In the context of this data breach, “ZDR” means that the Fireworks does not retain the data. It doesn’t mean that Notion can’t retain the data. Notion needing to retain its own data is not relevant here at all, and it’s confusing to the reader. The question is whether Notion did a good job ensuring that Fireworks used ZDR for this data.
Then they sort of try to blame Fireworks for not enforcing ZDR properly, right after they said that they themselves didn’t mean to use ZDR for that feature. See how they “imply” without actually stating: “Fireworks provided us with a contractual commitment to zero data retention for our use of its platform … Looking forward, we are doing an audit of critical vendors … to verify that they abide by data minimization practices, including zero data retention policies.”