Spend a couple of words for uneducated folks?

[ghost]

Hi, coming from a non-IT field I would like to ask the owner of the code if they could spend a couple words talking about the security of this new format.

In simple words, what does it mean that this format is safe?

Does it mean I can convert any malicious .ckpt model that I find and once it is .safetensors it can't infect my machine anymore? Or is this format only protecting against some kinds of threats? if so, which ones? How concerned should I be running randomg converted .ckpts found over the internet once they are made .safetensors?

Thank you

[Narsil]

Hello !

I put in bold the tl;dr non technical speech. Unfortunately I am a technical person and cannot help myself but be specific at the same time (which does imply some jargonning, sorry).
If you are interested, and/or do not understand things I am alluding to here, please ask.

In simple words, what does it mean that this format is safe?

It means you should never be scared to open these files.

Contrary to Pytorch pickled files, which can contain arbitrary code and steal all your bitcoins.

Does it mean I can convert any malicious .ckpt model that I find and once it is .safetensors it can't infect my machine anymore?

Reading a malicious pickle file will infect your machine (depending on what actually goes in there tbh). So there is nothing preventing you from being infected if you ever read a malicious pickled file.
Personally I used colab to inspect unknown pickle files (so I can play around with them and worst case it infects google, not my computer). You can then save them back within the colab and those new files are safe (in the sense it won't execute code on your behalf when you load the model).
You can also use https://huggingface.co/spaces/safetensors/convert to convert some hf.co models automatically (it still needs some work to function on most CompVis format files out there and handle various differences I've seen in the wild). This would infect huggingface, not you.

Or is this format only protecting against some kinds of threats? if so, which ones? How concerned should I be running randomg converted .ckpts found over the internet once they are made .safetensors?

The threats currently avoided is Arbitrary Code Execution (ACE: people can do whatever they want with your machine). And DOS (Denial of Service: your computer stops responding or crashes) a much lesser kind of attack, usually it just tries to fill your memory or make your computer do unnecessary work to prevent it doing it's regular job, but it's not a means (on its own at least) for an attacker to gain control of your machine or steal your bitcoins.

How concerned should I be running random converted .ckpts found over the internet once they are made .safetensors?

Random .safetensors found on the internet should never execute anything on your machine on your behalf when you load them.
They are a machine learning model, pure data, nothing more.

That's the first goal I had in mind when creating this.

Actually if you don't trust this repo's code, you can always load them yourself manually https://gist.github.com/Narsil/3edeec2669a5e94e4707aa0f901d2282 . This snippet is meant to be simple to adapt and pretty efficient with torch.
Technically you still rely on python to not have any security defects but overall if you trust Python (and torch in this particular snippet) you're good to go.

And if you don't trust Python, you can use any other language to use those files
That's part of the security, the design of the files is meant to be trivial so that you don't have to trust me or complex programs to be able to use them. You basically need to trust at least one JSON parser to be correct. This library's one is implemented in Rust, so it should be free of most classic use after-free, buffer overflows which are classical threats: https://www.quora.com/Does-the-Rust-programming-language-prevent-buffer-overflow-exploits

This repo does add some nice checks for you that are not in the snippet, which prevent loading huge json files (also a DOS), or intentionally malformed script that could cause DOS. It's hard to say never in security, but let's say nothing weird should happen when using those files. They contain tensors, so when you load you should get tensors of roughly the same size they are on disk.

[ghost]

Hi! First of all sorry for the double posting. Secondly, I want to thank you greatly for this detailed answer, the clarity that you use to write it and the speed of the delivery. All this for free, to a stranger. Kudos!

What you're telling me it's honestly so liberating, as I was very concered with opening random ckpts on AUTOMATIC,'s webui, and to know that i can basically run any .safetensors without any fear that i will execute malicious code, basically i can just feel safe, is honestly so good to hear. I know this is not useful but i wanted to send some virtual thanks and make you feel appreciated for this, which i believe is very good work.

I'm already using the safetensor! I manually modified AUTOMATIC's WEBUI with the files i found in pattonim's commit here .

Then I take the ckpts, and use the colab you provided here to that user, all I do is slightly modify it to work with the ckpt i have in drive (or !wget from huggingface for example). On a side note, the NAI ckpt you uploaded in huggingface for that user might be illicit (i think it's stolen proprietary software), i suggest you look into it or remove it from hugginface to be safe.

By the way, sometimes the free colab has not enough ram (12gb total) to convert .ckpts that are bigger than 4gb (I recently converted a 6bg one), but I found out a workaround that lets you use 25 gbs of ram instead of the usual 12gbs for free, let me know if you need it!

They are a machine learning model, pure data, nothing more. This is what most people actually think that normal .ckpts do. With the fact that safetensors can even support merges, I have 0 doubts that this will be largely adopted as soon as it's implemetned in AUTOMATIC's and will be greatly appreciated by the community, once people realize what it is and how easy it is to use.

[Narsil]

Thank you for the kind words.

By the way, sometimes the free colab has not enough ram (12gb total) to convert .ckpts that are bigger than 4gb (I recently converted a 6bg one), but I found out a workaround that lets you use 25 gbs of ram instead of the usual 12gbs for free, let me know if you need it!

Really interesting (and surprising). There shouldn't any reason to use more than 2X the memory. To be fair I didn't really optimize writing in general in here, so there might be room for improvement. torch.load unfortunately loads the whole thing in memory by default.
It's actually technically possible to load each tensor one at a time when using custom functions instead of torch.load IIRC. I could look it up if more than 10 ppl are interested. (And if any reader want to do it, it's just a question of reading torch.load source code and not loading all the tensors at once, it is going to create copies where some tensors share some underlying data, which might not be desirable).

Anyway, you most certainly don't require all this memory to copy things around, but it might require creating some custom code for it.

[CodeByRRR]

Hello, Thank you really for the answer here. its neat !
One question, as you mentioned it is designed to avoid - ACE and DOS, how it is ensuring that ? Is it because it is storing pure data (in BYTES) and not storing any computation graph or so.
If you can please kindly, shed some light on how safetensors ensuring avoidance of ACE and DOS. Please feel free to answer in technical heavy way and you can also refer to code snippet, if needed.

Thank you in advance and appreciation.

[Narsil]

ACE and DOS, how it is ensuring that ?

Warning notice: It's impossible to prove the absence of flaws.

While it's impossible to prove there's no error, (It could be in the compiler, in the hardware, in the OS etc...) here are a list of steps taken here to assure no ACE:

Use RUST, don't eval code. That's about it.

torch pickle uses eval on the file, so attackers can write pretty much what they want to execute code on a given machine.
safetensors doesn't . That seems easy enough. BUT code written in low level languages can be vulnerable to https://en.wikipedia.org/wiki/Buffer_overflow and https://en.wikipedia.org/wiki/Dangling_pointer which can enable ACE. Rust the language is not vulnerable to this by virtue of it's semantics: https://doc.rust-lang.org/rustc/exploit-mitigations.html https://www.quora.com/Does-the-Rust-programming-language-prevent-buffer-overflow-exploits
Now, as previously stated, it's impossible to be 100% sure it's good because the compiler could be bugged, the standard library (Which uses unsafe could be bugged, or even PyO3 which are creating the link with Python could be bugged. But there's a strong probability that it's not vulnerable. If a vulnerability exists, it's guaranteed to be linked to use of unsafe (which this library make a single use of which should be correct. unsafe isn't inherently bad, but it needs to be used when the compiler by itself cannot make some guarantees, and the responsability lies in the hand of the developer.

To assure no DOS is triggered:

This one is first of all a much smaller guarantee, and not even guaranteed till the end of time. If usability can be vastly improved and is required for mass adoption, we could potentially lower that barrier for usability. I don't see that happening, but never say never.

• serde_json is used under the hood, and that comes already with a bunch of protection, like stack overflow protection etc.. (https://www.reddit.com/r/rust/comments/8mjait/is_serialization_considered_a_security_risk/)
• The structure is highly organized, removing the need for arbitrary structure parsing which could grow faster than the file size.
• header size is capped to 100Mo https://github.com/huggingface/safetensors/blob/main/safetensors/src/tensor.rs#L10 meaning even if somehow JSON could cause DOS the damage would be relatively limited (since models are usually much larger than this).
• Lots of checks during reading to make sure the data part of the file is read only once and never copied. (This provides both zero-copy and protection against DOS)

Hope that helps.
Cheers.

[CodeByRRR]

Thank you very much !!!
But is it possible that the safetensor model file could be tampered/corrupted. Attacker can change the weights/values (aka, tensors bytes). So in this case, it is not ACE or DOS but the model wont be true anymore. And during load time, user wont know that model was tampered.
Please help me understand.

Big thanks once again.

[Narsil]

So in this case, it is not ACE or DOS but the model wont be true anymore

Absolutely !
If attacker has access to a file, he can modify weights at will.
The only protection from that is MitM protection, and a good base is usually for the distributor of weights to send both the file and the checksum of the file through different means:

Like so : https://huggingface.co/bert-base-uncased/blob/main/model.safetensors

Here you can download the file, and you can see the expected sha on the page. An attacker would need to takeover hf.co to be able to mess with your weights to modify both the file and the checksum.