SoylentNews is people
SoylentNews
At least 100 instances of malicious AI ML models were found on the Hugging Face platform, some of which can execute code on the victim's machine, giving attackers a persistent backdoor.
Hugging Face is a tech firm engaged in artificial intelligence (AI), natural language processing (NLP), and machine learning (ML), providing a platform where communities can collaborate and share models, datasets, and complete applications.
JFrog's security team found that roughly a hundred models hosted on the platform feature malicious functionality, posing a significant risk of data breaches and espionage attacks.
This happens despite Hugging Face's security measures, including malware, pickle, and secrets scanning, and scrutinizing the models' functionality to discover behaviors like unsafe deserialization.
[...] The analysts deployed a HoneyPot to attract and analyze the activity to determine the operators' real intentions but were unable to capture any commands during the period of the established connectivity (in one day).
(Score: 3, Informative) by mth on Saturday March 02 2024, @10:47PM (1 child)
You can use Python with the safetensors format. Using pickle as a distribution format was never a good idea.
(Score: 2, Informative) by grant on Sunday March 03 2024, @03:36AM
Yeah, using pickle is the issue for the vulnerability in the python based stuff, not python itself. But, with the *.cpp stuff, you get safe models AND easy setup.:
For plain CPU only inference, there are zero deps, just:
make
You're done setting things up. You can now run inference on any llama family model you have downloaded.
For vulkan, If your system can already run vulkan stuff like games, there is only a single dep to install (assuming Debian):
apt install libvulkan-dev
make LLAMA_VULKAN=1
If you have never run vulkan stuff before, you may also need to:
apt install mesa-vulkan-drivers vulkan-tools
No coda/miniconda + virtual env with a million deps and files you need to edit to change the back-end (everything assumes cuda by default) that, if they are documented anywhere but random websites in an incomplete form, I failed to find that documentation.
Maybe I was holding the python stuff wrong out of ignorance about the platform, but llama.cpp and friends are so easy to setup it is probably an improvement even for folks who are familiar with the python way. For folks like me not deep into python, it was a huge improvement.