Captain

joined 1 year ago
MODERATOR OF
ai_infosec
sorted by: new top controversial old
4
Identifying AI-generated images with SynthID (www.deepmind.com)
6
Thinking about the security of AI systems (www.ncsc.gov.uk)
6
GitHub - google/model-transparency (github.com)
8
Universal and Transferable Attacks on Aligned Language Models (llm-attacks.org)
7
Cybercriminals train AI chatbots for phishing, malware attacks (www.bleepingcomputer.com)
7
Adversarial suffixes that circumvent the alignment of open source LLMS, ChatGPT, Claude, Bard, and LLaMA-2 (twitter.com)
8
(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs (twitter.com)
7
Intro to ML Safety - Free course (course.mlsafety.org)
3
Army looking at the possibility of 'AI BOMs' (defensescoop.com)
7
Socket AI – using ChatGPT to examine every npm and PyPI package for security issues (socket.dev)
 

A very interesting approach. Apparently it generates lots of results: https://twitter.com/feross/status/1672401333893365761?s=20

4
OWASP Top 10 for LLMs - 0.5 (owasp.org)
submitted 1 year ago* (last edited 1 year ago) by Captain@infosec.pub to c/ai_infosec@infosec.pub
0 comments fedilink
13
Most popular generative AI projects on GitHub are the least secure (www.csoonline.com)
 

They used OpenSSF Scorecard to check the most starred AI projects on GitHub and found that many of them didn't fare well.

The article is based on the report from Rezilion. You can find the report here: https://info.rezilion.com/explaining-the-risk-exploring-the-large-language-models-open-source-security-landscape (any email name works, you'll get access to the report without email verification)

In Escalating Order of Stupidity in c/ai_infosec@infosec.pub
[–] Captain@infosec.pub 2 points 1 year ago

My take so far is that there isn't really any great options to protect against prompt injections. Simon Wilson presents an idea here on his blog which could is a bit interesting. NVIDIA has open sourced a framework for this as well, but it's not without problems. Otherwise I've mostly seen prompt injection firewall products but I wouldn't trust them too much yet.

Who is behind this instance and how is it financed? in c/infosecpub@infosec.pub
[–] Captain@infosec.pub 1 points 1 year ago (6 children)

Looks like you're right. It's not mentioned on that page but here he says he's the one running it.

view more: next ›