TechTakes

1436 readers

195 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago

MODERATORS

dgerard@awful.systems

105

in absolutely the funniest outcome so far, you can send data to an LLM that pops a Remote Code Execution vulnerability (mastodon.social)

submitted 7 months ago* (last edited 7 months ago) by dgerard@awful.systems to c/techtakes@awful.systems

13 comments fedilink hide all child comments

courtesy @self

preprint: https://arxiv.org/pdf/2309.02926
blackhat abstract: https://www.blackhat.com/asia-24/briefings/schedule/index.html#llmshell-discovering-and-exploiting-rce-vulnerabilities-in-real-world-llm-integrated-frameworks-and-apps-37215
Tong Liu's related research: https://scholar.google.com/citations?hl=en&user=egWPi_IAAAAJ

can't wait for the crypto spammers to hit every web page with a ChatGPT prompt. AI vs Crypto: whoever loses, we win

you are viewing a single comment's thread
view the rest of the comments

[–] zogwarg@awful.systems 6 points 7 months ago

From reading the paper I'm not sure which is more egregious, the frameworks that pass code and/or use exec directly without checking, or the ones that rely on the LLM to do the checking (based on the fact that some of the CVEs require LLM prompt jailbreaking)

If you wanted to be exceedingly charitable, you could try and make the maintainers of said framework claim that "of course none of this should be used with unsanitized inputs open to the public, it's merely a productivity boost tool that you would run on your own machine, don't worry about possible prompts being evaluated by our agent from top bing results, don't use this for anything REAL."