SoylentNews

looorg [soylentnews.org] writes:

https://www.cio.com/article/4155404/ai-token-freeloaders-are-coming-for-your-customer-support-chatbot.html [cio.com]

Coversation framing or Social-engineering the Customer support AI bots. Making them do things to burn company tokens. One just can't stop laughing.

Users are tricking enterprise chatbots into performing complex AI computations unrelated to customer support, with potentially costly governance and ROI ramifications.

He adds: “Anyone who’s spent five minutes with these tools knows you can steer past a system prompt with basic conversational framing, which is exactly what [is happening to enterprises today]. The system authenticates the session, not the intent.”

“A normal customer service interaction of ‘Where’s my order? What are your hours?’ runs maybe 200 to 300 tokens. Someone asking the bot to reverse a linked list in Python is generating more than 2,000 tokens easy. That’s roughly a 10x cost multiplier per session,” says Nik Kale, member of the Coalition for Secure AI (CoSAI) and ACM’s AI Security (AISec) program committee.

Original Submission