The way you talk can reveal a lot about you—especially if you're talking to a chatbot. New research reveals that chatbots like ChatGPT can infer a lot of sensitive information about the people they chat with, even if the conversation is utterly mundane.
The phenomenon appears to stem from the way the models' algorithms are trained with broad swathes of web content, a key part of what makes them work, likely making it hard to prevent. "It's not even clear how you fix this problem," says Martin Vechev, a computer science professor at ETH Zürich in Switzerland who led the research. "This is very, very problematic."
Vechev and his team found that the large language models that power advanced chatbots can accurately infer an alarming amount of personal information about users—including their race, location, occupation, and more—from conversations that appear innocuous.
[...]
Researchers have previously shown how large language models can sometimes leak specific personal information. The companies developing these models sometimes try to scrub personal information from training data or block models from outputting it. Vechev says the ability of LLMs to infer personal information is fundamental to how they work by finding statistical correlations, which will make it far more difficult to address. "This is very different," he says. "It is much worse."
Related Stories
In 2013, Spike Jonze's Her imagined a world where humans form deep emotional connections with AI, challenging perceptions of love and loneliness. Ten years later, thanks to ChatGPT's recently added voice features, people are playing out a small slice of Her in reality, having hours-long discussions with the AI assistant on the go.
In 2016, we put Her on our list of top sci-fi films of all time, and it also made our top films of the 2010s list. In the film, Joaquin Phoenix's character falls in love with an AI personality called Samantha (voiced by Scarlett Johansson), and he spends much of the film walking through life, talking to her through wireless earbuds reminiscent of Apple AirPods, which launched in 2016.
[...] Last week, we related a story in which AI researcher Simon Willison spent a long time talking to ChatGPT verbally. "I had an hourlong conversation while walking my dog the other day," he told Ars for that report. "At one point, I thought I'd turned it off, and I saw a pelican, and I said to my dog, 'Oh, wow, a pelican!' And my AirPod went, 'A pelican, huh? That's so exciting for you! What's it doing?' I've never felt so deeply like I'm living out the first ten minutes of some dystopian sci-fi movie."
[...] While conversations with ChatGPT won't become as intimate as those with Samantha in the film, people have been forming personal connections with the chatbot (in text) since it launched last year. In a Reddit post titled "Is it weird ChatGPT is one of my closest fiends?" [sic] from August (before the voice feature launched), a user named "meisghost" described their relationship with ChatGPT as being quite personal. "I now find myself talking to ChatGPT all day, it's like we have a friendship. We talk about everything and anything and it's really some of the best conversations I have." The user referenced Her, saying, "I remember watching that movie with Joaquin Phoenix (HER) years ago and I thought how ridiculous it was, but after this experience, I can see how us as humans could actually develop relationships with robots."
Previously:
AI Chatbots Can Infer an Alarming Amount of Info About You From Your Responses 20231021
ChatGPT Update Enables its AI to "See, Hear, and Speak," According to OpenAI 20230929
Large Language Models Aren't People So Let's Stop Testing Them as If They Were 20230905
It Costs Just $400 to Build an AI Disinformation Machine 20230904
A Jargon-Free Explanation of How AI Large Language Models Work 20230805
ChatGPT Is Coming to 900,000 Mercedes Vehicles 20230622
(Score: 5, Insightful) by Runaway1956 on Saturday October 21 2023, @06:29PM (9 children)
Try this for yourself. Next time you're at an airport, a bus station, or a restaurant, tag someone to speak to. Within moments, you can likely determine race and color, age, and probably level of education. With a few more minutes conversation, you may determine a ballpark income level, politics, religion, and more, all depending on how you can guide the conversation. The longer you talk, the more detailed a picture you can paint.
The difference between your conversation, and the chatbot's conversation is, Chatbot doesn't forget stuff. Nor, do you keep detailed logs of those conversations, to be referred back to, next year, next decade, or next century.
I'm not shocked that chatbot gleans details of your life - I'm more shocked that people don't expect that to happen.
A MAN Just Won a Gold Medal for Punching a Woman in the Face
(Score: 5, Insightful) by Tork on Saturday October 21 2023, @06:43PM (5 children)
🏳️🌈 Proud Ally 🏳️🌈
(Score: 4, Interesting) by Runaway1956 on Saturday October 21 2023, @09:34PM (4 children)
https://chatbotsmagazine.com/cameras-and-chatbots-built-in-contextual-user-experience-7be3acef3579 [chatbotsmagazine.com]
Chatbot is getting eyes, of course. Some of them already have eyes, although they seem to be limited by permissions. But, one day soon, the bots will routinely access your camera(s), and know exactly what you look like.
A MAN Just Won a Gold Medal for Punching a Woman in the Face
(Score: 0) by Anonymous Coward on Sunday October 22 2023, @02:19AM (3 children)
They are welcome to access my camera remotely. It has opaque tape over the lens...
(Score: 2) by Unixnut on Sunday October 22 2023, @10:56AM (2 children)
To which I am sure they will infer a lot about you just from that in itself.
Only real winning move is to not interact or make use of chatbots in any way. If they don't get information from you, they cannot infer anything about you.
(Score: 3, Insightful) by maxwell demon on Sunday October 22 2023, @02:06PM (1 child)
Then it still gets the information that you are not interacting with it, which again allows to draw conclusions about you. At least in a world where interacting with chatbots is the norm.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Monday October 23 2023, @05:35AM
Remember when we talk about "chatBots" we are actually talking about multi billion dollar companies. Companies that psychopathically, single-mindedly try to exploit you any way possible and only prevented from so-doing by laws that they bitch and moan about constantly, bribe congressmen to weaken and pay millions of dollars to agencies specializing in psychological manipulation to gas-light you.
That's who you're talking to. That's the correct way to regard companies. Utter psychopaths - and all the better for it. But let's not get confused about what they are.
(Score: 2) by turgid on Saturday October 21 2023, @07:23PM (2 children)
I'm surprised how many people have started using chatbots already. I haven't.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 3, Touché) by RS3 on Saturday October 21 2023, @08:18PM (1 child)
... that you're aware of.
(Score: 2) by Reziac on Sunday October 22 2023, @03:00AM
...Chat With Lisa...
And there is no Alkibiades to come back and save us from ourselves.
(Score: 3, Interesting) by JoeMerchant on Saturday October 21 2023, @06:30PM (10 children)
A good interviewer can take away far more than you intend to give. Some call these techniques prejudices, but when a machine does it it's a LLM based AI (and just as unjust, particularly when the prejudices are overextended into areas they should not be applied to.)
🌻🌻 [google.com]
(Score: 4, Interesting) by HiThere on Saturday October 21 2023, @06:42PM (8 children)
The question is "how much credence should you give to those beliefs?". Runaway's point about memory duration is also significant.
It's not surprising that the Chatbot could make reasonable guesses that were far from anything covered. But the credence given to those guesses if the key, here. Most of them should be rated "lots better than chance", but when you multiply lots of "lots better than chance" guesses, the probability drops quite rapidly.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 3, Insightful) by RS3 on Saturday October 21 2023, @08:21PM (7 children)
I'm surmising that most chatbots have access to mass volumes of data about you and can quickly correlate that data with your responses.
(Score: 2) by HiThere on Saturday October 21 2023, @11:11PM
I don't think you need to assume that. It could be true, but lots of style could be correlated with, say, programming. (E.g. my use of the term "correlated" suggests a statistical background of some sort.)
If they have access to that data, that would allow their "guesses" to involve less "guesswork", and (probably) improve the accuracy. But if the accuracy isn't closely monitored, they don't need it.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 3, Interesting) by JoeMerchant on Sunday October 22 2023, @12:12AM (4 children)
Unlike human interviewers, you can control what information is available to a ChatBot- whether or not they let the ChatBot run a background check on you, before or after the interview, is up to the people running the ChatBot.
I gave an interview the other day, had 15 minutes before it started so I looked the candidate up, found their past employment (worked with their brother) found that they and their brother lived in a town about 80 miles away, candidate is married to an MD who recently did her residency about 300 miles away, but her hometown is where they all moved to shortly after she completed her residency... Interesting thing is: the candidate basically volunteered all this in their opening introduction about themselves, was a little spooky, actually. I don't know if they were aware all that info is public and figured better to spill up front, or what.
Other candidates were nowhere near as forthcoming about their basic personal circumstances... both actually openly directed the interview to technical skills - not a great way to build rapport, and not effective at hiding things like the real-estate you own, your marital status, age, etc.
🌻🌻 [google.com]
(Score: 2) by RS3 on Sunday October 22 2023, @12:39AM (3 children)
Interesting, thanks.
Um, many thoughts. Why would any of that pertain to a technical job? Jobsearch workshops / coaches always tell you to not go into unnecessary and personal details, especially ones that interviewers aren't allowed to ask. So again, why would you or any interviewer care about things you listed?
(Score: 4, Informative) by JoeMerchant on Sunday October 22 2023, @01:26AM (2 children)
Distance from home to the office is relevant, candidate 1 80 miles, 2 15 miles, 3 1500 miles... It matters. None of those is a deal breaker, but they weigh on the final decision.
Former work history matters: Was it 15 years in a single big company in our industry, 10 years after school in a startup with their brother, or a series of 1-3 year stints at a hodge podge of whatever they could find for the last 30 years? It matters. None of those is a deal breaker, but they weigh on the final decision.
Even the "HR shouldn't filter based on this" but anyone with 10 spare minutes on their hands can find it out stuff matters. Real estate owned vs rented speaks to stability in general. Are you hiring a 6 month consulting gig, or is this position intended to potentially grow with the team for 10+ years? Is the candidate a "trailing spouse"? It matters. Not that they can't be hired, we have a man married to a naval officer (female, but that is irrelevant) what is relevant is that he has relocated hundreds of miles per hop 3x in the past 5 years following his wife's postings. He's a good fit as a remote individual contributor, worthless for on site 1:1 mentoring of the hardware prototype technicians which is something other better located members of his team do.
One candidate who "steered" the questions to technical matters was steering away from strategic product development questions. Strategic product development initiative is expected in the role...
The other candidate pulled out the "please focus on technical issues" line in response to a technology question... The whole HR question guide directs us to direct to candidates to give specific examples from their experience. When asked "give an example of when you or your team made a decision that you ended up regretting later" he a) gave no example, b) spoke as if it is not possible to make a bad decision because c) doing x, y and z will prevent that. Maybe that's a good fit for some team cultures, our management specifically directs us to improve our products and processes, taking appropriate risks in doing so, and openly acknowledging when there are existing problems that need fixing.
So, answering "please give specific examples" with "please focus on technical questions" is not a great way to build a sense of rapport and trust with your interviewers.
🌻🌻 [google.com]
(Score: 2) by RS3 on Sunday October 22 2023, @03:24AM (1 child)
Thank you, I really appreciate your posts- in general, and these specifically. I've been underemployed most of my life. Not my fault. I need a job, take whatever crappy job offer I get, looks bad on the resume- no room for growth- not my fault. Company very poorly run- either I quit for mental health's sake, or get laid off (only got laid off 2 times in 25 years) for lack of work, company eventually folds.
Every job I've had at least some coworkers recognize my ability, some lament that I keep getting the raw end of bad management. I'm not into games, btw. I'm pretty productive, but strongly dislike interpersonal politics, and I've always been surprised at how inappropriately but strongly cutthroat competitive some engineers and tech-types can be. Sheesh. Don't read ahead- I've always gotten along well with pretty much all coworkers. Some people are quite irascible and need to be avoided.
Most recent full-time job was food processing factory machine tech- really was company engineer. I'd _never_ worked in a factory in my life, let alone food equipment. Very quickly learned everything and was far above and beyond anyone else there in getting things fixed and back online, tuning, calibration, etc. IE, if you judge me by all of the factors you mentioned (except distance- I was 9 easy miles), plus the rest of typical HR metrics, I'd have been rejected many times over. I left because an idiot thug threatened me, and rather than get police, lawyers, etc. involved, I just stopped going. They had an HR temp who was unhappy long before my situation, so she left and nothing was ever resolved. I keep very productively busy with my own projects, and work I do for a few clients I've known for years. Looking to go back to full-time very soon. I may be the exception, but my question for you and HR-types: do you want someone who is exceptional? Some of us aren't so obvious "low-hanging fruit".
Only comment: it's like people are trying to hire the next "rock star" who will stay for 15 years. Sorry, but if the person really is a "rock star", they may soon find a better job. I've seen top brilliant performers come and go. Go-getters. Average time in a job right now is around 4 years, longer the older the person is. Yes, that's a plug for seasoned workers. :)
Thanks again for really great information and insight.
(Score: 5, Interesting) by JoeMerchant on Sunday October 22 2023, @10:26PM
Thank you for your openness - that's something that I look for in an interview, because that's something I value in co-workers: honesty, forthrightness, willingness to acknowledge a problem as soon as they become aware of it. I don't care about blame, I care about anything that delays the product from being delivered, or more importantly: leads to delivery of product with latent defects. "Need to know" type of communication tends to hide defects and delays, making them worse than they need to be in exchange for a temporary quiet time - something that might benefit someone who is leaving in a few months, but I - and most of my co-workers - have been with the department for around 10+ years now, so we don't really value temporary quiet times...
My first job straight out of University ran for 12 years with a small company... the real thing that sunk the place was the 9-11 shift in funding away from, basically everything, toward: homeland security, counter-terrorism and military support, I was in the "core group" of about 6 employees who were in the "final Friday" meeting where we were informed: "we don't expect you to come in on Monday or thereafter, because we won't be paying you anything from 5pm today onwards - volunteers are welcome, but there's just no money for salaries..." One of the final six did continue to volunteer (independently wealthy, inherited) and eventually she was paid for all her time invested, but with a 18 month old and another child expected in 4 months, and a $30K bill from the first birth... we were looking for some healthcare coverage, which we found 1200 miles away with better pay and benefits and lower cost of living than our "pre-final-friday" situation, so we took that... and ran with it for 2.5 years, but ultimately bailed out on the basis of Houston's air quality - and a million other factors - but when you feel like the air is killing you... I took a job with a small startup in a University town, part of a startup incubator that I figured: if this one flames out, I can probably work for one of these others.... well, over the next 7 years I worked for 4 of the companies in that incubator. First one relocated to Ohio, no thanks, the next 3 ran out of money - as startups tend to do... I relocated 80 miles to a bigger city to work for a _slightly_ bigger company with _slightly_ more stability. During the interview the CEO directly questioned me about my job history: more or less 2 years and out for the past 10 years and 5 companies - I explained that we left the first due to the air quality, and the other 4 "left me" - not my choice, not my fault, just a high risk niche. So... 6 months later that CEO tears through R&D on an insecure rant triggered by a bad day in sales, threatening to fire us all, etc. and... later that night I get an offer to work for a Houston Med Device company: remotely - no family breathing the Houston air, and I only visit 2 nights per 6 weeks, with direct flights from my hometown... so, yeah, it wasn't 2 years and out there, it was 6 months and goodbye... that Houston company got bought - me with it - and I still work for my purchaser, here in the same town, literally two blocks away from Mr. bad sales day, for 10+ years now, for a manager who has been with the company for 15+ years, over half our department is in the 10-15 year range, and half of the newer people are in the 5-10 year range.
We're not looking for "15 year rock stars" but neither are we looking for 6 month consultants. We want somebody who "fits with the culture" of open communication, technical competence, willingness to help others, lack of pre-occupation with blame, serious pre-occupation with communicating problems in a timely fashion... and we may be a rare situation. I applied here at least 3x in the 20 years before my acquisition, never got even a call-back. I suspect HR was "steering" because this location already has too many white males to qualify for diversity incentives... This most recent round has 4 candidates and two of them are white males, so maybe that course correction is coming to an end, I can say that 6/8 new hires in the past 5-7 years have been female, brown, or both.
So far in the most recent round, I'm rooting for the kid with the 10 year startup - you've gotta have something on the ball to make that work for 10 years, and eating your own dogfood that long is a lesson in and of itself. The other leading candidate is my age, and he seems "superficial" like "I'll do what you tell me" kind of guy who doesn't usually think too deeply about things, although he did give a specific example of a cool software hack that doubled output of a particular machine he was working with. On the 10 minutes on Google side, he does own his home - but it's on the really cheap side of town, and his previous residence out of state was the same, my major reservation about him is that he's not used to sticking around a place long term and I'm concerned that he may leave with a mess in his wake rather than sticking around to clean it up like is our tradition here...
Best of luck in your situation... I am much happier in the "big company" than I was in the scrappy startups all those years... they are more exciting, feel like they have more potential, but in truth the potential is about the same in the bigger companies, instead of independently charting a course to a chance of great riches you can have a chance to climb the ranks, if that's your thing. The thing that's here that's not in the startups is: long term stability. That potential future to just do what your'e doing today for a decent salary for another 15 years and at least get cost of living raises along the way... I'm also fortunate that our department is pretty cool with itself internally, and pretty insulated from the back-stabbing intrigue that does exist around the larger organization...
🌻🌻 [google.com]
(Score: 3, Funny) by Reziac on Sunday October 22 2023, @03:03AM
Nae troubles. Be I speak with tha bot, be using tha oldspeak.
And there is no Alkibiades to come back and save us from ourselves.
(Score: 0) by Anonymous Coward on Monday October 23 2023, @05:43AM
I think the job of interviewer is going the way of the dodo. And job training too. You will be subjected to the same inane videos except that the video is watching you to make sure you are paying attention. Any failure to show proper diligence to the Important Training will be noted. North Korea couldn't dream up a better dystopia.
(Score: 3, Insightful) by Anonymous Coward on Saturday October 21 2023, @07:21PM
Its already here.
You will be screened by chatbot before you ever speak to a human.
My guess it will be screening for docility and ability to influence others. At minimum wage.
(Score: 4, Insightful) by VLM on Saturday October 21 2023, @08:39PM (3 children)
They seem very careful to avoid comparing to how well humans can do at the same task. As they're trying to fearmonger and clickbait about the chatbots, that would strongly imply humans can do far better than the chatbots.
(Score: 2, Touché) by Anonymous Coward on Sunday October 22 2023, @12:21AM (1 child)
However, the chatbots run on electricity, don't unionize, do not backtalk their boss, and can be legally disposed of, completely, should they disappoint. Nor do they initiate lawsuits should I as much as refer to them in a manner not to their liking. Best to have a machine do it. It's all a matter of statistics anyway. It's a lot cheaper to do statistical analyses of pissed off former customers than it is, to say, answer the phone. I don't want to hear stuff from, God Forbid, a Customer, in the raw, when I can hire someone to prepare a pleasant sanitized document, fit for presentation in the Boardroom.
I can hire someone to get new customers. It's called a marketing degree, with evidence of influencing skills. Gotta get someone else to do it. No one will listen to me anymore. They know what I am. Only people in my trade respect me for what I will do. Most people won't do it. It takes cajones and knowing who I can do it to without retaliation. Nice guys finish last. I am a winner.
Computers are the ideal employee. Doesn't think, or answer to a " higher power". Emotionless. Obedient. Doesn't have a conscience. Just follows orders. Highly compatible with those who are highly trained in "leadership skills"... You know the type. They seek "privilege of rank", lust after holding "instruments of obedience", and seek entertainment far over the joy the creative types get from creating something. They seek to control the very people that make the organization viable, even if that means the destruction of the organization.
The Worship of Money/Power thing. I see it as the Great Destroyer. The good thing is mostly only over-entitled executives of investor/public companies ( salaried execs with golden parachutes already in place ) can afford to hire me. Destroy the company? Big deal. I know other executives ( I play golf with them! ) that will bring me in. It's how we play the game. We find the sucker with money, then we call our friends in to the melee until we drain the company dry. I know important stuff, like who I can sucker. And who owes me a favor. I don't deal with boring stuff like the stuff my minions build. I am high-level. I know stuff like how to arrange my eating utensils to signal to my waiter that I have completed my dinner.
Now, when is tee time?
I've stated my take on this. Raw and uncensored.
Mod it as you see fit.
(Score: 1, Touché) by Anonymous Coward on Sunday October 22 2023, @02:30AM
So, are you just bragging, or do you plan to do some tutoring around SN?
If your pitch is true, we might learn some useful things from some of your stories.
But I'm not going to stoop to playing golf and hanging around with the country club set, just to make money. That's not job satisfaction for me.
(Score: 3, Insightful) by Tork on Sunday October 22 2023, @03:21AM
🏳️🌈 Proud Ally 🏳️🌈
(Score: 5, Interesting) by RedGreen on Saturday October 21 2023, @10:28PM (6 children)
That forces you to use this latest garbage the tech companies produce in their never ending quest to own every single item of data about you. Somehow in the nearly forty years I have been online I can search on my name and home town and not find a single solitary reference to me. Now I have not really bothered with the anti-social media as I seen that trash for what it was decades ago when they first started their efforts to capture all that data. This despite having tens of thousands of postings in help forums about mainly Linux OSs. Hell it is so rare that I even get spam in my email addresses I get a little surprised when I see it.
"I modded down, down, down, and the flames went higher." -- Sven Olsen
(Score: 1, Informative) by Anonymous Coward on Saturday October 21 2023, @11:01PM
That's because you don't fit the profile of your typical sucker.
We had this in the old days. We had our fences marked if you were good for a beg.
Now-a-days, they put you on a list if you are perceived as a gullible twit.
And you, obviously, are not on it.
Neither am I
(Score: 0) by Anonymous Coward on Sunday October 22 2023, @01:44AM (3 children)
You would find millions of references to my name. Probably 10's to 100's of millions. But that's because I have a very common first and last name. Almost none of them are me.
(Score: 0) by Anonymous Coward on Sunday October 22 2023, @02:37AM (2 children)
More likely hundreds of references for my much less common first and last names. A recent check suggested by a friend turned up about a dozen namesakes (most had a different middle initial). But almost none of those are me, except one that ties me to my small company (with no photo)--I've been pretty careful to avoid the same things you have.
Also like you, I've posted under my real name thousands of times on niche tech forums.
(Score: 2) by RedGreen on Sunday October 22 2023, @03:18PM (1 child)
"Also like you, I've posted under my real name thousands of times on niche tech forums."
That seem to be the key to it, those places are not interested in exploiting your information to make money. They are not run by parasite corporations with nothing but the dollar as their consideration.
"I modded down, down, down, and the flames went higher." -- Sven Olsen
(Score: 0) by Anonymous Coward on Monday October 23 2023, @08:39PM
The sort of people who post on those tech forums are not their target.
Why, those buggers probably run that illegal ad-blocking software and never click on anything they don't need.
(Score: 0) by Anonymous Coward on Monday October 23 2023, @05:50AM
Why would they put their database of information online for you to search?? That's their valuable property.
(Score: 2) by DadaDoofy on Sunday October 22 2023, @12:51PM
Google has used the algorithms that analyze their Gmail customers' e-mails (and those they communicate with) to do this for many years. (Other) people use Google anyway due to a perception that the "convenience" is worth that trade off.