There is some fear about using AI for solving problems which is not necessarily an issue with LLMs, but with the companies that run them. The underlying issue seems to be the data mining that is the source of profit for many companies, Google, Microsoft, Meta. Opting out of Instagram’s use of the content you post to enhance their AI model requires providing an example of where it was used, virtually impossible for the average user.
If you install Windows 11 and click “agree” to all the defaults, you have given Microsoft the right to capture everything you type into your keyboard. Verifying stories on the internet is also impossible, but one second hand story I recall talked about an employee using ChatGPT to help write a company’s annual financial report, and then discovering all the information was now discover-able to anyone. Unfortunately, I can’t provide a source for this story, but I’m mentioning it as a possible point to consider when using ChatGPT for sensitive tasks.
This isn’t a platform for conspiracy theories and I’m not wishing to start down that path, but the WWW has become less reliable on many levels for accurate information and using many platforms gives up personal information in a way little understood by end-users. This to me is the real source of the suspicion of using AI because it has become integrated into social media and operating systems to a degree not fully apparent.
The true value of GNU/Linux and open source software is the ability to disengage from this.
Many (if not all) LLMs just want to keep on talking to you, no matter what. I admit that also I use more and more of these tools (they are really great to summarize and structurize long texts), but as long as you don’t have an initial prompt like “stick to the facts; if you have no or insufficient information on a topic, say so; don’t make things up; give sources for everything; etc.” you can always easily step into these pitfalls.
The lesson the kid absorbed was clear: write less creatively, use simpler vocabulary, and don’t sound too good, because sounding good is now suspicious.
In another discussion someone made the distibction between “describe” and summarize and I think that’s an important one. AI doesn’t actually summarize texts, but rather generates a statistically-reasonable description. Sometimes that can be useful, but it isn’t really a summary because it’s not backstopped by comprehension and synthesis.
Because it’s not a summary it can miss the point or end up with a lot of words that say little.
What about an AI-help category in the forum? Surely somebody can/will set up a zeroclaw or whatever that is just there to answer questions. Yes users could just as easily use claude.ai or whatever directly but this way it would add to a library of knowledge that is available to all. Interesting things might come from it while 95% is the same questions over and over.
The worst thing about AI is people using it to gaslight you in arguments. Instead of admitting they’re wrong, they just prompt-engineer a confirmation bias. We’re basically teaching LLMs to be as stubborn as we are. If we keep feeding them ego, we’re just building a giant hallucination machine.
