AI written books

[Quoth]

Originally Posted by gmw

The same could be said of raising baby humans.

No.


Also comparing regular software, FOSS or not to AI trustworthiness is an insult to programmers.
@gmw: you missed the point of Schneier's article.

[gmw]

Originally Posted by Quoth

[...]
Also comparing regular software, FOSS or not to AI trustworthiness is an insult to programmers. [...]

You were questioning supply-chain trust issues (Google vs outsourced), which is what I was responding to. The trust issue with that has little to do with the programmers involved, open or closed, conventional or AI.

Still, it is worth looking at our programmers. Let me get a mirror... nope, I wouldn't trust him either.


Being well intentioned does not make a programmer an expert in security. Being open-source makes code available for review but does not guarantee that any review takes place. Being closed source doesn't ensure that resources will be allocated to review code. There are many examples of bugs showing up after decades. I've found decades-old bugs in my own code! Such examples prove that review has not been comprehensive, and my experience suggests it has been very selective indeed. Given that, do you really think that being open source is a guarantee against supply-chain attacks such as the Solar Winds debacle? The layers of vulnerability in any major conventional software project, open or closed source, boggle the mind!

And that's all with largely deterministic conventional code. Now take a look at AI and despair. Well, I would. But no, there are many intrepid people out there carefully studying AI systems and trying to understand how to understand them. People like this are probing the trustworthiness of both conventional and AI systems. It's not an insult, it's just accepting the reality.

[Quoth]

I'm just a messenger.
The problem is that Chatbots are inherently unreliable. Citations and references can simply be a lie.

Nonetheless, he said academics could still look for clues that a student had used ChatGPT. Perhaps the biggest of these is that it does not properly understand academic referencing – a vital part of written university work – and often uses “suspect” references, or makes them up completely.

https://www.theguardian.com/technolo...ten-by-chatbot

[gmw]

Originally Posted by Quoth

I'm just a messenger.
The problem is that Chatbots are inherently unreliable. Citations and references can simply be a lie.

https://www.theguardian.com/technolo...ten-by-chatbot

It's an odd sort of essay. In parts trying to say how good the Chatbots are, and how difficult it is getting to detect them, but then offering various tells: “The use of English and quality of grammar is often better than from a student.”, and the one you quoted about references.

So which is it? Is using an unreliable chatbot the reliable way to get thrown out of Uni, or have chatbots become the new way of teaching: you can get the answer from a chatbot but then you have to learn your subject so you know whether the chatbot is right; your main bonus being that its grammar is probably better than yours.


It would be interesting to know if the reference issues were deliberate. It seems like a reasonable way to offer some sort of protection: force anyone wanting to produce academic quality results to at least validate and correct the references. For general use, the occasional redundant reference is not going to be a huge issue.

[DuckieTigger]

Originally Posted by gmw

It would be interesting to know if the reference issues were deliberate. It seems like a reasonable way to offer some sort of protection: force anyone wanting to produce academic quality results to at least validate and correct the references. For general use, the occasional redundant reference is not going to be a huge issue.

They are intentional. The training data is never disclosed, and if it did, we would scream bloody murder for all the bias they put in. All hiding under the "truth check" umbrella.

The only true test would be to train the AI by giving it full access to the world wide web, or a subset with a short list of domains. Then see what it learned, whatever it spits out as answer should be able to be verified by manually searching for the answer with search engine instead of a LLM.

[Quoth]

But for more complex questions, that capability is less useful than it seems. My friend Dan just published his first book, and ChatGPT can’t tell me anything about it, but Bard will happily summarise the reviews (“mixed … praised for its timely and important message”) and give specific quotes from the New York Times (“a passionate and well-researched argument for why cars are making our lives worse”). Unfortunately, it made it all up: the quotes are fake, and the reviews don’t exist. But even a canny user could be caught out, since Bard can search for real reviews and quote them accurately – it just doesn’t want to.

"want" isn't the correct term.
https://www.theguardian.com/technolo...medy-of-errors

Are these really just "playthings"? Just like Eliza, Ractor, Parry, Alice etc, but more complex and environmentally damaging due to computing resources for the LLM.

[Apache]

https://www.insidehook.com/daily_bri...paign=campaign
The article talks about over 200 books listed on Amazon that credit ChatGPT as an author.
Apache

[j.p.s]

Isn't the tendency to make stuff up rather than admit lack of knowlege one aspect of AI that closely matches human behavior?

[jhowell]

The potential writers strike this spring could be AI’s big break!

[rcentros]

Originally Posted by j.p.s

Isn't the tendency to make stuff up rather than admit lack of knowlege one aspect of AI that closely matches human behavior?

Only if you're talking about human liars or propagandists.

[j.p.s]

Originally Posted by rcentros

Only if you're talking about human liars or propagandists.

I think that is kind of harsh. (I am not condoning lieing, propagandizing, or even fear of saying "I don't know.", but I can understand why someone might be experiencing that fear.)

[gmw]

Originally Posted by j.p.s

Isn't the tendency to make stuff up rather than admit lack of knowlege one aspect of AI that closely matches human behavior?

Yup. Although in the case of ChatGPT it might be argued that making stuff up is part of its programming, sort of like telling a kid sitting a test: just do the best you can.

[Quoth]

Originally Posted by j.p.s

Isn't the tendency to make stuff up rather than admit lack of knowlege one aspect of AI that closely matches human behavior?

Only a minority of humans. And why would people spend Millions creating a computer system that mimics the worst of human behaviour? What use is it?

[gmw]

Originally Posted by Quoth

Only a minority of humans. And why would people spend Millions creating a computer system that mimics the worst of human behaviour? What use is it?

1) They ain't finished yet. This is not the end product, and was never intended to be. A lot has happened in this field in just the last few years, and there is more to come. For now they are enlisting the general public to do their testing for them, to iron out some of the kinks and to help gain acceptance. Plus it may whet some appetites ready for what comes next.

2) Seems like lots of people are finding uses even for this (producing art, answering essay questions, writing forum posts , and so on), and that worries a lot of people. Which harks back to the previous point about acceptance. The current imperfect model makes a good time to learn how people will respond to such a thing as an AI: the abuses, the fears, the niches that even the builders haven't seen coming.

[DiapDealer]

Originally Posted by Quoth

And why would people spend Millions creating a computer system that mimics the worst of human behaviour? What use is it?

Predicting what the worst humans might do before they do it?