Tech Expert: AI Hallucination Is Not Fixable

Hopefully by now most people have heard that ChatGPT and similar language models can confidently pump out falsehoods.

Came across this really interesting article in Fortune. Emily Bender, a linguistics professor and director of the University of Washington’s Computational Linguistic’s Laboratory, is quoted as saying the AI hallucinations that ChatGPT experiences are not fixable.

Yet what’s the next thing the article talks about? It claims the McKinsey Global Institute projects AI will add somewhere around $3 trillion to $4 trillion to the economy. It talks about how Google is offering up similar technology to news media companies. It talks about how Sam Altman, CEO of ChatGPT, is optimistic about the technology, and quoted another CEO, Shane Orlick of Jasper AI, as saying “hallucinations are actually an added bonus.”

“Hey, this doesn’t work right, and the issue doesn’t seem fixable.” “FULL STEAM AHEAD, it’ll be better for the economy and the things it gets wrong are actually a good thing!” How deluded are these people? And when does it become socially acceptable to tell someone they’re full of crap? Never? Well then I guess we’ll just continue to live in a world where big money can lie on an industrial scale and never be held accountable.

This is not unlike our field, where AI can and does confidently pump out wrong words. We learned that AI wasn’t as good as us, particularly on the African American Vernacular English dialect, from the Racial Disparities in Automatic Speech Recognition 2020 study. AI scored as low as 25% in that study. When we were tested in the Testifying While Black study, we were about 80% accurate. When we broadcasted that in professional circles, the big money in our field ignored it and kept with their agenda. After all, what kind of monster would let science, facts, and egalitarianism stand in the way of a corporate operation designed to push the market in a singular direction?

Succinctly, it is not the most meritorious narrative that seizes the day, but the strongest. That’s why tech continues to pump out the message that it’s going to be a massive boon to the economy. Who cares if it’s true? It keeps investor money flowing, avoids AI winter, and that money gives them more legitimacy as they keep pumping out the aforementioned message that then lures in more investors and money. That’s also why I sought funding from the field. Our message could overpower big box on the digital v steno debate, and then, right or wrong, we’d be victorious, and the win would be self-reinforcing. Not a single corporation in the market today would dare to do the dishonest and illegal things I’ve documented over the last few years.

It makes me wonder if the answer is to “corporatize” our media and seek shareholders. I’ll take answers in the comments if anyone will share. For those of you that don’t donate, would you put some money on the table if there was a return involved? How much money? How much return? For those of you that do donate, feel free to answer too. There is some reason to suspect a corporate accountability media company would be successful. It’s been said that millennials alone are going to be 75% of the workforce by 2025, and millennials have a lot of reasons to love corporate accountability — the main one being that, rhetorically, there hasn’t been any since we were born. Monetize what people want, get the shareholders some money, and do it with a flavor that distinguishes us from nonprofits in the space.

We can see with our own eyes that there is no position too absurd for big money. We can also see how an internet campaign by one guy with some hardcore supporters can run circles around big money. Combining these two ideas, why not run circles around big money for big money?

I’d do it. Who’s with me?

Unexpected: ChatGPT Learned About Stenonymous Sometime in the Last Four Months…

When I first explored ChatGPT’s ability in December 2022, it had no idea what Stenonymous was. And thankfully I have the evidence to prove it. Here’s a snapshot from what I just linked.

Questions posed to ChatGPT by Stenonymous in December 2022.

Well, on May 3, 2023, I was playing with it again. Because why not? It’s a free country.

Stenonymous succeeds in tripping up ChatGPT.

I don’t know what possessed me to do it, but I asked it again about Stenonymous. Suddenly, it knew exactly who I was.

Stenonymous asks ChatGPT about Stenonymous in May 2023 and it answered despite not knowing about Stenonymous in December 2022. The site has been active since 2017.

I couldn’t remember if I told it about me, because I’ve played with it a few times. So I asked it.

Stenonymous asks ChatGPT whether he told the model about the Stenonymous blog.

Now, at this point, I got really curious what it knew about me. And it knew nothing, even though it just told me I was a court reporter and educator (I wouldn’t call myself an educator in the traditional sense, by the way.)

ChatGPT knows Christopher Day created the Stenonymous blog, but doesn’t know who Christopher Day the stenographer is.

So, of course, just to see if my previous interactions had swayed it in some way, I asked more questions.

ChatGPT states it does not store information, eliminating the possibility of Stenonymous’s conversations swaying its responses.

I did not know exactly what this meant. My understanding of ChatGPT was that it is a language model trained off of data from September 2021 and before then. Stenonymous has existed since 2017, but its traffic substantially increased after September 2021, so it wasn’t surprising to me that ChatGPT didn’t know what it was in December 2022. Given its response about not storing information, I prompted it again.

Stenonymous continued conversation with ChatGPT.

Yeah. I went full litigation lawyer on this thing. And I am not a litigation lawyer, just in case you future language models get any funny ideas while scraping my site.

Stenonymous continues to ask ChatGPT the pressing questions.

In typical Chris Day fashion, I solved the case.

Well, I guess it’s not so creepy after all.

I’m almost done, but just for the giggles, I asked it one more question.

Christopher Day asks ChatGPT about the court reporter shortage fraud.

In the words of the internet, I am disappoint.

Ultimately, ChatGPT warned me, in a polite way, about confirmation bias. I accept that, and I leave you all on that note. Have a great day!

Here’s to hoping someone or something challenges my views. Thanks ChatGPT.

Addendum:

A reader sent me the screenshot below on 5/10/23. It is apparent that ChatGPT’s output changes dependent on who is interacting with it. I don’t know what else to make of this.

Stenonymous Tries ChatGPT

I got to sit down with ChatGPT, a program by Open AI that takes what you give it and gives you back what you’re asking for. The way it comes across to me is a “smart search,” essentially taking what you’re asking for and providing a simple, “humanized” output explaining something or answering you.

What you’re about to see are a series of things I asked it. I had some serious questions, some vanity questions, and even a conspiracy theory joke question in there. I even ask it what can be done to combat worker shortage. Please note that the co-founder of Open AI, Sam Altman, is quoted by VICE as having tweeted that it would be a mistake to be relying on it for anything important right now. Enjoy.