Generative AI in 2024: Growing Pains and Fatal Flaws

For many years, AI research muddled through cycles of promise, hype, and unmet expectations. Even after the advent of artificial neural networks and machine learning, which showed that computers could be “taught” to more-or-less reliably find patterns in unstructured data, AI applications had narrow areas of expertise and niche use cases. There were numerous successful examples in areas such as image identification and handwriting interpretation, but there was no “killer app” to capture the public’s attention.

And then, along came ChatGPT.

ChatGPT was, and remains today, the most prominent example of a new class of AI algorithms— the large language model (LLM), generally known as “generative AI.” LLMs are trained on immense quantities of text data and can respond to text or spoken prompts with generated text, images, or video. They take natural-language processing to the next level, and the results make it easy to believe there is some spark of human-like intelligence going on . . .

or that your interaction is with an actual intelligent entity and not a computer at all.

How big a deal is generative AI? Not long ago, company leadership at most businesses would be challenged to say what “AI” stood for. Generative AI now has C-level executives talking about how AI can be used to reduce costs, increase efficiency, increase revenues, and gain competitive advantage.

But generative AI is not without issues—issues that leaders need to be aware of. Some of the issues are inherent in the design and training of the models, and some are related to how generative AI can be misused. At best, these can be described as “growing pains” that will subside as the technology matures. At worst, they are fatal flaws that will keep the technology out of the enterprise at all.

In this article, we discuss some of these issues and some of the ways they can be mitigated.

Generative AI’s Issues

The main issues with generative AI fall into four categories:

• Hallucination
• Plagiarism
• Bias
• Deepfakes

Let’s look at each of these in detail.

Hallucination

As mentioned earlier, LLM applications such as ChatGPT are trained on vast quantities of text, often scraped from sources on the public internet. For the most part, if you ask ChatGPT a question, it will come up with the right answer. But sometimes it will generate a response that is altogether wrong and will even “argue” with you about it, often to the point where it implies that you are the crazy one, not it.

Plagiarism

For both text and visual generative AI, certain prompts can elicit responses that appear to violate someone’s copyright or trademark. Sometimes it’s an image of a well-known video game, movie character, or text that is an almost word-for-word reproduction of a published newspaper article.

An ongoing controversy now rages over whether generative AI models should be allowed to be trained on others’ copyrighted works or whether such training is considered “fair use.” Some legal actions regarding this issue are already in the courts.

Bias

One of the very early AI applications (not generative AI) was a model that would help medical-school admissions officers make decisions regarding applicants. The idea was that the computer could make these decisions without the racial or cultural biases that humans bring to the decision-making process. A close examination of the algorithm’s performance showed that it was just as biased as the humans. Why? Because it had been trained on past decisions made by humans. Garbage in, garbage out.

LLMs are also susceptible to bias. To the extent that we start to rely on generative AI applications for decision making, we need to be careful that we are not teaching them the wrong ideas.

Deepfakes

Recently, Elon Musk was seen on a YouTube livestream, exhorting viewers to transfer their cryptocurrency into a certain account with the promise that they would get double their coins back within minutes.

Except it wasn’t Elon Musk. It was a sophisticated, realistic rendition of Elon Musk’s likeness and voice—known as a deepfake—and it was a scam intended to separate viewers from their crypto. It’s not known how many viewers fell for the ruse.

Deepfaked photos and videos of high-profile personalities, celebrities, and political figures proliferate at an alarming rate and will get only worse as the tools to create them mature.

What Can Be Done?

Can anything be done to save generative AI from itself, to allay concerns about these issues before they paint generative AI in particular—and AI in general—as too risky for the enterprise?

These are thorny problems, to be sure. Deepfakes, in particular, are troubling because they are intentional. Miscreants with strong motivations to do harm are staying ahead of any attempts to thwart them.

But smart people around the world have begun to tackle the problem. Here are some solutions that have been proposed or are in active development:

• Better-curated training data: It’s easy to scrape text and images from the internet and use it to train a generative AI model, but in too many cases developers are not examining the sources of the data to determine the extent to which it contains incorrect information, biases, or copyrighted materials. To counter bias, at least one group is developing an LLM that intentionally includes data from and about racial minorities.
• Watermarks: Protected images can be generated with “watermarks”—subtle alterations to certain pixels that can be read by software and indicate their copyright status and ownership. In theory, LLMs could filter out images not released to the public domain.
• Retrieval-augmented generation (RAG): The RAG technique, introduced by researchers in 2020, can detect bias and reduce hallucinations because they corroborate multiple sources of information.
• Deepfake-detection algorithms: Several AI algorithms designed to detect deepfakes have been proposed, and some of them are quite effective. The trick will be to keep them up to date as deepfakes become more subtle and sophisticated.

How TensorWave Can Help

All of the issues discussed in this article have been with us since before generative AI was a thing, and in some cases, even before computers were in wide use. Generative AI just made it easier to propagate them.

Only so much can be done with the solutions proposed so far, and the landscape now evolves so fast that something that seems a sure-fire solution might be rendered obsolete before it can be implemented.

However, they are important issues, and we as a society should address them. Some are more problematic than others, so we should prioritize the biggest ones first.

About TensorWave

TensorWave is a cutting-edge cloud platform designed specifically for AI workloads. Offering AMD MI300X accelerators and a best-in-class inference engine, TensorWave is a top-choice for training, fine-tuning, and inference. Visit tensorwave.com tensorwave.com to learn more.