Driving Enterprise Transformation With
Generative AI

In the fast-evolving landscape of artificial intelligence (AI), one term is increasingly making waves among enterprises worldwide: Generative AI. This cutting-edge technology holds immense potential to revolutionize how businesses operate, innovate, and stay ahead of the competition.

However, scaling Generative AI with the right level of governance is not a given. Let’s delve into the realm of enterprise GenAI, exploring its significance, the transformative impact it promises, and the indispensable tools to harness its power effectively.

Generative AI is fundamentally changing the way we interact with technology, blurring the lines between human creativity and machine intelligence. Unlike conventional AI models that are trained to perform specific tasks or make predictions based on existing data, Generative AI has the remarkable ability to create new content that is often indistinguishable from human-created work.

In order to understand how Generative AI will reshape the enterprise, it’s important to first understand how it works and how to build a Generative AI use case. Let’s start by defining some of the most important technologies and concepts related to Generative AI.

Large Language Models (LLMs)

OpenAI’s ChatGPT, Microsoft’s Bing, and Google’s Gemini are specific products built on top of a class of technologies called Large Language Models (LLMs).

LLMs work by reviewing enormous volumes of text, identifying the ways that words relate to one another, and building a model that allows them to reproduce similar text. Importantly, when asked a question, they are not “looking up” a response. Rather, they are producing a string of words by predicting which word would best follow the previous, taking into account the broader context of the words before it. In essence, they are providing a “common sense” response to a question.

While the most powerful LLMs have shown their ability to produce largely accurate responses on an astonishingly wide range of tasks, the factual accuracy of those responses cannot be guaranteed.

That’s because the responses from LLMs, again, are not looked up from memory. They are generated on the fly based on 175 billion weights (see the Neural Network section). They can provide no guarantee that its output is right, only that it sounds right.

Neural Networks

A neural network is made up of a large number of “neurons,” which are simple mathematical formulas that pass the results of their calculations to one or more neurons in the system. The connections between these neurons are given “weights” that define the strength of the signal between the neurons. These weights are also sometimes called parameters.

A neural network could be very small. For example, a basic one could have six neurons with a total of eight connections between them. However, a neural network could also be very large, as is the case of the LLMs. These may have millions of neurons with many hundreds of billions of connections between them, with each connection having its own weight.

Transformer Architecture

We already know that an LLM is a type of neural network. More specifically, LLMs use a particular neural network architecture called a transformer, which is designed to process and generate data in sequence, like text.

Developed in 2017 by researchers at Google, a transformer is based on the idea of “attention,” whereby certain neurons are more strongly connected (or “pay more attention to”) other neurons in a sequence.

Since text is read in and read out in a sequence, one word after the other, with different parts of a sentence referring to or modifying others (such as an adjective that modifies the noun but not the verb) it is also no coincidence that an architecture that is built to work in sequence, with different strengths of connection between different parts of that sequence, should work well on text-based data.

Hallucination

Hallucination refers to the generation of content or patterns that appear realistic but are entirely generated by the AI model and do not correspond to any real-world data. While hallucination in Generative AI can lead to creative and imaginative outputs, in an enterprise context, it is troublesome.

Indeed, it poses significant risks and challenges, ranging from misinformation and reputational damage to compliance issues and security vulnerabilities. The potential for hallucination with Generative AI underscores the importance of robust governance frameworks, ethical guidelines, and rigorous validation processes to mitigate the potential impact of AI content generation on business operations and outcomes.

Fine-Tuning

In the context of LLMs, fine-tuning refers to the process of adjusting and optimizing the parameters of the pre-trained model to better suit a specific task or domain. Fine-tuning an LLM involves updating its parameters based on a smaller, task-specific dataset or a specialized corpus relevant to the target domain.

This process allows the model to adapt its understanding and text generation to better align with the nuances and requirements of the specific task or domain. By fine-tuning the LLM, researchers and practitioners can tailor its capabilities to excel in tasks such as text summarization, sentiment analysis, language translation, or any other natural language processing (NLP) task, resulting in improved performance and more accurate predictions for the target application. This also includes tasks such as Generative AI summarization and Generative AI NLP — read this blog for a rundown of these and other Generative AI technical methods.

Retrieval Augmented Generation

Retrieval Augmented Generation (RAG) is a smart way for a language model to generate better responses. It does this by first looking up relevant information from a big database based on the question or context you give it.

Then, it uses this information along with its own knowledge to more accurately answer questions. This helps RAG give responses that make more sense and are more useful compared to just relying on its own knowledge.

The integration of Generative AI into enterprise workflows unlocks opportunities across various domains, including product design, content creation, decision making processes, and much more. By harnessing the power of AI to generate images, novel ideas, designs, and solutions, businesses can streamline operations, drive innovation, and deliver unparalleled value to customers.

In fact, a Forrester survey of 220 AI decision makers at large companies in North America revealed that 83% are already exploring or experimenting with Generative AI.

But what applications and use cases can benefit from Generative AI? What are the top Generative AI solutions to consider? There are a few critical things to keep in mind.

Tradeoffs Between LLMs and Other Models

Using LLMs can be computationally intensive. In the case of a model with 175 billion weights, for each “token” (that’s a word or a piece of a word) that it outputs, it needs to run 175 billion calculations… each and every time. Why make such a large model then? Especially when a smaller language model is very good at the task for which it is designed?

The largest models are like a smartphone: They are convenient in that they pack a lot of functionality into a single product. This means that you can use the same model for a variety of tasks. It can translate, it can summarize, it can generate text based on a few inputs. Like a smartphone, it’s the one solution that you need to handle a vast array of tasks, though this flexibility comes at a price.

If all you need is a stopwatch, you can find a far less expensive option than a smartphone. Similarly, if you need a solution to a specific task, you may be better off selecting a small, task-specific model instead.

When Do LLMs and Generative AI Make Sense?

If you’re interested in testing the usefulness of an LLM inside your organization (like Heraeus did using LLMs for their lead process), seek an application that balances the following:

• Risk tolerance: If this is the first time you’re using Generative AI technology, choose a domain where there is a certain tolerance for risk. The application should not be one that is critical to the organization’s operations and should, instead, seek to provide a convenience or efficiency gain to its teams.
• Human review: A law firm has been quoted as saying that they are comfortable using this technology to create a first draft of a contract, in the same way they would be comfortable delegating such a task to a junior associate. This is because any such document will go through many rounds of review thereafter, thus a Generative AI assistant might minimize the risk that any error could slip through. For example, Whataburger uses LLMs to hear what customers are saying, with humans focusing on reviews to improve the customer experience.
• Text (or code) intensive: It’s important to lean on the strengths of these models and set them to work on text-intensive or code-intensive tasks — in particular those that are “unbounded,” such as generating sentences or paragraphs of text. This is in contrast to “bounded” tasks, such as sentiment analysis, where existing, purpose-built tools will provide excellent results at lower cost and complexity.
• Business value: As always, and perhaps especially when there is a lot of excitement around a new technology, it is important to come back to basics and ask whether the application is actually valuable to the business and how the Generative AI value chain fits in. LLMs can do many things, whether those things are valuable or not is a separate question.

Going from POC to LLM in production in the enterprise, beyond the simple web interface provided by products like ChatGPT, can be done in one of two ways:

1. Making an API call to a model provided as a service, such as the GPT-3 models provided by OpenAI, including the gpt-3.5-turbo or gpt-4-0314 models that powers ChatGPT. Generally, these services are provided by specialized companies like OpenAI, or large cloud computing companies like Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure.
2. Downloading and running an open-source model in an environment that you manage. Platforms like Hugging Face aggregate a wide range of such models.

Each approach has advantages and drawbacks, which we’ll explore in the following sections. And remember, both options allow you to choose from smaller and larger models, with tradeoffs in terms of the breadth of their potential applications, the sophistication of the language generated, and the cost and complexity of using the model.

Advantages of Using a Model-as-a-Service via API

Some companies (such as OpenAI, AWS, and GCP) provide public APIs that can be called programmatically. This means setting up a small software program, or a script, that will connect to the API and send a properly formatted request.

The API will submit the request to the model, which will then provide the response back to the API, which will in turn send it back to the original requester.

There are several advantages to this approach:

1. Low barrier to entry: Calling an API is a simple task that can be done by a junior developer in a matter of minutes.
2. More sophisticated models: The models behind the API are often the largest and most sophisticated versions available. This means that they can provide more sophisticated and accurate responses on a wider range of topics than smaller, simpler models.
3. Fast responses: Generally, these models can provide relatively quick responses (on the order of seconds) allowing for real-time use.

Limitations to Using a Model-as-a-Service via API

The use of public models via API, while convenient and powerful, has limitations that may make it inappropriate for certain enterprise applications:

1. Data residency and privacy: By nature, public APIs require that the content of the query be sent to the servers of the API service. In some cases, the content of the query may be retained and used for further development of the model. Enterprises should be careful to check if this architecture respects their data residency and privacy obligations for a given use case.
2. Potentially higher cost: Most of the public APIs are paid services, whereby the user is charged based on the number of queries and the quantity of text submitted. The companies providing these services usually provide tools to estimate the cost of their use. They also often provided smaller and cheaper models which may be appropriate for a narrower task.
3. Dependency: The provider of an API can choose to stop the service at any time, though such a decision is typically rare for a popular (and money-making) service. Smaller companies providing such services may be at risk of insolvency. Enterprises should weigh the risk of building a dependency on such a service and ensure that they are comfortable with it.

Advantages of Self Managing an Open-Source Model

Given the drawbacks of using a public model via API, it may be appropriate for a company to set up and run an open-source model themselves. Such a model can be run on on-premises servers that an enterprise owns or in a cloud environment that the enterprise manages.

Advantages of this approach include:

1. Wide range of choice: There are many open-source models available, each of which presents its own strengths and weaknesses. Companies can choose the model that best suits their needs. That said, doing so requires some familiarity with the technology and how to interpret those tradeoffs.
2. Potentially lower cost: In some cases, running a smaller model that is more limited in its application may allow for the desired performance for a specific use case at much lower cost than using a very large model provided as a service.
3. Independence: By running and maintaining open-source models themselves, organizations are not dependent on a third-party API service.

Tradeoffs to Self Managing an Open-Source Model

While there are many advantages to using an open-source model, it may not be the appropriate choice for every organization or every use case for the following reasons:

1. Complexity: Setting up and maintaining a LLM requires a high degree of data science and engineering expertise — beyond that required for simpler machine learning (ML) models. Organizations should evaluate if they have sufficient expertise, and if those experts have the necessary time to set up and maintain the model in the long run.
2. Narrower performance: The very large models provided via public APIs are astonishing in the breadth of topics that they can cover. Models provided by the open-source community are generally smaller and more focused in their application, though this may change as ever-larger models are built by the open source community.

Using LLMs responsibly requires similar steps and risk management considerations as the use of other ML and AI technologies:

• Understand how the model is built. All ML and neural networks, including those used in LLMs, are influenced by the data that they are trained on. This can introduce biases that need to be corrected for.
• Understand how the model will impact the end user. LLMs in particular present the risk that an end user may believe that they are interacting with a human when, in fact, they are not. Dataiku recommends that organizations disclose to end users where and how these technologies are being used. Organizations should also provide guidance to end users on how to interact with any information derived from the model and provide caveats for the overall quality and factual accuracy of the information.
• Establish a process for approving which applications are appropriate uses for each technology. The decision of whether or not to use a particular model for a given application should not be made by individual data scientists. Instead, organizations should set up principles for which technologies should — and should not — be used in what contexts, with a consistent review process by leaders who are accountable for the outcomes.
• Keep track of which technology is being used for which application. AI Governance principles are meant to both prevent problems from arising and to ensure that there is an auditable history of which technology has been used in the event that a problem occurs and needs to be reviewed. The best enterprise Generative AI tools, including AI platforms, like Dataiku, that offer strong governance features, can help serve as that central control tower.

As enterprises embark on their journey towards harnessing the power of Generative AI, one platform stands out as a comprehensive solution: Dataiku. Renowned for its advanced features, scalability, and ease of use, Dataiku empowers organizations to unleash the full potential of AI across diverse use cases and industries.

Dataiku offers a holistic approach to Enterprise AI, combining data preparation, modeling, deployment, and management within a single, unified Generative AI platform. With its intuitive interface and extensive library of pre-built algorithms, Dataiku enables smooth user experiences and helps rapidly prototype and deploy Generative AI models, driving innovation and efficiency across the enterprise.