Deploying an on-premise Instance of ChatGPT, does it worth it?

AI Usage in Industry will gain more and more in popularity with Generative AI

There are a number of potential use cases for language models like ChatGPT in closed industrial environments. Here are a few examples:

1. Quality control: AI language models can be trained to identify patterns and anomalies in manufacturing data, helping to identify quality control issues before they become major problems. For example, an AI language model could analyze sensor data from a production line and flag any instances where a particular product is outside of expected tolerances.
2. Predictive maintenance: By analyzing historical sensor data and other inputs, AI language models can help predict when machinery is likely to fail or require maintenance, enabling proactive maintenance and reducing downtime. This can be especially useful in environments where equipment failure can be costly or dangerous.
3. Natural language processing: Language models like ChatGPT can be used to analyze and understand natural language text data, such as maintenance reports or customer feedback. This can help identify trends and insights that might be difficult to detect manually, and can help inform decision-making.
4. Knowledge management: In large industrial organizations, it can be challenging to manage and organize knowledge effectively. AI language models can be used to automatically classify and tag documents, making it easier to search and retrieve information. They can also be used to create virtual assistants or chatbots that can help employees find the information they need more quickly and easily.
5. Speech recognition: In noisy or challenging environments, such as manufacturing floors or construction sites, it can be difficult to communicate effectively. AI language models can be used to improve speech recognition, enabling more accurate voice commands and reducing the need for manual input.

Running a standalone instance of ChatGPT:

The minimum requirements for running ChatGPT or any large language model in a closed environment would depend on a number of factors, including the size of the model, the amount of data being processed, and the specific use case. However, in general, you would need:

1. Hardware: Running a large language model like ChatGPT requires significant computational resources, so you would need a powerful server or cluster of servers to support it. This might include GPUs or other specialized hardware to accelerate training and inference.
2. Software: You would need to install the appropriate software for training and running language models, such as TensorFlow or PyTorch. You would also need to install the specific model you want to use, such as the GPT-3.5 model used by ChatGPT.
3. Data: In order to train a language model, you would need a large amount of text data to use as input. The specific amount of data required would depend on the size of the model and the complexity of the task.
4. Access controls: Because language models can be sensitive and may contain proprietary or confidential information, it’s important to implement appropriate access controls to ensure that only authorized users can access and use the system.
5. Maintenance and monitoring: Once the system is up and running, you would need to implement ongoing maintenance and monitoring to ensure that it continues to function properly and address any issues that arise. This might include monitoring system performance, addressing security vulnerabilities, and updating software as needed.

Hardware recommendations?

For a first testing setup, it’s possible to run ChatGPT on a single high-end server or workstation with a GPU. Here are some general hardware recommendations to get started:

1. CPU: A modern, high-end CPU is recommended, such as an Intel Core i7 or i9 or an AMD Ryzen processor.
2. GPU: A GPU is essential for running ChatGPT efficiently. A powerful GPU like the NVIDIA GeForce RTX 2080 or higher is recommended for best performance.
3. Memory: At least 16GB of RAM is recommended, though more may be needed for larger models or larger datasets.
4. Storage: You’ll need enough storage to store the model and any training data. At least 500GB of SSD storage is recommended, though again, this may vary depending on the specific model and dataset.

It’s worth noting that these are just general recommendations and the specific hardware requirements will depend on the size and complexity of the model being used, as well as the amount of data being processed.

Minimum quantity of data needed to train the model?

The minimum amount of data needed to train a language model like ChatGPT can vary depending on the specific use case and the size and complexity of the model being used. However, in general, larger language models require larger amounts of training data to achieve good results.

For example, the original GPT model released by OpenAI was trained on a dataset of over 40GB of text data, while the larger GPT-2 and GPT-3 models were trained on datasets of 1.5TB and 570GB, respectively.

In order to get started with training a smaller language model, you could start with a smaller dataset of a few GBs or less. However, it’s important to keep in mind that the quality of the outputs may be limited by the amount and quality of the training data used. In general, the more data that is available, the better the performance of the language model is likely to be.

Does it worth it?

Several Cloud based ready to use AI solutions are already there in the market, Microsoft is proposing a GPT model that can be clustered and used for your own data only, other providers are also proposing similar solutions. Does it worth the pain to go with an OpenAI solution and deploy your own AI internaly? It is an open question, just be ready to assume the consequences for both cases, as any IT solution, AI will require hiring talents, maintaining the solution and managing the access rights. It will also require a lot of trainings and a certain time before getting the first results. And as any other solution, administration and a dedicated training program to your work force in order to make the transition a successful Journey are a must!