Many small and medium-sized enterprises (SMEs) already have more usable company data than they are aware of. They are found, for example, in ERP or CRM systems, in Excel lists, production plants, service reports, PDF documentation, emails, or on shared drives. Employees' experiential knowledge can also be crucial for an AI application.

In this context, Retrieval-Augmented Generation (RAG) becomes interesting. Here, no separate AI model is trained with company data. Instead, an AI assistant searches internal documents for information that is provided to the language model for its answer. It is a low-barrier method to make knowledge accessible, make decisions data-based, or accelerate innovation without having to train costly AI models yourself.

Which data is needed depends on the specific AI use case. At the same time, the available data basis decides which AI applications can be sensibly implemented in your company. No extensive data management concept is needed for this. More important is a clear thread: What should the AI be able to do, what information does it need, and is this information already available?

For the entry into AI, your company does not need a fully cleaned data stock, nor does it automatically require a new data platform or additional software. Individual, targeted improvements in data quality gradually create a robust data basis that brings noticeable benefits to your company and secures its competitiveness.

Why is data important for AI?

A solid data basis is not only worthwhile for use in AI: It makes analyses more reliable, increases transparency in processes, and improves decisions. If you use data systematically, you can also develop new products and offers faster and strengthen competitiveness.

The topic is particularly relevant when...

• you want to introduce a specific AI application,

• you want to make internal company knowledge usable with AI,

• you want to systematically evaluate customer or service data or

• need to merge information from multiple systems.

If you are unsure whether your data is sufficient for the planned use, an early examination is advisable. You can clarify this based on a specific use case. This allows you to assess what data is missing and how much effort is needed for a viable data basis.

Utilizing company knowledge: Retrieval-Augmented Generation (RAG)

Anyone wanting to use AI with company data quickly encounters terms like RAG, training, or fine-tuning. Which approach is suitable depends on what the AI should do with the data. The choice significantly influences the effort and costs.

If you want an AI assistant to answer questions about products, processes, or documentation, you usually don't need to train your own model. With RAG, the system searches for suitable information in the released company data and forwards it together with the question to a language model. The model formulates the answer from this.

This brings several advantages. New or changed knowledge can be used without retraining a model. The AI only accesses data you have released. If the system names the sources used, you can verify any answer. It is therefore crucial that relevant information is current and findable.

Making predictions: Training with historical data
If AI is to predict machine failures or sales volumes, it needs suitable historical training data. It's not just the amount that counts. Thousands of machine values are of little help if there is no documentation of actual defect occurrences. A smaller, well-documented data set can be significantly more valuable.

Fine-tuning: not automatically necessary
In fine-tuning, an existing AI model is adapted for a specific task. To make company knowledge usable, this is often not necessary. If fine-tuning is recommended to you, ask: What specific problem does it solve, and why isn't access to existing information enough?

External AI services: Check conditions
Even if data is not used to train a model, it may be stored or otherwise processed. For confidential or personal information, you should therefore check the conditions of the specific service, tariff, and contract.

Create a good data basis for AI in 5 steps:

• Step 1 – Start from the use case
  Don't start with the question "What data do we have?" but with "What should AI be able to do in our company?" If it should, for example, answer service questions, it may need manuals, repair reports, and spare parts information. In contrast, sales forecasts require sales figures, time periods, and other influencing factors. First, create a short list of the information actually needed for the desired task. This way, you avoid having your company process data that doesn't matter for the project.

• Step 2 – Check what is already available
  Then specifically search for this information. It's worth looking beyond the classic databases: Relevant business data may be in the ERP system, but also in Excel files, PDFs, emails, SharePoint, a network drive, or with individual employees. Note where the needed information is, who can access it, and what is still missing. A simple table is enough for an initial overview—your company doesn't need new software for this.

• Step 3 – Check the quality with real examples
  Don't check the data quality abstractly, but based on 20 to 30 typical cases from your company's everyday work. For a service assistant, these could be real questions from customer service. Do you find a current and clear answer to these questions in the available documents? Different product names, outdated price lists, duplicate customer entries, or conflicting work instructions highlight what needs improvement before a pilot project.

• Step 4 – Prepare only the required data
  Your company doesn't need to clean all the data first. Focus on the information necessary for the selected application case. Remove outdated versions, unify important designations, and fill in missing information. Simultaneously clarify who will keep this data up to date in the future and who is allowed to access it. For personal, confidential, or otherwise sensitive information, it should be clarified in advance whether and under what conditions they may be processed with the intended AI solution.

• Step 5 – Test small and learn from it
  First test the application with a limited data set and real tasks from everyday work. If the AI provides only twelve usable answers to 20 typical questions, check the other eight cases: Was some information missing? Was it outdated? Could the AI not clearly assign it? Or is the problem not with the data at all? Before your company invests money in new infrastructure or external support, it should first be checked whether the application case can be tested sensibly with the existing data and systems. The result can also be that the effort for this application case is not worth it—then it makes more sense to choose another application case than to invest further time and money in data preparation.

You can find this and many more pieces of information on the AI compass topic "Data" in our download portal: