Artificial Intelligence (AI) is increasingly making its way into the world of work.

Microsoft, Google, and even the conservative business network LinkedIn are launching AI-assisted products. But how do we actually distinguish between a good and a bad AI?

The American company Datadog has a solution: their AI monitoring tool called LLM Obs (LLM stands for "Large Language Model") is designed to help assess and compare the quality of AI models.

The Handelsblatt visited Datadog's headquarters in New York and reports on a company that is already valued at nearly 30 billion dollars. The company was founded by Oliver Pomel and Alexis Lê-Quôc in 2010.

Datadog provides its customers with dashboards as a virtual overview of all cloud services in use. This allows programmers to keep track of server loads, data flows and security risks. But now the company wants to take it a step further and also monitor the quality of AI models.

The AI monitoring tool LLM Obs is currently still in beta phase and is being tested by selected corporate customers. It analyzes the performance, quality, cost and other measurements of AI models to quickly identify and fix potential problems.

How Businesses Can Handle Time and Resources More Efficiently. Datadog Product Manager Yrieix Garnier emphasizes the importance of cost control in AI models. In a world where new AI models are coming to market every day, it is crucial for companies to select the best and most cost-effective models. Datadog is open to partnerships and has not yet set prices for their program.

Datadog-Chef Junaid Ahmed compares the potential of his AI monitoring tool to a large furniture retailer. For instance, it could use various AI models on its website and compare their performance in different use cases with LLM Obs. This way, companies could specifically select the right models for their individual needs.

The AI offerings from companies like OpenAI, Anthropic, Cohere, Google, and open-source models are just the beginning, according to Ahmed. In the future, there will be a whole universe of different AI models that may even complement each other.

To keep an overview in this world, it is important to know how well the implemented models are working. This is precisely where Datadog's tool comes in, aiming to help companies receive more meaningful answers from their AIs.

The project is still in its early stages, but Datadog sees great potential in AI monitoring. In the best case scenario, programs like LLM Obs could help make AI use even more effective and efficient. In the future, monitoring of AIs could even become a major business field for companies like Datadog.