Introduction

Artificial Intelligence (AI) has been at the forefront of technology developments, especially with the latest advances in Large Language Models (LLMs). However, we must note that a significant evolution concerning AI is taking place. Cutting-edge AI is no longer exclusively the domain of single, huge models, but rather of compound systems incorporating LLMs and other sophisticated components.

While single-modeled AI systems have remarkable capabilities, they come with huge limitations. They lack the versatility and adaptability and often fail to adjust to dynamic environments. The concept of compound AI systems has emerged to tackle such limitations.

What is a Compound AI System?

To efficiently manage AI activities, a system known as a Compound AI System (CAS) integrates various components, such as databases, retrievers, AI models, and other external tools. However, the Transformer-based LLM and other prior AI systems depend mainly on one AI model, and CAS leverages the combination of several tools. We define a CAS as a system that manages AI tasks with different interacting components. This includes numerous calls to models, retrievers, or external tools. In contrast, an AI Model is just a statistical model, e.g., a Transformer predicting the next token in the text.

Why Use Compound AI Systems

Even though AI models are continually getting better, more and more state-of-the-art results are obtained using compound systems. Why is that? We have several distinct reasons:

Some tasks are easier to improve through system design

While LLMs appear to stick to excellent scaling laws that yield better results with more computing, in many applications, scaling provides lower returns in comparison to the cost of building a compound system. For example, consider a scenario where an LLM solves coding contest problems thirty percent of the time, and tripling its training budget increases this to thirty-five percent. Now, this is not good enough to win a coding contest! In contrast, designing a system that samples from the model multiple times can increase the performance to eighty percent. Even more importantly, iterating a system design is much faster than having to wait for training runs. In a high-value application, developers need to use every tool available to enhance the AI quality. Thus, they use system ideas in addition to scaling. We frequently witness this with LLM users, where a good LLM creates a good but unreliable first demo, and engineering teams then go on to methodically boost the quality.

Systems can be dynamic

Machine Learning (ML) models are integrally restricted as they are trained on static datasets, making their intelligence fixed. Thus, developers combine models with other components, such as search and retrieval, to include timely data. In addition, training lets a model review the whole training set. Thus, more complex systems are required to build AI applications with access controls (e.g., answer a user’s questions based only on files the user can access).

Improving control and trust is easier with systems

Neural network models alone are difficult to control. While training influences them, it is impossible to guarantee that a model can avoid certain behaviors. Using an AI system instead of a model helps developers control behavior, e.g., by filtering model outputs. Similarly, even the best LLMs cannot always offer perfect results. However, if we take the example of a system combining LLMs with retrieval, then such a system can increase user trust by automatically verifying facts.

Performance goals vary widely

Each AI model has a set quality level and cost, but applications need to alter these parameters. In some applications, such as inline code suggestions, the best AI models are too expensive, so tools like Github Copilot use smaller models and various search heuristics to display results.

What kind of Components are in a Compound AI System?

A compound AI system includes several key components, where each component plays a significant role in the system. The components can vary based on what kind of tasks the system tackles. Following are some of the main components of a CAS:

• Large Language Model (LLM): An LLM component analyzes the user’s text description, understanding the desired content, style, and artistic elements.
• Image generation component: This component leverages the LLM’s interpretation to generate a series of candidate images.
• Diffusion model: Likely used in a text-to-image system to refine the initial image outputs.
• User feedback integration: Allows users to provide feedback on generated images through text prompts or by selecting preferred variations.
• Ranking and selection component: Uses ranking algorithms to select the best image from the generated options.

What are the Advantages of Compound AI Systems?

• Better performance for complex tasks: For many complex tasks, a well-designed compound AI system can collate the capacities of its components and this collaborative outcome is usually better than a single, large model.
• Increased flexibility and adaptability: Compound AI systems are flexible to different tasks and situations. For example, individual components can be adjusted or replaced to enhance performance for a specific scenario.
• Dynamic integration of new data from different sources: Compound AI systems can incorporate new data sources or tools more easily. For instance, a real-time data stream or a specific database can be connected when required.
• Better control and trust: Compound AI systems offer greater control over the model’s outputs. Output validation, filters, or human-in-the-loop approaches can be included to ensure the system functions according to desired requirements.

Conclusion

Compound AI systems represent a significant leap in AI development. By leveraging the power of collaboration between diverse components, CAS can tackle complex problems that were beyond the reach of traditional AI. As CAS technology continues to evolve, we can expect to see more innovative applications emerge, shaping the future of different industries and scientific fields.

Aretove can help enhance complex AI systems by offering tailored AI solutions to address your unique business challenges. Our expertise spans the entire AI project lifecycle, from initial strategy development to deployment and maintenance. We have a team of seasoned data scientists and AI experts to design sophisticated models, that ensure that the models are optimized for performance and scalability. By providing comprehensive data management services, we ensure the quality and integration of datasets crucial for effective AI functioning. Additionally, Aretove is committed to ethical AI practices and regulatory compliance, mitigating risks related to bias and data security.