No Code GenAI Agents Workflow Orchestration: AutoGen Studio with Local Mistral AI model
Free link – Please help to like this LinkedIn post
AutoGen is a framework developed by Microsoft and designed to simplify the development of multi-agent applications, particularly in orchestrating LLM agents.
Multi-agent applications involve systems where multiple LLM or multi-modal agents or entities interact with each other in the whole workflow to achieve specific goals or tasks. These agents can be LLM agents, retrieval agents, or other agents capable of making independent decisions, function calling, or taking actions.
If you want to know more about AutoGen, you can refer to my previous article: AutoGen In-depth yet Simple.
Mistral AI is a French AI company founded by former Meta and Google researchers in April 2023. It focuses on producing open large language models (LLMs) and emphasizes the importance of open-source AI models.
In this article, we will be focusing on the revolutionary fusion of AutoGen Studio's intuitive, no-code platform with the locally integrated Mistral AI model. This combination is not just about making it easier to bring AI to application; it's also about facilitating how we interact with, deploy, and benefit from different generative AI agents in many real-life industry workflows.
Navigating AutoGen Studio's Interface:
Autogen Studio installation
AutoGen Studio is open source code here, and can be installed via pip.
pip install autogenstudioRunning Autogen Studio
Once installed, run the web UI by entering the following in your terminal:
autogenstudio ui --port 8082
AutoGen Studio presents a streamlined and user-friendly interface that facilitates the creation and management of multi agents AI applications. The interface is split into several sections, including Skills, Models, Agents, and Workflows, each playing indispensable role in application development.
Skills
In the Skills section, users can develop and store Python functions that agents will use to solve tasks. It is a programming environment where the functionalities of the application are defined and refined.
Models
The Models section is where users can configure and manage AI models, including GPT-4 and other local or custom models.
Users can set up models like Mistral AI, indicating the studio's support for local AI model integration and management. This flexibility allows for the use of powerful models like OpenAI's GPT-4, alongside proprietary or specialized models tailored to specific tasks.
Agents
The Agents section is designed for configuring reusable agents. Users can create and define the roles of various agents such as local assistants, primary agents, and user proxies, orchestrating how they interact within the app's workflow.
Workflows
The Workflows section is where the magic happens. Users can design complex workflows that define how tasks are handled by the system. This is where all the components come together: Skills provide the logic, Models offer the AI's intelligence, Agents act upon tasks, and Workflows connect all into a coherent system.
The platform's interface is intuitive, with clearly labeled sections and a visual layout that makes navigation simple. Whether adding a new agent, model, or skill, the process is straightforward, often requiring just a few clicks. The use of vibrant colours and modern design elements make the experience pleasant and easy to use.
Local Mistral AI Model Integration
Integrating local Mistral AI models into AutoGen Studio's ecosystem is quite easy and straightforward. Users can add their local models to the studio and define how they interact with other components within the app. This is particularly useful for those who need to combine the power of multi modal models like GPT-4 vision, GPT 3.5, GPT 4 turbo and other local models like Mistral AI.
In summary, AutoGen Studio is a comprehensive and accessible tool for building multi-agents AI-driven applications, accommodating a wide range of AI models and custom user-defined skills, making it a versatile choice for developers and businesses as well.
Setting up Local Mistral AI Model:
Setting up a local Mistral AI model involves several steps, each of which prepares the system to run large language models locally, such as the Mistral model by Ollama. Here's an explanation of each step and why we need each step:
- Install Ollama: Ollama is a platform that provides access to large language models. Installing Ollama is the first step because it is the environment within which the Mistral model will run. It allows you to customize and create your own models or run existing models like Llama 2, Code Llama, and Mistral. You can download the ollama here: https://ollama.ai/
- Run Ollama Mistral: Following command below from Ollama platform that starts the Mistral model, making it ready to process requests. This command initialises the processes that set up the model, load the necessary data, and start the services that will handle the processing of the queries.
ollama run mistral- Install LiteLLM: The LiteLLM library is a tool that facilitates the running of language models from http endpoint. Installing or upgrading LiteLLM ensures that you have the latest version of the library, which is important for compatibility and functionality reasons.
pip install litellm --upgrade- Install Gunicorn: Gunicorn is a Python WSGI HTTP Server for UNIX systems that's used to run Python web applications. It is a prerequisite for running the LiteLLM proxy, which allows you to make local HTTP requests to your language model.
pip install gunicorn- Run LiteLLM with Ollama/Mistral Model: This step actually initialises the model and gets it ready to run. This command tells LiteLLM to use the Mistral model provided by Ollama. It prepares the model for interaction, allowing you to start making queries and receiving responses.
At the end this process, local Mistral model is starting on 0.0.0.0:8000 with 1 workers.
litellm --model ollama/mistralEach of these steps is necessary to set up a complete working environment where the AI model can be interacted with locally. This local setup allows developers to test and integrate AI models without the need to constantly communicate with cloud services, which can be beneficial for development speed, offline functionality, and cost-saving on API calls.
Workflow Orchestration with AutoGen and Mistral AI:
In the scenario where we want to write a Python script to plot a sine wave and save it as a ‘sine_wave.png' file, AutoGen and Mistral AI can be used together to create a streamlined workflow. The AutoGen Studio serves as the interface for building and managing the multi-agent application, while Mistral AI provides the processing power through its local model.
Here's how the orchestration works using the capabilities of both platforms:
1. Creating the Workflow:
- In AutoGen Studio, a new workflow is initiated, designated for the specific task of generating a sine wave image.
- We configure the workflow with a name, such as "Local mistral Agent Workflow".
- The workflow specification includes the sender and receiver agents, where ‘userproxy' could be the sender initiating the request, and ‘primary_assistant' is the receiver that processes the request and provides the output.
- The role of user proxy will be interacting with user and executing the code generated by primary_assistant. If there is any error in the code send the error to primary_assistant agent.
- The primary_assistant agent is empowered by local Mistral AI model, it will generate code in our scenario. If error, reajust the code according to the execution error sent by user proxy agent.
2. Model Configuration:
- A model named ‘mistral local model' is set up within the AutoGen Studio to interact with the Mistral AI.
- The model's API endpoint would be configured to the local server address where the Mistral AI is running (e.g., http://0.0.0.0:8000)..)
- This setup allows the AutoGen Studio to send data to the Mistral AI model and receive the processed results.
3. Running the Session:
- In the AutoGen Studio's ‘Playground' interface, the user starts a new session by selecting the "Local mistral Agent Workflow".
- The user inputs the command or request to generate for the sine wave plot.
- The request is sent by the user proxy agent to the primary assistant agent, which leverages the Mistral AI model. It generates the Python script, and the user proxy agent executes the code. If an error occurs, it is sent back to the primary assistant agent. The primary assistant agent readjusts the code and sends it back to the user proxy agent until the code works and creates the sine wave plot image.
4. Outcome and Review:
- Once the script is run by Mistral AI, the result: ‘sine_wave.png' file is generated and saved to the disk.
- The AutoGen Studio interface displays the session's progress and the outcome, which includes two files: ‘sine_wave.png' file and the python code generated to plot the sine_wave.
- The user can then review the output, publish the workflow for broader use, delete the model if we want to start over, or iterate further to refine the script and the workflow's parameters.
Conclusion:
AutoGen Studio together with local Mistral AI models shows a major advance in orchestrating GenAI agents' workflows. This architecture benefits the intuitive design of AutoGen Studio and the excellent performance of Mistral AI, building an environment multi-agents will be able to contribute.
Advantages of Using AutoGen Studio with Local Mistral AI Models:
- Easy Integration: The ease with which local Mistral AI models integrate into AutoGen Studio simplifies the process of leveraging different LLM models into one deployment, streamlining the development.
- Customization and Flexibility: Developers can tailor AI models to suit specific requirements, offering a custom-fit solution that fits perfectly with the different needs of one single project.
- Enhanced Performance: Running AI models locally with AutoGen reduces latency and improves response times, which is crucial for real-time applications and sensitive workflows.
- Data Privacy and Security: Local execution of AI models gives full control over data, a critical factor for sensitive or proprietary information that cannot be exposed to external environments.
- Cost-Efficiency: By minimizing reliance on cloud-based AI services, organizations can significantly cut costs associated with data transfer and LLM API usage.
- Offline Capabilities: Local Mistral AI models ensure that the system remains operational even without internet connectivity, thus allowing the offline capability of AI-driven solutions.
- Construct Complex AI Workflows with a UI: Users can craft well-designed AI-driven workflows with ease, leveraging the no-code interface of AutoGen Studio to visualize and manage the entire process.
In this article, the integration of AutoGen Studio with local Mistral AI models is a transformative development in the world of GenAI workflows. It offers a user-friendly, flexible, and cost-effective solution that not only meets the growing demand for advanced AI applications but also make the creation of AI applications easier than ever.
