Ollama
What is Ollama?
Ollama is a lightweight and efficient server for running large language models (LLMs) on your local machine or in the cloud. It supports popular models like Llama, Mistral, Gemma, and many others.
The one-click deployment includes Open WebUI, a web-based interface for interacting with the models.
Deployment
Deploy Ollama as a one-click service from the service catalog. After deployment, pull models using the Ollama CLI or the Open WebUI interface. For GPU acceleration, ensure your server has appropriate drivers and pass through GPU devices in the compose configuration.