BerriAI/litellm - GitHub
LiteLLM offers a universal solution for integrating various large language model (LLM) APIs into your applications by using a consistent OpenAI format. This tool allows seamless access to multiple pro
14,126
Votes
23,429
Views
6,534
Bookmarks
About
LiteLLM offers a universal solution for integrating various large language model (LLM) APIs into your applications by using a consistent OpenAI format. This tool allows seamless access to multiple providers such as Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, and Replicate, among others, without the need for adapting to each provider's specific API style. LiteLLM's features include input translation to different providers’ endpoints, consistent output formats, common exception mapping, and load balancing for high-volume requests. It supports over 100 LLM APIs, making it an indispensable tool for developers looking to leverage AI language models across different cloud platforms, all with the ease of OpenAI-style API calls.
Key Features
- Consistent Output Format: Guarantees consistent text responses across different providers.
- Exception Mapping: Common exceptions across providers mapped to OpenAI exception types.
- Load Balancing: Capable of routing over 1k requests/second across multiple deployments.
- Multiple Providers Support: Access to 100+ LLM providers using a single OpenAI format.
- High Efficiency: Translates inputs efficiently to provider's endpoints for completions and embeddings.
You may also like
More tools in Other
PureCode.ai
A tool to automate coding tasks through codebase-aware code generation.
@kuki_ai
Welcome to the world of Kuki, an award-winning artificial intelligence designed to bring entertainment to the digital age. Dive into engaging conversations with AI that's crafted to provide not just r
AI Dungeon
AI Dungeon is a text-based adventure game where you lead the story and the AI creates the world around you. It offers endless possibilities by generating unique characters, settings, and scenarios bas
Verbacall
A platform that automatically answers, qualifies, and follows up on calls 24/7.
SuperU AI
A nocode tool to create voice AI agents for customer communications.
Hermes 3
A tool to perform complex creative and analytical tasks with long-term context.
PrompTessor
A tool that optimizes text for clarity, tone, and grammar without requiring prompt engineering skills.
MuseNet
MuseNet is a groundbreaking deep neural network that revolutionizes the way music is composed. With MuseNet, you have access to a powerful tool that can generate beautiful and intricate 4-minute music
YOUS
YOUS is a cutting-edge platform that revolutionizes the way people communicate and connect with each other. With its innovative AI-based translator, YOUS enables seamless audio and video calls between
Spikes Studio
Spikes helps creators extract captivating shorts from any video. Automatically get a title, description, hashtag recommendations, auto-captions, AI styling and quickly edit. Grow and give your audienc
G3D.AI {Jedi}
G3D.AI {Jedi} is a generative AI tool for game creation that enables game creators to build beautiful and novel games in a fraction of the time. With a suite of tools designed to supercharge creativit
Zeitpub
Zeitpub revolutionizes the publishing industry by offering an advanced AI writer that creates SEO-optimized and plagiarism-free content at an incredible pace. With just the click of a button, users ca