pip install 'litellm[proxy]'

model_list:
  - model_name: <modelID>
    litellm_params:
      model: openai/<modelID>
      api_base: <https://api.venice.ai/api/v1>
      api_key: "<api_key>"

litellm --config ~/Desktop/litellmtest/config.yaml --detailed_debug

23:44:33 - LiteLLM Proxy:DEBUG: proxy_server.py:1505 - loaded config={
    "model_list": [
        {
            "model_name": "llama-3.3-70b",
            "litellm_params": {
                "model": "openai/llama-3.3-70b",
                "api_base": "<https://api.venice.ai/api/v1>",
                "api_key": "<api_key>"
            }
        }
    ]
}
LiteLLM: Proxy initialized with Config, Set models:
    llama-3.3-70b

~/litellmtest % curl --location '<http://0.0.0.0:4000/chat/completions>' \\
  --header 'Content-Type: application/json' \\
  --data ' {
  "model": "llama-3.3-70b",
  "messages": [
      {
      "role": "user",
      "content": "tell me about ai"
      }
  ]
}'

23:46:34 - LiteLLM Proxy:DEBUG: proxy_server.py:3518 - Request received by LiteLLM:
{
    "model": "llama-3.3-70b",
    "messages": [
        {
            "role": "user",
            "content": "tell me about ai"
        }
    ]
}

{"id":"chatcmpl-77fc96ac3707555543746381628efd66","created":1741063594,"model":"llama-3.3-70b","object":"chat.completion","system_fingerprint":null,"choices":[{"finish_reason":"stop","index":0,"message":{"content":"Artificial Intelligence (AI) refers to the development of computer systems that can perform tasks that would typically require human intelligence, such as learning, problem-solving, decision-making, and perception. The field of AI has been rapidly advancing in recent years, with significant breakthroughs in areas like machine learning, natural language processing, and computer vision.\\n\\n**History of AI:**\\nThe term \\"Artificial Intelligence\\" was coined in 1956 by John McCarthy, a computer scientist and cognitive scientist. However, the concept of creating machines that can think and learn dates back to ancient Greece, with myths like Pygmalion and Galatea. The modern field of AI began to take shape in the 1950s, with the development of the first AI program, called Logical Theorist, by Allen Newell and Herbert Simon.\\n\\n**Types of AI:**\\nThere are several types of AI, including:\\n\\n1. **Narrow or Weak AI:** This type of AI is designed to perform a specific task, such as facial recognition, language translation, or playing chess. Narrow AI is the most common type of AI and is used in many applications, including virtual assistants like Siri and Alexa.\\n2. **General or Strong AI:** This type of AI is designed to perform any intellectual task that a human can. General AI is still in the early stages of development, but it has the potential to revolutionize many industries and aspects of life.\\n3. **Superintelligence:** This type of AI is significantly more intelligent than the best human minds, and it has the potential to solve complex problems that are currently unsolvable.\\n\\n**Machine Learning:**\\nMachine learning is a key component of AI, and it refers to the ability of a computer system to learn from data without being explicitly programmed. Machine learning algorithms can be divided into three categories:\\n\\n1. **Supervised Learning:** In this type of learning, the computer system is trained on labeled data, and it learns to make predictions based on the input data.\\n2. **Unsupervised Learning:** In this type of learning, the computer system is trained on unlabeled data, and it learns to identify patterns and relationships in the data.\\n3. **Reinforcement Learning:** In this type of learning, the computer system learns by interacting with an environment and receiving rewards or penalties for its actions.\\n\\n**Applications of AI:**\\nAI has many applications in various industries, including:\\n\\n1. **Healthcare:** AI is used in medical diagnosis, personalized medicine, and healthcare management.\\n2. **Finance:** AI is used in trading, portfolio management, and risk analysis.\\n3. **Transportation:** AI is used in self-driving cars, traffic management, and route optimization.\\n4. **Education:** AI is used in personalized learning, intelligent tutoring systems, and educational software.\\n5. **Customer Service:** AI is used in chatbots, virtual assistants, and customer service automation.\\n\\n**Challenges and Risks:**\\nWhile AI has the potential to revolutionize many industries and aspects of life, it also poses significant challenges and risks, including:\\n\\n1. **Job Displacement:** AI has the potential to automate many jobs, leading to significant job displacement and social disruption.\\n2. **Bias and Discrimination:** AI systems can perpetuate bias and discrimination if they are trained on biased data or designed with a biased perspective.\\n3. **Cybersecurity:** AI systems can be vulnerable to cyber attacks, which can compromise their security and integrity.\\n4. **Ethics and Governance:** AI raises significant ethical and governance questions, including issues related to accountability, transparency, and human oversight.\\n\\n**Future of AI:**\\nThe future of AI is exciting and uncertain, with many potential breakthroughs and challenges on the horizon. Some of the potential developments in AI include:\\n\\n1. **Increased Autonomy:** AI systems may become increasingly autonomous, making decisions and taking actions without human oversight.\\n2. **Improved Natural Language Processing:** AI systems may become more proficient in natural language processing, enabling them to communicate more effectively with humans.\\n3. **Enhanced Computer Vision:** AI systems may become more proficient in computer vision, enabling them to perceive and understand their environment more effectively.\\n4. **Merging with Other Technologies:** AI may merge with other technologies, such as the Internet of Things (IoT), blockchain, and quantum computing, to create new and innovative applications.\\n\\nOverall, AI has the potential to revolutionize many industries and aspects of life, but it also poses significant challenges and risks. As AI continues to evolve and improve, it is essential to address these challenges and risks, and to ensure that AI is developed and used in a responsible and ethical manner.","role":"assistant","tool_calls":null,"function_call":null}}],"usage":{"completion_tokens":938,"prompt_tokens":183,"total_tokens":1121,"completion_tokens_details":null,"prompt_tokens_details":null},"service_tier":null,"prompt_logprobs":null,"venice_parameters":{"web_search_citations":[]}