How to Build a Custom Chat Interface with LibreChat and Venice API

How to Build a Custom Chat Interface with LibreChat and Venice API

Create a custom LibreChat interface with Venice API on decentralized infrastructure. Follow this step-by-step guide to build your own private AI chat platform with multi-model support, user management, and complete configuration control.

Venice.ai

Venice provides powerful open-source AI models that can be integrated with various applications. In this guide, we'll show you how to create your own custom chat interface using LibreChat and the Venice API, all deployed on Akash's decentralized infrastructure.

What is LibreChat?

LibreChat is an open-source chat interface that allows you to create your own AI-powered chat application. When integrated with Venice's API, you get a fully customizable chat platform that maintains privacy while providing access to state-of-the-art open-source models.

With LibreChat you can create your own chat interface that utilizes the Venice API. LibreChat also offers the ability to add RAG functionality to provide a knowledge base specific to your use case. This is perfect for creating a chatbot that is specifically tailored to your application.

Setting Up Your LibreChat Deployment

We'll be hosting LibreChat on Akash through this example. Akash is a decentralized compute marketplace, connecting users who need computational resources to providers who supply those resources.

Prerequisites

Before starting the integration, you'll need:

  • Venice API key (via Pro account, VVV staking or topping up your account with USD)

  • Akash account with a funded wallet ($AKT tokens)

  • MongoDB Atlas account (for database storage)

Step-by-Step Deployment Guide

  1. Go to Akash.network and click “Deploy Now” on the top right corner, which will bring you to console.akash.network

  1. To use Akash, you will need a Keplr or Leap Cosmos wallet with $AKT tokens on the Cosmos chain. We recommend you following the Akash guides to properly setup and fund those wallets. The simplest method of funding is to purchase $AKT on Coinbase, and sending the funds directly to the akash wallet that has been setup

  2. Once funded, you will see the $USD value of your tokens on the top right corner within the wallet connection on the Akash console

  1. You will need to manually add your SDL file within akash. First click “Deploy”

  1. Then click, “Run Custom Container”

  1. Delete everything within the “YAML” tab, and replace with the following below:

---
version: "2.0"
services:
  service-1:
    image: kylecohen01/libre:3.3
    expose:
      - port: 3080
        as: 3080
        to:
          - global: true
    env:
      - HOST=0.0.0.0
      - MONGO_URI= <#MongoDB Atlas URI>
      - DOMAIN_CLIENT=http://0.0.0.0:3080
      - DOMAIN_SERVER=http://0.0.0.0:3080
      - NO_INDEX=true
      - CONSOLE_JSON=false
      - DEBUG_LOGGING=true
      - DEBUG_CONSOLE=false
      - ENDPOINTS='Venice'
      - PROXY=
      - VENICE_API_KEY= <#Venice API Key>
      - DEBUG_PLUGINS=true
      - CREDS_KEY= <#Creds Key>
      - CREDS_IV= <#Creds IV>
      - SEARCH=true
      - MEILI_NO_ANALYTICS=true
      - MEILI_HOST=http://0.0.0.0:7700
      - MEILI_MASTER_KEY= <#Meili Master Key>
      - OPENAI_MODERATION=false
      - BAN_VIOLATIONS=true
      - BAN_DURATION=1000 * 60 * 60 * 2
      - BAN_INTERVAL=20
      - LOGIN_VIOLATION_SCORE=1
      - REGISTRATION_VIOLATION_SCORE=1
      - CONCURRENT_VIOLATION_SCORE=1
      - MESSAGE_VIOLATION_SCORE=1
      - NON_BROWSER_VIOLATION_SCORE=20
      - LOGIN_MAX=7
      - LOGIN_WINDOW=5
      - REGISTER_MAX=5
      - REGISTER_WINDOW=60
      - LIMIT_CONCURRENT_MESSAGES=true
      - CONCURRENT_MESSAGE_MAX=2
      - LIMIT_MESSAGE_IP=true
      - MESSAGE_IP_MAX=40
      - MESSAGE_IP_WINDOW=1
      - LIMIT_MESSAGE_USER=false
      - MESSAGE_USER_MAX=40
      - MESSAGE_USER_WINDOW=1
      - ILLEGAL_MODEL_REQ_SCORE=5
      - CHECK_BALANCE=false
      - ALLOW_EMAIL_LOGIN=true
      - ALLOW_REGISTRATION=true
      - ALLOW_SOCIAL_LOGIN=false
      - ALLOW_SOCIAL_REGISTRATION=false
      - ALLOW_PASSWORD_RESET=false
      - ALLOW_UNVERIFIED_EMAIL_LOGIN=true
      - SESSION_EXPIRY=1000 * 60 * 15
      - REFRESH_TOKEN_EXPIRY=(1000 * 60 * 60 * 24) * 7
      - JWT_SECRET= <#JWT Secret>
      - JWT_REFRESH_SECRET= <#JWT Refresh Secret>
      - ALLOW_SHARED_LINKS=true
      - ALLOW_SHARED_LINKS_PUBLIC=true
      - APP_TITLE=LibreChat
      - HELP_AND_FAQ_URL=https://LibreChat.ai
profiles:
  compute:
    service-1:
      resources:
        cpu:
          units:
            - 2
        memory:
          size: 2gb #tuning of these values for optimization required
        storage:
          - size: 8Gi  #tuning of these values for optimization required
  placement:
    dcloud:
      pricing:
        service-1:

  1. Now is time to configure your deployment. There are 3 key areas of this file that are important: A) Venice API Key, B) MongoDB Atlas URI C) Secret Credentials

  • Venice API Key: To generate an API key you can follow these instructions https://docs.venice.ai/welcome/guides/generating-api-key. You can get an API key through Venice with a pro account (explorer tier), by staking VVV to get ongoing access to daily compute (via VCUs), or by depositing USD (both will enable paid tier).

  • MongoDB Atlas URI: MongoDB offers a forever free tier of Atlas that should be sufficient for this use case. You will need to create an account, and create a cluster for database management. LibreChat provides details instructions for this here

  • Secret Credentials: Go to the LibreChat credentials generator here to generate new credentials that will be used in this section

  1. Enter the Venice API Key, MongoDB URI, and the secret credentials into the Akash SDL file.

  1. Click “Create Deployment”

  1. Click continue to put 0.5 AKT into “escrow” of the deployment. You can add more funds later if deployed corrects.

  1. Click OK/Sign when your connected wallet asks for the wallet signature

  1. Choose a provider from the list that populates. We recommend only using “Audited” providers by Akash. Click “Accept Bid” to continue

  1. You will be asked again to Approve a transaction in your associated wallet

  1. When the deployment starts, you will see similar logs within the “Events” section of your deployment. This shows the successful download and startup of the docker image

  1. Check the status of the Agent in the “Logs” section. It can take a few minutes for the agent to startup and begin seeing information in the logs section. If you miss the startup messaging, you can also click “Download logs” to see exactly what happened during the build process. The image below shows that the build was successful

  1. Next, click the “Leases” button to see the ports that the agent is hosting on, you will need to copy these down. Click the link to port 3080 within the “Forwarded Ports” section

  1. If LibreChat is running and hosting properly, you will see this page in your web browser

  1. LibreChat uses a login based UI to maintain separation between user conversations. Sign up and log in to continue. When logged in, you will be brought to the home UI. This has already been configured for the Venice API

  1. You can choose the models available in the Venice API by clicking on the model dropdown

  1. Now you can interact with Venice.AI through the LibreChat interface. Send a message in the chat UI to ensure the integration was successful

  1. Congratulations on utilizing LibreChat to host a Venice Chat Interface

Resources and Next Steps

Venice's API integration with LibreChat provides a powerful, private chat interface without data collection or content restrictions. To learn more and get support:

Remember, Venice's API access through VVV staking provides ongoing, private access to AI capabilities without per-request fees or data collection.

Try it today and create your own customized AI chat interface!

Volver a todas las publicaciones
Room