serge

• Purpose: Serge is a self-hosted chat interface designed to run LLM (Large Language Model) models locally using llama.cpp , eliminating the need for API keys.
• Target Audience: Users looking for a private, self-hosted solution to interact with LLMs without relying on external services.
• Key Features:
  • Frontend: Built with SvelteKit.
  • Backend: Uses FastAPI + LangChain for the API, integrating with llama.cpp via Python bindings.
  • Storage: Redis for chat history and parameter storage.
• Deployment: Supports Docker and Docker Compose for easy setup.
• Environment Variables: Customizable settings for database, authentication, and session management.
• Platform Support: Works on Windows (with Docker Desktop and WSL2) and other platforms.
• Memory Consideration: Requires sufficient RAM to run LLaMA models effectively.
• Community Support: Active Discord community for assistance.

Note: Svelte is prominently used for the frontend.