LLaMa2 GPTQ

Question Answering AI who can provide answers with source documents based on Texonom.

  • Made with Langchain
  • Chat UI support made by Streamlit Web Component
  • Rye based python package management

Get startedOpen on GitHub →
Bash
$git clone https://github.com/seonglae/llama2gptq$cd llama2gptq$git clone https://huggingface.co/datasets/texonom/llama2gptq db$rye sync$streamlit run chat.py
Click to copy

What's included

Langchain

Langchain made it easy to Prompt engineering of referencing source documents.

GPTQ

CUDA based int4 Model quantization make model available to run in local environment.

ChromaDB

Vector database make LLaMa2 GPTQ provide responses with reference documents

Web UI

Chat UI provided for conversation with private AI without any external API