Self-Host
Learn about how to self-host Khoj on your own machine.
Benefits to self-hosting:
- Privacy: Your data will never have to leave your private network. You can even use Khoj without an internet connection if deployed on your personal computer.
- Customization: You can customize Khoj to your liking, from models, to host URL, to feature enablement.
Setup
These are the general setup instructions for self-hosted Khoj.
- Make sure python and pip are installed on your machine
- Check the Khoj Emacs docs to setup Khoj with Emacs
It's simpler as it can skip the server install, run and configure step below. - Check the Khoj Obsidian docs to setup Khoj with Obsidian
Its simpler as it can skip the configure step below.
For Installation, you can either use Docker or install the Khoj server locally.
If you want to use the offline chat model and you have a GPU, you should use Installation Option 2 - local setup via the Python package directly. Our Docker image doesn't currently support running the offline chat model on GPU, making inference times really slow.
Installation Option 1 (Docker)
Prerequisites
- Install Docker Engine. See official instructions.
- Ensure you have Docker Compose. See official instructions.
Setup
Use the sample docker-compose in Github to run Khoj in Docker. Start by configuring all the environment variables to your choosing. Your admin account will automatically be created based on the admin credentials in that file, so pay attention to those. To start the container, run the following command in the same directory as the docker-compose.yml file. This will automatically setup the database and run the Khoj server.
docker-compose up
Khoj should now be running at http://localhost:42110. You can see the web UI in your browser.
Installation Option 2 (Local)
Prerequisites
Install Postgres (with PgVector)
Khoj uses the pgvector
package to store embeddings of your index in a Postgres database. In order to use this, you need to have Postgres installed.
- MacOS
- Windows
- Linux
- From Source
Install Postgres.app. This comes pre-installed with pgvector
and relevant dependencies.
- Use the recommended installer.
- Follow instructions to Install PgVector in case you need to manually install it. Windows support is experimental for pgvector currently, so we recommend using Docker.
- Follow instructions to Install Postgres
- Follow instructions to Install PgVector in case you need to manually install it.
Create the Khoj database
Make sure to update your environment variables to match your Postgres configuration if you're using a different name. The default values should work for most people. When prompted for a password, you can use the default password postgres
, or configure it to your preference. Make sure to set the environment variable POSTGRES_PASSWORD
to the same value as the password you set here.
- MacOS
- Windows
- Linux
createdb khoj -U postgres --password
createdb -U postgres khoj --password
sudo -u postgres createdb khoj --password
Install package
Local Server Setup
- Make sure python and pip are installed on your machine
- Check llama-cpp-python setup if you hit any llama-cpp issues with the installation
Run the following command in your terminal to install the Khoj backend.
- MacOS
- Windows
- Linux
# ARM/M1+ Machines
MAKE_ARGS="-DLLAMA_METAL=on" python -m pip install khoj-assistant
# Intel Machines
python -m pip install khoj-assistant
# 1. (Optional) To use NVIDIA (CUDA) GPU
$env:CMAKE_ARGS = "-DLLAMA_OPENBLAS=on"
# 1. (Optional) To use AMD (ROCm) GPU
CMAKE_ARGS="-DLLAMA_HIPBLAS=on"
# 1. (Optional) To use VULCAN GPU
CMAKE_ARGS="-DLLAMA_VULKAN=on"
# 2. Install Khoj
py -m pip install khoj-assistant
# CPU
python -m pip install khoj-assistant
# NVIDIA (CUDA) GPU
CMAKE_ARGS="DLLAMA_CUDA=on" FORCE_CMAKE=1 python -m pip install khoj-assistant
# AMD (ROCm) GPU
CMAKE_ARGS="-DLLAMA_HIPBLAS=on" FORCE_CMAKE=1 python -m pip install khoj-assistant
# VULCAN GPU
CMAKE_ARGS="-DLLAMA_VULKAN=on" FORCE_CMAKE=1 python -m pip install khoj-assistant
Local Server Start
Before getting started, configure the following environment variables in your terminal for the first run
- MacOS
- Windows
- Linux
export KHOJ_ADMIN_EMAIL=<your-email>
export KHOJ_ADMIN_PASSWORD=<your-password>
If you're using PowerShell:
$env:KHOJ_ADMIN_EMAIL="<your-email>"
$env:KHOJ_ADMIN_PASSWORD="<your-password>"
export KHOJ_ADMIN_EMAIL=<your-email>
export KHOJ_ADMIN_PASSWORD=<your-password>
Run the following command from your terminal to start the Khoj backend and open Khoj in your browser.
khoj --anonymous-mode
--anonymous-mode
allows you to run the server without setting up Google credentials for login. This allows you to use any of the clients without a login wall. If you want to use Google login, you can skip this flag, but you will have to add your Google developer credentials.
On the first run, you will be prompted to input credentials for your admin account and do some basic configuration for your chat model settings. Once created, you can go to http://localhost:42110/server/admin and login with the credentials you just created.
Khoj should now be running at http://localhost:42110. You can see the web UI in your browser.
Note: To start Khoj automatically in the background use Task scheduler on Windows or Cron on Mac, Linux (e.g with @reboot khoj
)
Setup Notes
You can use Khoj with a custom domain as well. To do so, you need to set the KHOJ_DOMAIN
environment variable to your domain (e.g., export KHOJ_DOMAIN=my-khoj-domain.com
or add it to your docker-compose.yml
). By default, the Khoj server you set up will not be accessible outside of localhost
or 127.0.0.1
.
To expose Khoj on a custom domain over the public internet, use of an SSL certificate is strongly recommended. You can use Let's Encrypt to get a free SSL certificate for your domain.
To disable HTTPS, set the KHOJ_NO_HTTPS
environment variable to True
. This can be useful if Khoj is only accessible behind a secure, private network.
2. Configure
- Go to http://localhost:42110/server/admin and login with your admin credentials.
Configure Chat Model
Configure OpenAI or a custom OpenAI-compatible proxy server
- Go to the OpenAI settings in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key and server API base URL. The API base URL is optional - it's only relevant if you're using another OpenAI-compatible proxy server.
- Go over to configure your chat model options. Set the
chat-model
field to a supported chat model1 of your choice. For example, you can specifygpt-4-turbo-preview
if you're using OpenAI.- Make sure to set the
model-type
field toOpenAI
. - The
tokenizer
andmax-prompt-size
fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.
- Make sure to set the
Configure Offline Chat
- No need to setup a conversation processor config!
- Go over to configure your chat model options. Set the
chat-model
field to a supported chat model1 of your choice. For example, we recommendNousResearch/Hermes-2-Pro-Mistral-7B-GGUF
, but any gguf model on huggingface should work.
- Make sure to set the
model-type
toOffline
. Do not setopenai config
. - The
tokenizer
andmax-prompt-size
fields are optional. Set them only when using a non-standard model (i.e not mistral, gpt or llama2 model) when you know the token limit.
Share your data
- Select files and folders to index using the desktop client. When you click 'Save', the files will be sent to your server for indexing.
- Select Notion workspaces and Github repositories to index using the web interface.
Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.
3. Download the desktop client (Optional)
You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time.
To download the latest desktop client, go to https://download.khoj.dev and the correct executable for your OS will automatically start downloading. You can also go to https://khoj.dev/downloads to explicitly download your image of choice. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step.
To use the desktop client, you need to go to your Khoj server's settings page (http://localhost:42110/config) and copy the API key. Then, paste it into the desktop client's settings page. Once you've done that, you can select files and folders to index. Set the desktop client settings to use http://127.0.0.1:42110
as the host URL.
4. Install Client Plugins (Optional)
Khoj exposes a web interface to search, chat and configure by default.
The optional steps below allow using Khoj from within an existing application like Obsidian or Emacs.
Setup host URL
To configure your host URL on your clients when self-hosting, use http://127.0.0.1:42110
. This is the default port for the Khoj server. Note that localhost
will not work.
5. Use Khoj 🚀
You can head to http://localhost:42110 to use the web interface. You can also use the desktop client to search and chat.
Upgrade
Upgrade Khoj Server
- Local Setup
- Docker
- Emacs
- Obsidian
pip install --upgrade khoj-assistant
Note: To upgrade to the latest pre-release version of the khoj server run below command
From the same directory where you have your docker-compose
file, this will fetch the latest build and upgrade your server.
docker-compose up --build
- Use your Emacs Package Manager to Upgrade
- See khoj.el package setup for details
- Upgrade via the Community plugins tab on the settings pane in the Obsidian app
- See the khoj plugin setup for details
Uninstall
Uninstall Khoj Server
- Local Setup
- Docker
- Emacs
- Obsidian
# uninstall khoj server
pip uninstall khoj-assistant
# delete khoj postgres db
dropdb khoj -U postgres
From the same directory where you have your docker-compose
file, run the command below to remove the server to delete its containers, networks, images and volumes.
docker-compose down --volumes
Uninstall the khoj Emacs, or desktop client in the standard way from Emacs or your OS respectively
You can also rm -rf ~/.khoj
to remove the Khoj data directory if did a local install.
Uninstall the khoj Obisidan, or desktop client in the standard way from Obsidian or your OS respectively
You can also rm -rf ~/.khoj
to remove the Khoj data directory if did a local install.
Troubleshoot
Dependency conflict when trying to install Khoj python package with pip
- Reason: When conflicting dependency versions are required by Khoj vs other python packages installed on your system
- Fix: Install Khoj in a python virtual environment using venv or pipx to avoid this dependency conflicts
- Process:
- Install pipx
- Use
pipx
to install Khoj to avoid dependency conflicts with other python packages.pipx install khoj-assistant
- Now start
khoj
using the standard steps described earlier
Install fails while building Tokenizer dependency
- Details:
pip install khoj-assistant
fails while building thetokenizers
dependency. Complains about Rust. - Fix: Install Rust to build the tokenizers package. For example on Mac run:
brew install rustup
rustup-init
source ~/.cargo/env - Refer: Issue with Fix for more details
Khoj in Docker errors out with "Killed" in error message
- Fix: Increase RAM available to Docker Containers in Docker Settings
- Refer: StackOverflow Solution, Configure Resources on Docker for Mac
Footnotes
-
Khoj, by default, can use OpenAI GPT3.5+ chat models or GGUF chat models. See this section on how to locally use OpenAI-format compatible proxy servers. ↩ ↩2