Skip to main content

Self-Host

Learn about how to self-host Khoj on your own machine.

Benefits to self-hosting:

  1. Privacy: Your data will never have to leave your private network. You can even use Khoj without an internet connection if deployed on your personal computer.
  2. Customization: You can customize Khoj to your liking, from models, to host URL, to feature enablement.

Setup​

These are the general setup instructions for self-hosted Khoj. You can install the Khoj server using either Docker or Pip.

Offline Model + GPU

If you want to use the offline chat model and you have a GPU, you should use Installation Option 2 - local setup via the Python package directly. Our Docker image doesn't currently support running the offline chat model on GPU, making inference times really slow.

1A. Install Method 1: Docker​

Prerequisites​

  1. Install Docker Engine. See official instructions.
  2. Ensure you have Docker Compose. See official instructions.

Setup​

  1. Get the sample docker-compose file from Github.
  2. Configure the environment variables in the docker-compose.yml to your choosing.
    Note: Your admin account will automatically be created based on the admin credentials in that file, so pay attention to those.
  3. Now start the container by running the following command in the same directory as your docker-compose.yml file. This will automatically setup the database and run the Khoj server. 
    docker-compose up

Khoj should now be running at http://localhost:42110! You can see the web UI in your browser.

1B. Install Method 2: Pip​

Prerequisites​

Install Postgres (with PgVector)​

Khoj uses Postgres DB for all server configuration and to scale to multi-user setups. It uses the pgvector package in Postgres to manage your document embeddings. Both Postgres and pgvector need to be installed for Khoj to work.

Install Postgres.app. This comes pre-installed with pgvector and relevant dependencies.

Create the Khoj database​

Make sure to update your environment variables to match your Postgres configuration if you're using a different name. The default values should work for most people. When prompted for a password, you can use the default password postgres, or configure it to your preference. Make sure to set the environment variable POSTGRES_PASSWORD to the same value as the password you set here.

createdb khoj -U postgres --password

Install Khoj server​

Install Khoj Server​

Run the following command in your terminal to install the Khoj server.

# ARM/M1+ Machines
MAKE_ARGS="-DLLAMA_METAL=on" python -m pip install khoj

# Intel Machines
python -m pip install khoj
Start Khoj Server​

Before getting started, configure the following environment variables in your terminal for the first run

export KHOJ_ADMIN_EMAIL=<your-email>
export KHOJ_ADMIN_PASSWORD=<your-password>

Run the following command from your terminal to start the Khoj backend and open Khoj in your browser.

khoj --anonymous-mode

--anonymous-mode allows you to run the server without setting up Google credentials for login. This allows you to use any of the clients without a login wall. If you want to use Google login, you can skip this flag, but you will have to add your Google developer credentials.

On the first run, you will be prompted to input credentials for your admin account and do some basic configuration for your chat model settings. Once created, you can go to http://localhost:42110/server/admin and login with the credentials you just created.

Khoj should now be running at http://localhost:42110. You can see the web UI in your browser.

Note: To start Khoj automatically in the background use Task scheduler on Windows or Cron on Mac, Linux (e.g. with @reboot khoj)

2. Configure​

Login to the Khoj Admin Panel​

Go to http://localhost:42110/server/admin and login with the admin credentials you setup during installation.

CSRF Error

Ensure you are using localhost, not 127.0.0.1, to access the admin panel to avoid the CSRF error.

DISALLOWED HOST Error

You may hit this if you try access Khoj exposed on a custom domain (e.g. 192.168.12.3 or example.com) or over HTTP. Set the environment variables KHOJ_DOMAIN=your-domain and KHOJ_NO_HTTPS=false if required to avoid this error.

Note

Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.

Configure Chat Model​

Setup which chat model you'd want to use. Khoj supports local and online chat models.

Multiple Chat Models

Add a ServerChatSettings with Default and Summarizer fields set to your preferred chat model via the admin panel. Otherwise Khoj defaults to use the first chat model in your ChatModelOptions for all non chat response generation tasks.

Configure OpenAI Chat​
Ollama Integration

Using Ollama? See the Ollama Integration section for more custom setup instructions.

  1. Go to the OpenAI settings in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key and server API base URL. The API base URL is optional - it's only relevant if you're using another OpenAI-compatible proxy server.

example configuration for openai processor

  1. Go over to configure your chat model options. Set the chat-model field to a supported chat model1 of your choice. For example, you can specify gpt-4o if you're using OpenAI.
    • Make sure to set the model-type field to OpenAI.
    • The tokenizer and max-prompt-size fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.
    • If your model supports vision, set the vision enabled field to true. This is currently only supported for OpenAI models with vision capabilities.

example configuration for chat model options

Configure Anthropic Chat​
  1. Go to the OpenAI settings in the server admin settings to add an OpenAI processor conversation config. This is kind of a misnomer, we know. Do not configure the API base url. Just add your API key and give the configuration a friendly name.
  2. Go over to configure your chat model options. Set the chat-model field to a supported chat model by Anthropic of your choice. For example, you can specify claude-3-5-sonnet-20240620.
    • Make sure to set the model-type field to Anthropic.
    • The tokenizer and max-prompt-size fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.
Configure Offline Chat​

Offline chat stays completely private and can work without internet using open-source models.

System Requirements:

  • Minimum 8 GB RAM. Recommend 16Gb VRAM
  • Minimum 5 GB of Disk available
  • A CPU supporting AVX or AVX2 instructions is required
  • An Nvidia, AMD GPU or a Mac M1+ machine would significantly speed up chat response times

Any chat model on Huggingface in GGUF format can be used for local chat. Here's how you can set it up:

  1. No need to setup a conversation processor config!
  2. Go over to configure your chat model options. Set the chat-model field to a supported chat model1 of your choice. For example, we recommend bartowski/Meta-Llama-3.1-8B-Instruct-GGUF, but any gguf model on huggingface should work.
  • Make sure to set the model-type to Offline. Do not set openai config.
  • The tokenizer and max-prompt-size fields are optional. You can set these for non-standard models (i.e not Mistral or Llama based models) or when you know the token limit of the model to improve context stuffing.

Share your data​

You can sync your files and folders with Khoj using the Desktop, Obsidian, or Emacs clients or just drag and drop specific files on the website. You can also directly sync your Notion workspace.

3. Use Khoj 🚀​

Now open http://localhost:42110 to start interacting with Khoj!

4. Install Khoj Clients (Optional)​

Khoj exposes a web interface to search, chat and configure by default.
You can install a Khoj client to sync your documents or to easily access Khoj from within Obsidian, Emacs or your OS.

  • Khoj Desktop:
    Install the Khoj Desktop app.

  • Khoj Obsidian:
    Install the Khoj Obsidian plugin.

  • Khoj Emacs:
    Install khoj.el

Setup host URL​

Set the host URL on your clients settings page to your Khoj server URL. By default, use http://127.0.0.1:42110 or http://localhost:42110. Note that localhost may not work in all cases.

Upgrade​

pip install --upgrade khoj

Note: To upgrade to the latest pre-release version of the khoj server run below command

Uninstall​

# uninstall khoj server
pip uninstall khoj

# delete khoj postgres db
dropdb khoj -U postgres

Troubleshoot​

Dependency conflict when trying to install Khoj python package with pip​

  • Reason: When conflicting dependency versions are required by Khoj vs other python packages installed on your system
  • Fix: Install Khoj in a python virtual environment using venv or pipx to avoid this dependency conflicts
  • Process:
    1. Install pipx
    2. Use pipx to install Khoj to avoid dependency conflicts with other python packages. 
      pipx install khoj
    3. Now start khoj using the standard steps described earlier

Install fails while building Tokenizer dependency​

  • Details: pip install khoj fails while building the tokenizers dependency. Complains about Rust.
  • Fix: Install Rust to build the tokenizers package. For example on Mac run: 
    brew install rustup
    rustup-init
    source ~/.cargo/env
  • Refer: Issue with Fix for more details

Khoj in Docker errors out with "Killed" in error message​

Advanced​

Self Host on Custom Domain​

You can self-host Khoj behind a custom domain as well. To do so, you need to set the KHOJ_DOMAIN environment variable to your domain (e.g., export KHOJ_DOMAIN=my-khoj-domain.com or add it to your docker-compose.yml). By default, the Khoj server you set up will not be accessible outside of localhost or 127.0.0.1.

Without HTTPS certificate

To expose Khoj on a custom domain over the public internet, use of an SSL certificate is strongly recommended. You can use Let's Encrypt to get a free SSL certificate for your domain.

To disable HTTPS, set the KHOJ_NO_HTTPS environment variable to True. This can be useful if Khoj is only accessible behind a secure, private network.

Footnotes​

  1. Khoj, by default, can use OpenAI GPT3.5+ chat models or GGUF chat models. See this section on how to locally use OpenAI-format compatible proxy servers. ↩ ↩2