Running Sentient from Source (Self-Host)

This page contains a detailed process for building and running Sentient from the source code on our GitHub repo.

šŸ§° Getting Started (Updated for Web App)

Welcome! This guide will walk you through setting up the Sentient project for local development. Choose your operating system below and follow the steps carefully.

1. Prerequisites

Before you begin, ensure you have the following software installed on your system:

  • Git: To clone the repository.

  • Node.js: v18.x or later. Download here.

  • Python: v3.11 or later. Download here.

  • MongoDB: The community server. Download here. Ensure it's installed and running as a service.

  • Redis: For the Celery message broker.

    • Windows: Install Windows Subsystem for Linux (WSL) and then install Redis inside your WSL distribution (e.g., sudo apt-get install redis-server).

    • Linux/macOS: Install via your package manager (e.g., sudo apt-get install redis-server on Debian/Ubuntu, or brew install redis on macOS).

2. Initial Setup

First, clone the repository and set up the Python and Node.js environments.

# 1. Clone the repository
git clone https://github.com/existence-master/Sentient.git
cd Sentient

# 2. Set up the Python backend
cd src/server
python3 -m venv venv

# Activate the virtual environment
# On Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# On Linux/macOS:
source venv/bin/activate

# Install all Python dependencies
pip install -r requirements.txt
cd ../.. # Return to the project root

# 3. Set up the Next.js frontend
cd src/client
npm install
cd ../.. # Return to the project root

3. Environment Configuration

You need to create two .env files for the backend and frontend. Copy the template files and fill in the required values.

  1. Backend Server:

    • Copy src/server/.env.template to a new file named src/server/.env.

    • Fill in the values. You will need to create an Auth0 application and get API keys for services like Google, GitHub, etc.

  2. Frontend Client:

    • Copy src/client/.env.template to a new file named src/client/.env.

    • Fill in the values. Most of these should match the Auth0 configuration from your backend .env file.

Here are the updated templates to guide you:

Updated `src/server/.env.template`
# --- LLM Provider ---
# Choose your local LLM provider. "OLLAMA" is recommended for local development.
LLM_PROVIDER="OLLAMA"
OLLAMA_BASE_URL="http://localhost:11434"
OLLAMA_MODEL_NAME="qwen3:4b" # or qwen2, etc.
NOVITA_API_KEY=
NOVITA_MODEL_NAME=

# --- Auth0 Configuration ---
# Values from your Auth0 Application and API settings.
AUTH0_DOMAIN=<your-auth0-domain>
AUTH0_AUDIENCE=<your-auth0-api-audience>
AUTH0_MANAGEMENT_CLIENT_ID=<your-auth0-m2m-app-client-id>
AUTH0_MANAGEMENT_CLIENT_SECRET=<your-auth0-m2m-app-client-secret>

# --- Database ---
MONGO_URI="mongodb://localhost:27017/"
MONGO_DB_NAME="sentient_dev_db"

# --- Task Queue (Celery) ---
# For local dev, a local Redis instance is used.
CELERY_BROKER_URL="redis://localhost:6379/0"
CELERY_RESULT_BACKEND="redis://localhost:6379/0"

# --- 3rd Party APIs ---
# You will need to create projects/apps on these platforms to get keys.
NEWS_API_KEY=
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
GOOGLE_API_KEY=
GOOGLE_CSE_ID=
UNSPLASH_ACCESS_KEY=
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=

# --- Encryption ---
# Generate secure random hex strings for these.
# AES_SECRET_KEY: 64 hex characters (32 bytes)
# AES_IV: 32 hex characters (16 bytes)
AES_SECRET_KEY=<generate_a_64_char_hex_string>
AES_IV=<generate_a_32_char_hex_string>

# --- WhatsApp API (WAHA - Optional) ---
WAHA_URL="http://localhost:3000"
WAHA_API_KEY="admin"

# --- MCP Hub Internal URLs (for local development) ---
MEMORY_MCP_SERVER_URL=http://localhost:8001/sse
GMAIL_MCP_SERVER_URL=http://localhost:9001/sse
GCAL_MCP_SERVER_URL=http://localhost:9002/sse
GDRIVE_MCP_SERVER_URL=http://localhost:9003/sse
GDOCS_MCP_SERVER_URL=http://localhost:9004/sse
GOOGLE_SEARCH_MCP_SERVER_URL=http://localhost:9005/sse
SLACK_MCP_SERVER_URL=http://localhost:9006/sse
ACCUWEATHER_MCP_SERVER_URL=http://localhost:9007/sse
QUICKCHART_MCP_SERVER_URL=http://localhost:9008/sse
NOTION_MCP_SERVER_URL=http://localhost:9009/sse
GITHUB_MCP_SERVER_URL=http://localhost:9010/sse
PROGRESS_UPDATER_MCP_SERVER_URL=http://localhost:9011/sse
NEWS_MCP_SERVER_URL=http://localhost:9012/sse
CHAT_TOOLS_MCP_SERVER_URL=http://localhost:9013/sse
GSLIDES_MCP_SERVER_URL=http://localhost:9014/sse
GSHEETS_MCP_SERVER_URL=http://localhost:9015/sse
GMAPS_MCP_SERVER_URL=http://localhost:9016/sse
GSHOPPING_MCP_SERVER_URL=http://localhost:9017/sse
JOURNAL_MCP_SERVER_URL=http://localhost:9018/sse

# --- Supermemory MCP ---
SUPERMEMORY_MCP_BASE_URL=https://mcp.supermemory.ai/
SUPERMEMORY_MCP_ENDPOINT_SUFFIX=/sse
Updated `src/client/.env.template`
# URL of the main FastAPI backend server
NEXT_PUBLIC_APP_SERVER_URL="http://localhost:5000"

# --- Auth0 Configuration ---
# These should match your Auth0 Application settings.
# AUTH0_SECRET should be a long, random, secure string.
AUTH0_SECRET=<generate_a_secure_random_string_for_the_session_cookie>
AUTH0_BASE_URL="http://localhost:3000"
AUTH0_ISSUER_BASE_URL="https://<your-auth0-domain>"
AUTH0_CLIENT_ID=<your-auth0-client-id>
AUTH0_CLIENT_SECRET=<your-auth0-client-secret>
AUTH0_AUDIENCE=<your-auth0-api-audience>
AUTH0_SCOPE='openid profile email offline_access read:chat write:chat read:profile write:profile manage:google_auth read:tasks write:tasks read:notifications read:config write:config admin:user_metadata read:journal write:journal'

4. Running the Application

We provide startup scripts to launch all the necessary services.

Windows (Using PowerShell)

The provided start_all_services.ps1 script will open multiple terminals to run all backend services and the frontend client.

  1. Set Execution Policy (if needed): You may need to allow scripts to run. Open PowerShell as an Administrator and run:

    Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

  2. Run the script: In a regular PowerShell terminal at the project root, simply run:

    .\start_all_services.ps1

    This will:

    • Start MongoDB and Redis.

    • Start all MCP microservices.

    • Start the Celery background workers.

    • Start the main FastAPI API server.

    • Start the Next.js frontend development server.

Linux / macOS (Using Bash)

Create a new script named start_all_services.sh in the root of the project with the content below. This script will run all services in the background and log their output to a logs directory.

#!/bin/bash

# --- Sentient All-in-One Startup Script for Linux/macOS ---
# This script starts all backend services and the frontend client.
# It runs processes in the background and logs their output to a `logs/` directory.

# Exit on any error
set -e

# --- Cleanup function to kill all background jobs on exit ---
cleanup() {
    echo -e "\n\ shutting down all services..."
    # The 'jobs -p' command lists the PIDs of all background jobs.
    # The '-' before the PID in kill sends the signal to the entire process group.
    kill -TERM -- -$(jobs -p) 2>/dev/null
    echo "All services stopped."
}

# Register the cleanup function to run on script exit (including Ctrl+C)
trap cleanup EXIT

# --- Define Paths ---
PROJECT_ROOT=$(pwd)
SRC_DIR="$PROJECT_ROOT/src"
SERVER_DIR="$SRC_DIR/server"
CLIENT_DIR="$SRC_DIR/client"
MCP_HUB_DIR="$SERVER_DIR/mcp-hub"
VENV_ACTIVATE="$SERVER_DIR/venv/bin/activate"

# --- Validate Paths ---
if [ ! -d "$SRC_DIR" ] || [ ! -d "$SERVER_DIR" ] || [ ! -d "$CLIENT_DIR" ]; then
    echo "Error: 'src', 'server', or 'client' directory not found. Please run from the project root."
    exit 1
fi
if [ ! -f "$VENV_ACTIVATE" ]; then
    echo "Error: Python virtual environment not found at '$VENV_ACTIVATE'. Please run the setup steps."
    exit 1
fi

# --- Activate Python Virtual Environment ---
# shellcheck source=/dev/null
source "$VENV_ACTIVATE"
echo "Python virtual environment activated."

# --- Create Log Directory ---
mkdir -p "$PROJECT_ROOT/logs"

# --- Start Databases (Assumes they are running as services) ---
echo -e "\n--- 1. Checking Databases ---"
# Check for Redis
if ! redis-cli ping > /dev/null 2>&1; then
    echo "āš ļø  Redis server not running. Please start it (e.g., 'sudo service redis-server start' or 'redis-server')."
    exit 1
else
    echo "āœ… Redis is running."
fi

# Check for MongoDB
if ! mongosh --eval "db.adminCommand('ping')" > /dev/null 2>&1; then
    echo "āš ļø  MongoDB server not running. Please start it (e.g., 'sudo service mongod start')."
    exit 1
else
    echo "āœ… MongoDB is running."
fi

# --- 2. Start MCP Servers ---
echo -e "\n--- 2. Starting All MCP Servers ---"
for server_path in "$MCP_HUB_DIR"/*/; do
    if [ -d "$server_path" ]; then
        server_name=$(basename "$server_path")
        module_name="server.mcp-hub.$server_name.main"
        echo "šŸš€ Launching MCP - ${server_name^^}..."
        python -m "$module_name" > "$PROJECT_ROOT/logs/mcp-$server_name.log" 2>&1 &
        sleep 0.5
    fi
done

# --- 3. Start Backend Workers ---
echo -e "\n--- 3. Starting Backend Workers ---"
echo "šŸš€ Launching Celery Worker..."
celery -A server.celery_app worker --loglevel=info --pool=solo > "$PROJECT_ROOT/logs/worker-celery.log" 2>&1 &
sleep 0.5

echo "šŸš€ Launching Celery Beat Scheduler..."
celery -A server.celery_app beat --loglevel=info > "$PROJECT_ROOT/logs/worker-beat.log" 2>&1 &
sleep 0.5

# --- 4. Start Main API Server and Frontend Client ---
echo -e "\n--- 4. Starting Main API and Client ---"
echo "šŸš€ Launching Main API Server..."
python -m server.main.app > "$PROJECT_ROOT/logs/api-main.log" 2>&1 &
sleep 3

echo "šŸš€ Launching Next.js Client..."
(cd "$CLIENT_DIR" && npm run dev) > "$PROJECT_ROOT/logs/client-nextjs.log" 2>&1 &

echo -e "\nāœ… All services are starting in the background."
echo "   - View logs in the 'logs/' directory."
echo "   - Press Ctrl+C to stop all services."

# Wait for all background processes to finish (which they won't, until you press Ctrl+C)
wait

How to run the script:

  1. Save the content above as start_all_services.sh in the root of your project.

  2. Make the script executable:

    chmod +x start_all_services.sh

  3. Run the script:

    ./start_all_services.sh
PreviousWelcome to SentientNextOnboarding

Last updated

Was this helpful?