Plan: Secondary RAG-Dedicated Project Migration

1. Objective

To bypass the "Spanner mode" restriction currently blocking serverless RAG Engine corpus creation in propane-will-491900-m5. We will spin up a clean, dedicated project for RAG experimentation and wire OSVauco to it as a secondary data source.

2. New Project Setup

Project Selection

Suggested Name: vauco-rag-lab-01 or osvauco-memory-node.
Organization: Ensure it is created under the same billing account as the primary project.

Step 1: Bootstrap Command

NEW_RAG_PROJECT="vauco-rag-lab-01"
BILLING_ACCOUNT=$(gcloud billing projects describe propane-will-491900-m5 --format="value(billingAccountName)")

# Create project
gcloud projects create ${NEW_RAG_PROJECT} --name="OSVauco RAG Lab"

# Link billing
gcloud billing projects link ${NEW_RAG_PROJECT} --billing-account=${BILLING_ACCOUNT}

Step 2: Minimal APIs

Enable only what is needed for RAG Engine and its data source:

gcloud services enable \
  aiplatform.googleapis.com \
  storage.googleapis.com \
  --project=${NEW_RAG_PROJECT}

Step 3: Cross-Project IAM

Grant the primary OSVauco Agent (from propane-will-491900-m5) access to the new RAG project:

PRIMARY_SA="jason.vauger@propane-will-491900-m5.iam.gserviceaccount.com"

# Grant Vertex AI User and Storage Admin on the NEW project
for ROLE in roles/aiplatform.user roles/storage.objectAdmin; do
  gcloud projects add-iam-policy-binding ${NEW_RAG_PROJECT} \
    --member="serviceAccount:${PRIMARY_SA}" \
    --role="${ROLE}"
done

3. RAG Setup & Sync Refactoring

We will use variants of our existing scripts, but modified to accept a --project override.

`setup_corpus_remote.py`

A variant of setup_corpus.py that:

Targets NEW_RAG_PROJECT.
Attempts the same "Serverless" PATCH in us-central1 or europe-west4.
If it succeeds, it saves the resulting corpus name to a shared secret in the Primary project.

`sync_corpus_remote.py`

A variant of sync_corpus.py that reads from the Primary project's GCS bucket but writes/indexes to the Remote project's RAG Engine.

4. Dual-Project Configuration Strategy

OSVauco will be updated to handle "Project Context Switching."

Environment Variables

The following overrides will be added to .env:

PROJECT_ID: propane-will-491900-m5 (Primary logic/infra)
RAG_PROJECT_ID: vauco-rag-lab-01 (Knowledge base source)
RAG_LOCATION: europe-west4 (Region for the remote corpus)

Code Updates

agents/rag/*: All RAG scripts and agent tools will default to RAG_PROJECT_ID (falling back to PROJECT_ID) during vertexai.init().
scripts/preflight.sh: Will check both projects.
- Infrastructure section -> Primary Project.
- Knowledge Base section -> Remote Project.
scripts/cost-lens.sh: Will aggregate costs from both projects if HEAVY_MODE is enabled.

5. Migration Timeline

Project Creation: Create vauco-rag-lab-01 and link billing.
IAM Wiring: Establish the cross-project service account trust.
Test Run: Run setup_corpus.py targeting the new project to verify if "Spanner mode" restriction is absent.
Integration: Update .env and preflight.sh to recognize the secondary project.

Senior Staff Engineer (Gemini CLI)

3.3 KiB Raw Blame History