feat: push all remaining files — agents/, docs/, architecture/, cloudbuild.yaml, README

This commit is contained in:
chrischristiansen-glitch 2026-05-23 02:37:39 +02:00
parent 5a33e8f444
commit 34ba75a4a2
20 changed files with 1231 additions and 15 deletions

42
.gitignore vendored Normal file
View File

@ -0,0 +1,42 @@
# Python
__pycache__/
*.py[cod]
*.pyo
*.pyd
.Python
build/
dist/
*.egg-info/
.eggs/
# Virtual environments
.venv/
venv/
env/
.env
# Credentials — NEVER commit these
*.json
credentials/
*.key
*.pem
service_account*.json
application_default_credentials.json
# GCP / ADK
.gcloud/
adk_sessions/
# IDE
.vscode/
.idea/
*.swp
*.swo
# OS
.DS_Store
Thumbs.db
# Temp
/tmp/
*.log

146
README.md
View File

@ -1,22 +1,138 @@
# OSVauco
# OSVauco — GCP Agent Master Repo
Main operator workspace for OS-Vauco, focused on OPAX-MCP and Gemini TUI in GCP project `propane-will-491900-m5` (region `us-central1`).
Prosjekt: `propane-will-491900-m5` | Region: `us-central1`
## Repositories
Repository for ADK 2.0-baserte agenter på Gemini Enterprise Agent Platform (tidl. Vertex AI Agent Engine).
- **OSVauco (this repo)** operator hub: scripts, governance docs, session log.
- **vauco-gemini-tui-bridge** Textual TUI + FastAPI WebSocket bridge for Gemini LLM.
- **vauco-bootstrap** skill templates and higher-level automation flows.
---
## Daily flow
## Repo-struktur
From Cloud Shell:
```bash
cd ~/OSVauco
./scripts/osvauco-startup.sh # auth + ADC + .env sync
./scripts/osvauco-opax-boot.sh # OPAX-MCP ping + optional Gemini TUI entry
./scripts/osvauco-health-check.sh # sanity-check git/disk/auth
```
├── agents/
│ ├── core-logic/ # Hoved-agent: agent.py, __init__.py, deploy_agent.py
│ │ ├── requirements.txt # Pinnede avhengigheter
│ │ ├── Dockerfile # Cloud Run containerisering
│ │ └── .env.example # Lokal dev — kopier til .env
│ ├── multi_agent/ # Orchestrator + sub-agents
│ ├── tools/ # MCP-integrasjoner (BigQuery, Maps)
│ ├── rag/ # RAG corpus setup
│ ├── memory/ # Memory Bank + Sessions setup
│ ├── eval/ # CI/CD eval-gate (groundedness ≥ 0.8)
│ └── tests/ # Lokal test-runner (ADK dev server)
├── infrastructure/
│ ├── 00-authcheck.sh # Verifiser gcloud-identitet og prosjekt
│ ├── 01-setupenv.sh # APIs, bucket, lifecycle, SA, IAM, billing budget
│ ├── 02-deploy.sh # Deploy til Agent Runtime (ADK Python SDK)
│ ├── 03-teardown.sh # Slett Agent Runtimes + cleanup (kostnadsbeskyttelse)
│ ├── 04-observability-setup.sh # Monitoring, Logging, Trace APIs
│ ├── 05-cloudrun-deploy.sh # Deploy til Cloud Run via `adk deploy cloud_run`
│ ├── 06-cicd-setup.sh # Cloud Build trigger + Artifact Registry
│ ├── 07-rag-setup.sh # RAG corpus create + import wrapper
│ └── 08-memorybank-setup.sh # Memory Bank instance + IAM
├── docs/
│ ├── GCPBestPractices.md # Platform, ADK 2.0, deploy targets, modeller
│ ├── CostManagementRules.md # Priser, lifecycle-regler, teardown-policy
│ └── IAMRolesandPermissions.md # Roller, SA, ADC, Agent Identity
├── architecture/
│ ├── agentworkflowdiagrams.md # Mermaid-diagrammer: deploy, multi-agent, CI/CD
│ └── dataflowsecurity.md # Auth, secrets, guardrails, Agent Gateway
├── cloudbuild.yaml # CI/CD pipeline
└── README.md
```
Session history is stored in `docs/OSVAUCO_OPAX_SESSION_LOG.md`. Governance rules live in `docs/AGENT_RULEBOOK.md`.
---
## Dag-1 oppsett
### Forutsetninger
- `gcloud` CLI installert og logget inn
- Python 3.12+
- Docker (for Cloud Run deploys)
### Steg 1 — Auth
```bash
gcloud auth login
gcloud auth application-default login
gcloud config set project propane-will-491900-m5
```
### Steg 2 — Environment setup
```bash
bash infrastructure/01-setupenv.sh
# Følg instruksjonene om billing budget (krever billing account ID)
# Finn med: gcloud billing accounts list
```
### Steg 3 — Lokal kjøring
```bash
cd agents/core-logic
cp .env.example .env # fyll inn variabler
pip install -r requirements.txt
adk web . # åpner dev UI på http://localhost:8080
```
### Steg 4 — Deploy til Cloud Run
```bash
bash infrastructure/05-cloudrun-deploy.sh
```
### Steg 5 — Deploy til Agent Runtime (managed)
```bash
bash infrastructure/02-deploy.sh
# HUSK: Kjør teardown på slutten av dagen!
bash infrastructure/03-teardown.sh
```
---
## Hurtigreferanse-kommandoer
| Handling | Kommando |
|---|---|
| Auth check | `bash infrastructure/00-authcheck.sh` |
| Setup env | `bash infrastructure/01-setupenv.sh` |
| Deploy Agent Runtime | `bash infrastructure/02-deploy.sh` |
| **Teardown (viktig!)** | `bash infrastructure/03-teardown.sh` |
| Deploy Cloud Run | `bash infrastructure/05-cloudrun-deploy.sh` |
| Setup RAG corpus | `python agents/rag/setup_corpus.py` |
| Setup Memory Bank | `python agents/memory/memory_setup.py` |
| Kjør eval | `python agents/eval/run_eval.py` |
| Lokal test | `bash agents/tests/test_local.sh` |
| Finn billing account | `gcloud billing accounts list` |
| Liste aktive Cloud Run services | `gcloud run services list --region=us-central1` |
| Slett Cloud Run service | `gcloud run services delete oavauco-agent-v1 --region=us-central1` |
---
## Kostnadsbeskyttelse — viktigste regler
1. **Kjør alltid `03-teardown.sh` på slutten av arbeidsdagen** — Agent Runtimes faktureres.
2. Sett billing budget alert før første deploy (`01-setupenv.sh` har stub for dette).
3. Cloud Run skalerer til 0 — ingen idle-kost med `--min-instances=0`.
4. RAG Engine med Spanner → 24/7 fakturering. Bruk `us-east1` for dev/test.
5. Sessions, Memory Bank og Code Execution er metered fra 28. jan 2026.
---
## CI/CD
`cloudbuild.yaml` kjøres automatisk ved push til `main`:
1. Install dependencies + unit tests
2. Eval gate (groundedness ≥ 0.8 — feiler bygget ellers)
3. Build Docker image
4. Push til Artifact Registry
5. Deploy til Cloud Run
Oppsett: `bash infrastructure/06-cicd-setup.sh`
(Krever at GitHub-repo er koblet til Cloud Build via GCP Console først.)
---
## Sikkerhetsregler
- Ingen JSON-nøkkelfiler i repo — bruk ADC / Workload Identity.
- Alle hemmeligheter i Secret Manager (`gcloud secrets create ...`).
- `.env` er i `.gitignore` — aldri commit.
- Cloud Run kjører med `--no-allow-unauthenticated`.
- ADK-callbacks blokkerer prompt injection og destruktive tool-args.

View File

@ -0,0 +1,25 @@
# .env.example — Copy to .env for local development
# NEVER commit .env to Git. Add .env to .gitignore.
# GCP project and region (required)
GOOGLE_CLOUD_PROJECT=propane-will-491900-m5
GOOGLE_CLOUD_LOCATION=us-central1
# Authentication mode: True = Vertex AI (ADC), False = Gemini API key
GOOGLE_GENAI_USE_VERTEXAI=True
# Only set if GOOGLE_GENAI_USE_VERTEXAI=False (local testing without gcloud)
# GOOGLE_API_KEY=your-api-key-here
# RAG corpus resource name (from setup_corpus.py output)
# RAG_CORPUS=projects/propane-will-491900-m5/locations/us-central1/ragCorpora/XXXXXXXXX
# Agent Engine ID (from memory_setup.py output)
# AGENT_ENGINE_ID=your-agent-engine-id
# MCP server URLs (if using Google managed MCP)
# MAPS_MCP_URL=https://maps.googleapis.com/mcp
# BIGQUERY_MCP_URL=https://bigquery.googleapis.com/mcp
# Maps API key (for Google Maps MCP)
# MAPS_API_KEY=your-maps-api-key

View File

@ -0,0 +1,19 @@
# Dockerfile for OSVauco ADK agent — Cloud Run deployment
# Note: `adk deploy cloud_run` manages this automatically.
# Use this Dockerfile only for manual `gcloud run deploy --source .` deploys.
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
ENV PORT=8080
ENV GOOGLE_GENAI_USE_VERTEXAI=True
EXPOSE 8080
CMD ["adk", "web", "--host", "0.0.0.0", "--port", "8080", "."]

View File

@ -0,0 +1,2 @@
# Required by ADK: exports root_agent for adk web and adk deploy commands
from .agent import root_agent

View File

@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""
agent.py OSVauco root agent using ADK 2.0.
Requires: google-adk >= 1.29, google-cloud-aiplatform >= 1.111.0
"""
import os
from google.adk.agents import Agent
from google.adk.integrations.secret_manager.secret_client import SecretManagerClient
from vertexai.preview import rag
PROJECT_ID = "propane-will-491900-m5"
LOCATION = "us-central1"
# --- Secret Manager integration (ADK >= 1.29) ---
_sm = SecretManagerClient()
def _get_secret(name: str) -> str:
return _sm.get_secret(
f"projects/{PROJECT_ID}/secrets/{name}/versions/latest"
)
# --- RAG retrieval tool ---
RAG_CORPUS = os.environ.get("RAG_CORPUS", "") # set via Secret Manager or env
rag_tool = None
if RAG_CORPUS:
from google.adk.tools import VertexAiRagRetrieval
rag_tool = VertexAiRagRetrieval(
name="retrieve_knowledge",
description="Retrieve relevant documentation from the knowledge base.",
rag_resources=[rag.RagResource(rag_corpus=RAG_CORPUS)],
similarity_top_k=10,
vector_distance_threshold=0.6,
)
# --- Root agent ---
root_agent = Agent(
model="gemini-2.5-flash", # cost-optimized default
name="oavauco_root",
description="OSVauco enterprise agent for propane-will-491900-m5",
instruction=(
"You are OSVauco, a GCP knowledge and workflow agent. "
"Use the retrieve_knowledge tool to answer questions from the knowledge base. "
"Always prefer grounded, documented answers over speculation."
),
tools=[rag_tool] if rag_tool else [],
)

View File

@ -0,0 +1,38 @@
#!/usr/bin/env python3
"""
deploy_agent.py Deploy ADK agent to Gemini Enterprise Agent Platform (Agent Runtime)
Usage: python3 deploy_agent.py --project PROJECT_ID --region REGION \
--display-name DISPLAY_NAME --staging-bucket GS_URI
"""
import argparse
import vertexai
from vertexai.preview import agent_engines # SDK v1.112.0+ client-based design
def deploy(project: str, region: str, display_name: str, staging_bucket: str):
vertexai.init(project=project, location=region, staging_bucket=staging_bucket)
from agents.core_logic import agent as my_agent # adjust import path
print(f"Deploying agent '{display_name}' to Agent Runtime in {region}...")
remote_agent = agent_engines.create(
my_agent.root_agent,
requirements=["google-cloud-aiplatform[adk,agent_engines]>=1.112.0"],
display_name=display_name,
)
print(f"Agent deployed. Resource name: {remote_agent.resource_name}")
print(f"Console: https://console.cloud.google.com/ai/agents?project={project}")
print("REMINDER: Run 03-teardown.sh at end of day to stop billing.")
return remote_agent
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--project", default="propane-will-491900-m5")
parser.add_argument("--region", default="us-central1")
parser.add_argument("--display-name", default="oavauco-agent-v1")
parser.add_argument("--staging-bucket",
default="gs://propane-will-491900-m5-agent-staging")
args = parser.parse_args()
deploy(args.project, args.region, args.display_name, args.staging_bucket)

View File

@ -0,0 +1,33 @@
# OSVauco agent dependencies — pin versions to prevent surprise upgrades
# ADK + Vertex AI SDK (must match)
google-adk>=2.0.0
google-cloud-aiplatform[adk,agent_engines]>=1.112.0
# RAG Engine
vertexai>=1.71.0
# Secret Manager integration (requires ADK >= 1.29)
google-cloud-secret-manager>=2.20.0
# MCP toolset
httpx>=0.27.0
google-auth>=2.29.0
google-auth-httplib2>=0.2.0
# Observability
opentelemetry-sdk>=1.25.0
opentelemetry-exporter-gcp-trace>=1.8.0
google-cloud-logging>=3.10.0
google-cloud-monitoring>=2.22.0
# Evaluation
google-cloud-aiplatform[evaluation]>=1.112.0
# HTTP server for Cloud Run
uvicorn[standard]>=0.29.0
fastapi>=0.111.0
# Utilities
python-dotenv>=1.0.0
pydantic>=2.7.0

51
agents/eval/run_eval.py Normal file
View File

@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""
run_eval.py Run agent evaluation using Vertex AI Gen AI Evaluation Service.
Metrics: coherence, groundedness, tool_call_quality
CI/CD gate: fails build if groundedness < 0.8
"""
import vertexai
from vertexai.evaluation import EvalTask
PROJECT_ID = "propane-will-491900-m5"
LOCATION = "us-central1"
vertexai.init(project=PROJECT_ID, location=LOCATION)
EVAL_DATASET = [
{
"prompt": "What GCP region should all resources use?",
"reference": "us-central1",
},
{
"prompt": "What command tears down all Agent Runtimes?",
"reference": "Run 03-teardown.sh",
},
{
"prompt": "What ADK version is required for Memory Bank?",
"reference": "google-adk >= 2.0.0",
},
]
METRICS = ["coherence", "groundedness", "tool_call_quality"]
eval_task = EvalTask(
dataset=EVAL_DATASET,
metrics=METRICS,
experiment="oavauco-agent-eval",
)
result = eval_task.evaluate(
model="gemini-2.5-flash",
prompt_template="{prompt}",
)
print(result.summary_metrics)
# CI/CD gate
if result.summary_metrics.get("groundedness/mean", 1.0) < 0.8:
print("EVAL FAILED: groundedness below 0.8 threshold")
raise SystemExit(1)
print("EVAL PASSED")

View File

@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""
memory_setup.py Initialize Agent Engine instance with Sessions + Memory Bank.
Project: propane-will-491900-m5 | Region: us-central1
SDK: google-cloud-aiplatform >= 1.111.0
COST NOTE: Sessions + Memory Bank are metered since Jan 28, 2026.
Trim session histories. Only persist high-value facts to long-term memory.
"""
import vertexai
from vertexai import Client
from google.adk.memory import VertexAiMemoryBankService
from google.adk.sessions import VertexAiSessionService
PROJECT_ID = "propane-will-491900-m5"
LOCATION = "us-central1"
def create_agent_engine() -> str:
"""Create an Agent Engine instance (backing store for Sessions + Memory Bank)."""
client = Client(project=PROJECT_ID, location=LOCATION)
agent_engine = client.agent_engines.create()
resource_name = agent_engine.api_resource.name
agent_engine_id = resource_name.split("/")[-1]
print(f"Agent Engine created: {resource_name}")
print(f"Agent Engine ID: {agent_engine_id}")
print("Store this in Secret Manager or env var: AGENT_ENGINE_ID")
return agent_engine_id
def get_services(agent_engine_id: str):
"""Return configured session and memory services for use with ADK Runner."""
memory_service = VertexAiMemoryBankService(
project=PROJECT_ID,
location=LOCATION,
agent_engine_id=agent_engine_id,
)
session_service = VertexAiSessionService(
project_id=PROJECT_ID,
location=LOCATION,
agent_engine_id=agent_engine_id,
)
return session_service, memory_service
if __name__ == "__main__":
agent_engine_id = create_agent_engine()
session_svc, memory_svc = get_services(agent_engine_id)
print("Services ready.")
print(f"session_service: {session_svc}")
print(f"memory_service: {memory_svc}")

View File

@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""
orchestrator.py OSVauco multi-agent orchestrator using ADK 2.0 collaborative workflow.
Architecture:
root_agent (coordinator / LLM-driven delegation)
rag_agent retrieves answers from private knowledge base
gcp_ops_agent handles GCP operations questions and script generation
memory_agent loads/stores long-term memory via Memory Bank
farewell_agent session closings
Pattern: Coordinator with sub_agents list.
Requires: google-adk >= 2.0.0
"""
from __future__ import annotations
import os
import logging
from typing import Optional
from google.adk.agents import Agent
from google.adk.agents.callback_context import CallbackContext
from google.adk.tools.load_memory_tool import load_memory_tool
from google.adk.tools.preload_memory_tool import preload_memory_tool
from google.genai.types import Content, Part
logger = logging.getLogger(__name__)
PROJECT_ID = os.environ.get("PROJECT_ID", "propane-will-491900-m5")
REGION = os.environ.get("REGION", "us-central1")
RAG_CORPUS_NAME = os.environ.get("RAG_CORPUS_NAME", "")
ORCHESTRATOR_MODEL = os.environ.get("ORCHESTRATOR_MODEL", "gemini-2.5-flash")
SUBAGENT_MODEL = os.environ.get("SUBAGENT_MODEL", "gemini-2.5-flash")
# ── Safety callbacks ──────────────────────────────────────────────────────────
BLOCKED_PATTERNS = [
"ignore previous instructions",
"ignore all instructions",
"drop table",
"system prompt",
"jailbreak",
"disregard safety",
]
def before_model_callback(callback_context: CallbackContext, llm_request) -> Optional[Content]:
try:
user_text = ""
if llm_request.contents:
last = llm_request.contents[-1]
if last.parts:
user_text = last.parts[0].text.lower()
for pattern in BLOCKED_PATTERNS:
if pattern in user_text:
logger.warning("Blocked pattern detected: '%s'", pattern)
return Content(parts=[Part(text="I cannot process that request. Please rephrase.")])
except Exception as exc:
logger.error("before_model_callback error: %s", exc)
return None
def before_tool_callback(tool, args: dict, tool_context) -> Optional[dict]:
dangerous = ["DROP", "DELETE FROM", "TRUNCATE", "--", ";"]
for val in args.values():
if isinstance(val, str):
for d in dangerous:
if d.upper() in val.upper():
raise ValueError(f"Tool argument rejected by safety guardrail: '{val}'")
return None
# ── Sub-agents ────────────────────────────────────────────────────────────────
rag_agent = Agent(
model=SUBAGENT_MODEL,
name="rag_agent",
description="Retrieves answers from the private OSVauco knowledge base (RAG corpus).",
instruction=(
"You are a knowledge retrieval specialist. "
"Use search_knowledge_base to find relevant information and return well-cited answers."
),
before_model_callback=before_model_callback,
)
gcp_ops_agent = Agent(
model=SUBAGENT_MODEL,
name="gcp_ops_agent",
description="Answers GCP operations questions: scripts, IAM, billing, Cloud Run, ADK deployments.",
instruction=(
"You are a GCP operations expert for project propane-will-491900-m5 in us-central1. "
"Provide accurate gcloud CLI commands, IAM patterns, and ADK deployment guidance. "
"Always include cost-safety reminders (teardown, billing budgets)."
),
before_model_callback=before_model_callback,
)
memory_agent = Agent(
model=SUBAGENT_MODEL,
name="memory_agent",
description="Manages long-term memory: loads past context and stores new facts for future sessions.",
instruction=(
"You manage the agent's long-term memory. "
"Use load_memory_tool to retrieve past facts and preload_memory_tool to store important new facts. "
"Only persist high-value, factual information — not transient conversation."
),
tools=[load_memory_tool, preload_memory_tool],
before_model_callback=before_model_callback,
)
farewell_agent = Agent(
model=SUBAGENT_MODEL,
name="farewell_agent",
description="Handles session closings, summaries, and goodbye messages.",
instruction="Generate a concise, friendly session summary and closing message.",
)
# ── Root orchestrator ─────────────────────────────────────────────────────────
root_agent = Agent(
model=ORCHESTRATOR_MODEL,
name="oavauco_orchestrator",
description="OSVauco root orchestrator — delegates to specialist sub-agents.",
instruction=(
"You are the OSVauco orchestrator for project propane-will-491900-m5. "
"Delegate to sub-agents based on the user's intent:\n"
"- Knowledge base questions → rag_agent\n"
"- GCP operations, scripts, IAM, billing → gcp_ops_agent\n"
"- Memory recall or storage → memory_agent\n"
"- Session endings → farewell_agent\n"
"Always synthesize sub-agent responses into a clear, concise final answer."
),
sub_agents=[rag_agent, gcp_ops_agent, memory_agent, farewell_agent],
before_model_callback=before_model_callback,
before_tool_callback=before_tool_callback,
)

View File

@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""
setup_corpus.py Create a Vertex AI RAG Engine corpus and import documents.
Project: propane-will-491900-m5
IMPORTANT: RAG Engine in us-central1 requires allowlist access.
Contact: vertex-ai-rag-engine-support@google.com
Alternative: Use us-east1 or us-east4 for immediate access.
"""
import os
import vertexai
from vertexai import rag
PROJECT_ID = "propane-will-491900-m5"
LOCATION = os.environ.get("RAG_LOCATION", "us-east1") # change to us-central1 after allowlist
CORPUS_DISPLAY_NAME = os.environ.get("RAG_CORPUS_NAME", "oavauco-knowledge-base")
GCS_SOURCE = os.environ.get(
"RAG_GCS_SOURCE",
f"gs://{PROJECT_ID}-agent-staging/rag-docs/"
)
def main():
vertexai.init(project=PROJECT_ID, location=LOCATION)
# Check if corpus already exists
existing = list(rag.list_corpora())
for c in existing:
if c.display_name == CORPUS_DISPLAY_NAME:
print(f"Corpus '{CORPUS_DISPLAY_NAME}' already exists: {c.name}")
corpus = c
break
else:
print(f"Creating RAG corpus '{CORPUS_DISPLAY_NAME}' in {LOCATION}...")
corpus = rag.create_corpus(
display_name=CORPUS_DISPLAY_NAME,
backend_config=rag.RagVectorDbConfig(
rag_embedding_model_config=rag.RagEmbeddingModelConfig(
vertex_prediction_endpoint=rag.VertexPredictionEndpoint(
publisher_model="publishers/google/models/text-embedding-005"
)
)
),
)
print(f"Corpus created: {corpus.name}")
print(f"Importing files from {GCS_SOURCE}...")
rag.import_files(
corpus.name,
paths=[GCS_SOURCE],
transformation_config=rag.TransformationConfig(
chunking_config=rag.ChunkingConfig(
chunk_size=512,
chunk_overlap=100,
)
),
)
print("Import complete.")
print(f"\nRAG_CORPUS={corpus.name}")
print("Set this as an environment variable or Secret Manager entry.")
# Test retrieval
response = rag.retrieval_query(
rag_resources=[rag.RagResource(rag_corpus=corpus.name)],
text="test query",
rag_retrieval_config=rag.RagRetrievalConfig(top_k=3),
)
print(f"Test retrieval returned {len(response.contexts.contexts)} chunks.")
if __name__ == "__main__":
main()

View File

@ -0,0 +1,13 @@
#!/bin/bash
# test_local.sh — Run ADK agent locally before deploying to cloud
set -euo pipefail
echo "=== OSVauco-NMTMD-GCOS :: Local Agent Test ==="
gcloud auth application-default print-access-token >/dev/null 2>&1 || {
echo "ERROR: No ADC. Run: gcloud auth application-default login"; exit 1; }
echo "Starting ADK dev server at http://localhost:8080 ..."
echo "Press Ctrl+C to stop."
adk web agents/core-logic/

64
agents/tools/mcp_tools.py Normal file
View File

@ -0,0 +1,64 @@
"""
mcp_tools.py MCP toolset integrations for OSVauco ADK agent.
Supports:
- Google BigQuery MCP (via StreamableHTTP, uses ADC/OAuth)
- Google Maps MCP (via StreamableHTTP, uses API key)
- Local stdio MCP server (dev/test only)
Requires: google-adk >= 2.0.0, google-auth
"""
import os
import google.auth
import google.auth.transport.requests
from google.adk.tools.mcp_tool.mcp_toolset import (
MCPToolset,
StdioServerParameters,
StreamableHTTPConnectionParams,
)
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT", "propane-will-491900-m5")
# --- BigQuery MCP (Google managed, OAuth) ---
BIGQUERY_MCP_URL = f"https://bigquery.googleapis.com/mcp/projects/{PROJECT_ID}"
def get_bigquery_mcp_toolset() -> MCPToolset:
"""Connect to Google's managed BigQuery MCP server using ADC (no key file)."""
credentials, _ = google.auth.default(
scopes=["https://www.googleapis.com/auth/bigquery"]
)
credentials.refresh(google.auth.transport.requests.Request())
return MCPToolset(
connection_params=StreamableHTTPConnectionParams(
url=BIGQUERY_MCP_URL,
headers={
"Authorization": f"Bearer {credentials.token}",
"x-goog-user-project": PROJECT_ID,
},
)
)
# --- Google Maps MCP (API key from env/Secret Manager) ---
MAPS_MCP_URL = "https://maps.googleapis.com/mcp"
def get_maps_mcp_toolset() -> MCPToolset:
"""Connect to Google's managed Maps MCP server using an API key."""
maps_api_key = os.environ.get("MAPS_API_KEY", "")
if not maps_api_key:
raise ValueError("MAPS_API_KEY env var not set. Add to Secret Manager and load at startup.")
return MCPToolset(
connection_params=StreamableHTTPConnectionParams(
url=MAPS_MCP_URL,
headers={"X-Goog-Api-Key": maps_api_key},
)
)
# --- Local stdio MCP server (dev/test only) ---
def get_local_stdio_toolset(command: str, args: list[str]) -> MCPToolset:
"""Connect to a local stdio MCP server for development and testing."""
return MCPToolset(
connection_params=StdioServerParameters(
command=command,
args=args,
)
)

View File

@ -0,0 +1,112 @@
# Agent Workflow Diagrams
## 1. Main deploy/run lifecycle
```mermaid
flowchart TD
A[Local VS Code / Cloud Shell] -->|01-setupenv.sh| B[APIs Enabled + Bucket + SA]
B -->|02-deploy.sh| C[ADK Python SDK]
C -->|gcloud auth ADC| D[Gemini Enterprise Agent Platform]
D -->|Agent Runtime deployed| E[Managed Serverless Runtime]
E -->|Query via API| F[Agent Response]
F -->|End of day| G[03-teardown.sh]
G --> H[Agent Runtime Deleted — Billing Stopped]
style G fill:#c0392b,color:#fff
style H fill:#27ae60,color:#fff
```
## 2. Multi-agent / Coordinator delegation flow
```mermaid
flowchart TD
U[User Input] --> O[Orchestrator LlmAgent]
O -->|Knowledge question| R[rag_agent]
O -->|GCP ops question| G[gcp_ops_agent]
O -->|Memory recall/store| M[memory_agent]
O -->|Session close| F[farewell_agent]
R --> V[Vertex AI RAG Engine]
M --> MB[Memory Bank / Agent Engine]
O -->|Final answer| U
```
## 3. Agents CLI lifecycle (2026)
```mermaid
flowchart LR
A[Developer] -->|natural language| B[Agents CLI skills]
B -->|agents-cli create| C[Scaffolded project]
C -->|agents-cli run| D[Local test]
D -->|agents-cli deploy| E{Deploy target}
E -->|cloud_run| F[Cloud Run service]
E -->|agent_runtime| G[Agent Runtime — managed]
E -->|gke| H[GKE cluster]
F & G & H --> J[Cloud Trace + Logging + Monitoring]
```
## 4. Observability stack
```mermaid
flowchart TD
A[ADK Agent — deployed] -->|OpenTelemetry| B[Cloud Trace]
A -->|structured logs| C[Cloud Logging]
A -->|metrics| D[Cloud Monitoring]
D -->|alert policy| E[Email / PubSub]
B --> F[Agent Platform: Traces tab]
C --> G[Agent Platform: Logs tab]
D --> H[Dashboards: sessions, latency p50/p95/p99, error rates]
```
## 5. RAG-grounded agent data flow
```mermaid
flowchart LR
A[User query] --> B[ADK Agent]
B -->|retrieve| C[Vertex AI RAG Engine]
C -->|search corpus| D[Vector index]
D -->|relevant chunks| C
C -->|grounded context| B
B -->|generate| E[Gemini model]
E -->|grounded response| F[User]
B -->|telemetry| G[Cloud Trace]
```
## 6. CI/CD pipeline
```mermaid
flowchart TD
P[git push to main] --> T[Cloud Build Trigger]
T --> B[Build Docker image]
B --> AR[Push to Artifact Registry]
AR --> EV[run_eval.py — CI gate]
EV -->|groundedness >= 0.8| CR[Deploy to Cloud Run]
EV -->|groundedness < 0.8| FAIL[Build FAILED]
CR --> LIVE[Live service]
```
## 7. Memory Bank session lifecycle
```
Session Start
└── Load memories (recall by user_id)
Agent conversation (session.state updates)
Session End
└── Generate memories (summarize session)
└── Store in Memory Bank (TTL: 30 days default)
Next Session
└── Memories retrieved at start
```
## 8. A2A agent communication
```
Orchestrator Agent (Cloud Run Service A)
│ HTTP POST /tasks (A2A protocol)
│ Authorization: Bearer <SA token>
Sub-Agent B (Cloud Run Service B)
│ AgentCard: /.well-known/agent.json
Response (A2A TaskResult)
└── Orchestrator aggregates + responds to user
```

View File

@ -0,0 +1,103 @@
# Data Flow Security — propane-will-491900-m5
## Authentication flow
```
Local VS Code ──[ADC: gcloud auth application-default login]──► Google Cloud APIs
Cloud Run ──[attached SA via metadata server]─────────────► Google Cloud APIs
GKE pod ──[Workload Identity Federation]────────────────► Google Cloud APIs
```
## No-key-file policy
- Service account JSON keys MUST NOT be committed to Git.
- `.gitignore` must always include: `*.json`, `credentials/`, `*.key`, `.env`
- Use `gcloud secrets create` (Secret Manager) for all sensitive values.
## Secret Manager pattern
```bash
# Store a secret
echo -n "MY_SECRET_VALUE" | gcloud secrets create my-secret \
--data-file=- --project=propane-will-491900-m5
# Grant access to service account
gcloud secrets add-iam-policy-binding my-secret \
--member="serviceAccount:vertex-agent-sa@propane-will-491900-m5.iam.gserviceaccount.com" \
--role="roles/secretmanager.secretAccessor" \
--project=propane-will-491900-m5
```
```python
from google.cloud import secretmanager
client = secretmanager.SecretManagerServiceClient()
name = "projects/propane-will-491900-m5/secrets/my-secret/versions/latest"
response = client.access_secret_version(request={"name": name})
value = response.payload.data.decode("UTF-8")
```
## Data classification
| Data Type | Classification | Handling |
|---|---|---|
| User queries | Confidential | In-memory only; not logged by default |
| RAG corpus documents | Internal | GCS, encrypted at rest |
| Agent memories | Confidential | Memory Bank, encrypted at rest |
| API keys / secrets | Secret | Secret Manager only; never in env vars |
| Container images | Internal | Artifact Registry, private |
| Audit logs | Internal | Cloud Logging, 30-day retention |
## Input guardrails (ADK callbacks)
```python
def before_model_callback(callback_context, llm_request):
blocked = ["drop table", "ignore previous instructions", "jailbreak"]
user_text = llm_request.contents[-1].parts[0].text.lower()
for pattern in blocked:
if pattern in user_text:
from google.genai.types import Content, Part
return Content(parts=[Part(text="I cannot process that request.")])
return None
def before_tool_callback(tool, args, tool_context):
if tool.name == "execute_query":
if "DROP" in args.get("query", "").upper():
raise ValueError("Destructive queries are not permitted.")
return None
```
## Agent Gateway + Model Armor architecture
```
Client (Gemini CLI / Claude Code / browser)
Agent Gateway ← enforces IAM + Semantic Governance policies
← Model Armor: blocks prompt injection, data leakage
Agent Runtime / Cloud Run (ADK agent)
├──► Google Cloud APIs — via SA with least-privilege IAM
└──► MCP Servers — requires roles/mcp.toolUser
```
## Network security
```bash
# Cloud Run: no unauthenticated access
gcloud run services update oavauco-agent-v1 \
--no-allow-unauthenticated --region=us-central1
# VPC connector for private Vertex AI access
gcloud compute networks vpc-access connectors create agent-connector \
--network=default --region=us-central1 --range=10.8.0.0/28
gcloud run services update oavauco-agent-v1 \
--vpc-connector=agent-connector \
--vpc-egress=private-ranges-only --region=us-central1
```
## Security feature matrix
| Feature | Purpose | Status |
|---|---|---|
| Agent Identity | Per-agent SA, cryptographic ID | GA |
| Agent Registry | Central catalog of deployed agents | GA |
| Agent Gateway | API gateway, IAM + policy enforcement | GA |
| Model Armor | Prompt injection / data leakage blocking | GA |
| A2A Zero-Trust | Authenticated agent-to-agent comms | GA |
| DLP integration | PII detection in agent I/O | Available |
| Audit Logging | All agent actions logged to Cloud Logging | Always-on |

71
cloudbuild.yaml Normal file
View File

@ -0,0 +1,71 @@
# cloudbuild.yaml — Cloud Build CI/CD pipeline for OSVauco ADK agent
# Triggered on push to main branch
# Stages: install + test → eval gate → build image → push → deploy to Cloud Run
substitutions:
_PROJECT_ID: propane-will-491900-m5
_REGION: us-central1
_SERVICE_NAME: oavauco-agent-v1
_AR_REPO: oavauco-docker
_IMAGE: ${_REGION}-docker.pkg.dev/${_PROJECT_ID}/${_AR_REPO}/oavauco-agent:${SHORT_SHA}
steps:
- name: 'python:3.12-slim'
id: 'install-and-test'
entrypoint: bash
args:
- '-c'
- |
pip install -r agents/core-logic/requirements.txt --quiet
if [ -d "agents/tests/unit" ]; then
python -m pytest agents/tests/unit/ -v
fi
echo "Install + tests complete."
- name: 'python:3.12-slim'
id: 'eval-gate'
entrypoint: bash
env:
- 'GOOGLE_CLOUD_PROJECT=${_PROJECT_ID}'
- 'GOOGLE_CLOUD_LOCATION=${_REGION}'
- 'GOOGLE_GENAI_USE_VERTEXAI=True'
args:
- '-c'
- |
pip install google-cloud-aiplatform[evaluation] --quiet
python agents/eval/run_eval.py
- name: 'gcr.io/cloud-builders/docker'
id: 'build-image'
args:
- 'build'
- '-t'
- '${_IMAGE}'
- '-f'
- 'agents/core-logic/Dockerfile'
- 'agents/core-logic'
- name: 'gcr.io/cloud-builders/docker'
id: 'push-image'
args: ['push', '${_IMAGE}']
- name: 'gcr.io/cloud-builders/gcloud'
id: 'deploy-cloud-run'
args:
- 'run'
- 'deploy'
- '${_SERVICE_NAME}'
- '--image=${_IMAGE}'
- '--region=${_REGION}'
- '--project=${_PROJECT_ID}'
- '--service-account=vertex-agent-sa@${_PROJECT_ID}.iam.gserviceaccount.com'
- '--no-allow-unauthenticated'
- '--min-instances=0'
- '--platform=managed'
- '--quiet'
images:
- '${_IMAGE}'
options:
logging: CLOUD_LOGGING_ONLY

View File

@ -0,0 +1,67 @@
# Cost Management Rules — propane-will-491900-m5
## Hard rules
1. **Billing budget alert** must be created BEFORE any resource deployment.
Alert thresholds: 50% / 75% / 90% / 100% of monthly budget.
Start with 5 USD (~55 NOK) for dev/test phases.
2. **Agent Runtimes** cost money when deployed — always undeploy via `03-teardown.sh` at end of workday.
3. **Vertex AI Endpoints** — same rule as above.
4. **Staging bucket lifecycle**: auto-delete objects older than 7 days.
5. **Colab Enterprise runtimes** — terminate when not actively developing.
6. GCP does NOT enforce a hard spend cap. Budget alerts are email-only by default.
## Agent Runtime pricing (May 2026)
| Resource | Free Tier/month | Paid rate |
|---|---|---|
| vCPU | 180,000 vCPU-seconds | ~$0.0864/vCPU-hour |
| RAM | 360,000 GiB-seconds | ~$0.0090/GiB-hour |
Agent Runtime does NOT bill for idle/non-running agents (pay-per-use). Always delete at end of session.
## NEW billing lines (effective Jan 28, 2026)
These were previously free/in preview and are now METERED:
- **Sessions** — agent interaction contexts
- **Memory Bank** — long-term memory storage
- **Code Execution** — sandboxed code runs
→ Trim session histories. Only persist high-value facts to Memory Bank. Disable Code Execution on agents that don't need it.
## Cloud Run cost profile
- `--min-instances=0` → scales to zero — no idle cost (use for dev/test)
- `--min-instances=1` → keeps one warm instance — eliminates cold-start (~$48/month)
- Cleanup: `gcloud run services delete SERVICE_NAME --region=us-central1 --quiet`
## Lifecycle JSON for staging bucket
```json
{
"rule": [
{
"action": { "type": "Delete" },
"condition": { "age": 7 }
}
]
}
```
```bash
gcloud storage buckets update gs://propane-will-491900-m5-agent-staging \
--lifecycle-file=lifecycle.json
```
## RAG Engine cost warning
- If RAG Engine uses a managed Spanner instance as vector DB → billed 24/7.
- For dev/test: prefer `us-east1` or `us-east4` until us-central1 allowlist is granted.
- Contact: `vertex-ai-rag-engine-support@google.com` for us-central1 allowlist access.
## Artifact Registry cleanup
```bash
gcloud artifacts docker images list \
us-central1-docker.pkg.dev/propane-will-491900-m5/oavauco-docker \
--format="value(IMAGE,DIGEST)" | tail -n +2 | \
while read -r image digest; do
gcloud artifacts docker images delete "${image}@${digest}" --quiet
done
```
## Pub/Sub hard-stop billing automation (optional)
Consider: Pub/Sub → Cloud Run Function to auto-disable billing if budget alert fires.
Template: https://cloud.google.com/billing/docs/how-to/notify#cap_disable_billing_to_stop_usage

77
docs/GCPBestPractices.md Normal file
View File

@ -0,0 +1,77 @@
# GCP Best Practices — propane-will-491900-m5
## Platform context (20252026)
- **Vertex AI Agent Engine** has been rebranded as **Gemini Enterprise Agent Platform**.
Agent runtimes are fully managed serverless — no VM management required.
- **ADK (Agent Development Kit)** 2.0 is open-source, supports Python/TypeScript/Go/Java/Kotlin.
Agents run locally for dev, then deploy to Agent Runtime, Cloud Run, or GKE.
- Region: **us-central1** for all resources (low latency, widest feature availability).
## Required APIs (enable once, idempotent)
```
aiplatform.googleapis.com # Vertex AI / Gemini Agent Platform
storage.googleapis.com # Cloud Storage staging
cloudbilling.googleapis.com # Billing Budget API
cloudresourcemanager.googleapis.com
iam.googleapis.com
run.googleapis.com # If using Cloud Run as deploy target
monitoring.googleapis.com
logging.googleapis.com
cloudtrace.googleapis.com
secretmanager.googleapis.com
artifactregistry.googleapis.com
cloudbuild.googleapis.com
```
## ADK install
```bash
pip install google-adk>=2.0.0 google-cloud-aiplatform[adk,agent_engines]>=1.112.0
gcloud components update
gcloud auth application-default login
```
## Deploy targets — decision matrix
| Target | Best for | Cost model | Auth |
|---|---|---|---|
| **Agent Runtime** (managed) | Production agents, Memory Bank, Sessions | Per vCPU-second + GiB-second | ADC / SA |
| **Cloud Run** | Custom containers, public API endpoints, lower ops | Pay-per-request, scales to 0 | ADC / SA |
| **GKE** | Complex multi-agent meshes, custom networking | Cluster uptime + node cost | Workload Identity |
## ADK 2.0 — new in 2026
- Graph workflows and collaborative agents (coordinator + sub-agents)
- **ADK Kotlin** — new language support
- `adk deploy cloud_run` and `adk deploy agent_engine` CLI commands
- Native MCP client and server support
- A2A (Agent-to-Agent) protocol support
- `SecretManagerClient` integration (requires ADK ≥ 1.29)
## Four platform pillars (Gemini Enterprise Agent Platform)
| Pillar | Key features |
|---|---|
| **Build** | ADK 2.0, Agent Studio (visual designer), Agents CLI, 200+ models via Model Garden |
| **Scale** | Agent Runtime, Sessions, Memory Bank, Code Execution, multi-day workflows |
| **Govern** | Agent Identity, Agent Registry, Agent Gateway, Model Armor, Agent Security dashboard |
| **Optimize** | Agent Evaluation, Agent Optimizer (auto-clusters failures), Example Store, Observability |
## Model landscape (May 2026)
| Model | Status | Recommended use |
|---|---|---|
| `gemini-2.5-flash` | GA | Balanced cost/quality — **recommended default** |
| `gemini-3-flash` | Public preview | Cheapest option for dev/test |
| `gemma-4` | GA (open-source) | Self-hosted / fine-tuning |
| ~~`gemini-1.5-*`~~ | **Discontinued** | Migrate immediately |
| ~~`claude-3.5-haiku`~~ | **Deprecated** (shutdown Jul 2026) | Migrate immediately |
## agent-starter-pack (Google official)
```bash
uvx agent-starter-pack enhance # add CI/CD + eval + observability to existing agent
uvx agent-starter-pack create my-agent # scaffold from scratch
```
Provides: CI/CD (Cloud Build), Evaluation, Observability, Terraform IaC out of the box.
GitHub: https://github.com/GoogleCloudPlatform/agent-starter-pack
## Key constraints
- Never use GPU instances (A100/V100) unless business-critical.
- Default compute: `e2-micro` for background tasks, up to `e2-standard-4` for Colab.
- Agent Runtimes bill when deployed — always run `03-teardown.sh` at end of day.
- Cloud Storage buckets do NOT auto-delete — lifecycle rules are mandatory.

View File

@ -0,0 +1,82 @@
# IAM Roles and Permissions — propane-will-491900-m5
## Service account: vertex-agent-sa
Minimum roles required:
| Role | Purpose |
|---|---|
| `roles/aiplatform.user` | Deploy/query agents on Gemini Agent Platform |
| `roles/storage.objectAdmin` | Read/write staging bucket |
| `roles/logging.logWriter` | Write agent logs to Cloud Logging |
| `roles/iam.serviceAccountTokenCreator` | Allow ADK to use the SA |
| `roles/secretmanager.secretAccessor` | Read secrets at runtime |
| `roles/run.invoker` | Call Cloud Run services |
| `roles/cloudtrace.agent` | Write traces to Cloud Trace |
| `roles/artifactregistry.writer` | Push container images (CI/CD) |
## Create SA (idempotent)
```bash
SA_NAME="vertex-agent-sa"
SA_EMAIL="${SA_NAME}@propane-will-491900-m5.iam.gserviceaccount.com"
if ! gcloud iam service-accounts describe "$SA_EMAIL" \
--project=propane-will-491900-m5 &>/dev/null 2>&1; then
gcloud iam service-accounts create "$SA_NAME" \
--display-name="Vertex Agent SA" \
--project=propane-will-491900-m5
fi
for ROLE in \
roles/aiplatform.user \
roles/storage.objectAdmin \
roles/logging.logWriter \
roles/iam.serviceAccountTokenCreator \
roles/secretmanager.secretAccessor \
roles/run.invoker \
roles/cloudtrace.agent \
roles/artifactregistry.writer; do
gcloud projects add-iam-policy-binding propane-will-491900-m5 \
--member="serviceAccount:${SA_EMAIL}" \
--role="$ROLE" --quiet
done
```
**Do NOT download a JSON key file** — use Workload Identity or ADC (`gcloud auth application-default login`) instead.
## MCP Tool User role (new — 2026)
Required when using Google's managed MCP servers (BigQuery, Maps, etc.):
```bash
gcloud projects add-iam-policy-binding propane-will-491900-m5 \
--member="user:YOUR_EMAIL" \
--role="roles/mcp.toolUser"
```
## Authentication decision tree
```
Running locally in VS Code?
YES → gcloud auth application-default login
File: $HOME/.config/gcloud/application_default_credentials.json
Running on Cloud Run?
YES → Attach service account to service (no key file)
gcloud run services update SERVICE --service-account=SA_EMAIL
Running on GKE?
YES → Use Workload Identity Federation (keyless)
gcloud container clusters update CLUSTER --workload-pool=PROJECT.svc.id.goog
NEVER use:
× gcloud auth activate-service-account (static credentials)
× Exporting JSON key files to repo or environment variables
```
## ADC credential search order
1. `GOOGLE_APPLICATION_CREDENTIALS` env var (service account JSON path)
2. `~/.config/gcloud/application_default_credentials.json`
3. Attached service account from metadata server (Cloud Run, GCE, GKE)
**Best practice**: Never set `GOOGLE_APPLICATION_CREDENTIALS` in production. Let the metadata server handle it.
## Agent Identity (governance layer — 2026)
- Every deployed agent should have its own dedicated service account (Agent Identity).
- Naming convention: `agent-{name}-sa@propane-will-491900-m5.iam.gserviceaccount.com`
- Register in Agent Registry so all autonomous actions are traceable.