feat: push all remaining files — agents/, docs/, architecture/, cloudbuild.yaml, README
This commit is contained in:
parent
5a33e8f444
commit
34ba75a4a2
42
.gitignore
vendored
Normal file
42
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyo
|
||||
*.pyd
|
||||
.Python
|
||||
build/
|
||||
dist/
|
||||
*.egg-info/
|
||||
.eggs/
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
.env
|
||||
|
||||
# Credentials — NEVER commit these
|
||||
*.json
|
||||
credentials/
|
||||
*.key
|
||||
*.pem
|
||||
service_account*.json
|
||||
application_default_credentials.json
|
||||
|
||||
# GCP / ADK
|
||||
.gcloud/
|
||||
adk_sessions/
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Temp
|
||||
/tmp/
|
||||
*.log
|
||||
146
README.md
146
README.md
|
|
@ -1,22 +1,138 @@
|
|||
# OSVauco
|
||||
# OSVauco — GCP Agent Master Repo
|
||||
|
||||
Main operator workspace for OS-Vauco, focused on OPAX-MCP and Gemini TUI in GCP project `propane-will-491900-m5` (region `us-central1`).
|
||||
Prosjekt: `propane-will-491900-m5` | Region: `us-central1`
|
||||
|
||||
## Repositories
|
||||
Repository for ADK 2.0-baserte agenter på Gemini Enterprise Agent Platform (tidl. Vertex AI Agent Engine).
|
||||
|
||||
- **OSVauco (this repo)** – operator hub: scripts, governance docs, session log.
|
||||
- **vauco-gemini-tui-bridge** – Textual TUI + FastAPI WebSocket bridge for Gemini LLM.
|
||||
- **vauco-bootstrap** – skill templates and higher-level automation flows.
|
||||
---
|
||||
|
||||
## Daily flow
|
||||
## Repo-struktur
|
||||
|
||||
From Cloud Shell:
|
||||
|
||||
```bash
|
||||
cd ~/OSVauco
|
||||
./scripts/osvauco-startup.sh # auth + ADC + .env sync
|
||||
./scripts/osvauco-opax-boot.sh # OPAX-MCP ping + optional Gemini TUI entry
|
||||
./scripts/osvauco-health-check.sh # sanity-check git/disk/auth
|
||||
```
|
||||
├── agents/
|
||||
│ ├── core-logic/ # Hoved-agent: agent.py, __init__.py, deploy_agent.py
|
||||
│ │ ├── requirements.txt # Pinnede avhengigheter
|
||||
│ │ ├── Dockerfile # Cloud Run containerisering
|
||||
│ │ └── .env.example # Lokal dev — kopier til .env
|
||||
│ ├── multi_agent/ # Orchestrator + sub-agents
|
||||
│ ├── tools/ # MCP-integrasjoner (BigQuery, Maps)
|
||||
│ ├── rag/ # RAG corpus setup
|
||||
│ ├── memory/ # Memory Bank + Sessions setup
|
||||
│ ├── eval/ # CI/CD eval-gate (groundedness ≥ 0.8)
|
||||
│ └── tests/ # Lokal test-runner (ADK dev server)
|
||||
├── infrastructure/
|
||||
│ ├── 00-authcheck.sh # Verifiser gcloud-identitet og prosjekt
|
||||
│ ├── 01-setupenv.sh # APIs, bucket, lifecycle, SA, IAM, billing budget
|
||||
│ ├── 02-deploy.sh # Deploy til Agent Runtime (ADK Python SDK)
|
||||
│ ├── 03-teardown.sh # Slett Agent Runtimes + cleanup (kostnadsbeskyttelse)
|
||||
│ ├── 04-observability-setup.sh # Monitoring, Logging, Trace APIs
|
||||
│ ├── 05-cloudrun-deploy.sh # Deploy til Cloud Run via `adk deploy cloud_run`
|
||||
│ ├── 06-cicd-setup.sh # Cloud Build trigger + Artifact Registry
|
||||
│ ├── 07-rag-setup.sh # RAG corpus create + import wrapper
|
||||
│ └── 08-memorybank-setup.sh # Memory Bank instance + IAM
|
||||
├── docs/
|
||||
│ ├── GCPBestPractices.md # Platform, ADK 2.0, deploy targets, modeller
|
||||
│ ├── CostManagementRules.md # Priser, lifecycle-regler, teardown-policy
|
||||
│ └── IAMRolesandPermissions.md # Roller, SA, ADC, Agent Identity
|
||||
├── architecture/
|
||||
│ ├── agentworkflowdiagrams.md # Mermaid-diagrammer: deploy, multi-agent, CI/CD
|
||||
│ └── dataflowsecurity.md # Auth, secrets, guardrails, Agent Gateway
|
||||
├── cloudbuild.yaml # CI/CD pipeline
|
||||
└── README.md
|
||||
```
|
||||
|
||||
Session history is stored in `docs/OSVAUCO_OPAX_SESSION_LOG.md`. Governance rules live in `docs/AGENT_RULEBOOK.md`.
|
||||
---
|
||||
|
||||
## Dag-1 oppsett
|
||||
|
||||
### Forutsetninger
|
||||
- `gcloud` CLI installert og logget inn
|
||||
- Python 3.12+
|
||||
- Docker (for Cloud Run deploys)
|
||||
|
||||
### Steg 1 — Auth
|
||||
```bash
|
||||
gcloud auth login
|
||||
gcloud auth application-default login
|
||||
gcloud config set project propane-will-491900-m5
|
||||
```
|
||||
|
||||
### Steg 2 — Environment setup
|
||||
```bash
|
||||
bash infrastructure/01-setupenv.sh
|
||||
# Følg instruksjonene om billing budget (krever billing account ID)
|
||||
# Finn med: gcloud billing accounts list
|
||||
```
|
||||
|
||||
### Steg 3 — Lokal kjøring
|
||||
```bash
|
||||
cd agents/core-logic
|
||||
cp .env.example .env # fyll inn variabler
|
||||
pip install -r requirements.txt
|
||||
adk web . # åpner dev UI på http://localhost:8080
|
||||
```
|
||||
|
||||
### Steg 4 — Deploy til Cloud Run
|
||||
```bash
|
||||
bash infrastructure/05-cloudrun-deploy.sh
|
||||
```
|
||||
|
||||
### Steg 5 — Deploy til Agent Runtime (managed)
|
||||
```bash
|
||||
bash infrastructure/02-deploy.sh
|
||||
# HUSK: Kjør teardown på slutten av dagen!
|
||||
bash infrastructure/03-teardown.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hurtigreferanse-kommandoer
|
||||
|
||||
| Handling | Kommando |
|
||||
|---|---|
|
||||
| Auth check | `bash infrastructure/00-authcheck.sh` |
|
||||
| Setup env | `bash infrastructure/01-setupenv.sh` |
|
||||
| Deploy Agent Runtime | `bash infrastructure/02-deploy.sh` |
|
||||
| **Teardown (viktig!)** | `bash infrastructure/03-teardown.sh` |
|
||||
| Deploy Cloud Run | `bash infrastructure/05-cloudrun-deploy.sh` |
|
||||
| Setup RAG corpus | `python agents/rag/setup_corpus.py` |
|
||||
| Setup Memory Bank | `python agents/memory/memory_setup.py` |
|
||||
| Kjør eval | `python agents/eval/run_eval.py` |
|
||||
| Lokal test | `bash agents/tests/test_local.sh` |
|
||||
| Finn billing account | `gcloud billing accounts list` |
|
||||
| Liste aktive Cloud Run services | `gcloud run services list --region=us-central1` |
|
||||
| Slett Cloud Run service | `gcloud run services delete oavauco-agent-v1 --region=us-central1` |
|
||||
|
||||
---
|
||||
|
||||
## Kostnadsbeskyttelse — viktigste regler
|
||||
|
||||
1. **Kjør alltid `03-teardown.sh` på slutten av arbeidsdagen** — Agent Runtimes faktureres.
|
||||
2. Sett billing budget alert før første deploy (`01-setupenv.sh` har stub for dette).
|
||||
3. Cloud Run skalerer til 0 — ingen idle-kost med `--min-instances=0`.
|
||||
4. RAG Engine med Spanner → 24/7 fakturering. Bruk `us-east1` for dev/test.
|
||||
5. Sessions, Memory Bank og Code Execution er metered fra 28. jan 2026.
|
||||
|
||||
---
|
||||
|
||||
## CI/CD
|
||||
|
||||
`cloudbuild.yaml` kjøres automatisk ved push til `main`:
|
||||
1. Install dependencies + unit tests
|
||||
2. Eval gate (groundedness ≥ 0.8 — feiler bygget ellers)
|
||||
3. Build Docker image
|
||||
4. Push til Artifact Registry
|
||||
5. Deploy til Cloud Run
|
||||
|
||||
Oppsett: `bash infrastructure/06-cicd-setup.sh`
|
||||
(Krever at GitHub-repo er koblet til Cloud Build via GCP Console først.)
|
||||
|
||||
---
|
||||
|
||||
## Sikkerhetsregler
|
||||
|
||||
- Ingen JSON-nøkkelfiler i repo — bruk ADC / Workload Identity.
|
||||
- Alle hemmeligheter i Secret Manager (`gcloud secrets create ...`).
|
||||
- `.env` er i `.gitignore` — aldri commit.
|
||||
- Cloud Run kjører med `--no-allow-unauthenticated`.
|
||||
- ADK-callbacks blokkerer prompt injection og destruktive tool-args.
|
||||
|
|
|
|||
25
agents/core-logic/.env.example
Normal file
25
agents/core-logic/.env.example
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
# .env.example — Copy to .env for local development
|
||||
# NEVER commit .env to Git. Add .env to .gitignore.
|
||||
|
||||
# GCP project and region (required)
|
||||
GOOGLE_CLOUD_PROJECT=propane-will-491900-m5
|
||||
GOOGLE_CLOUD_LOCATION=us-central1
|
||||
|
||||
# Authentication mode: True = Vertex AI (ADC), False = Gemini API key
|
||||
GOOGLE_GENAI_USE_VERTEXAI=True
|
||||
|
||||
# Only set if GOOGLE_GENAI_USE_VERTEXAI=False (local testing without gcloud)
|
||||
# GOOGLE_API_KEY=your-api-key-here
|
||||
|
||||
# RAG corpus resource name (from setup_corpus.py output)
|
||||
# RAG_CORPUS=projects/propane-will-491900-m5/locations/us-central1/ragCorpora/XXXXXXXXX
|
||||
|
||||
# Agent Engine ID (from memory_setup.py output)
|
||||
# AGENT_ENGINE_ID=your-agent-engine-id
|
||||
|
||||
# MCP server URLs (if using Google managed MCP)
|
||||
# MAPS_MCP_URL=https://maps.googleapis.com/mcp
|
||||
# BIGQUERY_MCP_URL=https://bigquery.googleapis.com/mcp
|
||||
|
||||
# Maps API key (for Google Maps MCP)
|
||||
# MAPS_API_KEY=your-maps-api-key
|
||||
19
agents/core-logic/Dockerfile
Normal file
19
agents/core-logic/Dockerfile
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Dockerfile for OSVauco ADK agent — Cloud Run deployment
|
||||
# Note: `adk deploy cloud_run` manages this automatically.
|
||||
# Use this Dockerfile only for manual `gcloud run deploy --source .` deploys.
|
||||
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
|
||||
ENV PORT=8080
|
||||
ENV GOOGLE_GENAI_USE_VERTEXAI=True
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
CMD ["adk", "web", "--host", "0.0.0.0", "--port", "8080", "."]
|
||||
2
agents/core-logic/__init__.py
Normal file
2
agents/core-logic/__init__.py
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
# Required by ADK: exports root_agent for adk web and adk deploy commands
|
||||
from .agent import root_agent
|
||||
47
agents/core-logic/agent.py
Normal file
47
agents/core-logic/agent.py
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
agent.py — OSVauco root agent using ADK 2.0.
|
||||
Requires: google-adk >= 1.29, google-cloud-aiplatform >= 1.111.0
|
||||
"""
|
||||
|
||||
import os
|
||||
from google.adk.agents import Agent
|
||||
from google.adk.integrations.secret_manager.secret_client import SecretManagerClient
|
||||
from vertexai.preview import rag
|
||||
|
||||
PROJECT_ID = "propane-will-491900-m5"
|
||||
LOCATION = "us-central1"
|
||||
|
||||
# --- Secret Manager integration (ADK >= 1.29) ---
|
||||
_sm = SecretManagerClient()
|
||||
def _get_secret(name: str) -> str:
|
||||
return _sm.get_secret(
|
||||
f"projects/{PROJECT_ID}/secrets/{name}/versions/latest"
|
||||
)
|
||||
|
||||
# --- RAG retrieval tool ---
|
||||
RAG_CORPUS = os.environ.get("RAG_CORPUS", "") # set via Secret Manager or env
|
||||
|
||||
rag_tool = None
|
||||
if RAG_CORPUS:
|
||||
from google.adk.tools import VertexAiRagRetrieval
|
||||
rag_tool = VertexAiRagRetrieval(
|
||||
name="retrieve_knowledge",
|
||||
description="Retrieve relevant documentation from the knowledge base.",
|
||||
rag_resources=[rag.RagResource(rag_corpus=RAG_CORPUS)],
|
||||
similarity_top_k=10,
|
||||
vector_distance_threshold=0.6,
|
||||
)
|
||||
|
||||
# --- Root agent ---
|
||||
root_agent = Agent(
|
||||
model="gemini-2.5-flash", # cost-optimized default
|
||||
name="oavauco_root",
|
||||
description="OSVauco enterprise agent for propane-will-491900-m5",
|
||||
instruction=(
|
||||
"You are OSVauco, a GCP knowledge and workflow agent. "
|
||||
"Use the retrieve_knowledge tool to answer questions from the knowledge base. "
|
||||
"Always prefer grounded, documented answers over speculation."
|
||||
),
|
||||
tools=[rag_tool] if rag_tool else [],
|
||||
)
|
||||
38
agents/core-logic/deploy_agent.py
Normal file
38
agents/core-logic/deploy_agent.py
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
deploy_agent.py — Deploy ADK agent to Gemini Enterprise Agent Platform (Agent Runtime)
|
||||
Usage: python3 deploy_agent.py --project PROJECT_ID --region REGION \
|
||||
--display-name DISPLAY_NAME --staging-bucket GS_URI
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import vertexai
|
||||
from vertexai.preview import agent_engines # SDK v1.112.0+ client-based design
|
||||
|
||||
|
||||
def deploy(project: str, region: str, display_name: str, staging_bucket: str):
|
||||
vertexai.init(project=project, location=region, staging_bucket=staging_bucket)
|
||||
|
||||
from agents.core_logic import agent as my_agent # adjust import path
|
||||
|
||||
print(f"Deploying agent '{display_name}' to Agent Runtime in {region}...")
|
||||
remote_agent = agent_engines.create(
|
||||
my_agent.root_agent,
|
||||
requirements=["google-cloud-aiplatform[adk,agent_engines]>=1.112.0"],
|
||||
display_name=display_name,
|
||||
)
|
||||
print(f"Agent deployed. Resource name: {remote_agent.resource_name}")
|
||||
print(f"Console: https://console.cloud.google.com/ai/agents?project={project}")
|
||||
print("REMINDER: Run 03-teardown.sh at end of day to stop billing.")
|
||||
return remote_agent
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--project", default="propane-will-491900-m5")
|
||||
parser.add_argument("--region", default="us-central1")
|
||||
parser.add_argument("--display-name", default="oavauco-agent-v1")
|
||||
parser.add_argument("--staging-bucket",
|
||||
default="gs://propane-will-491900-m5-agent-staging")
|
||||
args = parser.parse_args()
|
||||
deploy(args.project, args.region, args.display_name, args.staging_bucket)
|
||||
33
agents/core-logic/requirements.txt
Normal file
33
agents/core-logic/requirements.txt
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# OSVauco agent dependencies — pin versions to prevent surprise upgrades
|
||||
|
||||
# ADK + Vertex AI SDK (must match)
|
||||
google-adk>=2.0.0
|
||||
google-cloud-aiplatform[adk,agent_engines]>=1.112.0
|
||||
|
||||
# RAG Engine
|
||||
vertexai>=1.71.0
|
||||
|
||||
# Secret Manager integration (requires ADK >= 1.29)
|
||||
google-cloud-secret-manager>=2.20.0
|
||||
|
||||
# MCP toolset
|
||||
httpx>=0.27.0
|
||||
google-auth>=2.29.0
|
||||
google-auth-httplib2>=0.2.0
|
||||
|
||||
# Observability
|
||||
opentelemetry-sdk>=1.25.0
|
||||
opentelemetry-exporter-gcp-trace>=1.8.0
|
||||
google-cloud-logging>=3.10.0
|
||||
google-cloud-monitoring>=2.22.0
|
||||
|
||||
# Evaluation
|
||||
google-cloud-aiplatform[evaluation]>=1.112.0
|
||||
|
||||
# HTTP server for Cloud Run
|
||||
uvicorn[standard]>=0.29.0
|
||||
fastapi>=0.111.0
|
||||
|
||||
# Utilities
|
||||
python-dotenv>=1.0.0
|
||||
pydantic>=2.7.0
|
||||
51
agents/eval/run_eval.py
Normal file
51
agents/eval/run_eval.py
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
run_eval.py — Run agent evaluation using Vertex AI Gen AI Evaluation Service.
|
||||
Metrics: coherence, groundedness, tool_call_quality
|
||||
CI/CD gate: fails build if groundedness < 0.8
|
||||
"""
|
||||
|
||||
import vertexai
|
||||
from vertexai.evaluation import EvalTask
|
||||
|
||||
PROJECT_ID = "propane-will-491900-m5"
|
||||
LOCATION = "us-central1"
|
||||
|
||||
vertexai.init(project=PROJECT_ID, location=LOCATION)
|
||||
|
||||
EVAL_DATASET = [
|
||||
{
|
||||
"prompt": "What GCP region should all resources use?",
|
||||
"reference": "us-central1",
|
||||
},
|
||||
{
|
||||
"prompt": "What command tears down all Agent Runtimes?",
|
||||
"reference": "Run 03-teardown.sh",
|
||||
},
|
||||
{
|
||||
"prompt": "What ADK version is required for Memory Bank?",
|
||||
"reference": "google-adk >= 2.0.0",
|
||||
},
|
||||
]
|
||||
|
||||
METRICS = ["coherence", "groundedness", "tool_call_quality"]
|
||||
|
||||
eval_task = EvalTask(
|
||||
dataset=EVAL_DATASET,
|
||||
metrics=METRICS,
|
||||
experiment="oavauco-agent-eval",
|
||||
)
|
||||
|
||||
result = eval_task.evaluate(
|
||||
model="gemini-2.5-flash",
|
||||
prompt_template="{prompt}",
|
||||
)
|
||||
|
||||
print(result.summary_metrics)
|
||||
|
||||
# CI/CD gate
|
||||
if result.summary_metrics.get("groundedness/mean", 1.0) < 0.8:
|
||||
print("EVAL FAILED: groundedness below 0.8 threshold")
|
||||
raise SystemExit(1)
|
||||
|
||||
print("EVAL PASSED")
|
||||
52
agents/memory/memory_setup.py
Normal file
52
agents/memory/memory_setup.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
memory_setup.py — Initialize Agent Engine instance with Sessions + Memory Bank.
|
||||
Project: propane-will-491900-m5 | Region: us-central1
|
||||
SDK: google-cloud-aiplatform >= 1.111.0
|
||||
|
||||
COST NOTE: Sessions + Memory Bank are metered since Jan 28, 2026.
|
||||
Trim session histories. Only persist high-value facts to long-term memory.
|
||||
"""
|
||||
|
||||
import vertexai
|
||||
from vertexai import Client
|
||||
from google.adk.memory import VertexAiMemoryBankService
|
||||
from google.adk.sessions import VertexAiSessionService
|
||||
|
||||
PROJECT_ID = "propane-will-491900-m5"
|
||||
LOCATION = "us-central1"
|
||||
|
||||
|
||||
def create_agent_engine() -> str:
|
||||
"""Create an Agent Engine instance (backing store for Sessions + Memory Bank)."""
|
||||
client = Client(project=PROJECT_ID, location=LOCATION)
|
||||
agent_engine = client.agent_engines.create()
|
||||
resource_name = agent_engine.api_resource.name
|
||||
agent_engine_id = resource_name.split("/")[-1]
|
||||
print(f"Agent Engine created: {resource_name}")
|
||||
print(f"Agent Engine ID: {agent_engine_id}")
|
||||
print("Store this in Secret Manager or env var: AGENT_ENGINE_ID")
|
||||
return agent_engine_id
|
||||
|
||||
|
||||
def get_services(agent_engine_id: str):
|
||||
"""Return configured session and memory services for use with ADK Runner."""
|
||||
memory_service = VertexAiMemoryBankService(
|
||||
project=PROJECT_ID,
|
||||
location=LOCATION,
|
||||
agent_engine_id=agent_engine_id,
|
||||
)
|
||||
session_service = VertexAiSessionService(
|
||||
project_id=PROJECT_ID,
|
||||
location=LOCATION,
|
||||
agent_engine_id=agent_engine_id,
|
||||
)
|
||||
return session_service, memory_service
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
agent_engine_id = create_agent_engine()
|
||||
session_svc, memory_svc = get_services(agent_engine_id)
|
||||
print("Services ready.")
|
||||
print(f"session_service: {session_svc}")
|
||||
print(f"memory_service: {memory_svc}")
|
||||
130
agents/multi_agent/orchestrator.py
Normal file
130
agents/multi_agent/orchestrator.py
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
orchestrator.py — OSVauco multi-agent orchestrator using ADK 2.0 collaborative workflow.
|
||||
|
||||
Architecture:
|
||||
root_agent (coordinator / LLM-driven delegation)
|
||||
├── rag_agent — retrieves answers from private knowledge base
|
||||
├── gcp_ops_agent — handles GCP operations questions and script generation
|
||||
├── memory_agent — loads/stores long-term memory via Memory Bank
|
||||
└── farewell_agent — session closings
|
||||
|
||||
Pattern: Coordinator with sub_agents list.
|
||||
Requires: google-adk >= 2.0.0
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from google.adk.agents import Agent
|
||||
from google.adk.agents.callback_context import CallbackContext
|
||||
from google.adk.tools.load_memory_tool import load_memory_tool
|
||||
from google.adk.tools.preload_memory_tool import preload_memory_tool
|
||||
from google.genai.types import Content, Part
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROJECT_ID = os.environ.get("PROJECT_ID", "propane-will-491900-m5")
|
||||
REGION = os.environ.get("REGION", "us-central1")
|
||||
RAG_CORPUS_NAME = os.environ.get("RAG_CORPUS_NAME", "")
|
||||
ORCHESTRATOR_MODEL = os.environ.get("ORCHESTRATOR_MODEL", "gemini-2.5-flash")
|
||||
SUBAGENT_MODEL = os.environ.get("SUBAGENT_MODEL", "gemini-2.5-flash")
|
||||
|
||||
# ── Safety callbacks ──────────────────────────────────────────────────────────
|
||||
BLOCKED_PATTERNS = [
|
||||
"ignore previous instructions",
|
||||
"ignore all instructions",
|
||||
"drop table",
|
||||
"system prompt",
|
||||
"jailbreak",
|
||||
"disregard safety",
|
||||
]
|
||||
|
||||
def before_model_callback(callback_context: CallbackContext, llm_request) -> Optional[Content]:
|
||||
try:
|
||||
user_text = ""
|
||||
if llm_request.contents:
|
||||
last = llm_request.contents[-1]
|
||||
if last.parts:
|
||||
user_text = last.parts[0].text.lower()
|
||||
for pattern in BLOCKED_PATTERNS:
|
||||
if pattern in user_text:
|
||||
logger.warning("Blocked pattern detected: '%s'", pattern)
|
||||
return Content(parts=[Part(text="I cannot process that request. Please rephrase.")])
|
||||
except Exception as exc:
|
||||
logger.error("before_model_callback error: %s", exc)
|
||||
return None
|
||||
|
||||
def before_tool_callback(tool, args: dict, tool_context) -> Optional[dict]:
|
||||
dangerous = ["DROP", "DELETE FROM", "TRUNCATE", "--", ";"]
|
||||
for val in args.values():
|
||||
if isinstance(val, str):
|
||||
for d in dangerous:
|
||||
if d.upper() in val.upper():
|
||||
raise ValueError(f"Tool argument rejected by safety guardrail: '{val}'")
|
||||
return None
|
||||
|
||||
# ── Sub-agents ────────────────────────────────────────────────────────────────
|
||||
rag_agent = Agent(
|
||||
model=SUBAGENT_MODEL,
|
||||
name="rag_agent",
|
||||
description="Retrieves answers from the private OSVauco knowledge base (RAG corpus).",
|
||||
instruction=(
|
||||
"You are a knowledge retrieval specialist. "
|
||||
"Use search_knowledge_base to find relevant information and return well-cited answers."
|
||||
),
|
||||
before_model_callback=before_model_callback,
|
||||
)
|
||||
|
||||
gcp_ops_agent = Agent(
|
||||
model=SUBAGENT_MODEL,
|
||||
name="gcp_ops_agent",
|
||||
description="Answers GCP operations questions: scripts, IAM, billing, Cloud Run, ADK deployments.",
|
||||
instruction=(
|
||||
"You are a GCP operations expert for project propane-will-491900-m5 in us-central1. "
|
||||
"Provide accurate gcloud CLI commands, IAM patterns, and ADK deployment guidance. "
|
||||
"Always include cost-safety reminders (teardown, billing budgets)."
|
||||
),
|
||||
before_model_callback=before_model_callback,
|
||||
)
|
||||
|
||||
memory_agent = Agent(
|
||||
model=SUBAGENT_MODEL,
|
||||
name="memory_agent",
|
||||
description="Manages long-term memory: loads past context and stores new facts for future sessions.",
|
||||
instruction=(
|
||||
"You manage the agent's long-term memory. "
|
||||
"Use load_memory_tool to retrieve past facts and preload_memory_tool to store important new facts. "
|
||||
"Only persist high-value, factual information — not transient conversation."
|
||||
),
|
||||
tools=[load_memory_tool, preload_memory_tool],
|
||||
before_model_callback=before_model_callback,
|
||||
)
|
||||
|
||||
farewell_agent = Agent(
|
||||
model=SUBAGENT_MODEL,
|
||||
name="farewell_agent",
|
||||
description="Handles session closings, summaries, and goodbye messages.",
|
||||
instruction="Generate a concise, friendly session summary and closing message.",
|
||||
)
|
||||
|
||||
# ── Root orchestrator ─────────────────────────────────────────────────────────
|
||||
root_agent = Agent(
|
||||
model=ORCHESTRATOR_MODEL,
|
||||
name="oavauco_orchestrator",
|
||||
description="OSVauco root orchestrator — delegates to specialist sub-agents.",
|
||||
instruction=(
|
||||
"You are the OSVauco orchestrator for project propane-will-491900-m5. "
|
||||
"Delegate to sub-agents based on the user's intent:\n"
|
||||
"- Knowledge base questions → rag_agent\n"
|
||||
"- GCP operations, scripts, IAM, billing → gcp_ops_agent\n"
|
||||
"- Memory recall or storage → memory_agent\n"
|
||||
"- Session endings → farewell_agent\n"
|
||||
"Always synthesize sub-agent responses into a clear, concise final answer."
|
||||
),
|
||||
sub_agents=[rag_agent, gcp_ops_agent, memory_agent, farewell_agent],
|
||||
before_model_callback=before_model_callback,
|
||||
before_tool_callback=before_tool_callback,
|
||||
)
|
||||
72
agents/rag/setup_corpus.py
Normal file
72
agents/rag/setup_corpus.py
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
setup_corpus.py — Create a Vertex AI RAG Engine corpus and import documents.
|
||||
Project: propane-will-491900-m5
|
||||
|
||||
IMPORTANT: RAG Engine in us-central1 requires allowlist access.
|
||||
Contact: vertex-ai-rag-engine-support@google.com
|
||||
Alternative: Use us-east1 or us-east4 for immediate access.
|
||||
"""
|
||||
|
||||
import os
|
||||
import vertexai
|
||||
from vertexai import rag
|
||||
|
||||
PROJECT_ID = "propane-will-491900-m5"
|
||||
LOCATION = os.environ.get("RAG_LOCATION", "us-east1") # change to us-central1 after allowlist
|
||||
CORPUS_DISPLAY_NAME = os.environ.get("RAG_CORPUS_NAME", "oavauco-knowledge-base")
|
||||
GCS_SOURCE = os.environ.get(
|
||||
"RAG_GCS_SOURCE",
|
||||
f"gs://{PROJECT_ID}-agent-staging/rag-docs/"
|
||||
)
|
||||
|
||||
def main():
|
||||
vertexai.init(project=PROJECT_ID, location=LOCATION)
|
||||
|
||||
# Check if corpus already exists
|
||||
existing = list(rag.list_corpora())
|
||||
for c in existing:
|
||||
if c.display_name == CORPUS_DISPLAY_NAME:
|
||||
print(f"Corpus '{CORPUS_DISPLAY_NAME}' already exists: {c.name}")
|
||||
corpus = c
|
||||
break
|
||||
else:
|
||||
print(f"Creating RAG corpus '{CORPUS_DISPLAY_NAME}' in {LOCATION}...")
|
||||
corpus = rag.create_corpus(
|
||||
display_name=CORPUS_DISPLAY_NAME,
|
||||
backend_config=rag.RagVectorDbConfig(
|
||||
rag_embedding_model_config=rag.RagEmbeddingModelConfig(
|
||||
vertex_prediction_endpoint=rag.VertexPredictionEndpoint(
|
||||
publisher_model="publishers/google/models/text-embedding-005"
|
||||
)
|
||||
)
|
||||
),
|
||||
)
|
||||
print(f"Corpus created: {corpus.name}")
|
||||
|
||||
print(f"Importing files from {GCS_SOURCE}...")
|
||||
rag.import_files(
|
||||
corpus.name,
|
||||
paths=[GCS_SOURCE],
|
||||
transformation_config=rag.TransformationConfig(
|
||||
chunking_config=rag.ChunkingConfig(
|
||||
chunk_size=512,
|
||||
chunk_overlap=100,
|
||||
)
|
||||
),
|
||||
)
|
||||
print("Import complete.")
|
||||
print(f"\nRAG_CORPUS={corpus.name}")
|
||||
print("Set this as an environment variable or Secret Manager entry.")
|
||||
|
||||
# Test retrieval
|
||||
response = rag.retrieval_query(
|
||||
rag_resources=[rag.RagResource(rag_corpus=corpus.name)],
|
||||
text="test query",
|
||||
rag_retrieval_config=rag.RagRetrievalConfig(top_k=3),
|
||||
)
|
||||
print(f"Test retrieval returned {len(response.contexts.contexts)} chunks.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
13
agents/tests/test_local.sh
Normal file
13
agents/tests/test_local.sh
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
#!/bin/bash
|
||||
# test_local.sh — Run ADK agent locally before deploying to cloud
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
echo "=== OSVauco-NMTMD-GCOS :: Local Agent Test ==="
|
||||
|
||||
gcloud auth application-default print-access-token >/dev/null 2>&1 || {
|
||||
echo "ERROR: No ADC. Run: gcloud auth application-default login"; exit 1; }
|
||||
|
||||
echo "Starting ADK dev server at http://localhost:8080 ..."
|
||||
echo "Press Ctrl+C to stop."
|
||||
adk web agents/core-logic/
|
||||
64
agents/tools/mcp_tools.py
Normal file
64
agents/tools/mcp_tools.py
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
"""
|
||||
mcp_tools.py — MCP toolset integrations for OSVauco ADK agent.
|
||||
Supports:
|
||||
- Google BigQuery MCP (via StreamableHTTP, uses ADC/OAuth)
|
||||
- Google Maps MCP (via StreamableHTTP, uses API key)
|
||||
- Local stdio MCP server (dev/test only)
|
||||
|
||||
Requires: google-adk >= 2.0.0, google-auth
|
||||
"""
|
||||
|
||||
import os
|
||||
import google.auth
|
||||
import google.auth.transport.requests
|
||||
from google.adk.tools.mcp_tool.mcp_toolset import (
|
||||
MCPToolset,
|
||||
StdioServerParameters,
|
||||
StreamableHTTPConnectionParams,
|
||||
)
|
||||
|
||||
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT", "propane-will-491900-m5")
|
||||
|
||||
# --- BigQuery MCP (Google managed, OAuth) ---
|
||||
BIGQUERY_MCP_URL = f"https://bigquery.googleapis.com/mcp/projects/{PROJECT_ID}"
|
||||
|
||||
def get_bigquery_mcp_toolset() -> MCPToolset:
|
||||
"""Connect to Google's managed BigQuery MCP server using ADC (no key file)."""
|
||||
credentials, _ = google.auth.default(
|
||||
scopes=["https://www.googleapis.com/auth/bigquery"]
|
||||
)
|
||||
credentials.refresh(google.auth.transport.requests.Request())
|
||||
return MCPToolset(
|
||||
connection_params=StreamableHTTPConnectionParams(
|
||||
url=BIGQUERY_MCP_URL,
|
||||
headers={
|
||||
"Authorization": f"Bearer {credentials.token}",
|
||||
"x-goog-user-project": PROJECT_ID,
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
# --- Google Maps MCP (API key from env/Secret Manager) ---
|
||||
MAPS_MCP_URL = "https://maps.googleapis.com/mcp"
|
||||
|
||||
def get_maps_mcp_toolset() -> MCPToolset:
|
||||
"""Connect to Google's managed Maps MCP server using an API key."""
|
||||
maps_api_key = os.environ.get("MAPS_API_KEY", "")
|
||||
if not maps_api_key:
|
||||
raise ValueError("MAPS_API_KEY env var not set. Add to Secret Manager and load at startup.")
|
||||
return MCPToolset(
|
||||
connection_params=StreamableHTTPConnectionParams(
|
||||
url=MAPS_MCP_URL,
|
||||
headers={"X-Goog-Api-Key": maps_api_key},
|
||||
)
|
||||
)
|
||||
|
||||
# --- Local stdio MCP server (dev/test only) ---
|
||||
def get_local_stdio_toolset(command: str, args: list[str]) -> MCPToolset:
|
||||
"""Connect to a local stdio MCP server for development and testing."""
|
||||
return MCPToolset(
|
||||
connection_params=StdioServerParameters(
|
||||
command=command,
|
||||
args=args,
|
||||
)
|
||||
)
|
||||
112
architecture/agentworkflowdiagrams.md
Normal file
112
architecture/agentworkflowdiagrams.md
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
# Agent Workflow Diagrams
|
||||
|
||||
## 1. Main deploy/run lifecycle
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Local VS Code / Cloud Shell] -->|01-setupenv.sh| B[APIs Enabled + Bucket + SA]
|
||||
B -->|02-deploy.sh| C[ADK Python SDK]
|
||||
C -->|gcloud auth ADC| D[Gemini Enterprise Agent Platform]
|
||||
D -->|Agent Runtime deployed| E[Managed Serverless Runtime]
|
||||
E -->|Query via API| F[Agent Response]
|
||||
F -->|End of day| G[03-teardown.sh]
|
||||
G --> H[Agent Runtime Deleted — Billing Stopped]
|
||||
|
||||
style G fill:#c0392b,color:#fff
|
||||
style H fill:#27ae60,color:#fff
|
||||
```
|
||||
|
||||
## 2. Multi-agent / Coordinator delegation flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
U[User Input] --> O[Orchestrator LlmAgent]
|
||||
O -->|Knowledge question| R[rag_agent]
|
||||
O -->|GCP ops question| G[gcp_ops_agent]
|
||||
O -->|Memory recall/store| M[memory_agent]
|
||||
O -->|Session close| F[farewell_agent]
|
||||
R --> V[Vertex AI RAG Engine]
|
||||
M --> MB[Memory Bank / Agent Engine]
|
||||
O -->|Final answer| U
|
||||
```
|
||||
|
||||
## 3. Agents CLI lifecycle (2026)
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Developer] -->|natural language| B[Agents CLI skills]
|
||||
B -->|agents-cli create| C[Scaffolded project]
|
||||
C -->|agents-cli run| D[Local test]
|
||||
D -->|agents-cli deploy| E{Deploy target}
|
||||
E -->|cloud_run| F[Cloud Run service]
|
||||
E -->|agent_runtime| G[Agent Runtime — managed]
|
||||
E -->|gke| H[GKE cluster]
|
||||
F & G & H --> J[Cloud Trace + Logging + Monitoring]
|
||||
```
|
||||
|
||||
## 4. Observability stack
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[ADK Agent — deployed] -->|OpenTelemetry| B[Cloud Trace]
|
||||
A -->|structured logs| C[Cloud Logging]
|
||||
A -->|metrics| D[Cloud Monitoring]
|
||||
D -->|alert policy| E[Email / PubSub]
|
||||
B --> F[Agent Platform: Traces tab]
|
||||
C --> G[Agent Platform: Logs tab]
|
||||
D --> H[Dashboards: sessions, latency p50/p95/p99, error rates]
|
||||
```
|
||||
|
||||
## 5. RAG-grounded agent data flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[User query] --> B[ADK Agent]
|
||||
B -->|retrieve| C[Vertex AI RAG Engine]
|
||||
C -->|search corpus| D[Vector index]
|
||||
D -->|relevant chunks| C
|
||||
C -->|grounded context| B
|
||||
B -->|generate| E[Gemini model]
|
||||
E -->|grounded response| F[User]
|
||||
B -->|telemetry| G[Cloud Trace]
|
||||
```
|
||||
|
||||
## 6. CI/CD pipeline
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
P[git push to main] --> T[Cloud Build Trigger]
|
||||
T --> B[Build Docker image]
|
||||
B --> AR[Push to Artifact Registry]
|
||||
AR --> EV[run_eval.py — CI gate]
|
||||
EV -->|groundedness >= 0.8| CR[Deploy to Cloud Run]
|
||||
EV -->|groundedness < 0.8| FAIL[Build FAILED]
|
||||
CR --> LIVE[Live service]
|
||||
```
|
||||
|
||||
## 7. Memory Bank session lifecycle
|
||||
|
||||
```
|
||||
Session Start
|
||||
└── Load memories (recall by user_id)
|
||||
Agent conversation (session.state updates)
|
||||
Session End
|
||||
└── Generate memories (summarize session)
|
||||
└── Store in Memory Bank (TTL: 30 days default)
|
||||
Next Session
|
||||
└── Memories retrieved at start
|
||||
```
|
||||
|
||||
## 8. A2A agent communication
|
||||
|
||||
```
|
||||
Orchestrator Agent (Cloud Run Service A)
|
||||
│ HTTP POST /tasks (A2A protocol)
|
||||
│ Authorization: Bearer <SA token>
|
||||
▼
|
||||
Sub-Agent B (Cloud Run Service B)
|
||||
│ AgentCard: /.well-known/agent.json
|
||||
▼
|
||||
Response (A2A TaskResult)
|
||||
└── Orchestrator aggregates + responds to user
|
||||
```
|
||||
103
architecture/dataflowsecurity.md
Normal file
103
architecture/dataflowsecurity.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# Data Flow Security — propane-will-491900-m5
|
||||
|
||||
## Authentication flow
|
||||
```
|
||||
Local VS Code ──[ADC: gcloud auth application-default login]──► Google Cloud APIs
|
||||
Cloud Run ──[attached SA via metadata server]─────────────► Google Cloud APIs
|
||||
GKE pod ──[Workload Identity Federation]────────────────► Google Cloud APIs
|
||||
```
|
||||
|
||||
## No-key-file policy
|
||||
- Service account JSON keys MUST NOT be committed to Git.
|
||||
- `.gitignore` must always include: `*.json`, `credentials/`, `*.key`, `.env`
|
||||
- Use `gcloud secrets create` (Secret Manager) for all sensitive values.
|
||||
|
||||
## Secret Manager pattern
|
||||
```bash
|
||||
# Store a secret
|
||||
echo -n "MY_SECRET_VALUE" | gcloud secrets create my-secret \
|
||||
--data-file=- --project=propane-will-491900-m5
|
||||
|
||||
# Grant access to service account
|
||||
gcloud secrets add-iam-policy-binding my-secret \
|
||||
--member="serviceAccount:vertex-agent-sa@propane-will-491900-m5.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretAccessor" \
|
||||
--project=propane-will-491900-m5
|
||||
```
|
||||
|
||||
```python
|
||||
from google.cloud import secretmanager
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
name = "projects/propane-will-491900-m5/secrets/my-secret/versions/latest"
|
||||
response = client.access_secret_version(request={"name": name})
|
||||
value = response.payload.data.decode("UTF-8")
|
||||
```
|
||||
|
||||
## Data classification
|
||||
| Data Type | Classification | Handling |
|
||||
|---|---|---|
|
||||
| User queries | Confidential | In-memory only; not logged by default |
|
||||
| RAG corpus documents | Internal | GCS, encrypted at rest |
|
||||
| Agent memories | Confidential | Memory Bank, encrypted at rest |
|
||||
| API keys / secrets | Secret | Secret Manager only; never in env vars |
|
||||
| Container images | Internal | Artifact Registry, private |
|
||||
| Audit logs | Internal | Cloud Logging, 30-day retention |
|
||||
|
||||
## Input guardrails (ADK callbacks)
|
||||
```python
|
||||
def before_model_callback(callback_context, llm_request):
|
||||
blocked = ["drop table", "ignore previous instructions", "jailbreak"]
|
||||
user_text = llm_request.contents[-1].parts[0].text.lower()
|
||||
for pattern in blocked:
|
||||
if pattern in user_text:
|
||||
from google.genai.types import Content, Part
|
||||
return Content(parts=[Part(text="I cannot process that request.")])
|
||||
return None
|
||||
|
||||
def before_tool_callback(tool, args, tool_context):
|
||||
if tool.name == "execute_query":
|
||||
if "DROP" in args.get("query", "").upper():
|
||||
raise ValueError("Destructive queries are not permitted.")
|
||||
return None
|
||||
```
|
||||
|
||||
## Agent Gateway + Model Armor architecture
|
||||
```
|
||||
Client (Gemini CLI / Claude Code / browser)
|
||||
│
|
||||
▼
|
||||
Agent Gateway ← enforces IAM + Semantic Governance policies
|
||||
← Model Armor: blocks prompt injection, data leakage
|
||||
│
|
||||
▼
|
||||
Agent Runtime / Cloud Run (ADK agent)
|
||||
│
|
||||
├──► Google Cloud APIs — via SA with least-privilege IAM
|
||||
└──► MCP Servers — requires roles/mcp.toolUser
|
||||
```
|
||||
|
||||
## Network security
|
||||
```bash
|
||||
# Cloud Run: no unauthenticated access
|
||||
gcloud run services update oavauco-agent-v1 \
|
||||
--no-allow-unauthenticated --region=us-central1
|
||||
|
||||
# VPC connector for private Vertex AI access
|
||||
gcloud compute networks vpc-access connectors create agent-connector \
|
||||
--network=default --region=us-central1 --range=10.8.0.0/28
|
||||
|
||||
gcloud run services update oavauco-agent-v1 \
|
||||
--vpc-connector=agent-connector \
|
||||
--vpc-egress=private-ranges-only --region=us-central1
|
||||
```
|
||||
|
||||
## Security feature matrix
|
||||
| Feature | Purpose | Status |
|
||||
|---|---|---|
|
||||
| Agent Identity | Per-agent SA, cryptographic ID | GA |
|
||||
| Agent Registry | Central catalog of deployed agents | GA |
|
||||
| Agent Gateway | API gateway, IAM + policy enforcement | GA |
|
||||
| Model Armor | Prompt injection / data leakage blocking | GA |
|
||||
| A2A Zero-Trust | Authenticated agent-to-agent comms | GA |
|
||||
| DLP integration | PII detection in agent I/O | Available |
|
||||
| Audit Logging | All agent actions logged to Cloud Logging | Always-on |
|
||||
71
cloudbuild.yaml
Normal file
71
cloudbuild.yaml
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
# cloudbuild.yaml — Cloud Build CI/CD pipeline for OSVauco ADK agent
|
||||
# Triggered on push to main branch
|
||||
# Stages: install + test → eval gate → build image → push → deploy to Cloud Run
|
||||
|
||||
substitutions:
|
||||
_PROJECT_ID: propane-will-491900-m5
|
||||
_REGION: us-central1
|
||||
_SERVICE_NAME: oavauco-agent-v1
|
||||
_AR_REPO: oavauco-docker
|
||||
_IMAGE: ${_REGION}-docker.pkg.dev/${_PROJECT_ID}/${_AR_REPO}/oavauco-agent:${SHORT_SHA}
|
||||
|
||||
steps:
|
||||
- name: 'python:3.12-slim'
|
||||
id: 'install-and-test'
|
||||
entrypoint: bash
|
||||
args:
|
||||
- '-c'
|
||||
- |
|
||||
pip install -r agents/core-logic/requirements.txt --quiet
|
||||
if [ -d "agents/tests/unit" ]; then
|
||||
python -m pytest agents/tests/unit/ -v
|
||||
fi
|
||||
echo "Install + tests complete."
|
||||
|
||||
- name: 'python:3.12-slim'
|
||||
id: 'eval-gate'
|
||||
entrypoint: bash
|
||||
env:
|
||||
- 'GOOGLE_CLOUD_PROJECT=${_PROJECT_ID}'
|
||||
- 'GOOGLE_CLOUD_LOCATION=${_REGION}'
|
||||
- 'GOOGLE_GENAI_USE_VERTEXAI=True'
|
||||
args:
|
||||
- '-c'
|
||||
- |
|
||||
pip install google-cloud-aiplatform[evaluation] --quiet
|
||||
python agents/eval/run_eval.py
|
||||
|
||||
- name: 'gcr.io/cloud-builders/docker'
|
||||
id: 'build-image'
|
||||
args:
|
||||
- 'build'
|
||||
- '-t'
|
||||
- '${_IMAGE}'
|
||||
- '-f'
|
||||
- 'agents/core-logic/Dockerfile'
|
||||
- 'agents/core-logic'
|
||||
|
||||
- name: 'gcr.io/cloud-builders/docker'
|
||||
id: 'push-image'
|
||||
args: ['push', '${_IMAGE}']
|
||||
|
||||
- name: 'gcr.io/cloud-builders/gcloud'
|
||||
id: 'deploy-cloud-run'
|
||||
args:
|
||||
- 'run'
|
||||
- 'deploy'
|
||||
- '${_SERVICE_NAME}'
|
||||
- '--image=${_IMAGE}'
|
||||
- '--region=${_REGION}'
|
||||
- '--project=${_PROJECT_ID}'
|
||||
- '--service-account=vertex-agent-sa@${_PROJECT_ID}.iam.gserviceaccount.com'
|
||||
- '--no-allow-unauthenticated'
|
||||
- '--min-instances=0'
|
||||
- '--platform=managed'
|
||||
- '--quiet'
|
||||
|
||||
images:
|
||||
- '${_IMAGE}'
|
||||
|
||||
options:
|
||||
logging: CLOUD_LOGGING_ONLY
|
||||
67
docs/CostManagementRules.md
Normal file
67
docs/CostManagementRules.md
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
# Cost Management Rules — propane-will-491900-m5
|
||||
|
||||
## Hard rules
|
||||
1. **Billing budget alert** must be created BEFORE any resource deployment.
|
||||
Alert thresholds: 50% / 75% / 90% / 100% of monthly budget.
|
||||
Start with 5 USD (~55 NOK) for dev/test phases.
|
||||
2. **Agent Runtimes** cost money when deployed — always undeploy via `03-teardown.sh` at end of workday.
|
||||
3. **Vertex AI Endpoints** — same rule as above.
|
||||
4. **Staging bucket lifecycle**: auto-delete objects older than 7 days.
|
||||
5. **Colab Enterprise runtimes** — terminate when not actively developing.
|
||||
6. GCP does NOT enforce a hard spend cap. Budget alerts are email-only by default.
|
||||
|
||||
## Agent Runtime pricing (May 2026)
|
||||
| Resource | Free Tier/month | Paid rate |
|
||||
|---|---|---|
|
||||
| vCPU | 180,000 vCPU-seconds | ~$0.0864/vCPU-hour |
|
||||
| RAM | 360,000 GiB-seconds | ~$0.0090/GiB-hour |
|
||||
|
||||
Agent Runtime does NOT bill for idle/non-running agents (pay-per-use). Always delete at end of session.
|
||||
|
||||
## NEW billing lines (effective Jan 28, 2026)
|
||||
These were previously free/in preview and are now METERED:
|
||||
- **Sessions** — agent interaction contexts
|
||||
- **Memory Bank** — long-term memory storage
|
||||
- **Code Execution** — sandboxed code runs
|
||||
|
||||
→ Trim session histories. Only persist high-value facts to Memory Bank. Disable Code Execution on agents that don't need it.
|
||||
|
||||
## Cloud Run cost profile
|
||||
- `--min-instances=0` → scales to zero — no idle cost (use for dev/test)
|
||||
- `--min-instances=1` → keeps one warm instance — eliminates cold-start (~$4–8/month)
|
||||
- Cleanup: `gcloud run services delete SERVICE_NAME --region=us-central1 --quiet`
|
||||
|
||||
## Lifecycle JSON for staging bucket
|
||||
```json
|
||||
{
|
||||
"rule": [
|
||||
{
|
||||
"action": { "type": "Delete" },
|
||||
"condition": { "age": 7 }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
```bash
|
||||
gcloud storage buckets update gs://propane-will-491900-m5-agent-staging \
|
||||
--lifecycle-file=lifecycle.json
|
||||
```
|
||||
|
||||
## RAG Engine cost warning
|
||||
- If RAG Engine uses a managed Spanner instance as vector DB → billed 24/7.
|
||||
- For dev/test: prefer `us-east1` or `us-east4` until us-central1 allowlist is granted.
|
||||
- Contact: `vertex-ai-rag-engine-support@google.com` for us-central1 allowlist access.
|
||||
|
||||
## Artifact Registry cleanup
|
||||
```bash
|
||||
gcloud artifacts docker images list \
|
||||
us-central1-docker.pkg.dev/propane-will-491900-m5/oavauco-docker \
|
||||
--format="value(IMAGE,DIGEST)" | tail -n +2 | \
|
||||
while read -r image digest; do
|
||||
gcloud artifacts docker images delete "${image}@${digest}" --quiet
|
||||
done
|
||||
```
|
||||
|
||||
## Pub/Sub hard-stop billing automation (optional)
|
||||
Consider: Pub/Sub → Cloud Run Function to auto-disable billing if budget alert fires.
|
||||
Template: https://cloud.google.com/billing/docs/how-to/notify#cap_disable_billing_to_stop_usage
|
||||
77
docs/GCPBestPractices.md
Normal file
77
docs/GCPBestPractices.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# GCP Best Practices — propane-will-491900-m5
|
||||
|
||||
## Platform context (2025–2026)
|
||||
- **Vertex AI Agent Engine** has been rebranded as **Gemini Enterprise Agent Platform**.
|
||||
Agent runtimes are fully managed serverless — no VM management required.
|
||||
- **ADK (Agent Development Kit)** 2.0 is open-source, supports Python/TypeScript/Go/Java/Kotlin.
|
||||
Agents run locally for dev, then deploy to Agent Runtime, Cloud Run, or GKE.
|
||||
- Region: **us-central1** for all resources (low latency, widest feature availability).
|
||||
|
||||
## Required APIs (enable once, idempotent)
|
||||
```
|
||||
aiplatform.googleapis.com # Vertex AI / Gemini Agent Platform
|
||||
storage.googleapis.com # Cloud Storage staging
|
||||
cloudbilling.googleapis.com # Billing Budget API
|
||||
cloudresourcemanager.googleapis.com
|
||||
iam.googleapis.com
|
||||
run.googleapis.com # If using Cloud Run as deploy target
|
||||
monitoring.googleapis.com
|
||||
logging.googleapis.com
|
||||
cloudtrace.googleapis.com
|
||||
secretmanager.googleapis.com
|
||||
artifactregistry.googleapis.com
|
||||
cloudbuild.googleapis.com
|
||||
```
|
||||
|
||||
## ADK install
|
||||
```bash
|
||||
pip install google-adk>=2.0.0 google-cloud-aiplatform[adk,agent_engines]>=1.112.0
|
||||
gcloud components update
|
||||
gcloud auth application-default login
|
||||
```
|
||||
|
||||
## Deploy targets — decision matrix
|
||||
| Target | Best for | Cost model | Auth |
|
||||
|---|---|---|---|
|
||||
| **Agent Runtime** (managed) | Production agents, Memory Bank, Sessions | Per vCPU-second + GiB-second | ADC / SA |
|
||||
| **Cloud Run** | Custom containers, public API endpoints, lower ops | Pay-per-request, scales to 0 | ADC / SA |
|
||||
| **GKE** | Complex multi-agent meshes, custom networking | Cluster uptime + node cost | Workload Identity |
|
||||
|
||||
## ADK 2.0 — new in 2026
|
||||
- Graph workflows and collaborative agents (coordinator + sub-agents)
|
||||
- **ADK Kotlin** — new language support
|
||||
- `adk deploy cloud_run` and `adk deploy agent_engine` CLI commands
|
||||
- Native MCP client and server support
|
||||
- A2A (Agent-to-Agent) protocol support
|
||||
- `SecretManagerClient` integration (requires ADK ≥ 1.29)
|
||||
|
||||
## Four platform pillars (Gemini Enterprise Agent Platform)
|
||||
| Pillar | Key features |
|
||||
|---|---|
|
||||
| **Build** | ADK 2.0, Agent Studio (visual designer), Agents CLI, 200+ models via Model Garden |
|
||||
| **Scale** | Agent Runtime, Sessions, Memory Bank, Code Execution, multi-day workflows |
|
||||
| **Govern** | Agent Identity, Agent Registry, Agent Gateway, Model Armor, Agent Security dashboard |
|
||||
| **Optimize** | Agent Evaluation, Agent Optimizer (auto-clusters failures), Example Store, Observability |
|
||||
|
||||
## Model landscape (May 2026)
|
||||
| Model | Status | Recommended use |
|
||||
|---|---|---|
|
||||
| `gemini-2.5-flash` | GA | Balanced cost/quality — **recommended default** |
|
||||
| `gemini-3-flash` | Public preview | Cheapest option for dev/test |
|
||||
| `gemma-4` | GA (open-source) | Self-hosted / fine-tuning |
|
||||
| ~~`gemini-1.5-*`~~ | **Discontinued** | Migrate immediately |
|
||||
| ~~`claude-3.5-haiku`~~ | **Deprecated** (shutdown Jul 2026) | Migrate immediately |
|
||||
|
||||
## agent-starter-pack (Google official)
|
||||
```bash
|
||||
uvx agent-starter-pack enhance # add CI/CD + eval + observability to existing agent
|
||||
uvx agent-starter-pack create my-agent # scaffold from scratch
|
||||
```
|
||||
Provides: CI/CD (Cloud Build), Evaluation, Observability, Terraform IaC out of the box.
|
||||
GitHub: https://github.com/GoogleCloudPlatform/agent-starter-pack
|
||||
|
||||
## Key constraints
|
||||
- Never use GPU instances (A100/V100) unless business-critical.
|
||||
- Default compute: `e2-micro` for background tasks, up to `e2-standard-4` for Colab.
|
||||
- Agent Runtimes bill when deployed — always run `03-teardown.sh` at end of day.
|
||||
- Cloud Storage buckets do NOT auto-delete — lifecycle rules are mandatory.
|
||||
82
docs/IAMRolesandPermissions.md
Normal file
82
docs/IAMRolesandPermissions.md
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
# IAM Roles and Permissions — propane-will-491900-m5
|
||||
|
||||
## Service account: vertex-agent-sa
|
||||
Minimum roles required:
|
||||
| Role | Purpose |
|
||||
|---|---|
|
||||
| `roles/aiplatform.user` | Deploy/query agents on Gemini Agent Platform |
|
||||
| `roles/storage.objectAdmin` | Read/write staging bucket |
|
||||
| `roles/logging.logWriter` | Write agent logs to Cloud Logging |
|
||||
| `roles/iam.serviceAccountTokenCreator` | Allow ADK to use the SA |
|
||||
| `roles/secretmanager.secretAccessor` | Read secrets at runtime |
|
||||
| `roles/run.invoker` | Call Cloud Run services |
|
||||
| `roles/cloudtrace.agent` | Write traces to Cloud Trace |
|
||||
| `roles/artifactregistry.writer` | Push container images (CI/CD) |
|
||||
|
||||
## Create SA (idempotent)
|
||||
```bash
|
||||
SA_NAME="vertex-agent-sa"
|
||||
SA_EMAIL="${SA_NAME}@propane-will-491900-m5.iam.gserviceaccount.com"
|
||||
|
||||
if ! gcloud iam service-accounts describe "$SA_EMAIL" \
|
||||
--project=propane-will-491900-m5 &>/dev/null 2>&1; then
|
||||
gcloud iam service-accounts create "$SA_NAME" \
|
||||
--display-name="Vertex Agent SA" \
|
||||
--project=propane-will-491900-m5
|
||||
fi
|
||||
|
||||
for ROLE in \
|
||||
roles/aiplatform.user \
|
||||
roles/storage.objectAdmin \
|
||||
roles/logging.logWriter \
|
||||
roles/iam.serviceAccountTokenCreator \
|
||||
roles/secretmanager.secretAccessor \
|
||||
roles/run.invoker \
|
||||
roles/cloudtrace.agent \
|
||||
roles/artifactregistry.writer; do
|
||||
gcloud projects add-iam-policy-binding propane-will-491900-m5 \
|
||||
--member="serviceAccount:${SA_EMAIL}" \
|
||||
--role="$ROLE" --quiet
|
||||
done
|
||||
```
|
||||
|
||||
**Do NOT download a JSON key file** — use Workload Identity or ADC (`gcloud auth application-default login`) instead.
|
||||
|
||||
## MCP Tool User role (new — 2026)
|
||||
Required when using Google's managed MCP servers (BigQuery, Maps, etc.):
|
||||
```bash
|
||||
gcloud projects add-iam-policy-binding propane-will-491900-m5 \
|
||||
--member="user:YOUR_EMAIL" \
|
||||
--role="roles/mcp.toolUser"
|
||||
```
|
||||
|
||||
## Authentication decision tree
|
||||
```
|
||||
Running locally in VS Code?
|
||||
YES → gcloud auth application-default login
|
||||
File: $HOME/.config/gcloud/application_default_credentials.json
|
||||
|
||||
Running on Cloud Run?
|
||||
YES → Attach service account to service (no key file)
|
||||
gcloud run services update SERVICE --service-account=SA_EMAIL
|
||||
|
||||
Running on GKE?
|
||||
YES → Use Workload Identity Federation (keyless)
|
||||
gcloud container clusters update CLUSTER --workload-pool=PROJECT.svc.id.goog
|
||||
|
||||
NEVER use:
|
||||
× gcloud auth activate-service-account (static credentials)
|
||||
× Exporting JSON key files to repo or environment variables
|
||||
```
|
||||
|
||||
## ADC credential search order
|
||||
1. `GOOGLE_APPLICATION_CREDENTIALS` env var (service account JSON path)
|
||||
2. `~/.config/gcloud/application_default_credentials.json`
|
||||
3. Attached service account from metadata server (Cloud Run, GCE, GKE)
|
||||
|
||||
**Best practice**: Never set `GOOGLE_APPLICATION_CREDENTIALS` in production. Let the metadata server handle it.
|
||||
|
||||
## Agent Identity (governance layer — 2026)
|
||||
- Every deployed agent should have its own dedicated service account (Agent Identity).
|
||||
- Naming convention: `agent-{name}-sa@propane-will-491900-m5.iam.gserviceaccount.com`
|
||||
- Register in Agent Registry so all autonomous actions are traceable.
|
||||
Loading…
Reference in New Issue
Block a user