Saltar al contenido
BeryX
Volver a las respuestas para RFP

Documentación de BeryX

Este documento se publica en portugués y en inglés; esta es la versión en inglés.

BeryX MCP server

BeryX exposes data quality to AI agents through the Model Context Protocol (Claude Desktop, Claude Code, or any MCP client). The server is a bridge to the HTTP API — it authenticates as a normal user (workspace and RBAC respected) and is read-only on the sources; the only action with effects is requesting an RCA diagnosis (it creates an AI report).

Available tools

Tool What it does
list_datasets Workspace datasets with rows, size and score
get_dataset Detail: columns, open issues, suggestions
list_issues Issues with filters (dataset, severity, status); those with linked external tickets include tickets (key + status)
list_suggestions Improvement suggestions (heuristics + AI)
get_quality_overview Risk, €/month exposure and FinOps totals
get_verdict Verdict for CI gates ("can I publish?"): pass/warn/fail/unknown
get_dataset_context Unified context of a dataset in a single call (see below); sections optional
get_trust_decision Trust decision before consuming, by purpose (agente/rag/treino/bi): trusted/caution/untrusted/unknown, reasons, required_actions and ttl_seconds/valid_until — see TRUST_API.md. Every call is recorded in the audit trail (trust_decision_checked)
get_trust_facts Trust facts of the dataset in a stable envelope (schema_version 1): verdict with policy, agente decision, score and 7d trend, contract, policy, sensitivity counts, AI-readiness, freshness, ownership, lineage, issues and links. detail=minimal (default, no column names) or full — see api/trust-facts.md
list_domains Data products (domains) with owner, score and SLA attainment for the month
get_domain_readiness Readiness of a data product by use case (rag/ml/agente/bi): ready/ready_with_warnings/not_ready with blockers and warnings — see below
get_domain_verdict Verdict of a data product for CI gates ("can I publish the domain?"): pass/warn/fail/unknown aggregated from the datasets, with counts and the verdict of each dataset; with_policy (default true) takes the data policy into account. Same JSON as GET /api/v1/domains/{id}/verdict (card #148)
match_datasets Latest entity matching between two datasets (e.g. CRM customers × ERP customers): rows on each side, pairs, unmatched, ambiguous 1→N, coverage %, keys used and the proposed survivorship (never applied). other_dataset_id optional. It only reads the last completed run, does not launch matching (it is requested in Duplicates › Across datasets) and does not return record values. Team+ plan
diagnose_issue Requests RCA for an issue (returns report_id) — stdio only
search_knowledge Searches the workspace knowledge base (summaries of reports and RCAs, issue resolutions, decisions and notes): filters query, kind and dataset_id; up to 10 entries, without obsolete ones
get_report Status/Markdown content of an AI report
get_failing_rows_sql SQL query that lists the rows failing a rule (rule_id), in the source dialect or another one (dialect: postgres, mysql, mssql, bigquery, snowflake, databricks, sqlite), with quoted names and a limit (limit, default 100). It only returns the text: the MCP never runs it. A rule with no row-level translation (e.g. row_count_min): sql is null, with reason and message. Reads GET /api/v1/rules/{id}/failing-rows.sql (card #400)
get_lineage Lineage of a dataset ("what feeds X?"): upstream and/or downstream (direction) up to hops hops (1 to 3), each dataset with the path in the direction of flow (path: "A → B → C"), the edges with their origin (kinds: declared, dbt, openlineage, parsed, inferred), the linking columns and the column map when there is one, and the downstream exposures. Reads GET /api/v1/datasets/{id}/lineage/traverse (card #173)
impact_of What breaks if a dataset or a column changes (column_name optional): downstream datasets, exposures and destinations outside the warehouse and, with the column, the columns derived from it (columns, only with column lineage). Datasets and columns up to 3 hops (max_depth), exposures up to 6 (exposure_max_depth), as in the impact analysis of an issue. Reads GET /api/v1/datasets/{id}/lineage/impact
path_between Shortest path between two datasets (up to 6 hops): path, path_text ("A → B → C") and the edges; with no path from A to B it searches from B to A (direction: target_to_source). Reads GET /api/v1/datasets/{id}/lineage/path
find_datasets Compound search: query, layer (bronze/silver/gold), domain (id or name), has_pii, min_score and usage_class (hot/active/cold/dead) applied together; returns layer, score, usage class and the columns with PII (names only). Layer, PII and score filter the first 500 datasets that match the text, the domain and the usage class; with more, truncated: true and total only counts those 500. Reads GET /api/v1/datasets/find

The four traversal tools (card #173) are deterministic: they call no LLM, so they work with BYOK, with AI turned off and without a key. The assistant chat uses the same functions (backend/app/domain/lineage/traversal.py) and the same summaries. Caps: 40 datasets and 60 edges (truncated: true when they are hit), names only — never the owner, the email or the URL of an exposure. A dataset in a restricted domain the caller has no access to appears as "Dataset restrito" (restricted dataset), with an opaque id (restrito-N) instead of the real one and without columns. An edge with one end in another workspace is ignored.

Resources (read-only, application/json)

URI Content
quality://overview Same summary as get_quality_overview
datasets://catalog Dataset catalogue (same format as list_datasets)
datasets://{dataset_id} Detail of a dataset (same format as get_dataset)
datasets://{dataset_id}/verdict CI verdict of the dataset
datasets://{dataset_id}/lineage Direct lineage of the dataset (dataset-lineage): the result of get_lineage with 1 hop in both directions (card #173)
datasets://{dataset_id}/context Unified context of the dataset (same format as get_dataset_context)
datasets://{dataset_id}/trust-decision Trust decision for the agente purpose (same JSON as the API and get_trust_decision)
datasets://{dataset_id}/trust-decision/{purpose} Fitness of the dataset at the point of consumption for purpose (agente/rag/treino/bi; ml = treino): the same decision, with required_actions and valid_until (card #151)
datasets://{dataset_id}/trust-facts Trust facts of the dataset (dataset-trust-facts, detail=minimal): the same JSON as the API and get_trust_facts (card #153)
domains://catalog Data products of the workspace: id, name, owner, score, attainment_current_month_pct (same format as list_domains)
glossary://terms Business glossary (glossary, Team+ plan): terms with definition, synonyms, the approved columns and the quality inherited from them — score, open_issues_total and explanation (card #169)
domains://{domain_id}/readiness Readiness of the data product for bi (same JSON as the API and get_domain_readiness)
domains://{domain_id}/readiness?use_case=rag Readiness for another use case (rag/ml/agente/bi; treino = ml)
domains://{domain_id}/verdict CI verdict of the data product, with the data policy (domain-verdict): the same JSON as the API and get_domain_verdict (card #148)

Unified context per dataset (card #146)

get_dataset_context / datasets://{dataset_id}/context call GET /api/v1/datasets/{id}/context and return, in a single read, what used to require 6–8 calls. No LLM; limited to 6,000 characters (JSON) — when it does not fit, it cuts knowledge first and then schema and sets truncated: true with truncated_fields. Each section includes available; without data (e.g. a dataset with no scan) it comes with available: false and a reason instead of an error.

Section Content
identity name, schema/table, connector_type, medallion layer, domain {id, name}, owners by name only
schema columns: name, type, nullable, description, sensitivity and pii_type
quality latest score and dimensions, trend of the last 5 scores, open issues by severity/type, verdict, failed rules (without observed values)
contract has_active, observed_only, active version (number/status), compliance (status + up to 5 problems)
lineage upstream/downstream at 1 hop, count of external nodes, has_column_lineage, exposures name/type/maturity only
readiness overall AI-readiness verdict, by use case, and blockers (without samples)
trust trust score, grade, components with evidence, blockers
knowledge top-5 non-obsolete KB entries (body ≤ 300 chars) and an excerpt of the published README (≤ 600 chars)
usage usage class and queries_30d
freshness last_modified_at of the latest metadata snapshot and last_profiled_at

No sample value, email or URL leaves (free text is sanitised — [email], [url]).

The resources reuse the DataVaultClient and the summarize_* functions shared with the chat — tools and resources return exactly the same data.

Data product readiness

get_domain_readiness / domains://{domain_id}/readiness?use_case= proxy GET /api/v1/domains/{id}/readiness and return the API JSON as is (the chat tool returns the same, read directly from the session). An agent or pipeline that asks "is the Sales data product ready for RAG?" makes one call instead of N. States, codes and the CI gate: CI_QUALITY_GATE.md.

Readiness uses, per dataset, the same reading as the Trust API (#150) — the two responses never contradict each other — but has its own vocabulary (it aggregates per product):

Trust API (get_trust_decision, per dataset) Readiness (get_domain_readiness, per domain)
no_scan (decision unknown) no_scan — blocker
verdict_fail verdict_fail — blocker (and contract_invalid if the active contract failed)
verdict_warn verdict_warn — warning
readiness_blocker ai_readiness_blocked — blocker
readiness_low — (the score comes in datasets[].ai_readiness.score)
scan_stale stale_scan — warning (same rule: 2× the cadence)
critical_incident (incidents, with lineage) open_critical_issues — critical issues of the dataset itself
freshness_sla_breached slo_breach_open / attainment_below_objective — warnings, only with an SLA on the domain
pii_unmasked_gold, warning_incident — (use the Trust API per dataset)
— (the Trust API is per dataset) deprecated — blocker; draft — warning: data product lifecycle (card #168)

LangChain example — check the product's readiness before indexing (alongside the per-document gate in RAG_GATE.md):

from beryx import BeryX

readiness = BeryX().domain_readiness(VENDAS_DOMAIN_ID, use_case="rag")
if not readiness.ready:  # not_ready: do not index
    raise RuntimeError(readiness.summary())
vectorstore.add_documents(splitter.split_documents(loader.load()))

Installation

cd backend
pip install -e ".[mcp]"

Configuration (Claude Desktop / Claude Code)

{
  "mcpServers": {
    "datavault": {
      "command": "/caminho/backend/.venv/bin/python",
      "args": ["-m", "app.mcp_server"],
      "cwd": "/caminho/backend",
      "env": {
        "DATAVAULT_API_URL": "http://localhost:8000",
        "DATAVAULT_EMAIL": "utilizador@empresa.pt",
        "DATAVAULT_PASSWORD": "…"
      }
    }
  }
}

In Claude Code: claude mcp add datavault -- /caminho/backend/.venv/bin/python -m app.mcp_server (with the same environment variables).

Streamable HTTP mode (remote)

Production server — https://mcp.praxis.lbostuff.tech/mcp

The remote MCP is published (card #311): Cloud Run service datavault-mcp, managed TLS, nothing to install. All you need is a dvk_ API token (Team → API tokens, admins only) — from any machine, without this repository:

claude mcp add --transport http datavault https://mcp.praxis.lbostuff.tech/mcp \
  --header "Authorization: Bearer dvk_…"

In another MCP client with HTTP support (Cursor, VS Code, Codex, Claude Desktop via mcp-remote), the app's Connect your agent page generates the exact configuration — see AGENTES_CODIGO.md. Generic format:

{
  "mcpServers": {
    "datavault": {
      "type": "http",
      "url": "https://mcp.praxis.lbostuff.tech/mcp",
      "headers": { "Authorization": "Bearer dvk_…" }
    }
  }
}

Test with the MCP Inspector: npx @modelcontextprotocol/inspector → Streamable HTTP transport, URL https://mcp.praxis.lbostuff.tech/mcp, header Authorization: Bearer dvk_….

Authentication: every request must carry Authorization: Bearer dvk_…. Without a token, or with a session JWT, the response is 401. The token is validated against the API itself (revocation and workspace respected) and is forwarded on every call: each customer sees only their own workspace. If the API does not respond, 503. OAuth 2.1 comes in a later slice (MCPV2).

Read-only surface: over HTTP the server is stateless and does not expose diagnose_issue — and dvk_* tokens are read-only on the API (any method other than GET returns 403), so no tool or resource writes.

Host and origin (anti-DNS-rebinding): the SDK's protection only switches on by itself with a bind on localhost. The server validates the Host header itself, before authentication, against an allowlist: the host of DATAVAULT_MCP_PUBLIC_URL plus those of DATAVAULT_MCP_ALLOWED_HOSTS (comma-separated; in production, the deterministic run.app URL). Outside the list: 421. A request with Origin (browser) only passes if the origin is https:// on a host in the list (http:// only on loopback); otherwise 403. With a bind on 0.0.0.0 and an empty allowlist, the server refuses to start. With a bind on loopback, 127.0.0.1/localhost/::1 are accepted as in the SDK.

Known limits:

  • Revocation up to 60 s: positive token validations are cached per instance for 60 s (by hash — the token is never stored or logged). A revoked token may keep working until then.
  • Shared per-IP rate limit: the API applies a per-IP limit (1,200 requests/min per instance, DATAVAULT_API_RATE_LIMIT_IP_REQUESTS) and another per token (300/min). All MCP clients reach the API from the egress IPs of datavault-mcp, so they share the per-IP limit; the per-token limit remains each client's own. The MCP does not forward X-Forwarded-For (it would be spoofable). Each tool makes 1–3 requests to the API.
  • Cold start: min-instances 0 — the first request after a period without traffic takes a few seconds.

Operations (deploy)

deploy/gcp_deploy.sh (section 5e) creates the service from the backend image (mcp extra in CONNECTOR_EXTRAS) with python -m app.mcp_server --transport http --host 0.0.0.0 --port 8080:

SA datavault-mcp — only logging.logWriter and monitoring.metricWriter; no Cloud SQL, no secrets, no VPC
Env DATAVAULT_API_URL (https://api.${DOMAIN} — with ingress restricted to the LB, the api's run.app does not respond to a Cloud Run service without VPC), DATAVAULT_MCP_PUBLIC_URL, DATAVAULT_MCP_ALLOWED_HOSTS
Scale 1 vCPU · 512 MiB · concurrency 20 · min 0 · max MCP_MAX_INSTANCES (3)
Probe TCP startup on 8080 (MCP_STARTUP_PROBE) — all of /mcp requires Host and token
Domain In production, mcp.${DOMAIN} is a host on the edge Application Load Balancer (deploy/lb_apply.sh, DEPLOY_GCP.md §4); without the LB, map_domain datavault-mcp "mcp.${DOMAIN}" and CNAME mcp → ghs.googlehosted.com. (with DNS_ZONE/DNS_PROJECT the script creates it)

The api receives DATAVAULT_MCP_PUBLIC_URL=https://mcp.${DOMAIN}/mcp, and that is what makes GET /ide/agent-config and the "Connect your agent" page generate the real configuration. CD updates the datavault-mcp image with the backend's. deploy/validate_prod.sh (step 16c, deploy/check_mcp_remote.sh) proves from outside, on every deploy: the advertised URL responds, without a token → 401, initialize and tools/list with a disposable dvk_ token (created, used and revoked) include get_verdict and get_trust_decision, and a Host outside the allowlist → 421 (via the service's legacy *.a.run.app URL, which reaches the container but is not on the list).

Your own server (self-hosted / development)

To run HTTP mode locally (endpoint /mcp):

DATAVAULT_API_URL=http://localhost:8000 \
python -m app.mcp_server --transport http --host 127.0.0.1 --port 8765

Also configurable via the environment: DATAVAULT_MCP_TRANSPORT=http, DATAVAULT_MCP_HOST, DATAVAULT_MCP_PORT. Exposed on another machine (bind 0.0.0.0, behind TLS), set DATAVAULT_MCP_PUBLIC_URL to the public URL — it is the Host allowlist. In the app, the same setting on the api makes "Connect your agent" point to it; when it is empty, the configs point to http://127.0.0.1:8765/mcp and the page shows the command above.

Recommendation (stdio, with DATAVAULT_EMAIL/DATAVAULT_PASSWORD): create a member user dedicated to the MCP (POST /auth/users, admins only) — members cannot delete connections or invite users.

Example questions for the agent

  • "Which dataset has the worst score and how much does it cost us per month?"
  • "Can the clientes dataset be published?" (uses get_verdict)
  • "Can I use gold.clientes in an agent? Who is the owner and what has gone wrong before?" (uses get_dataset_context)
  • "Can my agent read gold.clientes now? And for RAG?" (uses get_trust_decision — the answer also says how long it is valid for)
  • "What is known about the reliability of gold.encomendas? Who is the owner and is there unmasked PII?" (uses get_trust_facts)
  • "How many CRM customers do not exist in the ERP?" (uses match_datasets)
  • "Has this happened before? How did we resolve it last time?" (uses search_knowledge)
  • "Are there open critical issues? Diagnose the most recent one."
  • "What deduplication suggestions do we have with ready-made SQL?"