How to read this manual
This manual uses ASD-STE100 Simplified Technical English. The writing rules make each instruction short and clear. You can read it quickly, also when English is your second language.
The manual uses three types of safety notice. They come before the step that they apply to.
A WARNING tells you about a risk of wrong business numbers or exposure of personal data. Obey all WARNINGs.
A CAUTION tells you about a risk of damage to the service, the files, or the deployment.
A NOTE gives you more information to help you do the task.
Who does what
These are working roles. One person can have more than one role.
Owns the meaning
Defines terms, edits the Wren files, and compares results with evidence.
Approves definitions
Confirms financial definitions, status meanings, and reporting exceptions.
Owns the pipeline
Confirms the deployed dbt code, the source structure, and pipeline health.
Owns the server
Publishes files, restarts Wren, reads logs, and restores old versions.
The system in one picture
Data goes through two layers. The data layer moves and prepares the records. The meaning layer tells an AI what the records mean. You, the analyst, manage the meaning layer.
The kitchen prep
dbt cleans raw ingredients before service. It runs on a schedule, not when you ask a question.
The engine
Trino runs SQL and reads tables from many storage places with one language.
The translator
Wren changes business names like product_data into real table locations for Trino.
The universal plug
MCP lets any AI app use Wren tools with one standard connection.
A Wren build does not refresh data. A Wren restart does not correct a dbt calculation. If the source number is wrong, speak to the data engineer.
Terms in plain words
Type a word to filter the list. The third column gives an everyday comparison.
| Term | Plain meaning | Like… | In this project |
|---|---|---|---|
| Ontology | The agreed map of business things and how they connect. | A family tree for data. | A buyer places an order for a product. |
| Semantic layer | The layer that gives data business names and fixed calculations. | A restaurant menu over the kitchen. | Wren shows product_data, not a long table name. |
| Context | Everything the AI can read about the data before it writes SQL. | A briefing pack for a new staff member. | Models, relationships, rules, and SQL examples. |
| Model | A named set of records that Wren makes available. | A labelled drawer in a cabinet. | product_data, order_data. |
| Column | One named value in each record. | One box on a form. | product_uid, title. |
| Data type | The kind of value in a column. | "Number only" or "text only" on a form. | BIGINT = whole number. VARCHAR = text. |
| Grain | What one row represents. | "One line per receipt" or "one line per item on the receipt". | One row per product is not one row per invoice. |
| Primary key | A column that identifies exactly one row. | A national ID number. | product_data.id. |
| Join key | The value that connects records in two models. | The order number on a delivery slip. | content_id → product_data.id. |
| Relationship | A declared, checked connection between two models. | A road between two drawers. | Entries in relationships.yml. |
| Cardinality | How many records can match on each side of a join. | Many students, one teacher. | MANY_TO_ONE: many refund rows → one product. |
| Join | A SQL step that combines matching records. | Stapling the delivery slip to the order. | Add a product title to each transaction. |
| Fan-out | A join that copies rows, so totals become too large. | Counting one bill three times. | One invoice amount repeated on 3 transaction rows. |
| Dimension | A value that you group or filter by. | The labels on a pivot table's rows. | Product title, status, month. |
| Measure | A value that you calculate from many rows. | The total at the bottom of a column. | Sum of approved amounts. |
| Metric | A measure with an agreed business definition. | "Revenue" as finance defines it. | Needs amount, date basis, exclusions, currency. |
| Aggregation | A calculation over many rows. | Adding up a shopping list. | SUM, COUNT, AVG. |
| MDL | Modeling Definition Language. Wren's compiled description of all models. | The printed rulebook from many draft pages. | target/mdl.json. Never edit it by hand. |
| Metadata | Information that describes the data. | The label on a jar. | metadata.yml descriptions. |
| Reference SQL | SQL that builds a model's records from source tables. | A recipe for one drawer's contents. | ref_sql.sql in a model folder. |
| View | A saved query that acts like a table. | A saved filter on a spreadsheet. | event_refund_export. |
| Modeled SQL | SQL that uses Wren names. | Ordering from the menu. | SELECT id FROM product_data. |
| Physical SQL | SQL that uses real database locations. | Going into the kitchen yourself. | catalog.schema.table. |
| Catalog | A named data connection in Trino. | A building. | tiptip_datalake_prd_s7jc. |
| Schema | A group of tables inside a catalog. | A floor in the building. | data_staging, data_mart. |
| dbt model | Code that prepares one table or view. | A prep recipe in the kitchen. | content_svc_product_clean.sql. |
| Lineage | The path from source data to an answer. | A parcel tracking history. | App DB → dbt → Trino table → Wren model → answer. |
| Business rule | An instruction about meaning or use, in plain text. | A note in the staff handbook. | knowledge/rules/*.md. |
| SQL pair | A confirmed question with its approved SQL. | A worked example in a textbook. | knowledge/sql/*.md. |
| MCP | Model Context Protocol. A standard plug between AI apps and tools. | A USB-C port. | https://wren.tnexti.net/mcp. |
| Build | Converts project files into compiled MDL. | Spell-check plus print. | Proves the structure is valid. Not that numbers are right. |
| Dry plan | Shows the translated SQL without running it. | Reading the route before you drive. | MCP tool dry_plan. |
| Dry run | Asks the database to check SQL without returning rows. | A test start of the engine. | MCP tool dry_run. |
| Reconciliation | Comparing your result with a trusted result. | Matching a bank statement to a ledger. | Same period, same counts, same totals. |
| Freshness | How recently the data arrived. | The "baked on" date on bread. | A timestamp alone does not prove the pipeline finished. |
| PII | Personal data that can identify a person. | Anything on an ID card. | Names, emails, phone and ID numbers. |
| WIB | Western Indonesia Time, UTC+7. | Jakarta clock time. | Some dbt timestamps are already WIB. |
The words catalog and schema have two uses. In wren_project.yml they name the Wren namespace (data_mart). In a model's table_reference they name the real table location. These can be different. product_data reads from data_staging.
The current ontology
The project tiptip_poc has 20 models, 3 views, 8 relationships, 5 rule files, and 6 SQL examples. Fig. 2 shows the refund cluster. All its relationships are many-to-one.
| Domain | Objects | Rules file |
|---|---|---|
| Refunds and event cancellation | event_refund_transactions, event_ticket_holder_data, view event_refund_export | refund_model.md |
| Orders and bookings | order_data, product_booking_data, payment_data | order_booking_lifecycle.md |
| Vouchers and coins | wallet_coin_ledger_data | voucher_usage.md |
| War Room — Ticketing | war_room_revenue, war_room_operations, war_room_source_checks, war_room_creator_agency_history, war_room_report_reference, views war_room_report_components, war_room_reconciliation | war_room_ticketing.md |
| War Room — SatuSatu | war_room_satusatu_base, _components, _operations | war_room_satusatu.md |
Which file to change
Each type of change has one correct place. Use Fig. 3 to find it.
flowchart TD
Q{"What do you need to change?"}
Q -->|"Meaning of a column"| M1["models/{name}/metadata.yml
column description"]
Q -->|"Show a new source column"| M2["metadata.yml columns list
+ ref_sql.sql if the model has one"]
Q -->|"Connect two models"| R["relationships.yml"]
Q -->|"Reusable calculation"| C{"Is it shared by other
dashboards and pipelines?"}
C -->|"Yes"| D["dbt project
(with the data engineer)"]
C -->|"No, Wren only"| V["ref_sql.sql or views/{name}/"]
Q -->|"Date, status, currency convention"| K["knowledge/rules/{topic}.md"]
Q -->|"A confirmed worked example"| S["knowledge/sql/{question}.md"]
Q -->|"Evidence or test SQL"| E["queries/ or docs/"]
wren-project/
├── wren_project.yml Project settings (namespace, data source)
├── models/<model_name>/
│ ├── metadata.yml Source location, columns, descriptions, key
│ └── ref_sql.sql Source query (only some models)
├── relationships.yml Connections between models
├── views/<view_name>/metadata.yml Saved query over Wren models
├── knowledge/
│ ├── rules/<topic>.md Business instructions in plain text
│ └── sql/<question>.md Confirmed question + modeled SQL
├── tests/ Synthetic SQL behaviour checks
└── target/mdl.json Generated. Do not edit.A description or a rule does not force a calculation. The AI can ignore text. Put required calculations in executable SQL. Use the rule to explain the meaning and the exceptions.
Removing a column description does not remove access to the column. Database permissions still control access.
Change lifecycle
Every change follows the same route. Tasks A to J below follow this order.
flowchart LR A["A · Define
expected result"] --> B["B · Trace source
dbt + Trino"] B --> E["C–G · Edit
Wren files"] E --> H{"H · Validate
in staging"} H -->|"Fail"| E H -->|"Pass"| RV["Review
+ approve"] RV --> I{"I · Publish
+ restart"} I -->|"Build or answer fails"| J["J · Restore
previous version"] J --> I I -->|"Pass"| REC["Record version
+ evidence"]
Define the expected result
- Write the business question in one sentence.
- Name the person who owns the definition.
- Write what one output row represents (the grain).
- List the output columns.
- List the filters and the exclusions.
- Write the date column and the time zone (UTC or WIB).
- Write the currency and the rounding rule.
- Record the expected row counts and totals.
- Record the source and the date of the trusted result.
Business question:
Definition owner:
One output row represents:
Required columns:
Filters and exclusions:
Date column and time zone:
Currency and rounding:
Trusted comparison and date:
Expected counts and totals:
Known exceptions:
Affected models, views, rules, and SQL pairs:
dbt environment and deployed revision:
Reviewer:Trace a source in dbt and Trino
- Open the model file
models/<name>/metadata.yml. - If the model has
ref_sql.sql, open that file also. - Record each physical table that the model reads.
- Find the dbt model that builds each table.
- Read its SQL, configuration, macros, and column descriptions.
- Read the dbt tests for uniqueness and nulls.
- Ask the data engineer to confirm the deployed environment and revision.
- Run
DESCRIBEon the table in Trino. - Compare the Trino columns with the Wren column list.
- Record each difference before you change a definition.
dbt SQL contains {{ ref(...) }} and {{ source(...) }}. These are dbt instructions, not Trino SQL. Use the compiled SQL from the deployment to see the real query.
A dbt unique_key setting does not prove that the data is unique. Check the test results and the data.
Example: trace product_data
| Wren model | wren-project/models/product_data/metadata.yml |
|---|---|
| Trino table | tiptip_datalake_prd_s7jc.data_staging.content_svc_product_clean |
| dbt candidate | production-tencent/models/data_staging/contents_svc_clean/content_svc_product_clean.sql |
| Exposed now | id, product_uid, title |
DESCRIBE tiptip_datalake_prd_s7jc.data_staging.content_svc_product_clean;
-- Check a proposed primary key
SELECT
COUNT(*) AS row_count,
COUNT(id) AS nonnull_id_count,
COUNT(DISTINCT id) AS distinct_id_count
FROM tiptip_datalake_prd_s7jc.data_staging.content_svc_product_clean;For a large table, agree the scan scope with the data owner first. A sample does not prove uniqueness outside that sample.
Add a column to a model
- Make sure that Trino shows
status_nameas a text column. - Open
wren-project/models/product_data/metadata.yml. - Add the item below inside the existing
columnslist. - Keep the existing columns and
primary_key: id. - Write a description that tells what the column is not, if confusion is possible.
- Do Task F to add the matching rule.
- name: status_name
type: VARCHAR
properties:
description: >-
Product publication status from the cleaned dbt product model.
This is not an order, booking, or refund status.describe_model shows status_name. A query returns the same values as Trino.Make a new model only if necessary
First look in models/ for a model with the same source and grain. Extend that model if it exists. A new table model has this structure:
name: <model_name>
table_reference:
catalog: tiptip_datalake_prd_s7jc
schema: data_staging
table: <physical_table>
properties:
description: One row per <thing>. Use <key> to connect <other model>.
columns:
- name: id
type: BIGINT
properties:
description: Internal identifier.
primary_key: idShow only the columns that the approved analysis needs. Do not add PII columns without approval.
Use spaces, not tabs, in YAML. Use the real source data type. A model uses table_reference or reference SQL, not both.
Add or check a relationship
join_type | Meaning, from first model to second model |
|---|---|
ONE_TO_ONE | Each key matches at most one row on each side. |
MANY_TO_ONE | Many first-model rows match one second-model row. |
ONE_TO_MANY | One first-model row matches many second-model rows. |
MANY_TO_MANY | Many rows on both sides share the key. Use with great care. |
See the fan-out problem
| invoice | amount |
|---|---|
| INV-001 | 300,000 |
| invoice | ticket |
|---|---|
| INV-001 | T1 |
| INV-001 | T2 |
| INV-001 | T3 |
| invoice | ticket | amount |
|---|---|---|
| INV-001 | T1 | 300,000 |
| INV-001 | T2 | 300,000 |
| INV-001 | T3 | 300,000 |
| SUM(amount) | 900,000 ✗ | |
| Correct | 300,000 ✓ | |
Do not use DISTINCT to hide a fan-out. Correct the join, or calculate at the correct grain before the join.
Procedure
- Write the grain of both models.
- Count null keys on both sides.
- Check the "one" side for duplicate keys. Use the query below.
- Count the rows before the join and after the join.
- Compare the money totals before the join and after the join.
- Count the rows that have no match.
- If all checks pass, add the relationship to
relationships.yml.
-- Step 3: must return no rows
SELECT id, COUNT(*) AS key_rows
FROM product_data
GROUP BY id
HAVING COUNT(*) > 1
LIMIT 20;
-- Step 4: for MANY_TO_ONE + LEFT JOIN, both counts must be equal
SELECT COUNT(*) AS source_rows FROM event_refund_transactions;
SELECT COUNT(*) AS joined_rows
FROM event_refund_transactions t
LEFT JOIN product_data p ON t.content_id = p.id; - name: event_refund_transactions_product
models:
- event_refund_transactions
- product_data
join_type: MANY_TO_ONE
condition: event_refund_transactions.content_id = product_data.idApply the same date or event filter to both count queries. A relationship does not add a join to a view automatically. Write the join in the view SQL.
Add a calculation or a view
| Put it in | When | Example |
|---|---|---|
| dbt | Other dashboards or pipelines need the same logic. | Cleaning product status codes. |
ref_sql.sql | A direct table map cannot make the records you need. | payment_data: one metadata record per invoice. |
| Wren view | A saved query over existing Wren models. | event_refund_export. |
Change model SQL
- Open the model's
ref_sql.sql. - Check each source table and each join.
- Add the new expression to the SELECT list.
- Add the output column to
metadata.yml. - Check the grain and the totals again.
Add a view
- Make sure that no existing object answers the question.
- Make the folder
wren-project/views/<view_name>/. - Make the file
metadata.ymlin that folder. - Write the grain in the description.
- Write the SQL with Wren model names only.
name: published_product_directory
properties:
description: One row per product with the publication status Published.
statement: |
SELECT id, product_uid, title
FROM product_data
WHERE status_name = 'Published'For a financial metric, record the formula, date basis, grain, currency, exclusions, and rounding. Do not change VAT formulas or finance adjustments without owner approval.
A view does not make a Trino table. It does not refresh data. Check the exact status spelling in Trino before you filter on it.
Add a business rule
- Make the file
wren-project/knowledge/rules/<topic>.md. - Write one statement per line. Use short sentences.
- Write what the term means and what it does not mean.
- Verify each statement against data or the owner.
- Record the owner and the source in the change record.
## Product publication
Use product_data to resolve product identifiers.
Use id to connect product_data with transaction content_id.
Use product_uid to resolve an external event detail identifier.
Titles can repeat.
status_name describes product publication.
It does not describe payment, order, booking, or refund status.
Published status alone does not prove current ticket availability.get_instructions returns the rule.Rules guide the AI. They do not force it. Configure each AI client to call get_instructions before it queries.
Save a confirmed SQL example
- Make the file
wren-project/knowledge/sql/<question>.md. - Write the question in
nlas a user would ask it. - Write the approved modeled SQL in
sql. - Add a note about the limits of the example.
- Remove customer data, IDs from real cases, and credentials.
- If a definition changes, update all related examples in the same change.
---
nl: "List product identifiers and titles for products with publication status Published."
sql: |
SELECT id, product_uid, title
FROM product_data
WHERE status_name = 'Published'
ORDER BY id
source: user
datasource: trino
tags:
- products
- publication
---
This query uses product publication status.
It does not check ticket availability or payment status.
Check result limits before treating the returned list as complete.recall_queries can return the example for a similar question.MCP store_query is disabled in this deployment. Save examples only through reviewed files in Git. Recall can return a different example or none. Keep essential instructions in the rule file.
Validate a candidate
The production container reads ./wren-project directly. Do not edit files there to test. Use a separate checkout.
- In the candidate checkout, build to a temporary file.
- Fix each build error. Build again.
- Load the candidate into a separate staging Wren MCP service.
- Start a new chat with the staging service.
- Call
get_instructions. Look for the new rule. - Call
describe_modelfor each changed model. - Call
recall_querieswith the test question. - Call
dry_plan. Read the translated tables, joins, and filters. - Call
dry_runfor the proposed SQL. - Call
run_sqlfor the approved read-only checks. - Compare the results with the change record from Task A.
- If you changed War Room SQL, run the synthetic tests.
# Step 1 — build without replacing the production manifest
docker compose run --rm --no-deps --entrypoint wren wren-mcp \
context build --path /project --output /tmp/wren-validation-mdl.json
# Step 12 — synthetic checks (staging)
docker compose exec -T wren-mcp python /project/tests/test_war_room.py
docker compose exec -T wren-mcp python /project/tests/test_war_room_satusatu.pyAcceptance checks
| Check | Evidence you must have |
|---|---|
| Source mapping | Correct deployed table. Compatible data types. |
| Primary key | No duplicates. No nulls in scope. |
| Relationship | Cardinality, unmatched rows, row counts, and totals checked. |
| Meaning | Statuses, date basis, time zone, currency, rounding are correct. |
| Result | Counts and totals agree with the trusted result. |
| Edge cases | Empty results, nulls, refunds, and date boundaries behave as defined. |
| Context | MCP shows the new structure and rules. |
| Examples | Related SQL pairs still use valid objects. |
| Output | No hidden truncation. No unintended PII. |
A successful build proves the structure only. A successful dry run proves the SQL is valid only. Neither proves that the business number is correct. Only reconciliation proves that.
Publish the change
- Record the current deployed Git revision.
- Publish the reviewed source files to the deployment checkout.
- Restart the Wren MCP service.
- Read the startup log.
- Make sure that the log shows "Built:" before "Starting Wren MCP Server".
- Start a new chat with the deployed MCP service.
- Do steps 5 to 11 of Task H again.
- Record the release revision, the check time, and the result.
docker compose restart wren-mcp
docker compose logs --since=5m wren-mcp==> Building Wren context...
Built: <model-count> models, <view-count> views → /project/target/mdl.json
==> Starting Wren MCP Server on port 8080...If the log shows Context build initial attempt failed, the release failed. The server can still start with the old context. A running container does not prove that the new ontology is active.
Wren does not reload files automatically. Old chats can keep old tool results. Always test in a new chat. The public endpoint is https://wren.tnexti.net/mcp. Use the approved client sign-in.
Restore the previous version
- If reports use the wrong definition, stop those reports.
- Find the previous approved Git revision.
- Restore the models, views, relationships, rules, and SQL pairs together.
- Restart Wren with the commands in Task I.
- Make sure that the build succeeds in the log.
- Ask the previous acceptance questions in a new chat.
- Record the failure and the restored revision.
Do not restore only target/mdl.json. The source files and the compiled file must agree. A Wren restore does not undo a dbt data change.
Troubleshooting
| Symptom | Check | Action |
|---|---|---|
| New column is not in MCP | Build log, loaded MDL, active server. | Fix the build error. Restart. Test in a new chat. |
| Trino cannot find a column | Physical table structure and model SQL. | Correct the mapping, or request the upstream change. |
| Total increased after a join | Duplicate keys and grain on both sides. | Correct the relationship, or aggregate at the correct grain first. |
| SQL runs but the answer is wrong | Date basis, exclusions, status source, formula. | Correct the SQL or the rule. Reconcile again. |
| AI ignores a rule | Result of get_instructions and client instructions. | Make the client fetch rules. Put stable logic in SQL. |
| Recall returns an old example | Files in knowledge/sql/, memory index. | Correct or remove old pairs. Rebuild the index if used. |
| dbt code disagrees with Trino | Deployed revision, target, compiled SQL. | Solve the difference with the data engineer first. |
| Dates are 7 hours wrong | Upstream conversion and timestamp type. | Confirm UTC or WIB before you convert again. |
| A list looks incomplete | Tool row limits, row counts, export. | Confirm completeness before you use it as evidence. |
| Server starts after a failed build | Entrypoint warning, old manifest. | Treat the release as failed. Fix or restore (Task J). |
Project facts to keep
These meanings are already verified. Do not change them without the owner.
product_uid is the external ID. Refund rows use internal content_id. Resolve through product_data.id first. Titles can repeat.
order_data has one row per order ID. Its invoice ID is not a verified unique key.
event_refund_transactions can have many rows per invoice. Check the export grain.
product_booking_data takes the lowest booking ID per invoice.
payment_data picks metadata deterministically. It does not rank success or recency.
Order status, booking payment status, and transaction flags describe different records.
A source refund date does not prove that the payment provider settled the refund.
Some dbt staging timestamps are already WIB (UTC+7). Check before you convert.
SQL revenue inputs are not final Net Revenue. Report references are dated snapshots, not live data.
Review checklist
Tick each item before you ask for publication. Your ticks stay in this browser only.
References
In the repository
docs/analyst_ontology_guide.md | Markdown source for this manual |
docs/wren_context_workflow.md | Technical context workflow |
docs/data_transformation_wren_review_20261023.md | dbt review and source mappings (dated evidence) |
docs/refund_domain_mcp.md | Refund domain and accepted limits |
docs/war_room_ticketing.md | Ticketing models and validation |
docs/satusatu_model_improvement_proposal.md | SatuSatu models and validation |
queries/sample-query-using-product-uid.sql | Product identifier example |
Official documentation
- Wren OSS — Model your business
- Wren OSS — MDL project format
- Wren OSS — MCP tools
- dbt — Data tests
- Trino — DESCRIBE
- ASD-STE100 Simplified Technical English
Online documentation can describe newer features. For commands and syntax, use the pinned versions in this repository first.