Business Data Architecture: Transformations - Making Architectural Boundaries Enforceable

[AllanHald]

Understanding architecture is one thing. Enforcing it is another.

You understand the concepts - modeling layers organize data flow, data product types structure reuse, platforms own domains. But when a developer asks: "Can I add Revenue Growth % to my Definition layer data product?" - how do you answer?

Without explicit rules, the answer is judgment: "Probably not, that feels like semantic layer logic." With 10 developers making judgment calls, you get 10 different interpretations. Architectural boundaries dissolve.

Transformations make boundaries enforceable. They classify data transformations and specify which layers and data product types each can be used in. The question "Can I add Revenue Growth % to Definition Layer?" gets a concrete answer: "No. That's KPI Calculation - only allowed in Semantic Layer."

This isn't about one developer building one data product. It's about 50 developers across 5 platforms making consistent decisions over 3 years. The architecture stays intact because the rules are explicit, not implicit.

This post defines the transformation framework. It establishes which transformations are allowed in which scope (layer + data product type), provides the catalog structure for organizing them, and includes the process for defining new transformations. Let's start with what transformations actually are in Business Data Architecture.

Transformations

Transformations are the mechanism that makes the framework enforceable. They're logical patterns (not platform-specific code) that define what kinds of change can happen to data - categorized into two types based on their scope.

Two Transformation Categories

1. Business Transformations - Layer-specific

• These transformations implement business logic - changing data meaning, structure, or interpretation based on business rules. Each belongs to specific layers based on its purpose. Examples: Currency Conversion, KPI Calculation, Join Across Sources, Data Type Normalization.

2. Operational Transformations - Allowed everywhere

• These transformations support observability, audit trails, and operational metadata without changing business meaning. They're allowed in any layer because they track operational state, not business logic. Examples: Add Load Timestamp, Add Lineage Metadata, Add Process ID, Add Data Quality Metrics.

Transformation Anatomy

Whether operational or business, every transformation is defined using the same six-part structure. This consistency ensures that anyone in your organization can look at a transformation definition and immediately understand what it does, where it belongs, and what rules apply.

Here's what every transformation contains:

This structure makes transformations complete and actionable - you know what it does, where you can use it, and what rules you must follow.

What Transformations Are NOT

You now understand the anatomy - six elements that define any transformation. But before we show you what a populated transformation catalog looks like, there's a critical distinction to establish: transformations are logical patterns that describe change, not the code that executes it. Here's what that means:

Transformations are logical patterns that describe WHAT change happens to data. They are NOT:

• Artifacts - Not SQL views, dbt models, Python scripts, Databricks notebooks, or any platform-specific code
• Implementation details - Not HOW the change is executed, just WHAT change occurs
• Platform-specific - The same transformation (e.g., "Currency Conversion") can be implemented differently on Snowflake, SAP Datasphere, or Databricks

Example: "Currency Conversion" is a transformation - a logical pattern. It might be implemented as a Snowflake SQL view, a dbt model, or a Python notebook. The transformation defines WHAT (convert currency values), WHERE (which layers), and WHAT LIMITS apply (constraints). The artifact is HOW it's actually executed.

The relationship between transformations and artifacts is covered in our next post on artifact roles - which artifacts can execute which transformations.

You now understand what transformations are: logical patterns with six elements, categorized as operational or business, distinct from the code that implements them. But understanding the concept isn't enough - everyone needs to reference the same definitions.

The Transformation Catalog

How do you ensure 50 developers across multiple platforms all use the same transformation definitions?

A central catalog - your repository of defined transformations that serves as the single source of truth. Without it, each team invents their own "Data Masking" or "Currency Conversion" - same names, different implementations, inconsistent results.

The catalog could be a wiki, a repository, or a metadata system. What matters is that it's:

• Centralized - One place, not scattered across teams
• Accessible - All developers can find and reference it
• Governed - Changes go through a formal process
• Versioned - Track when transformations are added or deprecated

Catalog Structure

Here's what a populated catalog looks like, organized by layer and category. This is NOT exhaustive - these are examples showing structure.

Key observations:

• Ingestion layer: Not shown because it uses only operational transformations - preserving source data without business logic
• Define/Apply pattern: Reference data products define mappings once (e.g., Define Source Metric), and other data products apply them in their specific layers (e.g., Apply Source Metric Definition in Definition layer)
• Multi-layer transformations: Some business transformations appear in multiple layers with different purposes. For example, Access Control appears in both Semantic layer (row-level security for analytical data products) and Sharing layer (consumption-level security for all exposed data)

This catalog gives you a shared vocabulary and consistent patterns across your organization. But what happens when you need a transformation that doesn't exist yet?

Adding New Transformations

Adding a new transformation requires formal definition and communication to ensure consistent reuse across teams.

This five-step process ensures every transformation in your organization follows the same structure, gets reviewed for consistency, and becomes discoverable to all teams. It's the difference between 50 developers each inventing their own "Currency Conversion" and 50 developers reusing one well-defined transformation.

The transformation framework gives you the governance mechanism to keep architecture intact at scale. Transformations define what's allowed where, the catalog organizes them, and the process ensures they're defined consistently.

You now have the complete transformation framework: two categories (operational and business), six-element anatomy, a catalog structure to organize them, and a process to add new ones. This framework makes architectural boundaries enforceable - turning abstract concepts like "layer separation" into concrete rules developers can follow.

Equally important: transformations give you a common language across platforms. When a Snowflake developer and a Datasphere developer both implement "Currency Conversion," they're following the same logical definition - same purpose, same constraints, different code. This prevents business logic from fragmenting into platform-specific interpretations. One architectural definition, multiple platform implementations, consistent behavior.

What's next

Transformations define what logic belongs where (layer + data product type). But how do you ensure a SQL view executes only the transformations it's allowed to? Or that a stored procedure doesn't violate layer boundaries?

This is where Artifact Roles come in. You know what artifacts are - SQL views, stored procedures, graphical views. But not all artifacts can execute all transformations. The next post introduces roles that constrain which transformations an artifact can execute, ensuring your platform implementations respect the architectural boundaries you just defined.

This blog post is part of the Business Data Architecture for Modern Analytics Landscapes series.

⬅️  Previous Post | Next Post  ➡️