Overview of the Enterprise Data Warehouse (EDW)
What is an EDW?
Section titled “What is an EDW?”Trimble’s EDW is a centralized, integrated data repository that stores and manages Trimble’s historical and current business data from organizationally cross-cutting data sources.
Why EDW?
Section titled “Why EDW?”Scale
- Interoperable/consumer platform agnostic
- Avoids point to point connections to source systems
- Storage/Compute power is scalable based on the need
- Centralizes data for improved reporting and analysis on enterprise datasets
Governance
- Improves access control and discoverability
- Meets data privacy regulation requirements
- Enables data quality assessment and improvement
High-Level Architecture Diagram
Section titled “High-Level Architecture Diagram”
Trimble Platform utilizes a three-layered architecture for Enterprise Data Warehouse, with an optional Landing Area. This architecture progressively improves data quality through validation and transformation stages before data is published for analytical use.
Data Sources
Section titled “Data Sources”- Business Systems (ex. Oracle, Salesforce, EMS)
- Core Platform (ex. TID, IAM, API, Insights)
- Other sources broadly applicable to Trimble which make sense to centralize
Raw Data Layer
Section titled “Raw Data Layer”- This layer comprises raw data from source systems. Data objects within this layer are structurally identical to the source systems and maintain a complete history of all incoming data. For some of the data sources, data might go through a Temporary area called Landing Area to improve the process efficiency.
- Methods: Push or Pull; Incremental or Full Load
- Update frequency: Data source and use case dependent
- Contracts are in place with source system owners to ensure proper data access, quality, and compliance.
- Characteristics: Data is raw, untransformed, and often stored with high fidelity, serving as an immutable system of record for auditing and exploration.
Processed Data Layer
Section titled “Processed Data Layer”- Data from the raw data layer is cleansed, deduplicated, normalized, and validated for accuracy and consistency.
- Curated and Conformed
- Business rules are applied using transformations.
- Data quality, data privacy rules are applied in this layer.
Publish Data Layer
Section titled “Publish Data Layer”- Delivers highly curated and business-focused datasets that are ready for immediate use by business users.
- Star Schema models are built using the objects from this layer.
- Data quality, data privacy, and data retention rules are applied in this layer.
- Contains highly refined, semantically meaningful data, sometimes aggregated and filtered to meet specific business logic and reporting needs.
- some of the examples are: Enterprise Core Datasets, Conformed Dimensions, Summary Data Mart, and Subject Area Specific Data Marts
Landing Area
Section titled “Landing Area”- Landing Areas( AWS S3 and Snowflake) are optional and used as needed.
- These intermediary areas provide flexibility for introducing additional steps or transformations to streamline data flow and improve processing, but it’s only used when a clear benefit and efficiency is identified.