Skip to content

product
architecture

Agile Data Engine is a DataOps Management Platform. It is used for building, deploying, and running cloud data warehouses. It operates and orchestrates data loads from your preferred cloud storage to your target database.
horizontal-ruler-up
ADE Stack v4
horizontal-ruler-down

DataOps TOOLS for your Cloud Data Warehouse

Our DataOps Management platform (SaaS) provides the backbone for your data warehouse so that you can focus on building business value on top of it. You will get all this out-of-the-box with Agile Data Engine:

  • MFA-based User Authentication with optional Federated AD
  • Shared data product design experience for all of your data & analytics engineering teams
  • Continuous Delivery (CI/CD) with automatic pipelines to Dev, QA & Prod environments
  • Customer process connectivity to your overall CI/CD and orchestration and also to your data catalogs through service layer APIs
  • Workflow orchestration & data quality monitoring
  • Automatic schema changes and load/transform SQL & workflow code generation

core features

agile data engine architecture

ADE Product Architecture v4

  

Data products developed as packages

package_list

metadata-driven approach

Agile Data Engine's metadata-driven approach has three core concepts:

  • Entities - metadata object combining the data model and data loads into one. Information like keys and physical types are combined with permissions and load schedules. 
  • Packages - collections of entities. A Package is also the unit of commit and deployment, flowing through the CI/CD pipeline.
  • Workflows - generated automatically based on logical data lineage and schedules. Entity loads are logical mappings between entities (and transformations), and the scheduling defines how often data is loaded.

Everything is designed in the same web-based user interface and an environment shared by all developers and teams working with your data warehouse.

The data models, load definitions, and other information about the data warehouse content are stored in a central metadata repository with related documentation.

The actual physical SQL code for models and loads is generated automatically based on the design metadata. Also, load workflows are generated dynamically using dependencies and schedule information provided by the developers.

More complex custom SQL transformations are placed as Load Steps as part of metadata, to gain from the overall automation and shared design experience.

product
documentation
Documentation

horizontal-ruler-up

multi-cloud support

Agile Data Engine works with major cloud vendors and databases. When you face a need to change the cloud or the database platform, the data products’ lifecycle gets extended with us.

Snowflake Amazon Redshift | Azure Synapse | Azure SQL DB | Google BigQuery

Snowflake-logo-white
AWS
Azure
Google-Cloud
horizontal-ruler-down

next up