Dremio Unveils Full Flexibility with Data Catalog for Apache Iceberg Across All Environments

SANTA CLARA, Calif., Oct. 29, 2024 — Dremio has announced that its Data Catalog for Apache Iceberg now supports all deployment options—on-prem, cloud, and hybrid—making Dremio the only lakehouse provider to deliver full architecture flexibility.

Additionally, Dremio is announcing integrations with Snowflake’s managed service of Apache Polaris (incubating) and Databricks’ Unity Catalog managed service. This allows customers to choose the best catalog for their needs, while using Dremio to deliver seamless analytics across all data. Databricks and Snowflake customers can choose the catalog that makes the most sense for their business, reducing total cost of ownership (TCO) and avoiding unnecessary infrastructure expenses. By enabling full governance and security, the solution helps organizations unify their data infrastructure while maintaining control and optimizing performance.

The First Hybrid Data Catalog for Apache Iceberg

“Enterprises face extraordinary pressure to access, prepare, and govern distributed datasets for consumption by analytics and AI applications. To meet this demand, they need to catalog diverse data and metadata across data centers, regions, and clouds,” said Kevin Petrie, vice president of research at BARC. “Dremio is taking a logical step to enable this with an open catalog that is based on Apache Iceberg, the emerging standard for flexible table formats, thereby integrating with an ecosystem of popular platforms.”

Built on the open-source Project Nessie, Dremio’s Data Catalog for Apache Iceberg introduces key features and functionality that includes:

• Flexible and Open Interoperability: Supports all Iceberg engines, such as Dremio and Spark, through the Iceberg REST catalog API, providing a flexible, open and future-proof solution.
• Centralized Data Governance: Enables robust centralized data governance across all data, with role-based access control and powerful fine-grained access privileges to ensure data compliance and security.
• Automated Table Optimization: Automates table optimization tasks like compaction and garbage collection, enhancing performance and lowering storage costs.
• Data Branching and Versioning: Git-like branching and version control supports  experimentation, virtual development environments, and time-travel without data duplication while preventing risk to production data.
• Simplified Data Management: Reduces complexity and increases data management efficiency by addressing common pain points such as convoluted catalog setups, lack of governance controls, and insufficient maintenance tools.

Seamless Integration with Managed Service Catalogs

With augmented support for Snowflake and Databricks, Dremio is strengthening its ongoing commitment to deliver open, scalable, and flexible lakehouse architectures that streamline data integration and analytics across any environment. Now, Dremio customers no longer have to choose between vendors or architectures as they can integrate with their preferred catalog, deploy on-prem, in the cloud, or in a hybrid architecture while maintaining smooth interoperability across platforms to unite analytics without vendor lock-in.

“Flexibility is essential for modern organizations looking to maximize the value of their data. With expanded Iceberg catalog support across all environments, Dremio empowers businesses to deploy their lakehouse architecture wherever it’s most effective,” said Tomer Shiran, Founder of Dremio. “We’re 100% committed to giving customers the freedom to choose the best tools and infrastructure while reducing fears of vendor lock-in.”

About Dremio

Dremio is the unified lakehouse platform for self-service analytics and AI, serving hundreds of global enterprises, including Maersk, Amazon, Regeneron, NetApp, and S&P Global. Customers rely on Dremio for cloud, hybrid, and on-prem lakehouses to power their data mesh, data warehouse migration, data virtualization, and unified data access use cases. Based on open source technologies, including Apache Iceberg and Apache Arrow, Dremio provides an open lakehouse architecture enabling the fastest time to insight and platform flexibility at a fraction of the cost. Learn more at www.Dremio.com.

Source: Dremio