StreamSets Launches StreamSets Transformer

SAN FRANCISCO, September 05, 2019 — StreamSets, Inc., provider of the industry’s first DataOps platform for modern data integration, released today StreamSets Transformer, a simple-to-use, drag-and-drop UI tool to create native Apache Spark applications. Designed for a wide range of users — even those without specialized skills — StreamSets Transformer enables the creation of pipelines for performing ETL, stream processing and machine-learning operations. Now, data engineers, scientists, architects and operators gain deep visibility into the execution of Apache Spark while broadening usage across the business.

Apache Spark delivers on the promise of advanced data processing and machine learning at scale. But there are drawbacks. Developing and operating applications on Apache Spark is complex and requires hand-coding. It is typically restricted to developers and companies with mature data engineering and data science practices. In addition, users often have very limited visibility into how their Apache Spark jobs are running. StreamSets Transformer solves these issues. Its easy-to-use, logical user interface and rich tools for designing data transformations eliminate the complexity and need for specialized skills. Pipelines instrumented with StreamSets Transformer provide unparalleled visibility into every Spark execution. Equally important, developers now have a single tool to build both batch and streaming pipelines.

The key features of StreamSets Transformer include:

• Continuous monitoring — Unparalleled visibility into Apache Spark application execution
• Continuous data — Runs in both batch and streaming modes
• Progressive error handling — Finds where and why errors occur without the need for Apache Spark skills to decipher complex log files
• Execute on Apache Spark anywhere — Works in the cloud, Kubernetes or on premises
• Highly extensible — Higher order transformation primitives for the ETL developer, SparkSQL for the analyst, PySpark for the data scientist, and custom Java/Scala processors for the Apache Spark developer
• Sets-based processing — For ETL, machine learning and complex event processing

“With StreamSets Transformer, Apache Spark is finally available to a wide range of users, enabling visibility, monitoring and reporting for mission-critical workloads,” said Arvind Prabhakar, CTO of StreamSets. “In essence, StreamSets Transformer brings the power of Apache Spark to businesses, while eliminating its complexity and guesswork.”

“With StreamSets Transformer and Databricks integrated together, even more users can easily access the powerful capabilities of Delta Lake and our optimized Apache Spark for data science and analytics,” said Michael Hoff, senior vice president of Business Development and Partners at Databricks. “Especially as organizations migrate from legacy on premises platforms, our partnership will help them efficiently make that transition to manage their data and machine learning workloads in the cloud.”

StreamSets Transformer is available immediately. Please visit the StreamSets website for more information.

About DataOps
Analytics has modernized in our always-on, always-changing world. How you deliver data to drive analytics has to modernize, too. DataOps is a set of practices and technologies that operationalizes data management and integration to ensure resiliency and agility despite ceaseless change. It combines the DevOps principles of continuous delivery with the ability to tame data drift (unexpected and undocumented changes to data). By embedding these principles, DataOps makes it possible to deliver the continuous data needed to drive modern analytics and digital transformation.

About StreamSets
StreamSets built the industry’s first multi-cloud DataOps platform for modern data integration, helping enterprises to continuously flow big, streaming and traditional data to their data science and data analytics applications. The platform uniquely handles data drift, those frequent and unexpected changes to upstream data that break pipelines and damage data integrity. The StreamSets DataOps Platform allows for execution of any-to-any pipelines, ETL processing and machine learning with a cloud-native operations portal for the continuous automation and monitoring of complex multi-pipeline topologies.

Source: StreamSets