By MICHAEL CIRCLE
The Data Group News Service
Databricks, the data and AI company, announced the launch of Databricks LakeFlow on Monday, a new solution that unifies and simplifies all aspects of data engineering from ingestion to transformation and orchestration.
With LakeFlow, data teams can efficiently ingest data at scale from databases like MySQL, Postgres and Oracle, as well as enterprise applications such as Salesforce, Dynamics, Workday and NetSuite. Databricks also introduced Real Time Mode for Apache Spark, enabling ultra low latency stream processing.
"Data engineering is essential for democratizing data and AI within businesses, yet it remains a challenging and complex field," said Databricks CEO Ali Ghodsi. "LakeFlow addresses these challenges by simplifying all aspects of data engineering via a single, unified experience."
Key features of LakeFlow include:
— LakeFlow Connect provides scalable connectors for ingesting from databases and enterprise apps, integrated with governance capabilities.
— LakeFlow Pipelines allows implementing data transformation and ETL in SQL or Python, with real-time streaming enabled.
— LakeFlow Jobs automates orchestration, data health monitoring and delivery across the Databricks platform.
The solution is built on Databricks' Data Intelligence Platform, with integrations for governance and serverless compute. It incorporates technology from Databricks' recent acquisition of Arcion.
"LakeFlow eliminates the need for manual orchestration and unifies batch and stream processing," said Ghodsi. "It makes even the most complex data transformations simple to build and operate."
LakeFlow is entering preview soon, starting with LakeFlow Connect. A waitlist is available on the company's website.
Visit Introducing Databricks LakeFlow: A Unified, Intelligent Solution for Data Engineering - Databricks to learn more.
***
Want to stay in the know about Data Industry news? Subscribe to our weekly newsletter!
Each Monday we will send out the top stories of the week straight to your inbox. Best part? It’s FREE, courtesy The Data Group News Service.
