Microsoft launches Azure Databricks, a new cloud data platform based on Apache Spark

Microsoft Azure customers interested in parsing large amounts of data to improve their businesses will soon be able to use Azure Databricks, developed in consultation with big data startup Databricks and built around the Apache Spark framework.

Microsoft plans to announce the new service Wednesday at Microsoft Connect, a virtual event hosted by Scott Guthrie, executive vice president of the Microsoft Cloud and Enterprise Group. Azure Databricks will be available in a preview version for developers to use for processing and analyzing large streams of corporate data.

Databricks is a San Francisco-based startup founded by the team that developed the popular open-source Apache Spark data-processing framework at the University of California-Berkeley. Spark is one of the most widely used open-source projects in data analysis, and it allows users to run complex data simulations.

In a briefing ahead of Microsoft Connect, Guthrie called Azure Databricks “the first Spark-as-a-service of any of the cloud vendors.” The service could be used to model “real-time data patterns,” according to a Microsoft representative, such as measuring how hotel guests wander around a lobby as to best place furniture and guest services.

Azure Databricks will be integrated with a score of other Azure data-related services, including the Azure Cosmos DB database introduced earlier this year, as well as Azure SQL Data Warehouse and Azure Active Directory.

Microsoft will also announce that it is joining the MariaDB Foundation as a platinum member, yet another open-source project to receive Microsoft’s blessing this year. MariaDB is a version of the MySQL database that also counts Alibaba, Tencent, and Booking.com among its platinum sponsors, and Microsoft will announce a private preview of MariaDB on Azure at some point in the future.

And several other enhancements to its Visual Studio development environment will be discussed during Microsoft Connect, including the general availability of Visual Studio App Center, a new tool for Objective-C, Swift, Android Java, Xamarin and React Native developers.

Microsoft also introduced a collaborative development environment called Visual Studio Live Share, which will become available in 2018. It’s more or less like a Google Doc for developers, allowing multiple colleagues to work with code at the same time regardless of the individual development environment they are using.