Why Microsoft is combining all its data analytics products into Fabric

Microsoft is combining its existing data warehousing, business intelligence, and data analytics products into a single offering, dubbed Microsoft Fabric, the company announced Tuesday.

Analysts said this could help enterprises combine workloads while reducing IT integration overhead, complexity and costs.

The launch of a unified data analytics platform in the form of Microsoft Fabric can be seen as “riposte” to the massive sprawl of products in an enterprise’s modern data stack, according to Sanjeev Mohan, principal analyst at SanjMo.

“Microsoft realizes that their clients were looking to reduce the integration overhead and complexity. It’s not that the customers won’t spend, but they are looking for better value for their IT investment,” Mohan said, adding that if Fabric can deliver the right outcomes then it could be the “right” strategy for the public cloud service provider.

In addition to reducing IT complexity, Fabric can also help reduce costs, Mohan said.

“An integrated solution should cost less than a collection of best-of-breed specialized solutions. The pre-integrated Fabric also reduces integration costs and skills needed to learn different tools,” the principal analyst explained.

Microsoft, too, claims that purchasing and managing resources with Fabric is simpler.

“Customers can purchase a single pool of compute that powers all Fabric workloads. The universal compute capacities significantly reduce costs, as any unused compute capacity in one workload can be utilized by any of the workloads,” the company said in a statement.

Raw data, whether it is transactional, operational, or otherwise, according to Forrester’s principal analyst Boris Evelson, needs to go through steps such as sourcing, extracting, moving, integrating, cleaning, modeling, and cataloging before being turned into valuable information.

“Typically, some of the technologies supporting each of the steps come from different vendors and data or analytics professionals need to spend time on integration,” Evelson said, adding that while Fabric does not completely eliminate the need for integrating components, it reduces the integration time and effort, allowing data and analytics users to concentrate more on solving business problems and addressing business opportunities.

Microsoft Fabric provides seven core services and tools

Microsoft Fabric, according to the company, uses a unified architecture to provide a software-as-a-service (SaaS) experience to developers to help in extracting insights from raw data and present it to business users.

The new analytics suite comes with seven core modules and toolsets, including data connectors, data engineering tools, data workflows for data science, and analytics tools among others, it added.

Microsoft

One module currently in public preview, Data Factory, provides over 150 connectors to cloud and on-premises data sources, enabling a drag and drop experience for data transformation along with the ability to orchestrate data pipelines, the company said.

Microsoft has also bundled the Synapse Data Engineering module, also in public preview, which helps with model authoring in Apache Spark, it said.

Fabric also includes Synapse Data Science, an end-to-end workflow for data scientists to build sophisticated AI models, and Synapse Data Warehousing, combining lakehouse and data warehousing tools with the ability to run SQL on open data formats, the company said. Both are in public preview.

Another still in public preview is the Synapse Real-Time Analytics module, which will enable developers to work with streaming data and analyze large volumes of semi-structured data, Microsoft said.

Further, the company is also integrating a Power BI module in Fabric to help business analysts and business users of an enterprise generate insights from data with the backing of AI-based tools.

The Power BI experience is also deeply integrated into Microsoft 365, it said.

For Amalgam Insights’ principal analyst Hyoun Park, the addition of Power BI to Fabric closes a variety of holes in Power BI that made it look less enterprise-ready when compared to analytic platforms such as Qlik, TIBCO, or SAS.

Microsoft said Fabric will come with a Data Activator module that is aimed at real-time detection and monitoring of data. It will be able to trigger notifications and actions when it finds specified patterns in data, the company said, adding that the module was currently in private preview.

All the seven modules, according to the company, can be experienced on the company’s portal.

Azure OpenAI service added to Microsoft Fabric, Copilot coming soon

Microsoft is adding its Azure OpenAI service to Fabric and will soon integrate GPT-powered Copilot into the analytics platform, the company said.

“With Copilot in Microsoft Fabric, users can use conversational language to create dataflows and data pipelines, generate code and entire functions, build machine learning models, or visualize results,” it added.

The addition of the Azure OpenAI service and Copilot will greatly speed up the work done by data professions inured in Microsoft Azure by breaking down a lot of the barriers between solutions, said Bradley Shimmin, chief analyst at Omdia.

“It should also accelerate the adoption of Microsoft’s own foundational models and frankly all of the company’s machine learning tooling portfolio,” Shimmin added.

The move can also been seen as Microsoft’s continued efforts to integrate large language models’ APIs from OpenAI’s stable across its entire portfolio, Mohan said.

Enterprises can also combine large language models from Azure OpenAI service along with their own data to create their own conversational language experiences, the company said, adding that Copilot was not trained on an enterprise’s tenant data.

Fabric gets OneLake support, adopts Databricks’ Delta table format

All of Fabric’s data and workloads, according to Microsoft, are stored in a SaaS, multi-cloud data lake called OneLake, akin to 365 applications data being stored into OneDrive.

“Data is organized in a data hub, and automatically indexed for discovery, sharing, governance, and compliance. It provides a single, unified storage system for all developers, where discovery and sharing of data is easy with policy and security settings are enforced centrally,” the company said in a statement.

This helps avoid data silos due to y different developers provisioning and configuring their own isolated storage accounts, the company explained, adding that OneLake allows enterprises to virtualize data lake storage in ADLS Gen2, AWS S3, and Google Storage.

The introduction of OneLake, according to Forrester’s Evelson, will make it easier for enterprises to create their own mini data lake in minutes as opposed to days or weeks.

“True, to get that data lake ready for mission critical apps will still take time, but prototyping, proof-of-concepts, agile development will become easier,” the analyst said.

In addition, Fabric treats Delta on top of Parquet files as a native data format that is the default for all workloads.

“This deep commitment to a common open data format means that customers need to load the data into the lake only once and all the workloads can operate on the same data, without having to separately ingest it,” the company said, adding that this means that OneLake supports structured data of any format and unstructured data.

The adoption of the open-source format, according to Evelson, could be a huge time, effort and storage saver.

“While OneLake itself is not open source, but under the covers the data structure is based on an open source format called Parquet —a data format optimized for analytics. This means that a data lake, a data warehouse, and a BI platform (in this case Power BI) will use exactly the same format, and most importantly the same instance/version of the data,” Evelson said.

Further, Microsoft said that it is planning to introduce a universal security model for Fabric that is managed in OneLake to help enterprises manage data security across different data engines, modules or tools.

The model will ensure that all data engines or modules enforce the security model as they process queries or other tasks, the company said.

Will Microsoft be able to capitalize on Fabric?

The launch of Microsoft Fabric, according to analysts, could go either way in terms of its adoption and popularity.

“If every enterprise user of Office 365 get a copy of Fabric, just like they get a copy of Power BI currently with the E5 Office 365 license, it’ll have the same viral effect as Power BI,” Evelson said.

However, Constellation Research’s principal analyst Doug Henschen warned that Fabric’s success might not come overnight.

“We have to keep in mind that everything announced is still in preview and Microsoft has a mixed track record when it comes to success in areas such as data warehousing. Azure Synapse, most recently, has not seen huge adoption or customer acclaim as a lakehouse platform,” Henschen said, adding that enterprises typically don’t change data platforms quickly.  

Microsoft Fabric can be compared to the likes of Google DataPlex, SAP DataSphere and IBM Data Fabric, the analysts said.