Microsoft has announced Microsoft Fabric, which integrates existing solutions to make data analytics easier. From the 1st July 2023 all Power BI customers can use MS Fabric. In this article we will explain what Microsoft Fabric is and how it can benefit your business.
What is Microsoft Fabric
Microsoft Fabric is an end-to-end, SaaS-based, unified analytics platform designed to help organisations manage and leverage their data. It brings together various data sources and analytics tools in one integrated product. Microsoft Fabric leverages technologies such as Azure Data Factory, Azure Synapse Analytics, and Power BI, all of which are already part of the Microsoft Azure ecosystem.
With MS Fabric, both data professionals and end users can take advantage of the capabilities of their data. It provides a centralised environment where data sources can be integrated, transformed and managed.
OneLake
OneLake is an important part of Microsoft Fabric and acts as a data lake-as-a-service solution. It offers organisations the opportunity to avoid data silos and to centrally store and manage data without having to build or maintain a data lake themselves. Just as OneDrive serves as a cloud storage platform for files, OneLake functions as a data storage platform for data.
With OneLake no additional infrastructure management is required. This means that users have direct access to a scalable and organised data without having to arrange the provisioning and security themselves.
Lakehouse
An important part of the OneLake is the Lakehouse. The Lakehouse is a concept that refers to an integrated approach to data storage and processing, where structured and unstructured data are stored in a Data Lake and accessed as if they were in a traditional relational database.
Within a Lakehouse, data is organised into files, folders, and tables, creating a virtual database on top of the data lake. This allows users to perform queries and analysis using both the Spark engine and the SQL engine for big data processing. The Lakehouse concept leverages technologies such as Delta Lake, an open-source project that provides Atomicity, Consistency, Isolation, Durability (ACID) transactions for reliability and consistency when working with large data sets.
A Lakehouse can contain data in different formats, such as structured data (e.g. tables) and unstructured data (e.g. files). This data can be organised into folders within the data lake, giving users flexibility in managing and structuring their data sources.
By using a lakehouse, organisations can take advantage of the scalability and flexibility of a data lake, combined with the capabilities of traditional relational databases. This enables advanced analytics and processing of large amounts of data within the context of Microsoft Fabric.
Microsoft Fabric Components
Besides OneLake MS Fabric consists of these components:
- Data Factory (in preview), which offers 150+ connectors to cloud and on-premises data sources, drag-and-drop data transformation experiences, and the ability to orchestrate data pipelines
- Synapse Data Engineering (in preview), which offers authoring experiences for Spark, instant start with live pools, and the ability to collaborate
- Synapse Data Science (in preview), which provides an end-to-end workflow for data scientists to build advanced AI models, easily collaborate, and train, deploy, and manage machine learning models
- Synapse Data Warehousing (in preview), which provides a converged lakehouse and data warehouse experience on open data formats
- Synapse Real-Time Analytics (in preview), which enables developers to work with data coming in from the Internet of Things (IoT), telemetry, logs, and more, and analyse volumes of semi-structured data
- Power BI in Fabric, which provides data visualisation and AI-driven analytics. Data Activator (coming soon) offers real-time detection and monitoring of data and can trigger notifications and actions when it finds certain patterns in data – all without code.
MS Fabric and Power BI (Business Intelligence)
Power BI remains a powerful business intelligence and data analytics tool, but with the integration of Microsoft Fabric, it becomes even more powerful, providing seamless capabilities to take advantage of other features within Fabric, such as the Lakehouse and KQL Databases.
One of the new features in the Power BI workload is Direct Lake mode. This mode allows very large datasets to be analysed in Power BI directly from a data lake, without the need for a separate data warehouse or Lakehouse. This is possible by loading Parquet files directly from the data lake, without having to import or duplicate the data in a Power BI dataset.
The direct lake mode offers several advantages. Firstly, it allows users to analyse massive amounts of data in Power BI without the need for additional storage or transformation steps. This enables organisations to extract more value from their existing data lake infrastructure and analyse large data sets in real time.
Secondly, direct lake mode ensures that users always have access to the most up-to-date data in their data lake. Since the data is loaded directly from the data lake, changes and updates are reflected in Power BI reports and visualisations in real time. Unlike the performance of direct query, direct lake is super fast.
Tech-Wales complete IT Package for your business
At Tech-Wales we offer extended Managed IT Services for your business. We also offer award winning Managed IT Support, Cloud Computing Services and Business Applications Development. If you are looking to integrate MS Fabric into your business or you require any other business IT services call us today on 01639 326001 to discuss.