Power BI Architecture, Integration Points, Implementation Options Atlanta SQLSaturday BI Edition 1/9/2016 Melissa Coates Solution Architect, BlueGranite blue-granite.com Blog: sqlchick.com Twitter: @sqlchick
Agenda: 1. Brief Overview and History of Power BI 2. Implementation Options & Architecture Service architecture Components & integration Authoring environments Types of datasets 3. Bimodal BI: Use Cases for Power BI 4. Things to Consider Getting Started Power BI Architecture, Integration Points, Implementation Options Reuse of objects Types of gateways Sharing & collaboration Free vs Pro licensing
Overview & Evolution of Power BI
Components of Power BI Set of tools targeted primarily at analysts & power users to facilitate acquiring, cleansing, standardizing, modeling, presenting & sharing of data. ------------ Power BI Desktop ----------- Edit Queries Data & Relationships Report -- Web -- Power Query Power Pivot Power View, Excel PivotTable, Excel PivotChart ------------------ Excel ------------------ Reports, Dashboards, Sharing, Q&A Power BI Service Mobile Power BI Mobile Apps
Typical Users of Microsoft Reporting Tools Reporting Services Datazen PerformancePoint Visio Services Power BI Desktop Power Pivot Power Query Power Map Report Builder Power View Excel Corporate BI Self-Service BI
Evolution of Power BI V1: Power BI for Office 365 GA Release Feb 2014 Excel Add-Ins Integrated with Office 365 + SharePoint Online Data must be replicated in embedded data model (Power Pivot) Deprecated as of 3/31/2016; customers required to migrate to V2 V2: Power BI Dashboards GA Release July 2015 Excel Add-Ins or Power BI Desktop Independent Software as a Service Support for embedded data model or DirectQuery Toolsets are maturing; very fast release cycles Scope: Self-Service BI Scope: Self-Service BI + elements of Corporate BI, Analytics
Implementation Options & Architecture
Power BI V2 Primary Components Power BI Desktop Power BI Service Power BI Mobile Apps New tool of choice for creating queries, data model, and reports Excel Add-Ins The Software as a Service (SaaS) web portal component at powerbi.com for sharing & collaboration If someone says Power BI they may be referring to the Power BI Service specifically, or to the ecosystem as a whole. Native mobile apps (ios, Android, Windows) for viewing & interacting with dashboards
Power BI Service Architecture The Power BI Service is built on Azure. Azure 1 Web Front End cluster 1 handles connectivity & authentication 2 Back End cluster manages Power BI Service activities 2 5 4 3 3 User data is uploaded to Blob storage 4 5 Metadata is stored in SQLDB Public internet boundary Enhanced In-Memory Tabular DB to service queries Image source: https://powerbi.microsoft.com/en-us/documentation/powerbi-admin-power-bi-security/ Info about In-Memory Tabular DB: Pg 14-15 Applied Microsoft Power BI by Teo Lachev
Power BI Implementation Options On-Premises Cloud Services 1 2 3 4a 4b Power BI Desktop and/or Excel Add-Ins only Power Pivot for SharePoint Power BI for Office 365 (V1) Power BI (V2) with Power BI Service Pyramid Analytics Server (part of V2) V1 Deprecated 3/31/2016 4c Integrated with Custom Application (part of V2) Hybrid Environment
1 Excel Add-Ins &/or Power BI Desktop Only No usage of the Power BI Service or Mobile Apps
4a Power BI (V2) End-to-End
Authoring & Reuse of Power BI Objects A dashboard can contain elements from many reports. A report can reference data from one dataset. A single dataset can be reused among various reports.
Authoring Environments Power BI Desktop Excel Add-Ins Power BI Service New tool of choice for creating queries, data model, and reports Monthly release cycle for new features PBIX format Includes Power Query, Power Pivot, and Power View add-ins Much slower release cycle of new features due to deep Excel integration XLSX format The web portal component for sharing & collaboration Dashboards and Q&A must be created in the web; Reports can optionally be edited in the web Weekly release cycle for new features Tip: There s no download ability (PBIX or XLSX) yet from the Power BI Service. Since a report originally uploaded from PBI Desktop or Excel can be modified in the web, you need to be careful with where the original resides.
Deciding Which Tool to Use? Power BI Desktop Excel Add-Ins You run an older Excel version You have 32-bit Excel You want the newest features as quickly as possible You want to access SaaS connectors & other new sources Release cycle: new Power BI Desktop msi installer 1x / month * You already have an Excel 2013 click-torun version that supports the Power add-ins running in 64-bit mode Your analysts feel strongly about using Excel & don t mind waiting for features You want to use pivot tables, pivot charts, and cube functions (will require publishing to OneDrive) It s likely you ll upsize to SSAS Tabular * Although Power BI Desktop updates monthly, the Power BI Service (online) updates weekly. You have no control over new features being introduced to the Service.
Types of Datasets Embedded Dataset An in-memory columnar data structure. Limited to 250MB compressed in size which applies to storage limits. In this situation, the data is *stored* in the dataset and imported to the Power BI Service. Requires refresh schedule to stay current. Embedded datasets can be created various ways: Pull Power BI Desktop ( Get Data ) Power Pivot Excel add-in SaaS Connector (SaaS Content Pack) Push Streaming data (Azure Stream Analytics) API library Direct Query Live Connection Queries are live back to original data source. Useful for: Utilizing existing data investments Larger datasets that cannot or shouldn t be replicated When row-level security is required (via SSAS with EffectiveUserName) No refresh schedule is required. In this situation, the dataset in the Power BI service is a connection string only. DirectQuery is supported for: SQL Server (with Enterprise Gateway) SAP HANA (with Enterprise Gateway) Azure SQL DB Azure SQL DW Spark on HDInsight Live Connection is supported for: SSAS Tabular (with Enterprise Gateway) SSAS Multidimensional (with Enterprise Gateway)
Using a Dataset Published to Power BI Data stored & refreshed in the Power BI Service
Using DirectQuery or Live Connection Live queries to source (Note: tiles are cached in the Power BI Service as images in order to improve performance)
Power BI Gateways Personal Gateway Handles data refresh for data imported into the Power BI Service via Azure Service Bus. Each individual user sets up their own data refresh schedule. One gateway can be installed per user. Typically only needed for on-premises data sources (not usually for cloud sources). If user is admin on machine: runs as Service. Otherwise, runs on Application (which requires user to be logged in for refresh to succeed). In public preview. Enterprise Gateway Handles DirectQuery: SQL Server SAP HANA Handles Live Connection (explore live): SSAS (Tabular and Multidimensional) Will replace the SSAS Connector early reports are that speed is *significantly* better with Enterprise Gateway. Data refresh capabilities (in addition to DirectQuery & Live Connection) are coming soon.
Options for Sharing Content Share Dashboard Share a read-only dashboard from My Workspace with another user Intended for sharing with a few select colleagues Recommendation: use minimally; if employee leaves Group Group workspace for datasets, reports, and dashboards Intended for team collaboration O365 unified groups All group members are set to read or edit permissions 10GB size limit Organizational Content Pack Sets of datasets, reports, and dashboards Intended for broad delivery across numerous groups User discovers via Get Data User can personalize a copy in My Workspace (and even re-share) 1 person 1 group many groups
Workspaces, Sharing, and Collaboration
Power BI Service Limits Per user account or group: # of reports per dataset: 200 (Multi-page reports still count as 1) # of datasets: 200 Size for embedded dataset (non-direct connect): 250MB compressed # of dashboards: 100 # of reports: 40,000 Workspace data storage: 1 GB (Free license) 10 GB (Pro license) API calls: Single push: 10,000 rows Continuous push: 500 rows once per second Max rows per table: 5,000,000 (or 2,000,000 if continuous) Pending requests: 5 Per tenant: Global data storage: 10 GB x # of Pro licenses purchased Sharing to a group > 500 members is not supported
Who Needs a Pro License? Freemium model: Free license & Pro license Retail pricing of a Power BI Pro license is $9.99 / user / month Both producers and consumers need a Pro license if any paid features are being utilized. Paid features include: Data refresh more frequently than daily Data streams in excess of 10k rows/hour Data accessed via DirectQuery (ex: SSAS or SQL Server with Enterprise Gateway) Data accessed and refreshed via the Personal Gateway Storage > 1GB for data models published and refreshed in Power BI Service Publishing via organizational content packs Publishing to a group workspace Shared queries via the Data Catalog (not yet available in V2)
Power BI (V2) End-to-End
Bimodal BI
Bimodal BI & Analytics Two modes for development & delivery of information Agile Self-Service BI Business-driven Rapid delivery Less governance Exploration freedom Traditional Corporate BI Defined IT-driven processes Reliable Governed & secured Standardized
Use Cases for Power BI Power BI used for reporting only (data queries only; not replicated in Power BI) Report Only Requires direct connection to source data Reports from source w/ higher data volumes Reports from source w/ row level security (SSAS Tabular) Power BI used for data shaping, modeling, and reporting (data typically is replicated & refreshed in Power BI) Query Model Report Small individual self-service projects Data mashups (up to 250MB compressed in web) One-time or infrequent analysis On the fly reporting with Q&A Standalone SaaS connectors APIs to programmatically push data to Power BI
Bimodal BI: Where Power BI Can Fit Self-Service BI Corporate BI Excel Data Model (Power Pivot) Upsize Prototype SSAS Tabular Model Mashup of different data sources One-time or infrequent analysis Supply info needed very quickly Augment corporate BI solutions Standalone SaaS Connectors (ex: SalesForce, CRM) Application Integration (APIs) DW & Analytics Environment Analytical Machine Learning Data Warehouse Streaming Power BI Reporting Data Lake
Recap + Questions
Recap of Main Points Various options, each with its own strengths and shortcomings, will remain as viable choices for delivery: Power BI Desktop and/or Excel Add-Ins Only (no portal) Power Pivot for SharePoint (on-premises) Pyramid Analytics (3 rd Party on-premises) Power BI Service (SaaS) Evolving: Application integration and embedding in websites The Power BI ecosystem is continually evolving and growing in its role as a self-service BI tool, as well as elements of corporate BI A hybrid approach can be utilized to take advantage of existing BI/DW assets A purposeful Bimodal BI approach can attempt to balance many different agile and traditional needs
Things to Consider Getting Started Decision on using Power BI Desktop, or Excel, or both + training needs Use of Groups, including integration of O365 unified groups with AD Organizing content into Groups by subject area and/or user security boundaries Securing access to content and row-level security needs which can be met using Live Connection to SSAS using EffectiveUserName via Enterprise Gateway Types of users; who will be consumers vs publishers and is there a process for verification/approval of data models published Data source types and usage (imported to Service vs DirectQuery) Options for distributing, publishing, sharing content (file system, Power BI Service, application integration, mobile apps)
Resources To Follow Power BI Team Blog: http://blogs.msdn.com/b/powerbi/ Weekly Power BI Service Updates Monthly Power BI Desktop Updates Power BI Support Blog: http://blogs.technet.com/b/powerbisupport/ Power BI Community Site: http://community.powerbi.com/ Support & Documentation User Voice (ideas for product team) File Downloads GuyInACube YouTube Channel: aka.ms/guyinacube Short videos from Adam Saxton
Thanks for Attending! Melissa Coates Twitter: @sqlchick Blog: sqlchick.com Check Presentations & Downloads at sqlchick.com for this slide deck. Creative Commons License: Attribution-NonCommercial-NoDerivative Works 3.0