The Microsoft Business Applications Summit was last week and, in conjunction with this event, the October 2018 Release Notes were published. Additionally, an updated version (v 2.0) of the Planning a Power BI Enterprise Deployment whitepaper was also released. This event and updated documentation represents a truly a massive level of information to digest (thankfully I’m not actively writing a book on Power BI) but the good news is that videos from the event sessions are available on-demand and the documentation is well structured and easy to navigate.

*Given the volume of announcements I’m very thankful to not be actively writing a book on Power BI. Moreover, having reviewed a significant amount of the content, it seems that both the Power BI Cookbook and Mastering Power BI will remain as potentially useful resources, in the short term at least. 

Unified, Enterprise BI Platform

If I had to briefly summarize the very top takeaways I’d say that Power BI is in the process of becoming a unified, enterprise BI platform, inclusive of the administrative and governance capabilities large organizations require, in addition to continued improvements in existing capabilities in self-service BI, AI integration, and collaboration. The two primary examples of this include the ability to author large-scale, feature-rich semantic models typically reserved for Analysis Services via Power BI Desktop as well as the addition of paginated reports (aka operational, pixel-perfect SSRS reports) as first-class artifacts in Power BI.

Resources provisioned via Power BI Premium capacity will support the processing and query needs of these large data models and paginated report types. A closely related and essential feature also announced is the ability for a dataset in one workspace, potentially a ‘certified dataset’, to support the reports and dashboards in multiple separate workspaces. (In the current state, without an Analysis Services model, a separate or duplicate copy of a Power BI dataset is used in different workspaces)

Another major, potentially transformative enhancement to the Power BI architecture is Dataflows. Dataflows embed Power Query (M) expressions directly into the Power BI web service and allow these queries to leverage the storage and compute power of the Azure Data Lake. With scheduling and incremental refresh of these data flows configurable in the Power BI service, many new scenarios open up such as a large-scale (and prepped) source for a Power BI dataset, the re-use of a Dataflow’s output by a different Azure service (e.g. Azure Machine Learning), or the retrieval of existing Azure Data Lake storage assets.

Unlike enterprise data models and paginated reports, Power BI Premium won’t be a requirement for dataflows. A Power BI Pro license will be all that’s needed to get started though, per the release notes, advanced data refresh of the dataflow including incremental dataflow refresh will require Power BI Premium.

I’d strongly recommend watching the following session: Introducing Self-service data prep with dataflows.

*I’ve added the working title “Go with the Flow: Dataflows” to my backlog of future blog posts to further explain why this could be more significant than the PowerPivot for Excel Add-in was over eight years ago.

Administering and governing Power BI is clearly a top priority and this is reflected in a number of announcements including the new Admin APIs and PowerShell module, the new app workspaces (separate from Office 365 groups), visibility/monitoring of Power BI Premium resource utilization, more granular workspace roles (‘Contributor’ role), as well as monitoring of on-premises gateway clusters in the future.

You may watch Establishing and administering a Power BI environment in the enterprise as well as Distribute insights across the organization with Power BI for more details.

The announcements have been very well-received as they address many of the top gaps and customer requests. Before moving into greater detail, I would just caution that it appears we’re at least a few months away from major parts of the unified platform roadmap being delivered. Additionally, having experienced Microsoft’s prior attempts at a single BI platform (SharePoint integration), and to a lesser extent the earlier hybrid/composite modes of Analysis Services Tabular (1100-1103 compatibility level), it’s at least possible that the path ahead may not be as smooth as the PowerPoint slides suggest.

October 2018 Release Notes

I’ve grouped and ranked what I consider the top announcements and release note items, along with my initial thoughts/feedback, into the following four areas:

• Report Authoring
• Data Prep and Modeling
• Consume and Collaborate
• Administration

By no means is this list complete – for example I haven’t included PowerApps or embedded – thus you’re encouraged to research what’s important to you.

Report Authoring

In the Power BI and the Future for Modern and Enterprise BI presentation, Will Thompson, Senior Program Manager on the Power BI Desktop team, referred to the maturity of Microsoft Office several times and used the phrase “PowerPoint for Data” to describe the ease-of-use and perhaps industry-standard nature of Power BI Desktop in the future. This of course doesn’t mean that significant improvements aren’t coming, but it does seem clear that the finer points of the experience using the features rather than the features themselves is increasingly the priority.

You can view Creating pixel perfect reports in Power BI Premium for details on paginated reports coming to the Power BI service.

1. Paginated Reports (.RDL files) in the Power BI Service
   • Paginated (operational) reports serve a different purpose than the interactive visualization reports developed in Power BI Desktop and many organizations depend on them for at least part of their BI workload.
   • With Power BI Premium capacity, you’ll be able to publish and view both paginated (.RDL) and interactive (.PBIX) reports in the same Power BI portal.
   • There’s going to be limited support for SSDT to start – for example no shared data sources and reports must be manually uploaded to Power BI. There are several other report functionality limitations (e.g. Subreports, Linked Report Actions) as well.
     • Exactly how Application Lifecycle Management (ALM) will work with paginated reports in Power BI appears to be a TBD. More broadly, it was suggested that Report Builder may be updated with new development features rather than SSDT (just like data modeling).
   • The ability to migrate paginated reports from an on-premises SSRS/Power BI Report Server environment to a Platform-as-a-Service offering integrated with Power BI and thus other report types (Power BI, Excel) is by far the top reporting announcement.
2. Expression-based formatting
   • The ability to define a DAX expression to drive the formatting of Power BI report properties unlocks the customization often required to reflect business rules/logic and to create more powerful, intelligent experiences generally.
   • Custom expressions are supported for almost all paginated report objects in SSRS so I’m very happy to see this flexibility coming to Power BI reports.
3. Expand/Collapse PivotTable functionality for Matrix Visual
   • The ability to see the children/detail of one parent (e.g. States of a Country) while still viewing other parent members closes an important remaining gap with Excel.
   • I’ve written before in blogs and books on the advantages of Power BI Desktop over Excel for data analysis and reporting (e.g. greater scale, DirectQuery option, bidirectional cross-filtering, etc) so full support for PivotTable functionality further removes obstacles to migrating to Power BI Desktop.
4. Pre-built Report Themes
   • If you’ve ever had to create your own report theme by modifying properties in a JSON file you’ll surely welcome this feature.
   • The new pre-built themes (i.e. City Park, Classroom) will not surely not eliminate the need to customize to meet corporate standard color schemes. However, just like themes available in PowerPoint decks, the new themes will give report authors who aren’t tied to a single corporate standard options to format their reports according to the needs of the given report and its users.
5. Auto Alignment
   • Similar to the visual cues available in PowerPoint to easily align one object with another, report authors will have visual cues to easily align/snap their report visuals.

*The ability to copy specific values or selections (for pasting into other apps) and the ability to print and export from Power BI Desktop to PDF will also very useful.  

Data Prep and Modeling

I assume most MSBI professionals are at least surprised, if not stunned, at the announcement of using Power BI Desktop as the primary authoring tool for enterprise model projects. Readers of this blog may recall my January 25th post, Design Mode in Power BI Desktop, in which I described in detail the need to enhance Power BI Desktop to support large Power BI data.

What features did I suggest over six months ago? Multi-select property changes, schema focus, surfacing display folders in Power BI Desktop (asymetric design experience), and more. I have no idea if the product team read my blog post but these ideas aligned very closely with the modeling features being emphasized in last week’s presentations in Seattle.

In my imagination anyway, I sense that the engineering team would enjoy and even prefer working with me on these features and several other modeling ideas I’ve had since then. More importantly, customers would appreciate the tools and timelines. Introducing and explaining the features at events like Ignite, PASS, etc, however, may be a very different story but I digress…

The most important data prep and modeling feature/announcement (again, just my opinion) is Dataflows. Dataflows provides Power Query and thus Power BI models and potentially other Azure data services with the cloud-based, data lake computing power and storage/landing zone it’s needed.

Composite models and aggregated tables, whether built on top of dataflows or not, open up many new modeling scenarios such as multiple data sources and types and no-compromise DirectQuery solutions at scale.

1. Dataflows 
   • The data prep layer and one consistent schema (ie Common Data Model) is missing in Power BI, as Adi Regev noted in her presentation.
   • In the short term this is intended as self-service ETL/data prep for business analysts but it could easily become so much more.
   • Scheduled and incremental data refresh, Common Data Model integration,
2. Composite Models
   • The import (cached) vs. DirectQuery modeling decision has been a difficult trade-off in many scenarios. Although organizations may strive toward a single data source, in so many scenarios you need an additional source for the data model and you may also need or want this source to be accessed differently than your other other (import instead of DirectQuery or vice versa).
   • It will be interesting to see performance test results from DAX queries which have to utilize many-to-many relationships between import and DirectQuery sources or different DirectQuery sources.
     • I would have to think that A) there would be a severe penalty when multiple data sources must be queried and the results are merged together but we’ll see and B) you’d want to avoid bidirectional cross-filtering relationships across sources if at all possible.
3. Aggregated Tables 
   • You should watch Christian Wade’s Building a data model to support 1 trillion rows with Power BI Premium for great details on this feature.
   • The ability to ‘cover’ a large percentage of report queries with a relatively small, manageable cache of memory (optionally refreshed incrementally) at an aggregated level avoids the painful performance of DirectQuery over large sources or perhaps even sub-optimal DirectQuery sources.
   • I can see this feature, which builds on top of composite models, driving many models to become composite models thus limiting the memory footprint of the model as well as the query workload on the DirectQuery source.
   • Similar to performance with composite models, it will be interesting to see how this can/will work with row-level security (RLS). If security filters aren’t applied to the aggregated table(s), particularly in the big data scenario without relationships, this could limit/undermine the use cases.
4. Automated Partition Management
   • Incremental data refresh, currently in preview, has built-in partition management logic that deletes and merges partitions based on the configured refresh policy.
   • This addresses the real pain/time of developing custom PowerShell scripts, which likely aren’t nearly as optimal as the logic built into Power BI premium.
     • Rather than tweaking data refresh code, BI teams can focus on building out their data models to meet business needs.
5. Agile Application Lifecycle Management
   • The ALM Toolkit for Power BI demonstrated in the Building Enterprise grade BI models with Power BI Premium presentation looks very interesting, even at this early state.
   • The ability to set policies for each configuration (e.g. dev to test, test to prod), selectively deploy certain objects, easily compare the differences between two models, etc will allow BI teams to be responsive to changing business needs yet still have a process. (modern DevOps for BI).
6. Connectivity Parity with Analysis Services
   • With the XMLA Endpoint opened up to Power BI Premium workspaces, it will be possible to connect to these data models via common management (ie SSMS) and other BI reporting tools.
   • For example, paginated reports could be built on top of the same dataset which supports Power BI reports and as described earlier these reports could live in the same premium capacity and on the same web portal.
7. Scalability Parity with Azure Analysis Services
   • Rather than a 10GB dataset limit, you’ll be able to build a dataset as large as your Power BI Premium capacity. So if you have a P3 capacity, you could support a dataset up to 100GB in size in memory.

It was suggested that it may be 12 months before Power BI Desktop has all the modeling and ALM capabilities of SSDT for Visual Studio. More importantly, it was stated that Analysis Services, including both SSAS and Azure AS, will continue to receive new features with each release of SQL Server.

*A multi-dimensional version of Azure Analysis Services, which is currently exclusive to Tabular models, was described as Planned but without a timeline. Given other priorities, it seems unlikely that this will get any movement any time soon.

Consume and Collaborate

Ultimately it’s the user experience of interacting and d

1. Certified Datasets
   • It wouldn’t matter that you can build and incrementally refresh a huge Power BI dataset if the reports and dashboards referencing this dataset were limited to the same app workspace.
   • Per the Distribute Insights Across the Organization with Power BI presentation, not only can datasets support multiple workspaces, but datasets can be Promoted and marked as ‘Certified’ to represent the official and suggested dataset to be used.
   • Apparently access request workflows will be available for datasets and dataset owners can manage how the dataset will appear and grant access.
2. New Workspaces and Member Roles
   • With the new app workspaces which will not be tied to Office 365 groups, security groups of users (e.g. BI Report team), O365 groups, individuals, or distribution lists can be granted access to the workspace.
     • Currently security groups are limited to users of Power BI apps.
   • A ‘Contributor’ and a ‘Viewer’ role will be added to workspaces giving teams more granularity to align a user to the permissions required.
     • For example, you may add a Contributor or group of contributors to help build report content but you don’t want these uses to actually publish or update the app – that responsibility could be left to Admins or Members.
3. Comments
   • This feature has been available for the Power BI Report Server and now its coming the Power BI service for entire pages, dashboards, or even individual visuals.
   • Many users who are otherwise a bit timid to interact in the Power BI service will likely be comfortable in posting comments and possibly invoking notifications to others directly via @mentions.
4. Personalized Apps 
   • The ability for consumers of Power BI apps to save their own copy of a report and customize it in their own workspace will bring apps in line with the personalization feature of Content packs.
     • Additionally, this selective personalization avoids the problem of copying the entire content pack (all reports and dashboards) into My Workspace.
   • Content packs will not be supported by the new workspaces given this functi
5. Home Page
   • Perhaps with just a little coaching and/or supporting documentation, users will likely come to greatly appreciate their home page and the easy access to relevant content.

The ability for users to create their own personal bookmarks in the Power BI service, copying reports and dashboards across workspaces in the web service, and an improved Power BI web part for SharePoint (e.g. supporting dashboards) could all become hits as well.

Administration

This topic probably shouldn’t be last as administrators and organizational security and compliance requirements are essential for enterprise deployments. Some organizations have been ‘flying blind’ in some respects and this has limited their ability to adopt or promote Power BI throughout the organization.

Maybe the best news on this front is that Kay Unkroth, who helped build the Tabular Model Explorer for SSDT, is leading this area and several presentations highlighted upcoming improvements to help admins manage their environment.

1. Power BI Admin APIs and PowerShell Module
   • Most Microsoft admin pros are very comfortable with PowerShell so the new module (MicrosoftPowerBIMgmt) can be immediately useful.
   • Common questions such as finding the reports which are based on a dataset or the owner(s) of a dataset can be answered via the PowerShell cmdlets.
   • In the administering Power BI presentation, Kay provided an example of a data model schema reflecting the reports, dashboards, datasets, and workspaces for a Power BI tenant which could support custom reports not unlike the Power BI Usage Metrics Solution Template.
2. Power BI Premium Resource Monitoring
   • Power BI Premium will increasingly support more varied workloads ranging from enteprise BI models to self-service BI datasets to paginated reports. Additionally, as Premium capacity can be a significant expense, it’s important to have clear visibility to help provision just the resources necessary and to manage utilization of those resources.
   • A service app in Power BI (Power BI Premium Capacity Monitoring) will have a pre-built dashboard and report with data visualizations based on premium resource usage. This will only be 15 minutes delayed from the actual activity in the capacity (e.g. refreshes, queries).
     • Greater control and customization of the premium capacity data, perhaps allowing alerts based on various failures or duration metrics, is expected to be possible soon.
   • See the Deploying and Managing Power BI Premium capacities presentation for more detail on the new service app.
3. New Power BI Admin Portal
   • Manage the (new) workspaces in the Power BI Admin Portal, as they won’t be in the Office 365 Admin Center, and recover new workspaces if a user leaves the organization.
   • I didn’t find much detail or example of this new experience suggesting it’s at least several months away.
4. Gateway Cluster Monitoring
   • Improvements to the management APIs and PowerShell commands is already being worked on and new tenant-level administration APIs and PowerShell commands will be available later in 2018.
   • Governance and Monitoring was identified in the next planning cycle by the gateway team in their presentation.

Wrapping Up

Overall, the immediate, near and longer term roadmap for Power BI looks very bright across the platform. As mentioned at the beginning, this is a lot of information to take in and of course your scenario is unique (e.g. Embedded capacity) so be sure to check the resources and other sources for details.

Sorry for the delay – a few things came up the last 2-3 weeks. You may consider subscribing  (click ‘Follow’) if you find this content useful.