Which git workflow for Power BI development with pbip?

In my previous blog post about Power BI CI/CD & git integration, we discussed the basics of the .pbip format and how to set it up on a simple environment. In that environment, we were working with only a single Power BI workspace which was linked to a main branch in an Azure DevOps git repo.

But real word scenarios are often more complex than that. We’re often working with a dev, test and prod workspaces. How do we deal with the development of our Power BI report then? How many git branches do we need then?

Let’s try and answer these questions here!

Context

Ok, let’s say we’ve got an environment in which we have three workspaces: dev, test and prod. Before .pbip, our usual workflow typically looked something like this:

But we’ve also learned that there are many advantages (like Collaborative Power BI development) with working with the new .pbip format and Power BI git integration so we want to adapt our workflow. Let’s see what our new workflow could look like.

New workflow leveraging git integration and pbip

We’ve seen that in the previous workflow, the dev workspace is mainly used for personal validation. Azure DevOps branches are mainly useful for collaboration and synchronization with multiple developers. Therefore, there’s no need to link that Power BI workspace to an Azure DevOps branch. The dev workspace is not going to be connected to Azure DevOps at all in fact. This means that we can continue to use the Publish button from Power BI Desktop to move content to the dev workspace.

Things start to get different for the test workspace. In this workspace, what gets published needs to be tracked and version history needs to be kept. This means we need to link that workspace with an Azure DevOps branch. What gets published in this workspace needs to be tested by key users before going live to everyone. If both the test and prod workspaces were to be linked to the same Azure DevOps branch, both their content would be identical. Key users could not test the feature beforehand.

This means that our test and prod workspaces can’t be linked to the same Azure DevOps branch. Therefore, we’ll link the test workspace to an Azure DevOps branch called test. No direct publish from Power BI Desktop should be done. Instead, content should only be moved to this workspace by pull requests from feature branches to the test Azure DevOps branch.

Our prod workspace is going to be linked to an other Azure DevOps branch that we’ll call main (this branch is often called main or master, but it could also be called prod or whatever makes sense for you). When key users have validated our changes, we’re going to want to take the code from the test branch and push it to the main branch. Like always in Azure DevOps, we’ll do this by creating a pull request from test to main.

Note: if you have a large team, you might encounter cases where some features have been validated in test while some others still require proper testing. In this case, you just can’t push all of the code from test to main. What you’re going to want to do in this case is create pull requests between individual feature branches and the main branch. Make sure that your feature branches are granular enough (i.e. they only cover one feature ideally, or at least only features that are tested together) so that deployment can happen swiftly.

Here’s what the new workflow looks like.

Note: manual synchronization is still needed in that workflow. APIs exist (still in preview however) to automatically perform the synchronization, I’ll show in a blog post to come how we can leverage these APIs.

Implementing the workflow

Ok, let’s actually go and make changes to a report using that new workflow.

Environment set up

To try the full workflow we’ll need several things. First off, we’ll need 3 Power BI workspaces. I’m calling mine Demo pbip - dev, Demo pbip - test and Demo pbip - prod. The test and prod workpaces need to be in either a Fabric or a Premium capacity (others can’t be integrated with git). I’m personally using a trial Fabric capacity. I won’t cover workspace creation here; it’s pretty straightforward. Refer to the Microsoft documentation if need be.

We’ll also need an Azure DevOps repo with a test and main branch. Again, I won’t cover Azure DevOps repo creation. Plenty of stuff out there already explains this. I’m linking the Demo pbip - test workspace to the test branch and the Demo pbip - prod workspace to the main branch. See Manage a workspace with Git on how to do this.

Publishing to dev (steps 0 to 2)

These steps are pretty simple. They’re identical to the way we’re used to develop and publish Power BI reports. I open Power BI Desktop, I do my stuff, (connect to data sources, clean data, create my star schema, create my DAX measures and my visuals) in Power BI Desktop and I simply click Publish and select my Demo pbip - dev workspace.

I check that everything looks nice over in the service. It does. I’m happy.

Publishing to test (steps 3 to 6)

Ok things start to get trickier here.

First off, we need to set up our VS Code environment if we did not do so already. Check my blog Power BI CI/CD & git integration on how to do this. Over in VS Code, once we’ve cloned our repo, we create a local branch called feature/add_report_pbip in which we’ll add the report in .pbip format (the previously mentioned blog post also covers how to do this).

We commit and push our changes to Azure DevOps. Then, we create a pull request between our feature branch and the test branch.

Once our peers have reviewed our pull request. We complete it. Then, we head over to the Demo pbip - test workspace to synchronize the changes.

Now, it’s time for our key users to step in. They’ll take some time to validate the changes we’ve just pushed to the test workspace. If need be, we can go through steps 0 to 6 again to adapt to our key users’ requests.

Alright. We worked well. They’ve validated our changes. Let’s push to prod.

Publishing to prod (steps 7 to 9)

All of our features have been validated. Key users have asked us to share that report with everyone. It’s time to push to the prod workspace. To do so, we’ll create a pull request between the test and main branches. Again, it’s ok to so do because all of our changes from test have been validated by our key users. If some changes still needed testing while some others needed to be pushed to prod, we’d have to split our work into multiple feature branches and create multiple pull requests between these feature branches and the main branch.

The process here is going to be very close to what we did when we pushed to test. Our peers are going to validate (or not) our changes. We’ll complete the pull request when they do. Then we head over to the Demo pbip - prod workspace to synchronize the changes. And then, that’s it!

Conclusion

We’ve seen a possible git workflow which involved three environments. Other workflows could also perfectly work, depending on your team size and your development methods.

The main issue with this workflow is that is still involves manual synchronization in test and prod Power BI workspaces. This implies that someone still needs access to these Power BI workspaces. It could either by an admin or the report developer. Both options have their pros and cons.

If only an admin had access, he could quickly become a bottleneck in pushing reports to production. If a bug arises, it might also be harder for the report developer to fix that bug if he can’t have access to the report.

If the report developer directly had access to the prod workspace, that means he could still use the Publish button from Power BI Desktop and completely override the whole process. Maybe one day, we’ll be able to deactivate the Publish button in Power BI Desktop. In the meanwhile, let’s be cautious!

Thanks for reading!