The Microsoft Fabric Conference 2024 in Stockholm just finished last week and boy what an event! I had the chance to attend that first edition with some of my colleagues from Datatorii and I’m truly thankful for that great opportunity! In this blog post, I want to share my feedback on the event and the biggest news I’ve gotten from it.
The conference itself
That was my first time attending such a big event and I must say I loved it. It’s pretty awesome to see all these different tech fans gather in the same place and share their experience together. I think that discussing with others is as important (if not more) as actually attending the several sessions. I especially enjoyed the “Ask the Experts” stand in which one could talk with several MVPs. I’ve been playing with Fabric for more than 6 months now and I had quite a lot of questions to ask.
Among the several sessions I attended, the ones I enjoyed the most were the ones in which Fabric customers shared their feedbacks. They presented their architecture, the problems they had along the way and the workarounds they had to put in place to make up for the stuff that Fabric can’t yet do.
I felt like the key notes, emphasizing a lot on (way too much actually) Copilot and AI where basically big shows. AI and Copilot already bring a lot and will continue to do so. But I found disproportionate the amount of time spent on that matter, especially when the rest of the platform lacks some crucial functionnalities (like a decent CI/CD support, basic security features and support for Service Principals when working with APIs).
I also felt that the Roadmap sessions were not that interesting. Reading the monthly Fabric feature summary bascially does the same job.
Biggest news
A lot of new features were announced at the conference. Like really a lot. In the text below, I’ll briefly touch on the features I feel are the most important.
Data Engineering
- Fabric runtime 1.3 and Native Execution Engine are live: these enable faster spark execution in notebooks by providing support for the newer versions of Apache Spark and Delta and by setting up a new engine that runs in C++ (a bit like photon in the Databricks world)
- Fabric DP-700 certification: Microsoft announced a new certification dedicated to Fabric Data Engineers : the Implementing Data Engineering Solutions Using Microsoft Fabric; it’s still in beta as of today but I’ll be sure to take a look at it when it’s officially relased
- Warehouse improvements: I felt like the Fabric Warehouse was a lot behind its Lakehouse nemesis; I’m not so sure after all the news annoucements of new T-SQL commands support (like TRUNCATE, ALTER TABLE, …), shortcut integration (to come), more datatypes support (VARCHAR(max)), …
Data Factory
- New Copy job object in Fabric: I don’t feel like it’s a really big thing, but the new Copy job object will easy the task of data movement, especially for non-tech users (which is something to worry about for the tech users, not because non-tech users will take tech users’ jobs but because data could be copied significantly more than it should)
- Mirroring Databricks Unity Catalogs: this is probably one of the biggest news I heard at the conference; it’s now super easy to just bring in all the Databricks data (without any copy as it’s actually more a shortcut than an actual mirroring) inside a Fabric Lakehouse and build insights on top of it; I’m a big Databricks fan and this is a game chager as I feel many organizations will keep on using Databricks for their back-end
Platform
- Support for Service Principal authentication for Fabric REST APIs: it’s probably the biggest news, this enables admins and data architects to easily deploy and monitor components across the organization in a secure manner
- Sneak Peek of One Secutity: I got the chance to see a small demo of One Security; One Security is the service that will greatly simplify the set up of data-driven rules to limit who can see what data in a given Fabric environment; sadly, no dates were clearly communicated
Conclusion
This pretty much wraps up the event for me. Again I really enjoyed the conference and the awesome vibe that lived there for a few days. If I can, I’ll more than gladly participate in the one next year in Vienna.
Thank you for reading, stay tuned!