Extending build.cake with your (enterprise) functionality

For a few years now, I have been willing to write about this and I never really took the time. I spoke about it in a session during Visug XL 2019 and at that time, we were only getting started with Cake. Today, 1.5 years later, we have a set up a stable (unit test supported) build process that supports multiple development teams. In this blogpost, I want to share with you what we learned along the way and how you can accomplish this in your company too.

I will start with the basics (what is Cake) and take it from there. I will describe different approaches and will go into detail on why they can/cannot work. (The answer is (as usually) "it depends"). The most important thing to remember though is that you (and your org) will have to see what really works for you and decide based on your needs. You can always start with something and evolve into something else later on! And above al: don't be afraid to try something, learn and grow from there!

What is cake

Cake is a build system that allows you to describe your build process completely in a c# like manner in a so called "build.cake" file.

I prefer to refer you to the cake website to give you a better understanding, but below you can also find the most important takeaways (in the form of a screenshot of the website):

The idea is that you have a (huge) toolbox filled with functionality that can facilitate you in your build (and release) process. There is a big ecosystem of tools and integrations available and there is very little that you cannot achieve with it. So if you want to get started, there is a relatively short learning curve to get you started and you will be able to achieve impressive things!

What problem does Cake solve?

The main challenges with build platforms are typically that

• Troubleshooting issues is hard: I cannot count all the times that I did nothing but launch new (server side builds), just to be able to understand and (hopefully) solve yet another issue!
• If you want to move to another build system (eg: for better UX or integration), the you have to do all the work of setting up the builds again. This is really an issue and a problem of "vendor lock-in".

A schematic representation of this can be seen below:

Cake solves bot of these issues in a sense that you will be able to move your "build logic" closer to your code (eg: into your git repo). Doing this, you will be able to:

1. Build independently of your build system and you will also be able to reproduce builds on your (development) workstation. The benefit is that you will be able to see what is actually going on and that this will help you with solving issues faster (and this only because of a shorter feedback loop!)
2. You will be able to migrate from one build platform to another and this will only have a minor impact. To give you an example. We moved to cake almost 2 years ago, just to be able to move from TeamCity to Azure DevOps at some point in the future. And today, we are (finally) doing this migration with a very limited impact for the development teams! (big win!)

A schematic overview of what this looks like, can be seen below:

So instead of relying on a build process that is defined in a build environment (and that you can only execute in that context)

So what are your options?

Cake allows you to work in a few different ways. Below, I'll discuss the different approaches and point out (how I see) the pro's and con's:

Functionality in cake file

The easiest approach is to write a cake file with the functionality that you need for your project. A typical (basic) build.cake file can be found on the the cake getting started guide

This approach is really interesting in smaller organizations that do not have a lot of builds to manage. This is because you will typically use methods that are offered by Cake. (These can change over time as cake evolves)

On top of that, and certainly when you have more than one builds to manage, you will typically re-use build scripts and these all will have to be maintained. Therefore you will (probably sooner than later) evolve to an approach where you have your own "library" of shared functionality...

Common functionality in shared cake file

Once the number of builds grow, you will learn rather fast that the duplication of your build logic will give you a lot of work in the context of the management of these cake files. At this point, you will soon start evolving towards "the next level", which is the approach of the shared cake file that allows you to set up "a shared library" of functionality. You can define "functions" in a C# as an abstraction on top of the standard cake functionality. (To give an example, with those functions you can group cake functionality that always have to do the same in your context or you can ensure that specific settings are specified the way that you want it...) This is specifically useful if you want to provide a framework that can be used by several builds...

Using this shared functionality is as simple as importing the template file into your "build specific" cake file. You can then work with your own "custom" functionality.

This makes it easy to manage your builds in an environment that is not too big, whilst having some kind of stability.

One thing to be aware of is that modifying functionality in those "shared" cake files immediately has an effect everywhere the functionality is used, so be aware that there is a drawback on "shared cake files". They do not offer versioning and thus no version pinning for the "end user" of your shared functionality. We discovered this very fast and evolved very fast towards a cake add-in, which offers this higher level of stability!

Common functionality in cake add-in

A cake add-in allows you to provide your shared functionality in the form of a dll, which can be packaged, versioned (and published to a (private) Nuget feed). This allows your consumers to use your functionality in the same manner as they work in their development flow (if they are working in a .net environment)

Getting started is not that hard and it is explained on the cake website . The thing that we figured out at the time is how we had to do the Nuget packaging, versioning and testing so that we could provide the development teams a stable environment to work in.

The goal of this blogpost was initially to also describe the testing and packaging, but while I was browsing the cake website, I discovered that a pretty good documentation was added for the packaging part. This one can be found here

For versioning, we work with GitVersion and it allows us to do the versioning of our package based on the commit history of our git repo. (yes, this is also part of what is offered by cake. If you want to learn more about it, then check out this page )

Some things that we learned along the way:

1. you only need to create one cake library, it can contain everything that you need. (Don't waste your time with the introduction of overly complex structures with multiple cake add-ins if you don't need to. We started with this approach and came back from it pretty soon!)
2. Test test test! The number of use cases that you offer, combined with the freedom in parameters, results in a huge amount of possible scenario's that your add-in can cover. If you don't actively (unit)test your functionality, then you will discover a lot of issues along the way!
3. Think strategically: what you offer today, will most likely be around for a while. this means that you will have to support it that long. Therefore, if you have the choice to "fix" something, or think about a decent solution, then go for the latter!
4. In the same area, but also really important: changes in function signatures is "not done" out of the blue. If you really have to change a signature, then create a new one next to it and (of the old one has to disappear) make the old one obsolete. This allows your users to see that a functionality that they use, has become obsolete. If you do it well, then you can even signal what needs to be done by them to evolve with the changes that you introduced.
5. Sometimes it is a good idea to jump in and see what versions of your add-in are still used. We just did a migration of one build system to another (which was a breeze with cake) and I was surprised by the amount of different (and old versions) of our add in that were in active use. I took quite some time upgrading a lot of our cake files (so that they use the latest version of our add-in). To me, having to do this, isn't really a drawback as it allows for stability in the context of the teams:
   1. not having to "find" changed functionality over and over
   2. being able to evolve at your own pace
6. Abstraction is great if you want to be able to be future proof. In a sense, you can bundle an entire build process in one method call. This allows you to evolve the underlying code ( in your add-in) without too much impact on the "end users". The drawback of this approach is that your users won't really understand what happens and when things go wrong, they will show up on your doorstep! So, be aware of this, you want to manage for scale and manageability (if needed), but most likely you will also need to offer something that is less "black box", for the ones that want to have more control...

We use this way of working (cake add-in, with a gitversioned and unit tested Nuget library) in an environment with a few hundred build pipelines and for us, it really is a great solution!

Conclusion

Cake is a great tool/platform to manage your builds (and build environment). It really is flexible in the sense that you can do whatever you want with it in the context of your builds. You can also use in a lot of different ways and in different scales.(from a one person/team environment to an enterprise). It takes a bit effort to get started and get everyone on board, but once you have started, you will see very soon that it has a lot of added value. I'm a fan!

Link to presentation (Dec 2019)

As a final note: I presented this story during visug XL in December 2019. The slides were never put online, but I have uploaded them now as part of this post. You can find them here

Small note: Some of the drawings in this presentation are drawn from the Bikablo publications, www.bikablo.com

I can go on about this for a while, but I think that I shared the most important info! If you want to know, more, do not hesitate to contact me on (preferably) twitter !