Replacing roles in Puppet

We’ve been using Puppet for the better half of a decade now.

Puppet plays a vital role in managing our infrastructure. Every server we run is managed with Puppet. Beginning with Puppet wasn’t easy. Puppet has a pretty intense learning curve, especially when you’re not used to declarative development.

When you start learning a new language, whether it’s declarative like Puppet or imperative like Python, there is a lot of trial and error involved. Code is added organically and can grow fast— which often means you will be introducing a lot of technical debt.

And we were no exception.

When we started we didn’t really think about structure. We wrote unique code for most servers and repeated ourselves constantly. This lasted until 2014, when we started adding structure through roles and profiles.

Roles and profiles were first introduced in an article by Craig Dunn. Puppet needed a higher level of abstraction. At the time, Puppet didn’t have a lot of best practices on this particular subject and roles and profiles were the most popular solution. When we began implementing them we followed the letter of the law. We introduced new roles in our environment, added profiles to them and assigned these roles to our nodes. These were what Puppet now calls, granular roles.

But it never felt right. It felt like we were adding roles for the sake of having them, they didn’t have much purpose.

Implementing granular roles

Granular roles means that each type of node gets its own role. Our roles were looking like the one below:

class role::jenkins::master {
  include profile::base
  include profile::server
  include profile::jenkins::master
}
class role::jenkins::master {
  include profile::base
  include profile::server
  include profile::jenkins::master
}

Roles are meant to be simple and clean. Typically, you would only assign a single role to a server.

node 'master1.jenkins.example.com' {
  include ::role::jenkins::master
}
node 'master1.jenkins.example.com' {
  include ::role::jenkins::master
}

Let’s illustrate what happens in an environment with roles and profiles.

As you can see it adds another layer of abstraction. You can re-use the role for other servers and you never have to repeat yourself. And for some there will be benefits to that. In our case, not so much.

Our environment has lots of different servers. They don’t necessarily have things in common. Sometimes they do, but in most cases we can never re-use an existing role. We suffered from role-bloat, a lot.

We needed something else. Luckily, our server names are boring (read: predictable), which means we can take another approach.

Roles and regular expressions in node definitions

If we would use Jenkins (we don’t) our masters would be given names such as master1 and master2. So we don’t need specific node definitions as we can use regular expressions.

We can match our servers with the following snippet:

node /^master\d+\.jenkins\.example\.com$/ {
  include ::role::jenkins::master
}
node /^master\d+\.jenkins\.example\.com$/ {
  include ::role::jenkins::master
}

This is why we decided to cut out the explicit role definition and specify the profiles directly in the node itself.

node /^master\d+\.jenkins\.example\.com$/ {
  include profile::base
  include profile::server
  include profile::jenkins::master
}
node /^master\d+\.jenkins\.example\.com$/ {
  include profile::base
  include profile::server
  include profile::jenkins::master
}

Which results in the following flowchart:

This approach is also explained in the sixth approach of designing convenient roles article in the Puppet documentation. We’ve only added regular expressions into the mix.

In closing

We didn’t do a lot, really. We removed the explicit role definition and moved it into the node classifier. We think it has made our Puppet code cleaner and easier to read, but it works exactly as it did before.

With this solution we can easily find which roles have been added to a server, and make changes, whenever needed. And although it may not be a solution for everyone, it made managing our infrastructure a lot faster.