Engines Before Services

September 20, 2015

If you’ve ever worked on a poorly designed, monolithic Rails application you know how messy they can get. Fat controllers referencing models nested in folder structures three layers deep, filters used in base controllers that inherit from other base controllers, etc. Logic becomes strewn every which way because of entangled dependencies. If you change one thing you break another completely unrelated part of the system.

Applications are not designed this way from the beginning. No competent developer thinks an application such as this represents good design. This type of thing just tends to happen over time when features take precedent over code quality.

But all hope is not lost. Like a technical debt collector, it is your job to go in and clean up the mess. It’s time to bring some sanity back into the codebase allowing you and your fellow developers to be reunited with the simplicity and beauty that can be Ruby and Rails.

Micro.. what?

A popular topic these days when talking about working with legacy monolithic apps is breaking up the monolith into microservices. The basic idea is that you break your monolith into several smaller logical applications that each have a single purpose and are thus easier to work on and test.

I’m not going to really go into what a microservice is or how one would go about designing a microservice architecture. There are several other blog posts and videos to read and watch if you are interested.

One question that seems to arise whenever you begin to talk about microservices is: “how do you know what constitutes a good microservice?”. A usual followup is “ok.. I’m sold on this microservice stuff.. but where do we start?”. The concern is that if you start extracting a microservice and do it wrong, you may soon realize that in fact you’ve just created another mini-monolith.

I propose that when working with a monolithic Rails app, a good place to start is to first break your app up into a combination of smaller apps within the same codebase before diving straight into microservices.

Enter the Engine.

Choo Choo

Since Rails 3, Rails has shipped with the ability to create engines, which are basically smaller sub-applications that can be leveraged to add additional functionality to your existing app. Popular gems such as devise and RailsAdmin are actually Rails engines that can be ‘mounted’ into your application.

Engines are really just smaller Rails apps themselves. They even have a very similar structure:

app
│   ├── assets
│   │   ├── images
│   │   │   └── blorgh
│   │   ├── javascripts
│   │   │   └── blorgh
│   │   │       └── application.js
│   │   └── stylesheets
│   │       └── blorgh
│   │           └── application.css
│   ├── controllers
│   │   └── blorgh
│   │       └── application_controller.rb
│   ├── helpers
│   │   └── blorgh
│   │       └── application_helper.rb
│   ├── mailers
│   ├── models
│   └── views
│       └── layouts
│           └── blorgh
│               └── application.html.erb


Task Rabbit has written an EXTREMELY helpful article on how to structure your engines inside (or outside) of your existing Rails application. I definitely recommend checking it out before you get started: http://tech.taskrabbit.com/blog/2014/02/11/rails-4-engines/.

Why Engines?

As mentioned before, engines are usually used to add new functionality to an application, however in the case of breaking apart a monolith, we want to extract sections of our existing application into engines.

There are several reasons for breaking apart an application into a collection of engines:

  1. Defining Your Domain Boundaries - Extracting your application into several engines requires you to think about which pieces naturally belong together. These are essentially your application’s domain boundaries and they are important to define before you begin thinking about writing microservices. In order to reduce coupling and ‘crosstalk’ between services, it is important that services are sized correctly and contain the functionality required.
  2. Co-located Code - Closely related to the above point, extracting your code into engines requires that related code actually lives in the same place. Simply moving your code to their respective folders and files required by Rails when using engines will make sure that any code that is closely related ‘lives’ in the same place. This will be a huge help when making the switch from an embedded engine to an external microservice as it will allow you to more safely remove code when ready.
  3. Cleaning Out the Cruft - While I would recommend first extracting your application code into engines before doing any refactoring, this process does provide the opportunity for you to be able to clean up some technical debt as well. Extracting your existing application into several components really allows you to focus on each of those components one at a time, and in doing so you are able to hopefully pay down some of that debt that has accumulated over time.

How To Communicate

Ok, so you have successfully extracted some part of your application into it’s own engine. Now the hard work starts. How do you interface your ‘new’ engine with the rest of your application and vice-versa?

Let’s imagine that the engine you extracted is responsible for managing comments in your widely successful blogging application. Because you follow best practices, all of your comment engine code is now namespaced under the ‘Comments’ module like so:

module Comments
  class CommentController
    def new
      ....
    end

    def create
      ....
    end
  end
end


Ok so you can setup your main application routes to route incoming requests to create comments to your engine’s controller. But how do you communicate from your engine to your main application? For example, let’s imagine your main application is still responsible for managing the blog posts themselves. You could simply ‘reach’ into your existing application code and leverage the models/controllers that you know already exist and that Rails makes available to you.

For example you could do something like this in your engine:

module Comments
  class CommentController
    def new
      # Post model from main_app 'app/models/post.rb'
      @post = Post.find(params[:post_id])
      ....
    end
  end
end


Don’t do this! You have just done all of the hard work to extract your application into it’s separate components and now you mixing them back up again! This is what’s known as a violation of ‘Seperation of Concerns’ and is a big no-no in the microservice world.

Remember that you should now start thinking of your Comments engine as it’s own separate microservice, even though you know that the code still currently ‘lives’ in the same repository. This shift in thinking will be required when you actually do move your engine into it’s own application and communicate via HTTP or some other messaging protocol with your ‘core’ app, so you might as well start now. Thinking of your engines as different actual services will help you in the long run when they actually are.

A better way to communicate with your existing application is wrap all of your calling code in a ‘client’ or ‘service object’ as it’s known in the Rails world.

# app/services/post_service.rb
module Services
  class PostService
    def get_post(post_id)
      ....
    end
  end
end

# engines/comments/app/controllers/comments/comment_controller.rb
module Comments
  class CommentController
    def new
      @post = Services::PostService.get_post(params[:post_id])
      ....
    end
  end
end


This may seem like an arbitrary abstraction now, but having all of your post logic gathered in a single location will be extremely helpful once you do break the ties with your existing application. You won’t have to grep your entire code base for all references to the Post model for fear of breaking existing functionality, all of that logic will live entirely in the PostService. The PostService will become the only way you can get and create data about posts. It becomes your post client.

Final Thoughts Regarding Coverage

Before you even begin to extract your monolith into engines, you MUST make sure that you have the proper test coverage. You are going to be moving a lot of files and refactoring existing code, and doing so without the proper coverage is just plain reckless. It’s akin to performing shotgun surgery blindfolded in the dark.

While having 100% coverage for your entire application may not be achievable or cost effective, you have to have confidence that you won’t introduce regressions when beginning your refactoring. In short, you should make sure that all of the code you know you are going to be touching has as close to 100% coverage as possible.

If you’re thinking about using this pattern to start the move towards a microservice architecture I would first start by beefing up your existing test suite, and ONLY then begin moving code.

Hopefully this post sparked your interest in Rails engines or at least got you thinking about you might go about breaking apart your monolith while improving code quality at the same time.

Please drop me a line either below or on Twitter and let me know what you think!

Did you find this content helpful?


Let me send you more stuff like this! Unsubscribe at any time. No spam ever. Period.


Subscribe to MarkPhelps.me

* indicates required

Discussion, links, and tweets

Mark Phelps

I'm a Software Engineer in Durham, NC. I mostly write about Go, Ruby, and some Java from time to time.