How to assemble Rails frontend using Webpacker?

Frontend developer's tips for a painless migration from Sprockets
05 July 2017   10603

What's wrong with Sprockets

Sprockets, aka RoR Asset Pipeline, has become obsolete long time ago. Frontend is evolving with leaps and bounds, and Sprockets is not keeping pace. As a result we have plenty of problems. Some of them:

  • Limited support for new frontend tools. For example, you will face issues using PostCSS plugin or use a much faster SASS libsass compiler instead of written in Ruby.
  • No support for ES2015 transpiling
  • No support for javascript modules
  • No support for sourcemaps, which has existed for many years

Also, Sprockets configuration has low flexibility. Guess what Sprockets will do if you disable minification and pass through it some minified javascript. Sprockets will deminify the code and insert comments everywhere trying to emulate sourcemaps. I've never found how to disable this behaviour.

Ruby on Rails

Ruby on Rails (RoR) - a framework written in Ruby programming language.

Sprockets is being slowly developed. Version 4 was supposed to solve some problems and it was promised to be released with the Rails 5, but this has not happened so far.

At the same time, Sprockets is a rails-specific tool, but the frontend evolves by itself and the community prefers to use and create universal tools that don't have any specific limits. So, with help of node.js there were created task runners (for example, pretty popular gulp) and module bundlers (webpack, which is able to resolve dependencies between javascript modules, concatinate everything in the right order and remove the redundant code).

It is worth mentioning package managers in the context of frontend. If you use Sprockets, most likely you use gems to import any frontend-associated libraries. Because it's a common way. The problem is that authors of such libraries don't care about creating and maintaining gems, they use npm and bower repositories. So, you have to use a third-party gem and hope that it will be somehow maintained or you can use crutches like rails-assets.org.

All this has bothered me for a long time and not just me. As a result we had various attempts to tape gulp, webpack and Rails together. It worked but it also took time and nerves. And now it can be left behind.

What's great about Webpacker 

Webpacker is a Rails gem that provides smooth and standard integration with webpack module bundler and yarn package manager (it's like modernized npm). Optional integration with popular frameworks / libraries such as react, angular, vue is also available. The current version of Webpacker is 2.0 for the moment when I write this article.

Requirements are:

  • Rails 4.2+
  • Ruby 2.2+
  • node.js 6.4.0+
  • yarn

If you have Rails 5.1+ you can use the --webpack option, because Webpacker is delivered "out of the box" with this version. For example:

rails new webpacker-example-app --webpack
// or this to setup react or angular or vue 
rails new webpacker-example-app --webpack=react
rails new webpacker-example-app --webpack=angular
rails new webpacker-example-app --webpack=vue

In case of Rails 4.2+, you can install Webpacker by adding it to your Gemfile and running the following command after installing the gem: 

rails webpacker:install
rails webpacker:install:[react, angular or vue] (optional)

This command will check the compliance with the requirements and take care of creating: 

  • Scripts for running webpack and webpack-dev-server in the bin directory
  • The app/javascript directory
  • The config/webpack directory and webpacker.yml
  • The packages.json (analogue of Gemfile for npm) with necessary stuffing. Then it will run the packages installation with yarn, which will create the node_modules directory and the yarn.lock file

Default packages list includes Babel, so you will be able to use the newest Javascript syntax.

Setting everything up

Webpacker suggests to follow certain conventions, but you can change everything quite easily, look into the config/webpack directory and config/webpaker.yml.

Webpack Configuration

The config/webpack directory has corresponding configuration file for each Rails environment. Also there is shared.js file, that is common for all environments, and configuration.js file, which is responsible for processing settings from config/webpacker.yml.

The production.js configuration by default commands webpack to assemble all modules into single bundles, to minify the code, to add fingerprints to file names, generate sourcemaps and even compressed gzip versions. So that is even more than Sprockets able to do, but everything can be configured as you like. You can add or remove steps from an assembly process and you can modify them. For example, you don't need gzip versions. No problem, just remove them from the config.

The config/webpack/loaders directory, contains loaders which are responsible for processing and transforming files according to certain rules. They are similar to tasks from other build tools. The babel.js loader is responsible for .js and .jsx files and instructs the babel transpiler to convert code written with ES2015 syntax to ES5 code. The coffee.js loader is responsible for .coffee files and also turns code into ES5. Again, everything can be configured, added or removed.

Styles Processing

Webpacker was originally intended to solve problems with javascript, not for replacing Sprockets. It was assumed that Sprockets will continue handle styles and other assets. However, the styles code, just like javascript, can be handled by your favourite preprocessor's loader. Webpacker by default generates the config/webpack/loaders/sass.js file, which contains configuration for processing styles written in sass/scss. It is done to make working with styles easier for applications built with component approach, React.js or other similar tools that allows you to include styles directly in components. But you don't have to follow this approach, so I recommend to use webpack to assemble styles.

The Webpacker installation also integrates PostCSS into the assembly. It adds a lot of possibilites and flexibility to styles management. The PostCSS configuration is in the .postcssrc.yml file in a project's root. There are list of plugins:

  • precss, not so useful when we already have scss
  • autoprefixer, a great tool that allows you to forget about writing browser vendor prefixes

Unfortunatelly, there is no "out of the box" feature to conveniently include assets in css code. I mean functions like asset-url and asset-data-uri from Asset Pipeline. But there is a solution - the postcss-assets plugin. In order to install it execute yarn add postcss-assets -D. This will add the package to the packages.json list and install it. Then we need to add changes to .postcssrc.yml, my config looks like this:

plugins:
  postcss-smart-import: {}
  precss: {}
  autoprefixer: {}
  postcss-assets: {
    loadPaths: ['images', '../app/assets_src/images/'],
    basePath: 'public',
    cachebuster: true,
  }

Your loadPaths most likely will be different. I recommend to look at readme file and to play with configuration.

As a result, you will be able to use resolve and inline functions in your css code.

Directory structure

Webpacker creates the new app/javascript directory and it is suggested to store there all the javascript code and entry points for webpack (so called packs). The convention is that entry points are placed in the app/javascript/packs directory, and javascript modules are placed in the app/javascript.

For various reasons styles are expected to be placed in the same app/javascript directory. But the good thing is that it's easy to configure it in config/webpack/shared.js. Look for resolve.modules which is an array of paths used by webpack to search for files. The first item is resolve(settings.source_path). source_path here is taken from config/webpacker.yml. So, for my own project I've set the source_path: app/assets_src, and modified resolve.modules like this:

modules: [
  resolve(`${settings.source_path}/javascripts`),
  resolve(`${settings.source_path}/styles`),
  'node_modules'
]

Launch

Webpacker adds two scripts for running webpack:

  • bin/webpack for a one-time assembly
  • bin/webpack-dev-server for 

I recommend to check the content of this files.

Also, in order to build your assets in production mode, for example for testing purposes, you can pass the --config param with a full path to the corresponding config file.

Note that if you or your colleague has added some modules to packages.json it will require installation. Otherwise webpack will throw an error saying that a module is missing. Just run yarn command in the project's directory, just like bundle install but for npm packages.

Deploy

Webpacker adds a new task webpacker:compile, which runs every time when the assets:precompile is run. If you have completely abandoned Sprockets, then you can manually run the bundle exec rails webpacker:compile command during deploy.

Including bundles into views

Very similar to the Asset Pipeline. Use new javascript_pack_tag and stylesheet_pack_tag helpers. They will add necessary HTML tags with links to assembled packs.

Setting up Continuous Integration

Continuous integration

The practice of merging all developer working copies to a shared mainline several times a day

 

Vexor

Cloud Continuous Integration service, unlimited parallelis

If you are using a CI system, then of course you will need to configure it: add node.js and yarn.

At Evrone, we successfully use Vexor.io. So, the configuration for Vexor with Ubuntu server will look like this.

The vexor.yml file in the project's root will contain

rvm:
  - 2.3
node_js:
  - 6.11.0
 
before_install:
  - ./install_yarn.sh
 
install:
  - bundle install
  - yarn
  - bin/webpack

The install_yarn.sh script looks like this:

#!/usr/bin/env bash
set -ex
 
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
 
sudo apt-get update && sudo apt-get install yarn

Don't forget to make the file executable: chmod +x install_yarn.sh

Conclusion

I've moved my current project to Webpacker but I advise you to think carefully before doing the same. Think about if it will worth for you some time that the transition will require. Nevertheless, you can use it for all new Rails projects.

Charles Nutter. How to move your Ruby project to JRuby and why

Charles Nutter works on JRuby and JVM language support at Red Hat.
03 October 2018   1333

— How did you get into programming and into Ruby world?

— In 2004, I was working at a government contracting firm as a Java Enterprise Architect. I was in charge of a large mainframe conversion for the United States Department of Agriculture, which meant I spent a couple weeks a month in the Washington D.C. area. One of those trips coincided with RubyConf 2004, and since a close friend had been recommending I look at Ruby, I decided to attend. So there I was sitting in a Ruby conference without ever having learned Ruby...and I understood every piece of code, every example. I was amazed and vowed to find a way to bring Ruby into my Java world.

— Which projects are you working on now?

— My primary role is as co-lead of JRuby. This also means supporting several side projects like our native-library backend (Ruby's FFI library is maintained by us using this backend on JRuby) and our String encoding subsystem (an elaborate port of the logic from CRuby). I also do much of the outreach to the community and try to make sure our users are getting their issues addressed. There's always plenty to work on!

— Which one would have the biggest general impact from you opinion?

— I like to think that JRuby, while not the most popular JVM language, has at least helped to change the JVM platform. Because of our collaborations with Sun Microsystems, Oracle, and others, we have solid dynamic language support at the JVM level along with many other projects to support alternative languages. The JVM today is a much more hospitable home for non-Java languages than it used to be, and I hope we've played some small part in that change.

— Which languages are you writing on in your everyday life? Which one do you like most? Why?

— Most days I write in a mix of Ruby and Java, since JRuby is implemented using both. I like both languages for different tasks. Ruby is a better language for building applications that need to evolve and adapt quickly. Java is a great language for writing high-speed, reliable libraries and services. JRuby gives you the best of both worlds!

— Do you like to program in Java?

— I do, especially with all the language improvements that have been added recently, like lambdas (closures or blocks in Ruby) and the new "var" syntax for declaring local variables when the static type is unambiguous.

— What do you think about Rust?

— Rust is a great language! I did a lot of C++ development in my college years, and I can tell you right now if I'd had Rust available then I would have used it. I'm especially jealous of the static-typed ownership model, which helps avoid thread-safety issues like races and data corruption. I hope to see other languages adopt this pattern in the future.

— What do you think about the Ruby language perspective? Is it or its community dying?

— After all these years working on JRuby, I do still love Ruby syntax and the Ruby way of doing things. However I worry that the language is held back too much by limitations of its primary runtime. JRuby has been fighting to make true parallel threading a reality for Ruby developers, but still today the vast majority of Ruby services are run using multiple isolated processes, wasting tremendous amounts of CPU and memory resources. I believe this is due to the C API for writing Ruby extensions being so large and so invasive...it prevents many improvements -- including parallel threading -- from being developed. Hopefully we'll see this change some day.

— Which upcoming or not well-known features of Ruby language would rush in future?

— I look forward to strings becoming immutable-by-default, as they are in most other languages. Parallel programming would be much simpler if more of Ruby's objects supported pure-immutable or "deep freeze" semantics. It's a bit like the Rust ownership model...if you're going to be sharing an object across threads, choose the version of that object that you know can't be modified anymore. This extends to arrays, hashes, and just about every other mutable object in Ruby: we need to make it easier to lock down mutable data.

— Could you give me an advice on how to move my ancient monolithic project to JRuby? And should I?

— The first question really is whether such a move would benefit you. There's many good reasons to consider a move to JRuby:

  • Reducing CPU  and memory costs in a shared hosting environment by utilizing JRuby's true parallel threading
  • Deploying a Ruby application into a JVM-heavy environment, such as used by larger financial or government organizations
  • Needing access to libraries that only exist on the JVM, or that are more portable or scalable on the JVM than their Ruby or C equivalents
  • Getting a little performance boost out of CPU-heavy or concurrency-heavy applications.

— I would say if your application is scaling well and not costing you too much today, perhaps you don't need to make a move. But if you decide you need more out of Ruby, here's the process for migrating:

  • Do a clean bundle of your application, paying special attention to C extensions you may be using. You can also do this bundling *on* JRuby, and then deal with missing or unsupported libraries one by one.
  • For each extension, search for a JRuby equivalent. We have some pages on the JRuby wiki to help with this. Most popular libraries have JRuby versions. If no JRuby version exists, you may look for a pure-Ruby version (it might be fast enough on JRuby) or a JVM library (in Java or Scala or Clojure or whatever) that could be used as a replacement.
  • Once your bundle completes, you should have a working JRuby application! We've worked very hard on compatibility, and try to be responsive if users find new issues, but a successfully-bundled application is expected to work.

The steps beyond this involve deciding how to take advantage of your newfound power: how many threads to throw at a given server, what you're going to do with all the money you're saving, etc.

— What should nowadays students learn to become good programmers?

— When I was at university, my earliest computer science courses used the Scheme language, a Lisp-like functional language that's great for teaching the fundamentals of programming. I still recommend that serious new programmers work through at least some of the Structure and Interpretation of Computer Programs book from MIT. Beyond that, I'd say learn as many different and unusual languages as you can; they'll all give you new ideas and new ways to look at programming problems.

— How do you keep yourself motivated for programming? Have you ever been "burned-out"?

— Burn-out is a real problem in our industry, and working in open source brings with it huge amount of stress. We've all felt that way sometimes...too much work to do and not enough time to do it, missing out on time with family and friends, ignoring our own health so we can fix one more bug. These days I try to center myself by keeping up with hobbies: playing video and board games, learning to play guitar, studying foreign languages, and traveling around the world to meet new friends. There's always this nagging workaholic telling me to get back on the job, but I'm learning how to maintain the right balance.

— What do you think about Russia and what do you expect of the upcoming RubyRussia event?

— I love Russia, and my speaking trips the past few years have been some of the most rewarding of my life. This will be my fourth visit, having been to Saint-Petersburg, Moscow, and Novosibirsk (!!!) previously. I'm looking forward to returning to Moscow and meeting the RubyRussia community I've heard so much about!

Questions by Dmitry Matveyev PM at Evrone https://www.facebook.com/matveyev.d