Charles Nutter. How to move your Ruby project to JRuby and why

Charles Nutter works on JRuby and JVM language support at Red Hat.
03 October 2018   565

— How did you get into programming and into Ruby world?

— In 2004, I was working at a government contracting firm as a Java Enterprise Architect. I was in charge of a large mainframe conversion for the United States Department of Agriculture, which meant I spent a couple weeks a month in the Washington D.C. area. One of those trips coincided with RubyConf 2004, and since a close friend had been recommending I look at Ruby, I decided to attend. So there I was sitting in a Ruby conference without ever having learned Ruby...and I understood every piece of code, every example. I was amazed and vowed to find a way to bring Ruby into my Java world.

— Which projects are you working on now?

— My primary role is as co-lead of JRuby. This also means supporting several side projects like our native-library backend (Ruby's FFI library is maintained by us using this backend on JRuby) and our String encoding subsystem (an elaborate port of the logic from CRuby). I also do much of the outreach to the community and try to make sure our users are getting their issues addressed. There's always plenty to work on!

— Which one would have the biggest general impact from you opinion?

— I like to think that JRuby, while not the most popular JVM language, has at least helped to change the JVM platform. Because of our collaborations with Sun Microsystems, Oracle, and others, we have solid dynamic language support at the JVM level along with many other projects to support alternative languages. The JVM today is a much more hospitable home for non-Java languages than it used to be, and I hope we've played some small part in that change.

— Which languages are you writing on in your everyday life? Which one do you like most? Why?

— Most days I write in a mix of Ruby and Java, since JRuby is implemented using both. I like both languages for different tasks. Ruby is a better language for building applications that need to evolve and adapt quickly. Java is a great language for writing high-speed, reliable libraries and services. JRuby gives you the best of both worlds!

— Do you like to program in Java?

— I do, especially with all the language improvements that have been added recently, like lambdas (closures or blocks in Ruby) and the new "var" syntax for declaring local variables when the static type is unambiguous.

— What do you think about Rust?

— Rust is a great language! I did a lot of C++ development in my college years, and I can tell you right now if I'd had Rust available then I would have used it. I'm especially jealous of the static-typed ownership model, which helps avoid thread-safety issues like races and data corruption. I hope to see other languages adopt this pattern in the future.

— What do you think about the Ruby language perspective? Is it or its community dying?

— After all these years working on JRuby, I do still love Ruby syntax and the Ruby way of doing things. However I worry that the language is held back too much by limitations of its primary runtime. JRuby has been fighting to make true parallel threading a reality for Ruby developers, but still today the vast majority of Ruby services are run using multiple isolated processes, wasting tremendous amounts of CPU and memory resources. I believe this is due to the C API for writing Ruby extensions being so large and so invasive...it prevents many improvements -- including parallel threading -- from being developed. Hopefully we'll see this change some day.

— Which upcoming or not well-known features of Ruby language would rush in future?

— I look forward to strings becoming immutable-by-default, as they are in most other languages. Parallel programming would be much simpler if more of Ruby's objects supported pure-immutable or "deep freeze" semantics. It's a bit like the Rust ownership model...if you're going to be sharing an object across threads, choose the version of that object that you know can't be modified anymore. This extends to arrays, hashes, and just about every other mutable object in Ruby: we need to make it easier to lock down mutable data.

— Could you give me an advice on how to move my ancient monolithic project to JRuby? And should I?

— The first question really is whether such a move would benefit you. There's many good reasons to consider a move to JRuby:

  • Reducing CPU  and memory costs in a shared hosting environment by utilizing JRuby's true parallel threading
  • Deploying a Ruby application into a JVM-heavy environment, such as used by larger financial or government organizations
  • Needing access to libraries that only exist on the JVM, or that are more portable or scalable on the JVM than their Ruby or C equivalents
  • Getting a little performance boost out of CPU-heavy or concurrency-heavy applications.

— I would say if your application is scaling well and not costing you too much today, perhaps you don't need to make a move. But if you decide you need more out of Ruby, here's the process for migrating:

  • Do a clean bundle of your application, paying special attention to C extensions you may be using. You can also do this bundling *on* JRuby, and then deal with missing or unsupported libraries one by one.
  • For each extension, search for a JRuby equivalent. We have some pages on the JRuby wiki to help with this. Most popular libraries have JRuby versions. If no JRuby version exists, you may look for a pure-Ruby version (it might be fast enough on JRuby) or a JVM library (in Java or Scala or Clojure or whatever) that could be used as a replacement.
  • Once your bundle completes, you should have a working JRuby application! We've worked very hard on compatibility, and try to be responsive if users find new issues, but a successfully-bundled application is expected to work.

The steps beyond this involve deciding how to take advantage of your newfound power: how many threads to throw at a given server, what you're going to do with all the money you're saving, etc.

— What should nowadays students learn to become good programmers?

— When I was at university, my earliest computer science courses used the Scheme language, a Lisp-like functional language that's great for teaching the fundamentals of programming. I still recommend that serious new programmers work through at least some of the Structure and Interpretation of Computer Programs book from MIT. Beyond that, I'd say learn as many different and unusual languages as you can; they'll all give you new ideas and new ways to look at programming problems.

— How do you keep yourself motivated for programming? Have you ever been "burned-out"?

— Burn-out is a real problem in our industry, and working in open source brings with it huge amount of stress. We've all felt that way sometimes...too much work to do and not enough time to do it, missing out on time with family and friends, ignoring our own health so we can fix one more bug. These days I try to center myself by keeping up with hobbies: playing video and board games, learning to play guitar, studying foreign languages, and traveling around the world to meet new friends. There's always this nagging workaholic telling me to get back on the job, but I'm learning how to maintain the right balance.

— What do you think about Russia and what do you expect of the upcoming RubyRussia event?

— I love Russia, and my speaking trips the past few years have been some of the most rewarding of my life. This will be my fourth visit, having been to Saint-Petersburg, Moscow, and Novosibirsk (!!!) previously. I'm looking forward to returning to Moscow and meeting the RubyRussia community I've heard so much about!

Questions by Dmitry Matveyev PM at Evrone https://www.facebook.com/matveyev.d

A. Patterson: "I Wanna Write Ruby Extensions With Rust"

Interview with Aaron Patterson, Ruby Core Team and Rails Core Team member, senior software engineer at GitHub
24 September 2018   633

Aaron Patterson is Ruby Core team and Rails Core team member, senior software engineer at GitHub, Ruby Hero 2010, speaker at Rails Club 2014 and Ruby Russian 2018 and cat lover.

Aaron Patterson and his cat
Aaron Patterson and his cat

Let’s start with the standard question. What’s your personal ruby story? How did you get on this train? What did you achieve? How do you make this world better?

I’ve first discovered the language in 2006. I was a Java programmer at that time. Let’s go back a little bit further. I used to be a Perl programmer, and then I became a java programmer, but I didn’t wanna be a java programmer. Ha ha!

Why?

When I was a perl programmer we actually had our own web framework. We did a lot of the stuff that Rails did: уou can just change stuff, reload the page and test it. And everything just worked. When we switched to doing Java development, it was like - you have to re-compile everything, then it will take 10 minutes before you can test any changes you’ve made. I like dynamic languages like Perl better than Java. So, I was waiting for Perl 6 to come, and while I was waiting for Perl 6, I discovered Ruby. I was like: “Oh, wow! This is what I wanna do!” So, I just started doing Ruby in my free time, like, for side projects. You know, just fun programming stuff. That’s when I very first discovered it. I finally got a job doing Ruby programming in 2008.

That was already Rails?

Yeah, some friend of mine decided to start a startup. “We gonna use Rails. Do you wanna come with us and join our company?”. I was like: “Yeah, absolutely, I’d love to use Rails for my job!” That’s first time I’ve started it.

To be honest, I did not like my job when I was at that company. Ha-ha! So, what I did was - wherever I could, I wrote open source code at work. It was like: “OK, this project will take 2 days.” I got the project done in few hours and used the rest of time for open source code.

That’s what in Russia we call: “Don’t beat a lying person!” Haha! I’m just sitting here quietly, doing things. Please, leave me alone!

Yep! Haha! So, that’s where I’ve started doing a lot of open source work. At that job I’ve first started writing Nokogiri and working on my Ruby open source stuff. That’s where I’ve started with open source world. I’ve just kind of participating until one day I just got on the Ruby Core Team and Rails Core Team.

How did you eventually find yourself at the Rails Core Team?

We just kept finding bugs and developing Rails applications. We were running into bugs;  I’ve fixed the bugs and submit patches. I just kept submitting patches. Eventually, they got tired of me sending pull requests. Haha!

Just try to manage all this stuff yourself, right?

Yes, exactly! So, it was basically like a brute force attack! Haha!

Sounds reasonable! So, what’s your overall contribution to Rails?  

I worked a lot on basically all parts of the framework. I mostly work on Active Record. I personally like to do bug fixes and performance improvements. The reason I like doing that stuff is because it’s just doing everybody’s applications better. Everybody’s happy if the app gets better and you don’t have to do anything. That’s why I like working on that stuff.

You make some kind of “small” contributions which makes thing work. But didn’t you actually architecture “big” things?

Usually, whenever I do architectural stuff at Rails it will be stuff at internals. Like, how we do, for instance, URLs architecture, associations architecture, stuff inside the router, things like that. None of those things are necessarily exposed. They may be kind of expose to the user but they not like “Hey! This is the Thing! Go do this!”  I try to stick to that kind of stuff. I think it actually works well because I think David likes to do flashy awesome new features. I personally think: “OK, we’ll do those cool features. They are actually awesome!” Ha ha!

Yes, somebody has to do all the job! Your presentation at the conference will also touch some deep engineering parts of Ruby and Rails. What’s your presentation will actually be about?

I’m gonna talk about Ruby internals actually. I am still thinking what I’m gonna talk about.

GC, performance, everything, life, universe, 42?

I’m gonna talk about garbage collector, Ruby’s compilation process and bytecode. Basically, about the bytecode in a virtual machine and how those relate to the garbage collector. About some performance improvements I’ve made to the GC. I don’t think I’ll be talking much about Rails stuff at all. Haha!

Good note! Our conference was formerly known as “Rails Club”. Basically, after Matz said he’ll never be visiting the conference named after Rails, our organizers changed their minds and renamed the whole thing. So now we are “Ruby Russia”!

So, I will speak about Ruby internals.

In your opinion, what should Rails programmers do in their code to achieve better performance?

There are few different strategies. The first one, something like basically don’t do anything special. Just write your application. Get it done, get it out there, get customers, feedback, etc. As soon as you start doing that analyze the bottleneck. Don’t ever deal with any bottlenecks until you actually get customers using it. If you spend time working on bottlenecks, that are not actually a bottleneck - that’s basically a waste. You could’ve been writing new features. But I think a lot of people say what I’ve just said, so, let’s talk about actual performance stuff. First thing you do is just look at the database queries that the page is doing. It’s first line of attack - try to reduce the amount of time it takes for particular queries. Automating the queries, reducing the queries. You’ll be surprised how many times we forgot to add an index. Haha! So, add the index at the right place.

I’m doing the job interviews at our company and I understand how people forget about indexes and what are indexes at all. Why should one even bother about such a thing? OK, what do you think are other things which rubyists must know? What technical things rubyists should know to do his or her job better?

There’s a couple of things. One I think is Ruby the language itself. Learn the language very well. Other thing is learning UNIX well.

You are the first speaker I’ve interviewed who said you should know UNIX. Personally, I came to Ruby world from UNIX world. I was doing Linux, FreeBSD and tons of things with Perl. I came to Ruby as yet another Perl to do my sysadmin things, and then I’ve found it as a web language as well. You’re saying we should know UNIX. Why and how?

It’s important to learn about POSIX standards and the way they interact with the operating system because you gonna run into that stuff when you will start scaling. You…

… have to know who is General Failure and why he’s reading my file?

Haha, yes! You need to know what makes the difference in performance. Maybe, you don’t need to specifically memorize it but you need to know that they [syscalls — P.A.] exist and how to google them because you gonna run into this type of stuff. They are going to impact you because you are deploying your application to UNIX server so you need to learn how your application is going to interact with the OS that it’s being deployed to. Other important thing is if you learn those UNIX skills, you can take them to other languages too. If you ever have any problems, you can start there. That’s one thing I recommend people learn.

Do you think is it good for a rubyist to know another language? Is it possible to be a good Ruby programmer without knowing something outside Ruby alone?

That’s a good question. Honestly, I don’t know. All good Ruby programmers I know do know other languages. But I don’t know if you need to learn other languages to become a good Ruby programmer. I think it just happens that people learn other languages.

That’s good observation! By the medical point of view, the more languages people know, the farther their Alzheimer goes.

Haha!

After 40 you have to think about those things…

I’m getting close to 40s! I need to know that!

Let’s talk about Ruby itself. Ruby is a language with a great past. Is it also a language with a future? Week ago, I was at Saint Petersburg at the biggest IT conference I’ve ever seen in Russia. Saint Petersburg’s Ruby community wasn’t represented at the conference. I constantly had to do a big deal of apology for Ruby: that Ruby isn’t THAT dead, that Ruby is a language somebody still writes. Ruby has some kind of the  best web frameworks, things like that. Every major language on the market now has something as web development tools. Go, Rust, whatever. What is Ruby’s place in this ecosystem and does “Ruby with great past” have a future?

I think there’s couple of aspects answering that question. There are a lot of different languages that have web frameworks but I still believe that if you look at them from the perspective of developer ergonomics, Ruby is still going to come on top. It’s easy to use and easy to get something shipped. The problem is that it’s not new and shiny anymore. People wanna hop onto the next bandwagon. They wanna be on the next-to-Rails train.

They want a smell of a new car!

Yes! For future… There’s a lot of new developments go in Ruby, especially with a JIT and what Koichi works on, guilds. But the thing is, I would caution and say that Ruby definitely has a future but that’s the thing we all have to work for. If we put in an effort, the future’s definitely there.

Does Ruby have any perspective in areas other than web development? Or, do you know any examples where Ruby is being used these days outside of web development?

That’s a good question! It’s hard for me to answer because I’m so into… I’m only look at web development.

I’m asking because it’s pretty interesting to me as well. Guys of Python community like to boast their scientific success.

I know there’s a group working on Ruby scientific tools. But I think the main “other” application the people use Ruby for is systems administration.

How can we attract developers of other languages to our community?

That’s really good question! I think we just need to keep focusing on developer’s ergonomics, making developing web applications as easy as possible. We need to focus on lowering a bar for new developers who come onboard and write web applications. That’s the way we’ll get more new developers.

We came to this kind of a holy war question, about JavaScript. You know, they say that everything that could be rewritten in JavaScript will be rewritten in JavaScript. Do you feel like Rails will be also rewritten in JavaScript? We spoke about ergonomics of Ruby development. It’s best thing about Rails. One of the best known Russian programmers said that “many languages are good, but only Ruby has Rails”. But JavaScripters tend to question this situation. How can we battle with JavaScript? Or should we have symbiosis with that?

It’s very true that only Ruby has Rails. If you look at the web development frameworks for JavaScript, I don’t think that they compare very well to Rails. In terms of developer’s ergonomics. But the thing is - since we’re writing web applications, we need to work with JavaScript. We need to be a part or JavaScript community. It’s good for us to have symbiosis with that. If you can run any language on a server, why would you peak JavaScript? Haha! But it’s a fine language, I think, we need to work symbiotically. We still have developer’s ergonomics on our side, it’s still great at Rails community. So, you came to an IT conference and you had to represent Ruby there?

That was very informal, because I even didn’t have a t-shirt of my company or language. So I've just found a brightest group of young people which appeared to be the pythonists and we started chatting.

It’s better for us if we work together with other languages rather than being rival with them. Personally, I think programming in Ruby is so much easier and nicer than other languages. Why not? We’re talking about other programming languages and whether you should know them. I do think it’s important that Ruby developers learn other programming languages. Something like Java, Haskell, or something other functional like Elixir or Lisp, something like that. I think it's good to learn different paradigms because once you learn it, you can steal it and use it in your programming language. Nice thing about Ruby is that we can use different programming languages’ techniques in our programs.

Yes, we have, for example, instruments for functional programming or doing map/reduce/whatever.   

Yes, we’re able to use those things. If you use the language that encourage those, maybe you’ll learn a better way to do on the problem that you are trying to solve. I don’t know if you have to learn other programming languages to be good at Ruby, but learning other ones definitely helps me for sure. Honestly, I spend 50% of my time programming in C. Haha!

C makes your fingers stronger!

I program in C so others can program in Ruby. Haha!

Ruby internals are written in pure C, not ++?

Just C. It would be nice if more will be written in Ruby but honestly some of core stuff, for performance, we have to write it in C. One of the things I do is, we’re trying to do more memory profiling. So, I’m working at tools for doing memory profiling in Ruby. Since all the internals are in C, I have to write tools in C. I do a lot of C at work.

How does it go in Ruby with FFI and such?

FFI works pretty well. If you have a C library that only has one or two functions that you need… If it gets more complicated… It’s a lot harder. If you’re using FFI, you’re essentially writing C code that looks like Ruby. You still have to do weird stuff like memory management. I find personally, it’s easier to switch between those worlds if I use C to do the memory management, etc. and turn over to Ruby in other cases.

In Ruby, do we have interfaces to other languages?

Some interfaces with the JavaScript. I saw a guy who was doing a scientific work so he was interfacing with Python.

They’re interfacing directly with language runtime?

Yes, exactly. Not like shelling out or anything like that… His project is very experimental. When he’s giving demos, he says that “These things are working, but they are able to crash!”

I know a bunch of known ruby guys who went out to make Rust language. Why do you think, the people went there and how are they doing there?

I like Rust, I think it’s very good language. The reason why people are going to Rust…  because they want to have a systems programming language that has more safety than C does. It would be really, really awesome if Ruby will be rewritten with Rust. I’m personally a huge fan of Rust, I really like Rust.

Why could it be useful? Is it safer or quicker, or what?

I think it’s safer. I’m not sure whether or not it has better optimizations than C, but it’s definitely safer. So, that’s the thing I like upon it. When I’m writing C code, I’m pretty sure it won’t go SEGV but not that sure. But when I write rust code, I’m a lot more confident. When I’m writing C code I’m pretty sure it’s not gonna leak memory. With a Rust it’s clear you’re not gonna leak memory. That’s why I personally like Rust rather then C. I started learning Rust because I wanna write Ruby extensions with Rust. There’s a project called ‘Helix’, that lets you do that exactly. Whenever I write the C code it’s mostly like “OK, I have a C library and I have to access it from Ruby, writing a little bit of glue code.” Using Rust is overkill for something like that. In my ideal world, everything, whole system stuff will be re-written with Rust. Rust will be our new C. If you need something done quickly, you will write in Ruby. The operating system will be done in Rust. That will be awesome.

Is Rust mature enough to do the job?

I don’t know. I think so. Mozilla using it, they are pretty happy with it.

With Rust, what a chance do I have to see the registers of my CPU?

Haha, I don’t know! Hopefully low! It’s not fun when you see that information.

Especially, when you run something within the browser.

Yes. You get a crash notification and you like: “OK”. Haha! We got some C++ stuff at work and sometimes when I get those crash I just think: ”Huh…” Haha!

“I should program in the language, not macro assembler!” — That was favorite joke when I switched from C to Ruby…

You’re actually right! Whenever I’m writing in C, that’s what I have to think about. I’m not actually thinking about the thing that I’m doing. With Ruby I don’t have to think about that [low level — P.A.] stuff. I just focus on a logic of a program and I’m done with my job. That’s one of the reason I love the language so much! When I was a Java programmer on Java 1.3, that was before they had generics. Any time you had to write something like a map, like, say, collection, iterator, you’ll have to say “iterator-dot-next”, and then you’ll have to cast the thing, because collection didn’t know what was inside of it. Then you do the operation that you wanted to do. All the stuff around for just writing map. Then I’ve started learning Ruby, doing map in it was very similar to Perl…

… Oh, what a magic! I have an object right here with exact type I need?

Yes, exactly. In Java, I had to write 15 lines of code to do what I can do with one line at Ruby. If I was writing in Ruby, I was done with my job right now! Instead of writing all this crap! That realization made me incredibly depressed at work. I’ve spent so many hours on stuff I didn’t need to be doing!

Existential horror!

Yes, exactly! That was a turning point. I need to find a job programming Ruby. I can’t do this Java stuff for the rest of my life!

Can you tell that working in Ruby makes the programmer’s mind better?

I think if you’re able to spend more time focusing on high level tasks, goals of the program, then it helps you become better at abstract thinking. You get more practice thinking of the system as a whole rather than of tiny parts of the program. Like in C, I have to think about all those tiny little  things rather than the problem that I solve. In essence, you get more practice solving problems or solving high level tasks. I think it might make you better programmer.

I remember my own impression when I started back in some 90s. I tried to master OOP. I tried to do things at C++. I read the books and I’ve understood that “holy trinity of OOP”. Then I found myself at the beginning of macro assembler stuff again. Then I’ve tried to do Java and I did Perl for a living. But finally, I’ve found that only in Ruby understood what OOP means.

That makes sense. If you’ll think about other OO languages like C++, Java, that generation of OO languages. Not everything was an object. You still had ints, for example. You still had primitives and you had to deal with them differently than like with an object. In Ruby, really everything is an object. You have to do OOP. You get more practice, that’s making a lot of sense. I really didn’t think about it before you’ve brought it up.

The language was engineered so smoothly, that it just makes you thinking in this direction. That shapes your mind. Syntax explains what you’re doing.

I did OO stuff in perl. That’s mostly hack for OO-like stuff. Java, of course, does OO stuff. but it has non-object stuff too. Ruby is a first language where everything is really an object.

How would you encourage young programmers and how would you encourage older programmists too?

That’s good question! I think this is for both young and old rubyists. I personally think that Ruby is the only programming language that when I use it’s actually fun. For young programmers, who done other languages - give Ruby a try, because you’ll actually have good time. For older programmers - it’s nice to use other languages, so you can get a perspective how fun Ruby is, because if you use it long enough you might forget. When you go use something else, you will say - wow, I was having good time with Ruby!

I do some short switches to other languages for a weekend. After the weekend, I go back to work, open my Emacs with Ruby and I say: “Wow! What a glory to return back!”

Yes, I think it's good to go to other languages, do some work with them, get some perspective. It makes me better to come back. I feel like I’m coming home.

Meet Aaron and ask a question in person at Ruby Russia 2018!

Questions were asked by Pavel Argentov, developer from Evrone.