How to write a Ruby-related book - the tools

I have recently published the 1.0 release of the "Rails Refactoring: Rails Controllers" book. It took me over 1 year since I started. Over the time, I've experimented with many tools which helped me along the way.

This post is a summary of some of the tools.

The short version

iaWriter for the initial writing
vim/sublime for editing
LeanPub for PDF/epub/mobi generation
Github for storing the source files (markdown)
DPD for selling
MailChimp for email list
Trello for task management

The longer version

After I knew what to write about (the process how I came up with the "idea" is another topic - let me know if you're interested in reading about it) - I've had a bunch of notes, scattered across many tools - Evernote, Hackpad, MindNode, code repositories.

Despite all the temptations, I didn't try to find the best tool for writing. I know myself and that would result in endless experiments without any delivery. Obviously, there was a temptation to write such tools on my own. I'm very happy I didn't go this way.

The minimal requirements was to have a tool that generates a PDF.

Kitabu

The first tool that seemed to be good enough was  kitabu. It was a good tool - think Rails but for books, a whole framework.
They even give you the familiar "$ kitabu new mybook" tool.

Kitabu was a very programmer-friendly approach.

I eventually gave up on Kitabu. The problem was with the commercial PDF tool (PrinceXML) which is used by default. I was OK with paying for good software, but at that time, the license was about $2,000 which was a bit too high.

Scrivener

Then I switched to Scrivener. I loved this tool. I found it through some "real" writers forum. It's very popular among writers. It's a desktop app. It's great for notes collection, structuring, drafting. It's good at WYSIWYG. It has a built-in PDF generation. It supports export/import from markdown.

At some point, more people from my team started to help mi with the book. The Scrivener approach wasn't very team-friendly (or I wasn't able to use it). No easy way of two-way sync with markdown. Other people would have to be forced to use Scrivener as well. Also, it wasn't very version-control-friendly.

Leanpub

Meanwhile, Robert Pankowecki (with a little bit of my help) was working on the Developers Oriented Project Management book. He used Leanpub and this tool was very team-friendly. There was a repo, we could all contribute into. It generates PDF/mobi/epub (on their servers) via a Github hook.

The Github repo:

(as you see there were 300 commits so far and 13 people contributed to the book!)

I was sceptical at first, as they (at least at that time) didn't give emails from the book readers. I found it very limiting for marketing/sales activities. However, LeanPub lets you generate the PDF and take the files with you to sell somewhere else. That sounded perfect to us.
Over the last year, I've seen how Leanpub (the UI) keeps getting better. I may seriously consider switching to them as the selling tool as well at some point. They now also have an iOS app for reading your books.
So far, we're staying with Leanpub for our books (the third one is in progress).

My book is very code-heavy. LeanPub is great for it. They not only support syntax highlighting but they also let us present nicely the diff between two pieces of code.

One minor issue with Leanpub is that you need to wait for the PDF generation (30-60 seconds). I usually work locally, do frequent commits to the local repo. Every now and then I push to the github repo. This triggers a hook, which notifies Leanpub. After a while, I launch a simple bash script  - download_and_preview.sh - which downloads the newest PDF and opens Preview on my Mac (if it's already open then it reloads the file). This is a good-enough flow.

iaWriter

Leanpub lets me write in MarkDown. For some time, I've used vim for that. After a while, I realized that there are two aspects of writing. When I write the initial version of a chapter I need a full focus to let the flow work. iaWriter is a perfect tool for distraction-free writing, with a beautiful full-screen mode. This is where I do most of my initial writing.

When the draft of the chapter is ready, I need to do more editing (it's like refactoring but for normal text). This is when I usually launch either vim or sublime. The vim shortcuts lets me do editing faster.

When I'm done and ready for a new release of the book, I run the 'release' script:

This created a capistrano-like directory and creates a new .zip file.

DPD

I use the DPD system for selling. Unfortunately, they don't have the API to automatically upload a new .zip file and manually change the product configuration. I hate this part, as it's very error-prone. I even attached the wrong file to the wrong product once.

As for payments, DPD only supports PayPal (at least for a Polish company, last time we checked), so this is what I went with. I'm not a big fan of PayPal, but it didn't seem to be a huge problem for all the buyers.

Whenever I have a new release, I can send an email from the DPD panel to all the customers, which automatically generates the appropriate download URL.

As you may expect, DPD gives you all the reporting, charts, data which you may need.

Slack

Over the time, we've created notifications around the whole process. Whenever a new commit is pushed into the repo, a new notification appears on a special Slack channel. Whenever a new sale is made, we have a SalesPanda bot which notifies us about it to the same channel.

Summary

As you see, the whole process is a combination of multiple tools. In some places, they are integrated automatically, while some places are still manually updated. The perfect setup would be to have a Continuous Delivery and we're very close to such thing here. When the delivery is a PDF, the problem is that you would need to email people on every Delivery. This doesn't sound perfect, though. With this book, I think I've had 4 releases so far and this worked well - starting from February 2014 until now.

Feel free to email me at andrzejkrzywda@gmail.com if you need any help with writing a book. It might be overwhelming at first.

The landing page (pure html/css) of the book mentioned in this post is here: http://rails-refactoring.com

I'll refactor this later

Imagine this...

You're finishing your current ticket. The project manager keeps pinging you on IM, whether it's finished. This feature is important to a number of users. The business pressure is high. They want you to finish it ASAP.

"Could you deploy it before the lunch time?" - they beg.

You made some shortcuts in the code. It was just faster to put the code in the controller. You know how service objects are better, but this time it was just faster.

"I'll definitely be refactoring this later" - you think.

Just before the lunch, you push to master, run the deployment script. Everything seems to work. At the lunch, you're thinking how you're going to improve the code - extract the service object, add some more tests to cover the if branch that was added just at the last moment.

Back at the office. Everybody knows that you finished the previous ticket and the PM is happy. From the team perspective - you're now available for the next task.

"Could you help me debugging this problem?" - your colleague asks.

"We need the new reports in the admin panel" - the PM is back to you.

There's no way you can now fix the code shortcuts, you've made in the morning. You switched the context 3 times since then. There's just no time to do it. Other important things are waiting for you.

Did you just increase the technical debt? Is it your fault? You hate this kind of situations. You feel guilty.

Is there a way of avoiding such situations?

Can the code be debt-free?

In that story, you didn't manage to fix the code.
How it could have been done better?

Some people would say that you shouldn't deliver, before the code is great. The task is not finished, before the refactoring is finished.

"It's unprofessional" - they say.

There's some truth to that, but it's based on more assumptions.

"It's OK to have some technical debt, it's a pragmatic approach" - other people say. "Just document the problem somewhere" - they add.

Have you been in projects, where a special place for documenting the technical debt was created?

Some add it to the Pivotal/Redmine/wiki. Others try to keep a file in the repo - something like technical_debt.txt.

It can work, but it requires a huge discipline in the team. I used to be a big fan of this approach. After several approaches to it, now I think it's less realistic.

What are other options?

There's this "pragmatic" option to document the problem in the code comment. From what I see in the projects, I'm reviewing this may be the most popular option. I don't like it. I've looked at dozens of such comments, looked at the git repo history - the comments are almost always out of date. They're there and no one ever fixes the problem.

I really like another option. This practice alone isn't enough, but it's a good middle solution.

Document the hack in the git commit message.

Just write about the situation, why it was created like that, what were the time constraints. You may try to justify it a bit. You may explain how it can be fixed in the future.

Those messages are not part of the code, thus they don't make the code less readable. They are meta information, available to everyone, any time they want.

See a bad code? Just use your IDE to show the git logs for this place.

Bad code has other bad consequences. As developers, we like to point out the places, where code is wrong. Sometimes, a hack is implemented in 5 minutes (pragmatic), but then it takes 2 hours in the future for 2 people to discuss why it even got to the repository.

That's the real meaning of a debt. A 5-minute hack can turn into 2-hour discussion.

I've listed several approaches. I've tried them all. None of them is really good, right?

No hope then?

Well, there's another way of thinking. It's probably less spectacular and it requires better skills, but it can reduce most of the problems above.

First of all, there are hacks and hacks.

Some code should never be written in the first place. If you wrote a >100-lines method from scratch then it's a mess, not a code. Such code shouldn't go the repo. It doesn't matter if it's tested or not. There are some basic rules of coding that apply to our projects. It might be good for just spiking - writing some code to see if the concept can work. Afterwards it gets deleted.

Other cases are more on the technical debt side of things.

It's good to have such skills, that it doesn't make a time difference for you to write the code inside the action or in the service object. I just don't believe the time spent on typing those few characters/words more makes a difference. We're talking about seconds here, not even minutes.

More often, you're in situations, where you need to tweak some existing code to make the feature work. The original code wasn't yours - it doesn't even matter. What matters here is that this place requires a cleanup anyway. Let's say that you didn't have time to improve the existing code. There was no time for the Boy Scout rule. You've extended the existing action by some new 'if' statements. The whole path now works.

Now, this is the problematic situation. It's not your fault that the code was so bad at the beginning. You didn't improve it, though. You even made it worse. How much worse? It depends.

It's good to have a set of rules that all team agrees on.

Rails is so great, because it comes with conventions. As a team, you're not only choosing the web framework but also a convention framework. The problem is that at some point The Rails Way is not enough. This is the scary time when the team may not have enough guidelines. This is the time when it's not so clear anymore, where to put code and how to structure it.

That's why we coined the term The Next Way. It's a label to put some practices into. Service objects, form objects, adapters, repo objects. This is our way to solve the problem of missing conventions. It doesn't solve all of the problems, however it does help in limiting the length of the discussions.

The Next Way is just one possible set of conventions. We go with that but your team may choose anything else. What's important here is to have something that the team can agree on.

Thanks to The Next Way, we know what's the best format of a service object. Yes, there's lots of formats and discussing which one is best takes time.

So, The Next Way describes the goal. Your code is in point A and The Next Way code is Z. There's lots of steps in-between.

Being aware that improving code is a process helps a lot. Some would argue that being in point C is still bad. As long you as you all know that it's better than A - you can all agree that it's an improvement anyway.

Let's take an example here.

Imagine that your new feature requires displaying a new sentence in the view which shows the total value of the orders. The current view and action is very @ivar-heavy. There's no service object here, no repository objects as well. If you know The Next Way, then you know that removing @ivars is one of the steps we make to enable us for further steps.

In this interpretation, adding a new @ivar, like @total_order_value makes the code worse. It's not a big deal, but it's definitely not an improvement.

Now, what's good when you all have the commons set of techniques, like The Next Way is that everyone knows that it needs to be fixed. If you didn't have time to do it (like in the story we started with) in the first place - it doesn't really matter that much.

Why?

Because it's very easy to fix it. Once you know what is the goal, it's a matter of minutes to just turn the @ivar into an explicit rendering of a view following the "Explicitly Render Views with Locals" recipe.

Whoever next comes to this code and is more lucky with the business pressure can just fix it (with the help of the recipe if needed), commit, push, deploy. That's it. No lengthy discussions, no blaming, all clear.

I've seen many teams struggle when they entered the Beyond-The-Rails-Way phase. This phase is hard to avoid. When it happens, the team has problems with the consistency of the code. Sometimes the code looks like a collection of random blog posts. One technique here, another there. There are some good ideas, but there's no clear goal for all the code.

Once we've embraced The Next Way as a common set of techniques it was all easier. It is like The-Rails-Way-On-Steroids. It's more clear, where the code is. It's now clear when the code is in step C or closer to point Z.

What I'd like to encourage you to with this blogpost is to find your set of techniques in your team. This will help you and it will speed up your team work.

You can take what's in our book as a starting point and fork from there. The "Patterns" part of the book is all about The Next Way and its techniques. This is the description of the goal - the Z points. The "Recipes" chapter, on the other hand is all about the steps between the points.

Buy the book here

If you already bought the book, please consider starting the discussion in your team. Ask if it's clear what are the team conventions once The Rails Way is not enough.

Once you have the book, please don't just read it and put it away. The recipes need to be practiced regularly before they become a habit.

Start today - timebox 15 minutes, choose any action and try to apply the Recipe. I encourage you to start with the "Explicitly Render Views with Locals" one. It's a good start to get the feel of such changes.

There's a second episode of the Rails Refactoring Podcast - we're talking about the DNA of the Rails community and about The Rails Way. The whole episode is 35 minutes. We now also support RSS and iTunes, so subscribe to the podcast if you haven't done so yet :)