Thursday, September 17, 2015

Start from the middle

No comments:

The order in which requirements are implemented can have a huge impact on the success of the project. I often recommend a radical approach of starting from the middle to address this issue.

What does it mean?

It can work on different levels. It works on the whole project, but it can also work on a single feature. 

Don't focus on providing the data

Assume all the data involved in the feature already exists. Don't focus on the way the data is provided. Don't implement the importing feature at the beginning. Focus on what is possible to do with the data in the context of the project (or feature).

Let's say you implement a system which shows the project and its tasks (a project management tool). Starting from the middle would mean focusing on: 
  • how the project/task views looks like, maybe hardcode the HTML/CSS
  • what you can do with the tasks
    • prioritise
    • assign to people
      • all the consequences of assigning a task (emails, notifications)
    • complete the task

Consequently, you're NOT focusing on:

  • how the people are imported to the system (do they need to register? are they imported?)
  • how the projects are created
    • it's not a common task to start projects
  • authentication
  • persistence
You try to focus on the actions that are in everyday use of this project (potentially).

What if you really need to use the real data?

This obviously depends on the context of the system. Given the above example, you probably have 2 or 3 initial users of the system. Ask them to provide the data in any way they like. It might be Excel, Google Doc, just an email, whatever. You can quickly insert the data to the system, using the console/database.

Thanks to that you will see what kind of data those people *actually* use. Do they have 100 projects or just 3 of them? Do they have 5 employees or 500? This will guide you how to implement the importing, later on.

Most importantly, you don't waste time on implementing the less crucial parts of the app. Most likely, the importing is not your business advantage.

In the first days, you don't even need the proper persistence - focus on manipulating the existing data, but it doesn't need to be persisted (in the first days).

You don't need the proper authentication, yet. Everybody knows you're able to implement authentication, once it's actually the most important part. Focus on the risky parts, not on the ones, you know how to do.

Technical decisions

In Rails, we often don't know, what the app is about, but we know it's going to be Rails, Devise, RSpec, Postgres.

In Java, it's the same, Spring + JPA/Hibernate + Oracle, before we know anything here.

If it's a web app, consider starting with the frontend app, but don't go into Angulars/Embers or any other heavy framework. Make it a lightweight HTML + some simple views. They can be jQuery-based or React. You will change it later.

Starting from the middle is not only a prototyping tool. Starting from the middle is a business thinking tool. 

What in our app/feature is the most important/risky part?

How can we fake the other parts? How can we defer some of the less important decisions?


BTW, one practical tip - when a software project starts, many people care a lot about many things, especially the technical decisions. You may spend days debating rspec vs minitest or rails-api vs sinatra. Somehow, those are the most important decisions at the beginning. Everything changes after 2-3 months. Suddenly, introducing minitest is not a big deal. It's no longer so important that one part of our app will be React-based and not Angular. It still matters, but the discussions are not so heated. 
Use it to your advantage - suggest deferring decisions "until the next week/month". Try to move the discussion to the core of the requirements, at this stage of work.

Let's say, you're adding an analytical feature to the app. You want to collect a lot of data about the users behaviour and then implement a dashboard which show nice things.

Collecting analytical data is not really risky (maybe the performance part). Focus on what kind of nice things you want to display. What should be highlighted in red? Do we needs some alerts here? You will quickly realise that those things can guide the collecting phase. It will be easier to decide on the ways the collecting needs to be done. Do we need to collect data in batches? How real-time does it need to be?

Detecting actors early

What's most important, by focusing on how we want to use the data, we need to start thinking in terms of actors. Who is going to use the data? Do we have access to those people? The earlier we try to find them, the better. We try to localise the places, which are not under our (developers) control. Fix them first. Then you're on your own implementing the stuff that is fully under your control.

Sometimes, it's just a small gain. You start the feature by visualising the data. You send it to the people involved. They can start discussing it (often takes days). Meanwhile, you can work on the persistence or authentication foundations. When they're back with some ideas of changes, you will need to do them, but at least you were not fully blocked.

If you did it the other way round - spending 3 days on persistence/backend/authentication and then implementing the main screen - you would need to wait longer now. This is using concurrency to your advantage.

Another example - you have a working system. Now, it's time to make it a SaaS. You need to make it a multi-tenant app. This is a big deal. This is not a 3 days work. There's lots of things to consider. Starting from the middle would require thinking in terms of already existing data. Don't focus (yet) how to let new companies/tenants to register. Try to turn the existing system into a tenant system, from the middle. Does it mean, we need to assume we already have subdomains? How do we persist/isolate the tenants data? Try to hardcode other tenants data. This will all guide you how to turn the app into a SaaS in a safer way. 

Starting from the middle is pivot-friendly

If you work in a fast-changing environment there might be situations where a decision to drop the idea appears. You work on a feature X, you spend 1 week on it. Suddenly, the board decides that X is no longer important. We now need Y. If you started from the middle, there's no inconsistent features that you need to disable because they totally don't make sense. Yes, you still need to clean up the code, but at least there's no need to remove the import form, as there was no such thing yet.

I use this technique to blogging. I timebox 30 minutes and I start from the middle. What are the most important messages I want to send? Make them headers. Add 1-2 sentences to explain them. When the time is over, the blogpost is shippable. In fact, it's always shippable. If I have more time, I expand the explanation.

When to stop working in the middle?

Once you have all the unknowns clear - then it's time to go back, to the beginning. We're now totally sure what our app is for. How it solves the users problems. We know the actors. We know the data specifics.

Now it's a good start to implement the missing infrastructure. 

In one of my projects, we implemented the whole app as a Single Page App with no backend at all. We collaborated with the actors, the app was tested and validated. Once all the frontend was settled, we knew the data we needed. Implementing the backend took us only 2 days. It was crystal clear. We started with the client needs first, then we knew what we needed. If we started with the backend, we would be guessing about the needs.

This is a lot like TDD, but on a different level.

Gradual migration of persistence

Let's say you start with a JavaScript frontend. You show "the middle" screens. You implement the most important actions (frontend-only). This is enough for the basic testing. 

Now the users may want to actually "save" the data somehow, so that when they're back the next day, the data is still there.

You can implement it using the local storage. The simplest technique? Just dump all the data graph to the local storage after every change. It's not the proper way, but it solves the *current* problem.

Later on, the users may want to access the data from different devices. You now need to have some backend. You can even follow the same strategy and just dump the data to the backend as a json blob.

The next requirement is usually to be able to collaborate on this data with other users, so they can work on it at the same time. Now it might be time to be more granular with how the data is persisted. You can now extend the API with more endpoints for different actions.

Ultimately, you may need to make the collaboration smoother by pushing the data to the clients using WebSockets.

Code is not a stone

Starting from the middle is a great exercise for continuous refactoring. All the code keeps changing and you're not afraid to do so. It becomes a natural habit. Your work become a sequence of small steps, gradual improvements.

Outsourcing less important parts

Starting from the middle teaches about outsourcing some work to other tools or even other teams. Do we even need our own authentication? Is Twitter login enough? Do we need our own persistence? Do we need to implement the import forms? Maybe there's already a tool which the users can use to "click" all their data and then export it to xml? Maybe there are some standards in this industry?

Optimise for the developer brain

What I often see is a number of features in the "importing" process. Those are features that may be useful, but only at the beginning of the user lifecycle in the project. However, they complicate the UX forever. You have some checkboxes which make sense to be displayed on each line item, but it's only realistically use in the first days. Even if they're already implemented, I recommend considering to drop them. Code is maintenance cost. Even if the code is not changed that often it brings the mental overhead to load the codebase to the developer brain. 

It may sound stupid to optimise the features for the developer brain. However, it's less stupid, when you think that it's the developer who is then able to come up with simpler/faster solutions, thanks to the fact that the whole thing is now simpler.


Starting from the middle is like a framework of business thinking. It constantly challenges you on the best ways of using your time. 

Are you working in the middle now? 

DCI, DDD and the concept of roles

No comments:
All of the architectural approaches in software engineering aim to solve similar problems. One of the techniques that is often used is the concept of roles.

When you do OOP, you may use interfaces (especially with statically typed languages). Interfaces help us thinking in roles - in this context this is the role the object plays.

The DCI architecture is opposing the current class-oriented thinking. They aim to introduce the full OOP which is object based, not so class-based as we're used to now. To simplify things a bit, in DCI you should be able to implement an interface on an object basis, not on the class level.

For example, if you have an ebook object, you might want to temporarily treat it as "a digital product" in the Shipping context (where it's delivered via email to the customer). Not all ebooks are being delivered at the same time, so there's no point in all of them (at the class level) having the shipping abilities. For the time, when it's important to ship the digital product, the ebook has a new role. When it's used in another context (listed in a catalog?) it doesn't have the role - it doesn't have some methods.

That's the DCI conceptual way of using roles.

Now, what is the DDD way?

In DDD, we use the concept of bounded contexts. They represent the boundaries in which a different language is used. In a way, it's similar to DCI Contexts.

In DDD, we would have the Catalog context and the Shipping context as well. Similarly we will have the concept of ebook and digital products. They are not called roles, here, though. Also, it's probably more of an implementation detail, but most DDD implementations would have different objects (different identities) for the ebook (as in Catalog) and for the digital product (as in Shipping).

The ebook and the digital product represent the same thing and very likely they are aggregate roots in their bounded contexts. Keeping them as separate objects (often, with their own separate persistence) requires some synchronisation mechanism. If the ebook title changes, it may need to be changed in all contexts. In DDD, domain events are one way of ensuring the (often eventual) consistency.

As you see, different kinds of architectures need to solve similar problems. Here, both DDD and DCI use the role-oriented thinking. The implementations are different, though.

Wednesday, February 18, 2015

Starting my journey with Volt

Let me start with a disclaimer that I'm new to the Volt framework. I probably get some things wrong. Don't rely on my opinions here :) The reason I'm blogging about it is to document my learning process. Also, by blogging I hope that dear readers will catch my mistakes and correct them in the comments. Thanks to that I'll learn more and can share a better knowledge in the future blog posts :)

I've played with Volt today. For those of you who don't know - Volt is a Ruby web framework. It's a bit unusual, as it's an isomorphic framework. The code is shared between the server and the client. It works in the browser thanks to the Opal - a ruby to javascript compiler (source to source). In a way, it's similar to Meteor.


I'm quite excited about Opal and the way it can influence the Ruby/Rails community. One year ago, I've recorded my suggestion/prediction that Opal can become the default JavaScript implementation in Rails apps. Here's the short (1 minutes) video of me science-fictioning about it, while walking in the forest near my home :)

I'm not following the Opal progress very closely, but from seeing what's possible in Volt, I feel quite confident about my last-year prediction.

I'll focus on Opal in another blog post. For now, let me come back to Volt.

The wow effect

I came to Rails very early, in 2004. It was due to the "wow effect". Being experienced with PHP, Java, ASP.NET it was amazing to see a result produced in such a short amount of time with so short codebase!

Over time, I learnt a lot about JavaScript frontends. Here is a 12-blog-posts summary of my lessons. I know what it takes to develop a nice Single Page Application (or however you prefer to call it).

This is the Volt wow effect. I'm old enough not be excited too easily with new shiny toys, but Volt does impress me. In almost no time, you've got a rich frontend, which works as you'd expect from a good JavaScript application - fast, interactive. You get the backend basically for free. It all autosyncs with the backend without any effort from your side. What's more, a new Volt app comes with authentication built-in for free.

Part of the wow effect is the fact that you don't need to run any database migrations to add new fields. It all relies on Mongo under the hood, so it's schema-less.

One of my roles is to be a Ruby/Rails teacher  / coach and I know how important it is to have a great "wow effect" for the students.

The developer happiness

Another thing that brought me to Rails was its developer oriented approach. The small things like console, like migrations (at that time it was huge), like seeing everything immediately on the screen without restarting the server. It was all there. It was all prepared to be very productive for the developer.

Volt is the same. There's lots of details that work very well. It's hard to mention them all. For me, it was great to see the browser with live reload out of the box. I change some Ruby code and it immediately pushes the code to the browser and the page is reloaded. I know there are more tools like that, but seeing it work with Ruby in the tool chain is just great.

The documentation

I followed the tutorial (try it, you'll get a better picture of it in 15 minutes, than from reading this post ;) ) and browsed through the documentation. All very precise, very simple. I didn't find any inconsistency with the actual behaviour.

The codebase

I only scanned some parts of the code and it seems like very well written code. I was curious how the "reactive DOM" thing works here. It was easy to find the right piece of code and understand how the "watching" works. When I browse the Rails codebase, I often feel lost as it depends on a lot of metaprogramming. I didn't feel the same here (yet).

BTW, yesterday I tried to run Opal with ReactJS but couldn't make it. There's probably no reason, why it wouldn't work, though, given more time. If you know anyone who did that successfully, I'd love to see the result :)


I'm not experienced enough to see all the details, but I love the focus on components, from the beginning. Often, in Rails apps we think that some pieces may be extracted as an engine/component/module/gem *one day* and it rarely happens. Here, we're encouraged to do it from the beginning. The Volt components can be packaged as gems, which is quite interesting.

The architecture of a Volt app

As much as I'm impressed by Volt being a great framework for scaffolding/prototyping (maybe even better than Rails one day?), I also have some doubts.
A lot of my work in the last years was about decoupling legacy Rails applications. I wrote a book about Refactoring Rails apps and I keep collecting techniques which make it easier. Coupling is bad.

Is sharing the code between server and client a coupling? The answer to this question will be very important for me.

In a way, good decoupling may result in great reusability. There are applications, where the server role is simply to keep the data and to ensure basic authentication/authorisation rules. There are however also apps, where the server logic is much different from the client applications.

Is reusability the goal in itself? I don't think so. It's more of a nice side effect of good separation of concerns.

What's the Volt model of code reuse? At the moment, I'm not really prepared to answer this question easily. It seems to be, that at any moment, you can easily switch off the coupling, when required. That's a good sign. It's all Ruby under the hood, so we can do what we want.


In case of frameworks, it's not always their authors vision, which drives how the apps are created. There's many good things in the Rails framework. Rails itself (nor the Rails core team) doesn't force you to write monolithic application - yet, that's what many people do. The main Volt author (Ryan Stout) seems to be very modularity-oriented. From what I see in the Gitter activities, this vision is shared by the early adopters. That's a good sign.

The future?

Arkency (my company) became experts in dealing with legacy Rails apps. We know how to decouple the app, step by step, in a safe manner. Are we going to become "legacy Volt apps experts" in 5 years from now? Time will tell... I have positive feelings about it. It's great to see new players in the Ruby community. There's no way it will become more popular than Rails, however competition is good. Volt is so different from Rails, that the Merb-case is not vere likely here ;)

I'm not recommending starting serious apps with Volt (yet). However, the framework is mature enough to be tried. In only 15 minutes you can build the Todo app from the tutorial. At least it will show you how the process of working can look, in the future and how different it is from your everyday flow.

As for me, I'm off to start another side-project with Volt now. Follow me on Twitter to see how it goes :)

Wednesday, February 11, 2015

From Rails to JavaScript frontends

No comments:
I've collected a number of my blog posts, which together may help you in the transition from Rails to JavaScript (or CoffeeScript) frontends. Many of the posts are about the mental transition you need to make along the way, while other posts are more technical.

I think it's best to read them in the order I put it here, as they present the road I took over the last 5 years.

1. Rails is not MVC

2. Frontend is a separate application

3. From backend to frontend, a mental transition

4. Is CoffeeScript better than Ruby?

5. JavaScript frontends are like desktop applications

6. Frontend first

7. Non-framework approach to JavaScript applications

8. Angular or Ember? Congratulations, you've already won

9. Single Page Applications - fight

10. Turn a Rails controller into a Single Page Application

11. How can Rails react to the rise of JavaScript frameworks?

12. Which JavaScript framework should I choose? (video)

If you enjoyed reading the blog posts, you may consider following me on Twitter.

Which JavaScript framework should I use? (video)

No comments:
Over the last 4-5 years, I keep being asked this question - Which JavaScript framework should I use?

2.5 years ago, I had a talk at the RuPy conference, where I'm trying to answer this question. I realised, I've never posted the video to my blog. 2.5 years is like a century in the IT world. Everything has changed, right? I believe that my answer is still correct and all what I said is still true.

Enjoy the 30 minutes of my lecture :)

Tuesday, February 3, 2015

Splitting a Rails controller


Splitting a controller into separate files may be a good idea in certain cases.

Let me start with the cases, when it's NOT a good idea. In case of a simple CRUD controller, this may be an overkill. The same applies to simple, scaffold-like controllers.

Some controllers are more resource-oriented - those may benefit from keeping the actions together in one file. People often use filters for reusing certain rules of accessing the resources here.

Other controllers may be more action-oriented (as opposed to resource-oriented). It's typical for the your-main-model-controllers. In those controllers, you will see more than the standard actions. They will contain interesting logic for 'update', 'create', 'destroy'. Those actions may have different logic. Often, they don't share that much.

The action-oriented controllers may benefit from splitting them into several files. I call them SingleActionControllers.

Please, note that the action-oriented controllers can contain a more complicated set of rules in the controller filters. Often, they create a full algebra of filters, where you need to find your way through the :except clauses and dependencies between filters. Refactoring those places requires a really precise way of working.

Apart from the filters problem, extracting a SingleActionController is easy and consists of the following steps:

1. A new route declaration above the previous (first wins)
post 'products' => 'create_product#create' 
resources :products, except: [:create]

2. Create an empty controller CreateProductController which inherits from the previous
3. Copy the action content to the new controller
4. Remove the action from the previous controller
5. Copy the filters/methods that are used by the action to the new controller
6. Make the new controller inherit from the ApplicationController
7. Change routes to add 'except: [:foo_action]'

You can also see a simple example of this refactoring here:

TDDing a unit, not a class

1 comment:
Some time ago, I blogged about "Unit tests vs Class tests", where I described how it differs when you think of a unit more as a collection of classes instead of a single class.

How can we TDD such a unit?

This reddit thread triggered me to explain it a bit more. The title of the thread was interesting: "How does one TDD a user and session model?"

Such question is a good example, how the class tests seem to be the default in the "unit testing" thinking. Instead of asking how to TDD the Authentication unit, we're asked how to TDD some single classes - which often depend on each other.

Here is my response:

One way to do it is to think one level higher. Don't focus on the User and Session classes, but focus on the whole feature/module that you're building. In this case, that's probably Authentication, right?

Think of Authentication as a class, which is a facade. It's very likely, that you will come up with User and Session classes, sooner or later. They will be more like implementation details.

You probably need to implement functions like:
  • register
  • logging in
  • logging out
  • change_password
  • change_email
  • change_login
  • remember_me
  • delete_user

Together they can make nice stories. You can start with a simple test like:
def test_wrong_credentials_dont_authenticate 
  authentication = 
  authentication.register_user("user", "password") 
  assert_raises(WrongCredentials) { authentication.authenticate("user", "wrong password") } 

(if you don't like exceptions, you can turn it into returning true/false instead)

One by one, you could build the whole feature this way, while being test-driven all the time.

In a Rails app, you will then use this Authentication object from the Rails controllers. This is also a good way of improving Rails controllers.

This approach is sometimes called top-down. You start with the expectation from the higher module (Authentication) and this drives you to implement the lower-level classes like User and Session.

Another approach is called bottom-up, where you start with the User and Session classes from the beginning and don't worry with the wrapping module, yet.

The different approaches may result in different tests. With the top-down approach you end up with module tests, while with bottom-up most likely you end up with a test per class.