Overview

Demystified CQRS

2 Comments

…and why it should be a fundamental principle of application design

Some incredibly useful principles or patterns for software design are ignored because of the misconceptions around them, e.g. that they are not useful, too theoretical or their implementation is too complex.

CQRS (Command Query Responsibility Segregation) seems to be one of them. Though being extremely powerful in LOB and collaborative contexts, architects and developers often think that it isn’t useful and easily applicable to their application. But the biggest obstacle for them seems to be the paradigm shift, not the technology.
The reason for this aversion is that this fundamentally very simple pattern is spoiled by many overcomplicating discussions and special technical solutions that are not the pattern’s initial purpose. This short article tries to clear up the misunderstandings and aims to show why CQRS should in fact be a fundamental pattern of almost every non-trivial application that handles the writing and reading of data.

What is it, basically?

CQRS is really a very simple thing: It separates one thing into two by creating two separate objects for writing and reading where before there was one.
That’s all, really. Trust me.

We don’t even have to talk about Eventual Consistency, Read Layers, Read Stores or Event Sourcing yet. You don’t even need a framework for implementing it, it’s dead simple. It’s the well-known (and hopefully used by you) CQS (Command Query Separation) principle, which says you should separate your queries from the commands. CQRS does the same, but on the responsibility and architecture level.

Why would you want to do that?

Why should you use CQRS? Simply because queries and commands are different responsibilities and have different semantics. Additionally their implementations can rarely be done with the same effectiveness in one object and with one single technology or approach.
Your database probably won’t be able to write the same amount of data it is able to read.

Most larger, complex or collaborative applications have these attributes:

  • Often reading data is much more frequent than writing. In some applications it is of great magnitude, some even have a ratio of 1:10.000 or more. Most web or mobile applications have a very high read ratio. But even in more “traditional” domains like financial systems or stock trading it is of great benefit.
  • Typically you often read much larger amount of data than you need to write. Usually you change one entity but read a lot of them.
  • Users accept a delay and slower response when saving data, but not data being compromised. Moreover users are very impatient when reading data.

Actually it’s almost trivial

First of all: You don’t have to use CQRS as a general architectural base.
It’s not an all-or-nothing or binary decision. Some of your contexts may need read optimization, for others it’s not that important. Though if you keep it that simple it probably won’t hurt and instead lead to cleaner code anyway. And that way you are prepared for the scenario when suddenly millions of users are starting to work with your application. Then you simply can scale out the read side to the maximum and optimize the write side for higher transactional throughput.

The most simple implementation of CQRS

The original concept and therefore the most simple implementation of CQRS is to divide an object that did write state and read data into two objects.

CQRS level 1

Here we go one step further and even cut out the reading part of the Repository into a “Reader”. The semantics of the repository is to return valid Aggregates for state change. But a reader does not need to return Aggregates, its return is e.g. a simple data transfer object (DTO).

Some may claim that there’s an even simpler implementation: by just splitting an Aggregate and adding a DTO, where the repository has the same interface as before but different returns on writing or reading. I wouldn’t call this CQRS but CQS – but that’s a matter of taste.

I hear you asking “Nothing comes for free, so what’s the trade-off?”. In fact, it has one or two, in that now you need almost twice as many classes as before and probably need to adjust your application services or interfaces to use them. If that is a serious problem, then you probably have some far more critical ones.

Separate models (and/or storage mechanisms)

The next evolutionary step is to have two separate models, one “real” domain model for state changes and one optimized for reading. You could also start with splitting your RDMBS into Tables and Views. Then the above Reader would read from the Views. One advantage of this is that you can have specialized views for every read context (similar to the concept of Persistent Views) or even for every UI.

CQRS level 2

It doesn’t matter if you go that sophisticated (and highly recommended) route, you now have two separate models that can be independently optimized for the need of their responsibilities – apart from the ability to deploy them separately.

An advancement to this level is to place a Cache in front of the read model. But this is all sugar on the cake. It depends on your application and requirements if you need that. And it comes with a price: You have to implement a rebuild of it, either by means of Domain Eventing or some other messaging solutions. But most important is: You can do that with moderate work and the gain can be tremendous.

Where’s this infamous Eventual Consistency problem?

Well, we don’t have it, yet. It’s one RDBMS, either with only tables or tables and views. Basically it’s a separation at code level and scaling with the RDBMS’ abilities.

The second level is sufficient for most enterprise applications. It provides a good balance between code and data complexity and performance. You can tweak it at various places, e.g. as already mentioned with a Cache in front of the Read Model or with different query mechanisms or techniques.

Formula 1 CQRS

If we still need higher performance or greater scalability on the read side we can go even further. The next step would be to separate the data e.g. into a transactional safe database and on the read side into a high-performant and scalable storage mechanism. Depending on your needs, there are many good solutions out there, e.g. NoSQL storage engines or Elasticsearch.

CQRS level 3

Now we are, finally, in the Eventual Consistency “hell”. When data is written or modified we need a way to sync our read storage to reflect the latest state. There are many approaches to achieve this and it heavily depends on your domain and on which one is the most suitable.

Often at this point, the term Event Sourcing comes into play. But this is only one possible solution. I will not dive into that topic, but there are many good sources and frameworks out there.

We had a really good experience by reengineering a huge CRM system with CQRS and without Event Sourcing. It was using MYSQL as RDBMS and Elasticsearch as read storage. They were synchronized by carefully using Domain Eventing and updating the read storage in the Event Handlers and by nightly batches. This way we managed to handle about three million very heavy queries (before some queries used 70 JOINs to get their result!) a day, without the need for gigantic or sophisticated infrastructure or hardware. There are examples out there where CQRS is being used even in far bigger domains.

The unseen Eventual Consistency spectre

Eventual Consistency is probably one of the scariest terms in today’s technology discussions, especially when mentioned in front of business leaders.
It is a “problem” of all distributed systems.
Well, and also of all non-distributed ones!

There is no such thing as (absolute) consistency between state and its representation. All the data you change is stale right after the change when being read or even in between two reads. It’s a matter of how narrow you define the latency between data change and reading it to define your domain’s understanding of consistency.

Funnily, one of the most common domain almost anyone of us knows are ATM’s. When using your credit card you enter the scary world of Eventual Consistency! It’s even more scary for the bank than it is for you: No customer would accept taking money out of an ATM on Barbados and having to wait for minutes to find out if it’s possible or not. So if this well-known domain, where money is involved, has worked since its beginning with Eventual Consistency, where does the problem really lie?

It’s your choice: make it eventual, partial or strongly consistent

I’ll suppose you have already sent a package to someone via a postal service. Did the guy at the counter ask you: “Normal or express? Guaranteed delivery? Return of package if the receiver isn’t available?” – So many choices.

As often, the business use case and domain logic decides if something has to be strictly consistent or is eventually enough – but not the technology behind it. If you design your system such, that consistency is depending on the operation, you have gained still a lot more choices than CQRS already offers.

That’s one example of what you can do when you go the CQRS way, by deciding which operation or domain should have what level of consistency. Depending on how critical the writing of the state is or if it’s more important to deliver results with the chance of losing some of them – you decide, not a technology that treats all business cases as if they were the same and need the same technology.

And you can decide that even in the simplest implementation of CQRS, e.g. by telling your RDMBS to use different locking mechanisms for specific operations. In more advanced implementations you could use different storage technologies even on operational levels.

The same goes for the reading or reporting of data. Maybe some part is hypercritical in regards of speed but another part brings up the problem of consuming a lot of storage space and a third one enters Big Data territory. In any case, simply use the approach and technology that solves the specific problem with the highest efficiency.

Conclusion: Why aren’t you using CQRS yet?

I hope to have demystified CQRS a bit, that was the intention of this article. It’s like many really good principles: simple but powerful. If you respect the initial intention behind it you can build highly scalable and adaptable systems with it, and it’s very easy to start small and scale later, when needed.

Also I hope to have shown that CQRS isn’t limited to those high-end applications DDD or event sourcing – but can be applied in any application that reads and writes data and wants to offer choices on how to optimize every single part separately.

As soon as you start to implement, it will unfold its powerful concept and often will give rise to new possibilities. Don’t overcomplicate it from start, but iterate – as recommended for almost all principles.

 

More content about Architecture

Kommentare

  • Mark Paluch

    1. June 2015 von Mark Paluch

    Your post provides a nice overview of separating concerns. The different models/data structures part could be more emphasized. There is much truth between the lines.

    CQRS (or CQS) addresses an important issue: You’re often better off not doing CRUD in where you have the strong semantics of what you read is what you save.
    But this is the point: Semantics. If one does CRUD, it also means CRUD. You’ve got to fetch the whole model before updating. Otherwise, empty fields are written to your data store. In CRUD, it’s hard to find out, why the save happens (besides the user hit save) and determining the real event behind. Who guarantees, that a change was triggered by the use case and not by the state of your data in your data store?

    If you update certain details of a record (partial update), you might want to use an appropriate model for it. A state machine is a nice example for that particular use case: Change workflow state to done. The read model and write model look like different, and the semantic is different. A clear intention can be derived from the action instead of just ‘save’.

    One size fits it all is not a valid approach. I vote strongly for doing the right things right. The right part depends on the purpose and the use cases of an application.

  • Nino Martincevic

    1. June 2015 von Nino Martincevic

    Hi Mark,

    thanks for you comment.
    Basically you are quite right, both with semantics and your example of the state machine.
    In a CRUD-based system one may see CQRS as not very helpful, just because what you’ve mentioned (what you read is what you save). But it’s not the complete truth as you are basically describing the difference between CRUD-based and intention revealing or task-based systems, but that’s another story.

    With that post I wanted to outline some another important issues. First, as most UIs and forms often show far more information that actually is saved. E.g. on a form where I can change my postal address, additional data, e.g. my complete profile could be shown (as read-only). But the actual data that has to be saved can be limited to one or few fields, even in CRUD you have partial updates quite often, why not?

    Secondly, because reading of data in most systems is triggered far more often than writing data, there is a general benefit with CQRS on such systems. You can optimize both sides differently, so far it does not matter if they use the same model or reveal or not intentions at saving.

    Most probably you cannot rely entirely on data being shown as the data that actually should and will be saved, all data you read is (or may be) stale right after reading it. Perhaps it’s not stale in some or even many systems, but that depends on the context and the domain, despite being CRUD-based or not. It’s quite unlikely in heavy collaborative systems, e.g. brokerage. In every case it’s just the SLA that defines the meaning of consistency.
    IMHO, it’s not even dogmatic to state that CQRS is suitable for almost any system, but a quite natural observation: there is always a delay and difference between the state that we see and the one that it should be (=save) or actually is. And so we should design both sides differently in most cases.

    Eventually I can only imagine very few scenarios where CRQS has no benefit, one of it being a context where all data, being read or written, is always of the same model. Or another, where the staleness of data rarely happens or is as theoretical that it doesn’t matter.
    Personally I’ve never worked in such environments…

Comment

Cancel reply

Your email address will not be published. Required fields are marked *