.NET Core + RabbitMQ = RawRabbit

Choosing a service bus that meets our demands is a crucial part when developing a distributed system. There are many services to choose from like MSMQ, Azure Service Bus or RabbitMQ and even more frameworks that you can use in your projects as an additional layer of abstraction that makes your coding much easier when it comes to dealing with the specific service bus implementation. In this article, I’d like to present how to use the RabbitMQ in .NET Core with the help of really nice RawRabbit library.


 

At first, you may ask – why RabbitMQ? I’ve had some good experience with this type of service bus in the past. But it’s totally up to you which one to use – if you’re looking for a top-notch reliability, you might consider MSMQ instead (at least this is what the guys behind NServiceBus told me when I was asking them about their opinions on service buses).

Speaking of NServiceBus – it’s probably the most mature framework in the .NET world and I really recommend trying it out (yet it’s not totally for free).
For my case, I did pick up the RawRabbit – before I tried to use Rebus but it didn’t work with .NET Core back then. There’s also a raw implementation of RabbitMQ client that works with .NET core, but it takes much more effort to get everything up and running, which is the reason why the libraries like mentioned RawRabbit are being developed – to help us (programmers) start using such tool like RabbitMQ within our code by using a set of helpful methods and classes.

How to start using RawRabbit? Once you installed RabbitMQ on your machine, just create a new .NET Core application and update package.json with the latest version of this library e.g. “RawRabbit”: “1.10.0” and “RawRabbit.vNext”: “1.10.0” .
There’s also a very detailed documentation, yet I’ll present some of my code extensions that make usage of this library even easier.

This is quite easy to connect to RabbitMQ instance, simply by writing the following code:

There are of course many more configuration properties, and you can even omit this part (passing busConfig into CreateDefault()) and use the default settings which are the same as the presented above.

Now, how do subscribe to some messages and also publish them? It’s also very easy task, but first, let me introduce two basic concepts:

  • Command – can be subscribed & processed only by a single consumer, produces Event.
  • Event – can be subscribed & processed by one or more consumers, may product another Command (saga and that sort of workflows).

Having this two important concepts in mind let’s define the following interfaces:

I’m not going to get into the details of CQ(R)S pattern. For dispatching the commands automatically via dependency injection (Autofac) I’m using the following code:

So, how do you subscribe to the message or publish a new one? Like this:

Ok, but how do we differentiate between the commands and events? The commands should be consumed only by a single consumer while the events may have many more consumers.
Take a look at the following code:

What does it do? Based on the message type (either command or event) it creates a new subscription. You may wonder why GetExchangeName() is not as trivial as .e.g returning just a typeof(T).Name? It turns out, that on the regular server this pattern works just fine, but once you put .NET Core application into Docker container it will get the same Assembly name as the other applications (e.g. “app” depending on the WORKDIR), therefore, this variation is required here to make it work correctly.
What eventually happens is that you will get the unique queues per instance of a different application that subscribes to the same event.
Whether you want to use here command/event handlers (like I did) is totally up to you.

Finally, within your application code, you may do the following:

The events subscribers will get their unique queues (e.g. you may have distinct microservices that react to the same events and they will all have their own queues) and the same will happen for the command subscribers – just make sure, that only a single application subscribes to the particular command type (there might be many instances of such application, though, and it won’t matter because of the same queue name).

41 Comments .NET Core + RabbitMQ = RawRabbit

  1. Lukasz

    Have you tried MassTransit? It’s also very mature framework which allows to switch easily between rabbitmq/memory/azure service bus.

    Reply
  2. Pingback: .NET Core + RabbitMQ = RawRabbit - How to Code .NET

  3. Pingback: Dew Drop - October 31, 2016 (#2356) - Morning Dew

  4. Mariusz

    It’s been a while since you are playing with .NET Core, would you recommend to use it instead off asp.net web api?

    Reply
    1. Piotr Gankiewicz

      With .NET Core you can use the Web API as well. Yes, I’d totally recommend using .NET Core if you have such possibility, and instead of Web API take a look at the NancyFX.

      Reply
  5. Pingback: RabbitMQ in .NET Core = RawRabbit - .NET Development & News

  6. Pingback: Compelling Sunday – 13 Posts on Programming and QA

  7. Pingback: The week in .NET – On .NET on CoreRT & .NET Native – Enums.NET – Ylands – Markdown Monster | .NET Blog

    1. angelina

      I am trying to push to a specific exchange and the default configs don’t seem to have any directions as to which queue i need to retrieve the messages on. I do see a hit from my app in RabitMQ but not messages are queued when I try and do the publish async method on the IClientBus. I am still reading through the rawrabbit docs.

      Reply
      1. Piotr Gankiewicz

        Hi Angelina,

        I don’t have a sample application, but you could check the source code of some apps that I’m working on which are available on GitHub. For example, please take a look at this project:
        https://github.com/noordwind/Coolector.Api/tree/master/Coolector.Api

        You can see a default RawRabbit config within appsettings.json file. In the Framework/Bootstrapper.cs the Bus Client is configured.
        For publishing messages, I’m using the mentioned extensions: https://github.com/noordwind/Coolector.Common/blob/master/Extensions/RawRabbitExtensions.cs – you can also take a look at some other projects e.g. Coolector.Services.Remarks, as they’re all part of the same application.
        You can also just send me an email message if you will not find anything useful in that code 🙂 – or feel free to ask the author of RawRabbit, as he quickly responds to the reported issues.

        Reply
        1. angelina

          This great, I am trying to find a solution for my .NET Core services to dispatch logs (not to files on the server) but to RabbitMQ and from that point I”ll have to find a way to send them to an aggregator like Splunk. I am in a state of finding the solution. Let me know if you know of anything.

          Thanks!

          Reply
          1. Piotr Gankiewicz

            I would create a separate service consuming the commands related to creating the logs (e.g. SaveLog) and use it to publish logs to Splunk or any other service (e.g. using HttpClient).

  8. Pingback: Compelling Sunday – 11 Posts on Programming and QA

  9. Krystian Sitarek

    Hi Piotr,
    Noob question here. I was looking at the source code at your Coolector GitHub. Can you explain me, why you decided to use HTTP API for services (eg. Mailing)? My guess is that it is used to query specific service. What I don’t understand is why you decided not to use Rabbit’s Request/Response approach to query your services.

    Reply
    1. Piotr Gankiewicz

      Hello Krystian,

      Service bus should never be used for the querying purposes. It is designed only for sending messages (requests), but not for asking about some particular resources (querying).
      I was wondering about the same thing and asked guys that develop NServiceBus – I had received the same answer :).

      Reply
      1. Krystian Sitarek

        To be honest, I’m little surprised with that. How to provide multiple instances of specific service (scaling instances for better performance) then?
        With console app I could theoretically just… launch another one on separate machine. The one that is least busy could send response. With HTTP API I’d need to balance load by “hand” or am I missing something?

        Reply
        1. Piotr Gankiewicz

          There are tools like Consul.io that will take care of that – orchestrating your services, load balancing, health-checking and so on. You could just use a regular load balancer of course, but it’s much better to use a tool designed especially for such purpose.

          Reply
          1. Krystian Sitarek

            Thanks, I’ll dig into this. I still have to collect right tools for the job I guess 😉

  10. Les

    Hi Piotr,
    Just wondering if you could help with my understanding.
    If using RawRabbit with the request/response (RPC) functionality and having RawRabbit provide a level of abstraction to achieve this, does this restrict you to having the producers and consumers both in .net core?

    I was wondering if the consumer side could be in another language like Java, but it seems you would need to use the RabbitMQ client library, and am not sure if that requires a deeper understanding of how RawRabbit works in order to respond as part of the RPC call.

    Thanks for any assistance.

    Reply
    1. Piotr Gankiewicz

      Hi Les,
      I’m not sure if I’ll be able to help you with providing the answer to such question related to the RPC. All I can say is that we’re using .NET Core microservices with RabbitMQ (RawRabbit library) and also other technologies like NodeJS (socket.io) connected to the service bus and it works great. For your scenario, you would probably need to use a default .NET RabbitMQ client, but feel free to ask https://twitter.com/pardahlman.

      Reply
  11. Pingback: ASP.NET Core 12 samples | Piotr Gankiewicz

  12. Domas

    How do you handle code sharing? If one service publishes Foo class object and another subscribes to Foo, do you have like a separate “Common” project for those classes or is there another way to handle that?

    Reply
      1. Domas

        Hmm, doesn’t that kinda defeat the purpose of separate services, since they all depend on the same common project? : )

        Reply
        1. Piotr Gankiewicz

          I wouldn’t say so, yet you can do it both ways – like we did and the other way (which is what we did in the first place) is to have messages project per service and the other services that would like to know about messages from different services can use it. It’s merely a library containing simple classes and has to be somehow shared between services :).
          Otherwise, you would have to define over and over the same classes for each service.

          Reply
  13. Pingback: netcore

  14. B

    I see that you’ve recently migrated to asp.net core 2.0. Did you use Docker support feature in VS or did you simply build your own dockerfile and docker-compose and run them using the docker shell?

    Also have you ever tried to run your solution in docker-toolbox? Did it work for you?

    Reply
      1. B

        Do you know if you can open a asp.net core 2.0 solution created in mac or linux on Windows machine? I’ve heard that its not compatible. So then how do your team members using windows OS work on the same code?

        Reply
  15. C

    Do you have any blog or video series for this two topics (scaling up and db replication)

    1. If i want to scale up any micro service (i.e. the identity micro service), how can i manage it using your architecture design?

    2. If i would manage to scale up any micro service, how can i ensure it that the databases have the same data? (replication? or should i have use event sourcing? if yes, than it would be nice to make a video about the implementation of event sourcing with net core)

    Reply
    1. Piotr Gankiewicz

      Hi,
      No video series yet, but with a friend of mine, we’re working on it.

      1. You only scale the number of instances of particular service – it can be done using any orchestration tool, such as Rancher, Kubernetes etc.

      2. Keep in mind, that each microservice may have its own database, however, it doesn’t mean that it’s database per instance – you should have a common database for N instances of the same microservice. Speaking of data replication, it’s more of the database task, so ideally you should use provided tools. Event sourcing is great, but it’s not really about replicating the data :).

      Reply
      1. C

        Hello Piotr,
        thank you for your quick reply! 🙂

        “No video series yet, but with a friend of mine, we’re working on it.”
        – Nice to hear! Because “.NET Core Microservices” video series were really good!

        “Keep in mind, that each microservice may have its own database, however, it doesn’t mean that it’s database per instance – you should have a common database for N instances of the same microservice”:
        – In that case i don’t need to replicate dbs because i will have just single db for each microservice scope (not instance)

        “Event sourcing is great, but it’s not really about replicating the data”:
        – yeah! it should be a separate question 😀

        Thank you very much!

        Reply
  16. Priyal

    Hi, This is a very good post and thank you so much for sharing this. I have followed one of your O’Reilly video series and learnt a lot about these technologies. I tried to build a docker-compose file as an example you have given in the https://github.com/PacktPublishing/.NET-Core-Microservices location. However, I could not connect with the rabbit MQ docker container from the application and gave the following error.

    Exception has been thrown by the target of an invocation. —> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed —> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 127.0.0.1:5672

    I think the problem in here is we are trying to access the rabbitmq in local host while it is running in a different docker container. It would be great if you can help me to figure out how to communicate with the rabbitmq container in this setup.

    Thanks !

    Reply
  17. Pingback: .NET Blog

Leave a Reply to B Cancel reply

Your email address will not be published. Required fields are marked *