Protocol Buffers + gRPC + ScalaPB
My first week as a Scala developer was quite eventful. Following advice of my team-lead I decided to familiarize myself with some new concepts and technologies built around them. Here are some bits and pieces I've managed to pick about stuff I had never had to work with before. Take note that rather than comprehensive introductory article, this is more of a personal memo to help myself digest all of this by the very process of writing. I also assume this post can be useful to my future teammates.
Protocol Buffers
A protocol and set of instruments coming with it, allowing one to define DTOs in a way that is independent of language and platform. To do that, Google came up with it's own programming language called Proto3. Basically you plug-in required dependencies for Protocol Buffers, define .proto files with your models - called "messages" - and run protoc compiler to generate DTOs in a chosen programming language. This can be done with the help of a command-line, but in practice code generation and packaging is usually performed as part of your automated build process (Maven, Gradle, SBT).
Now one can ask "Why would I need something like that in the first place?" which is the very question I had myself. The whole point of Protocol Buffers is serialization / deserialization of structured data. Source code for generating or parsing stream of bytes, that represents our entities, is generated by compiler along with DTOs themselves. Messages are serialized into a binary wire format, which is compact, forward & backward compatible. This technology can be very useful for systems with modules (subsystems) written in different programming languages, but essentially working with the same or compatible models.
Google Remote Procedure Call
RPC is something I've never had to work with before. After reading several articles on the concept, I had enough knowledge on the tech to ask myself "Why do we need RPC if we have REST?". Further research and several useful articles later I had my answer. Basically if the goal of your API is to enable communication between two distributed components that you own and control, and processing efficiency is a major concern, then RPC is probably a better choice. However, if your primary objective is to make your software more malleable by breaking it down into components that are better isolated from each others' assumptions, or if your purpose is to open up your systems for future integration by other teams, then REST really looks like a better option.
I suspect that sometimes decision of RPC vs REST is anything but simple one to make. Both styles obviously have their own pros and cons. If we actively work with resources through CRUD operations and using HTTP methods feels natural, API should be public and as independent from other systems as possible, then REST is supposed to be a better approach. At the same time, we often don't need our subsystems to be that independent, especially when we own them, and using RPC services could often feel much more natural than trying to fit our needs into resource-based model of REST. Speed can be a major concern as well, though it's rarely the case in my experience, however limited it is at the moment.
Of course one cannot forget about the fact there are different RPC implementations. The one we use at our team is gRPC. It goes hand in hand with Protocol Buffers and allows one to use Proto3 to easily define RPC services: their name, request & response message types, etc. Like with DTOs, most of the stuff you'll need (e.g. traits for your services) is generated for you in a chosen programming language, then packaged as an artifact to use as a dependency. Of course, after that, one still has to define implementation for those services to do the actual work, but everything transport-related is handled by Protocol Buffers.
ScalaPB
It is also perhaps important to mention that while protoc supports many major programming languages, Scala support is provided by ScalaPB - a Protocol Buffer compiler for Scala. It is built on top of Google’s Protocol Buffer compiler to ensure perfect compatibility with the language specification and makes it possible for our team to write implementations for RPC services in Scala without having to worry about the fact that other parts of the system are written in other programming languages. Everyone's happy.
Additional Reading
Here are some links that can be useful to get a better understanding of all the concepts and technologies mentioned in this post. Those I've read myself, anyway :)