The past few posts we have discussed data migration, data consistency, orchestration and choosing one service at a time. In this post we’ll discuss investing in DevOps and cross team communication. You can read the past posts here:
Invest in DevOps
This item is critical. But in a microservices world, it becomes even more critical. It is crucial for the team to be able to easily check new changes to the particular microservices along with others. So, the CI pipeline might consist of the following steps:
1. Automatic build after each commit
2. Test execution
3. Automatic setup of an environment (latest version of your microservice against a specific set of dependent services)
4. Execution of automated tests
5. Automatic environment shutdown
Invest into infrastructure as a code approaches using, for instance, Terraform to fully automate setup/shutdown of an environment and provisioning tools like Ansible. Make sure that each engineer has a sufficient ecosystem, for instance via Jenkins — enough to deliver new robust functionality without tedious mechanics.
No DevOps, no micro services. Period.
Observability is much simpler for the monolithic app since there are less moving parts. However, in the micro services world, the more components you add to your complex distributed system the more things you need to proactively monitor and react on. Now, things can potentially go wrong anywhere, e.g. in your services, services, queues, storages, caches, etc. Definitely, a network will fail too. So make sure that your system provides enough information to discover the root cause of a possible issue. The first root cause analysis process for the first production issue will hurt a lot, therefore it makes sense to dedicate time beforehand and simulate RCA process based on staging issues for learning purposes. Also, before going to production confirm that your system is not silent at failure, provides alerts when something goes wrong, and verify that your tracing and monitoring data is actionable. You can leverage cloud provider ecosystem, proprietary products like NewRelic, Datadog or SignalFX, and create a custom approach using, for instance, Grafana/InfluxDB or ELK stack.
Communication is hard. Cross-team or cross-location communication is even harder.
There is a famous joke about communication and microservices:
Build microservices around teams that know how to collaborate. If you have concerns about any possible technological issues, please make sure that you don’t have any communication issues in advance. Talk to people, listen to them, lead by example, and invest in effective communication and collaboration as much as you can. It is priceless.
The more I code, lead and consult, the more I believe that projects fail because of people, not technologies. Humans are far more complex.