Many modern-day applications need to be built at an enterprise scale, sometimes even at an internet scale. Each application needs to meet scalability, availability, security, reliability, and resiliency demands.
In this article, I’m going to talk about some design patterns that can help you achieve the above-mentioned abilities with ease. I’ll be talking about each pattern, how to use that pattern in a cloud-native environment, and when to use it and when not.
Some of these patterns aren’t so new but are very useful in the current internet-scale cloud world.
Here’s the list of patterns I’ll be discussing in this…
Using containers for application development and deployment is very common these days.
While containers certainly bring a lot of value, they also bring some challenges like —
The most common solution to this problem is — using smaller distros!
Using lightweight distros like Alpine is a very common technique amongst the developers to avoid making the container image bulky. …
Data Discovery has become one of the most important capabilities that Enterprise Data Platforms must provide. Over the years many big companies like Airbnb, LinkedIn, Uber, Netflix, Lyft, etc. have talked about how they solved the data discovery problem by building an in-house metadata search engine.
With the rise in Analytics, Machine Learning & Data Science projects, data discovery has got the top priority in many data teams.
Even modern-day enterprise data architectures like Data Mesh talks about the importance of Data Catalogs.
Microservices is one most common architectural patterns to build highly scalable, distributed applications. This pattern describes delivering the system via small, independently releasable services. A service provides functionality over the network (typically via APIs).
The Google Trend below clearly shows rising interest in the topic over the last 5-6 years:
Many organizations have started adopting Site Reliability Engineering(SRE) practices to run their operations instead of traditional. The latest LinkedIn Job search show 190,000+ job openings for Site Reliability Engineers worldwide.
The article — Take the Green pill and I’ll show you how deep the rabbit hole goes gives in-depth insights into climate change. I really like the introduction piece where it says —
Weather is what you see outside every day, it’s changing pretty often.
Climate is the average change in weather over a long period, it changes slowly.
What we are seeing is the climate changing over time, it’s currently 1 degree hotter on average than pre-industrial times and that has a lot of people worried.
✅ In this article Stripe talks about how are they using a very simplistic approach called canonical logs to achieve flexible and lightweight observability.
✅ SoundCloud in their latest blog talks about guidelines to how to successfully hand over.
✅ This week I published an article talking about some techniques you can use which can be handy while you work on versioning your APIs.
✅ The latest issue of ACM Queue published a pretty interesting article which talks about a better framework to gauge developer productivity — SPACE
In my previous article on Designing REST APIs, I talked about using a simple versioning scheme for APIs. Over a period of time, I’ve realized that no matter how simple you keep your versioning scheme, it’s always a pain to move consumers from one version to another.
If you look around, the internet is flooded with opinions about API versioning schemes. When you use versioning, you need to think about maintaining backward compatibility for a certain time, until all consumers are moved to the new version. It takes a lot of energy, communications, and planning to make these things happen.
Open source has stormed the software world for ages. Plenty of companies not only use open-source projects but also contribute back to the communities. And the open technology world is growing every day. We can not imagine the software world without some projects like Linux, Postgres SQL, Hadoop, Kubernetes, Cassandra, Python, Tensorflow, and more — the list is unending.
In an enterprise world, it is essential to understand how open-source projects are being used. Some of these reasons are:
Envoy is an open-source proxy created at Lyft. As the world is moving towards a distributed microservice architecture, practitioners are observing difficulties in running operations in two main areas:
Envoy tries to solve these problems by making networks more transparent to applications. It also provides a way to better collect stats so that it’s easier to debug and trace the issues.
The most common way the Envoy proxy is deployed today is alongside applications as a sidecar. Just in case you are not aware of a sidecar pattern, then you can read about it in my previous article.