Articulating Architecture
One of the most important task when building a system is architecting it properly with all the use cases in mind. A loosely coupled architecture will help not only to make quick changes in the flow based on business needs, but also makes the lives of your team members easy to manage it. And by changes, I do not mean only logical changes in the flow of the application but also major ones like moving from Community Edition to Enterprise edition to one of your favourite databases which you rely on so heavily to retain data.
Almost missed it, I am a Solution Architect working with an organisation BDI Plus and my forte is to keep the software development cycle steady. I like to call this the “code to cloud” flow. I expertise in Digital Insurance products and have worked for more than 3 years bringing the idea of serving 3 carriers and a couple dozens of distributors to market within a span of countable days. I bring my expertise to any product development cycle, bending and coping the real life norms and policies to use. My role as a DevSecOps is also to ensure that customers can rely on our system with the most sensitive data to avoid any modern day security breach.
Enough said, I wanted to provide to you the knowledge I have gained over the years and some of the most common mistakes organisations often make when planning to go live into the market.
⏩ Drawings make it easier to understand rather than discussions — When you are designing a system from scratch, it is of utmost importance to jot down the thought process onto a paper. Do not only spend time with the technical team understanding the flow and they want to implement, but also with the business side to pen down how swift they want to keep the process flowing. List down all the flows using which you want to serve your customers. A proper understanding of where to keep the logic (on the application end or the system end), is highly necessary.
⏩ Kubernetes will not only orchestrate, but will allow you to gain some sleep — With the influence of Kubernetes growing so heavily in the field of technology, I strongly recommend everyone to use it! When the idea was first brought to me, it was indeed a little too hard to grasp. But once I started understanding how the a single management tool can take care of your system without any manual intervention, I became one of the followers of the “K8 community”. Gone are the days where you need to write thousands of lines of policies for your architecture to scale up and down. By allowing Kubernetes to take over, scheduling of pods (a core Kubernetes terminology) became a walk in the park. Containerisation has become the new trend. Whenever you are ready to ship your code in containers, you will understand the use of it. The encapsulation of details from a developer to DevOps is so significantly high that smaller teams can manage a huge infrastructure with ease. With multiple packages ready to build you a cluster, it is now a process of running a single command to bring it up for you. Want to think about a disaster recoverable system? Package the entire thing under a single Ansible and Terraform to bring the entire thing up and running with a few minutes.
⏩ Chunk it down, ‘monoliths’ makes lives harder — Imagine that you have a system that has class A logging mechanism and logs every little activity that it does. When a file of 10,000 lines of logs is provided to a developer to debug, he even sweats to figure out the exact point of failure. One of the best way to overcome this, is to break your code in smaller, more reliable and manageable code — giving raise to the concept of Microservices. Since Uber showcased their precise understanding of distributed system management years ago, the world was at awe. Today, this has become a trend. Microservices coupled with Kubernetes adds the cherry on top of the cake. Not only does the system scale as per the incoming traffic, you individual services scale up and down based on the need. People are focusing on a certain section of your application? Not a problem, your microservices handling that flow will scale up. Ensure that you’re restricting every pod with a threshold so that it understands that it requires more resources and let Kubernetes do the magic!
⏩ Keep your secrets a ‘secret’ — When thinking about protecting your valuable username and passwords to your databases and queues and buckets, architect it in keeping them under a single roof. Hashicorp’s Vault or even AWS KMS is the way to go. Also ensure that there is a constant key rotation from time to time to make the auditors look away while thinking about the security. Zero deployment when changing any credentials is the forward thinking way of managing a complex architecture. You think that you have accidentally showed the password over a recorded client meeting? 2 clicks to change it all.
⏩ Logs are a blessing as long as you’re rotating — Logging comes as a blessing in disguise when figuring out faults in the flow, but they can be a headache too. If not coupled with proper log rotation, the pods, the servers and the system will collect so much logs over time that it eventually cause its own demise. Also, have a global parameter set in order to control the level of logs you want to print. A production system will definitely not print PII information to gain a look from your auditors.
⏩ Your services should be dumb — When writing a service ensure that none of the parameters including the databases it is connecting to, or even the services it is communicating too is known to it. Nothing should be hardcoded in any service. Instead, create a global service that ensures it is always providing these details to every service. “Aren’t we creating SPOF in the system?” — true! But you can handle a single point of failure by protecting with affinity. Kubernetes will keep the service running with the heart it is trying to keep itself running. When changing the communication between the services, this single change will ensure the entire flow of the entire application is changes and adapted.
⏩ Rely on REST — Exposing every service publicly is an issue in itself. When you reveal too much, you’re giving more points of intrusions. Reveal only the services that are required, or else group them behind a single service to ensure that it is compromised in times of need and not the entire system. OAuth is slowly fading as a norm, while OAuth2 is becoming the star! You do not have the bandwidth to implement security? A Kong will do the trick for you. Internal communications between services doesn’t need to be public anyhow. KubeDNS and Kubernetes’ internal network is the low latency network you can rely on.
⏩ Do not protect your system, just make the door disappear — Having a WAF, ACL and IAM controlled system is something you have at the base. But is it enough? Imagine the scenario that you get a notification of your system being bombarded with DDOS. The time is not for you to analyse the loss, but to prevent the intruder to gain any more data. Have a kill switch instead, a single pop will make the architecture isolated within a private subnet and remove all internet connectivity for it. Our entire architecture runs on private subnets with not even public IPs attached to it. “How do we manage change of IPs when the DHCP takes over?” — we actually don’t, we allow K8 to do the job for us.
⏩ If you’ve an identity, you need to have a separate login — One of the major mistakes I did when creating an architecture is not creating individual entry credentials for everyone. I am not talking about having one login for departments of your organisation but individual (name wise) login to your systems. And the system logs everything and during the time of calamity, you know from where the issue arose. A RBAC, IAM enabled system is a must when controlling ACLs into your system. And it goes without saying, password rotation is at its base.
⏩ Always think of non-technical accessibility — In a real life scenario, when your business blooms, analysts and data scientist gains your attention because all they rely on is “data”. When allowing a user to look peep into the system or the database, ensure those are controlled. We relied on a VPN creation using Pritunl to allow users to login to our system, Pritunl is not protected by a username/password but also with 2FA that changes every 15 seconds.
⏩ Never forget the support system — When thinking of supporting the system and ensuring top of the class client management, do not forget to provide the support teams with the arsenal they need to ensure it. In addition to Grafana and Prometheus in place, a logging tool like ELK (even a EFK) or even DataDog is a must. This will help the support team to debug the issues coming in and report it to the level of developers to make necessary changes. Along with it, a ticket management tool is important enough to ensure that your clients always smile when they use your product.
Scoring a 98/100 score in security analysis can be easy if you pre-plan it. We here at BDI Plus also have self service portals for our clients to understand the flow of applications at the highest level.
“No system is perfect!” — an idiom rightly used in the industry. Everyday we try to bring in changes that will make our system close to that mark. Do not only focus on the destination, but enjoy the journey of creating a system, the system that you’ll own, the system you’ll carry throughout your career as a badge of honour!
If you want to talk about your system in details, reach out to me in most other social groups under the handle aemdeei. Will really be glad to hear about it more!
If you do find this interesting, drop a few claps. It encourages me to write more!
