By Marius Voila, Cloud Architect
There’s a good chance you’ve already come across the term Kubernetes. If not, you will soon see it appearing in blog posts, conferences and water-cooler conversations everywhere. But despite the increase in attention Kubernetes has received in recent times, adoption of the technology is still very low, especially among larger enterprises.
If the term is new to you: Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services. It was developed by Google – and I pride myself on having been a part of the Borg team that developed it – and later made public.
We are still in the early adoption phase, but the benefits of Kubernetes and containers are here to stay and could become a new industry hosting standard. For instance, it is predicted that the market for application container technologies will reach $4.3 billion in 2022, which is more than double the prediction for 2019.
Why we moved to Kubernetes
Kubernetes was open-sourced in 2014, the same year in which EclecticIQ was founded. Less than five years have passed since then. During this time, we have grown at a rapid rate, which made us realize quickly that we needed to look into new hosting solutions, before we became overrun by our own success.
The more success, the more data, the more cloud storage costs, the more maintenance and so on and so forth.
I am a big Kubernetes fan and have successfully implemented the technology for previous employers. That might have helped a bit to push the needle toward Kubernetes as the way to go for EclecticIQ, but our hosting team would have opted for embracing this technology anyway. Its capabilities were a great match for the requirements we had mapped out.
We had looked for a solution that would help us:
- Proof our infrastructure against the consequences of a growing client roster (not just short-term, but long-term growth)
- Deal with the enormous and constant increase of threat data that we ingest into our platform.
The result: a self-healing infrastructure that allows for (almost completely) hands-off maintenance and that further cut hosting costs by 70 percent!
Doesn’t sound too bad, does it? Let’s unpack it a little and look at how we achieved this:
Implementation phase
Simplified, we took the following steps for Kubernetes implementation:
- Containerize the platform (TIP) by taking tar packages and moving them into containers
- Create a helm chart to instruct Kubernetes how to deploy the platform and its requirements
- Design architecture and establish the matrix we wanted to observe
- Deployment
- Continuous improvement by monitoring the stats
Helm Chart
The benefits
- Self-healing: We don’t need to do anything. When we get an alert about a sick container, Kubernetes is already working on fixing it. The system auto-heals. It can spin up new containers and replace containers if they are corrupted. Maintenance could be reduced by up to 45 percent.
- Reduction in delivery time: Customer delivery was about a week. Now it is two hours.
- Eliminate naïve scaling: We run on spot instances: they require that you have to return instances within a matter of minutes. This is labor intensive and puts the hosting team on the spot. With our new infrastructure, we don’t have to deal with this problem anymore. That means we relieve pressure on our hosting team and reduce the risk of possible downtime.
- Reduction of server costs: As mentioned above, we were able to shave off 70 percent of the previous hosting bill by leveraging AWS spot instance.
The reason I wrote this post is not to boast about how we changed our hosting infrastructure with the help of a hyped technology. We wanted to share this insight because Kubernetes seems like it was made for the security industry in general, and for the cyber threat intelligence (CTI) industry in particular.
This is still a relatively new aspect of the cybersecurity industry for three reasons:
- The amount of threat data is growing at a rapid rate and it needs to be stored so threat analysts can draw conclusions by comparing new data with old.
- Security vendors need to avoid downtime at all costs.
- The CTI market is still in a rapid growth phase and, as mentioned before, Kubernetes helps you to cater for success.
We couldn’t be happier about having made the switch to our new architecture, both for the advantages it brings to our clients as well as to our own organization.
Now that companies such as VMWare are shifting toward the technology, it is only a matter of time before we see large-scale adoption of Kubernetes. We take pride that we have been at the forefront and have already brought a multitude of benefits from this to our customers.
