History of DevOps

First time pitched in Toronto Conference in 2008 by Patrick Debois and Andrew Shafer. He proposed that there is a better approach can be adopted to resolve the conflicts we have with dev and operation time so far. This again took a boom when 2 flickr employee delivered a seminar that how they are able to place 10+ deployment in a day. They came up with a proven model which can resolve the conflicts of Dev and operation having component build, test and deploy this should be an integrated development and operations process.

Definition of DevOps

DevOps is a practice culture having a union of people, process, and tools which enables faster delivery. This culture helps to automate the process and narrow down the gaps between development and IT. DevOps is a kind of abstract impression that focuses on key principles like Speed, Rapid Delivery, Scalability, Security, Collaboration & Monitoring etc.

A definition mentioned in Gartner

“DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture) and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology— especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.”

How DevOps Works with a toolchain?

Benefits of DevOps

Adding some benefits which we can see after adopting DevOps :

a) Horizontal and vertical growth: When I’m using “Horizontal and Vertical Growth” I’m keeping customer satisfaction on X, Business on Y2 and time on Y axis. Now the question is how it helps to populate growth in 2 axes, and my answer will be the quick turnaround time for minor and major issues. Once we adopt DevOps we scale and build in such fashion that in less time the graph shows a rapid jump.

b) Scalability & Quality: If a business starts reaching to more user we start looking to increase infrastructure and bandwidth. But in other hands, it starts popping up a question whether we are scaling our infra in right way and also if a lots people are pushing changes (Your code commits/builds) are having the same or great quality we have earlier. Both of the questions are somehow now reserved by the DevOps team. If your business pitch that we might be going to hit 2000+ client and they will be having billion of traffic and we are ready to handle, DevOps take this responsibility and says yes can scale infra at any point of time. And if the same time internal release team says I want to deliver 10 feature in next 10 days independently, DevOps says quality can be maintained.

c) Agility & Velocity: They key param of adopting DevOps is to improve the velocity of product development. DevOps enables Agility and if both are in sync we can easily observe the velocity. The expectation of end users are always high and at the same time, the deliverable time span is short. To achieve this we have to ensure that we are able to our rollout new features to customers at much higher frequencies otherwise your competitors may win the market

d) Improving ROI of Data: Having DevOps in an organization ensures that we can design a decent ROI from data at an early stage more quickly. If we will do a raw survey now Software industry is playing with data and have control over there a team should have an end to end control on data. And if we define DevOps it will help a team to crunch data in various ways by automating small jobs. Automation we can segregate and justify data and this helps to populate either in Dashboard or can present offline to a customer.

How DevOps enables CI/CD (Continuous Integration/Continuous Delivery)

When we say “continuous” it doesn’t translate that “always running” but of course “always ready to run”.

Continuous integration is nothing but the development philosophy and practices that drive teams to check-in code to version control system as often as possible. So keeping your build clean and QA ready developer's changes need to be validated by running automated tests against the build that could be a Junit, iTest. The goal of CI is to place a consistent and automated way to build and test applications which results in a better collaboration between teams, and eventually a better-quality product.

Continuous Delivery is an adjunct of CI which enables a facility to make sure that we can release new changes to your customers quickly in a sustainable way. Typical CD involves below steps:

a) Pull code from version control system like bitbucket and execute build.

b) Execute any required infrastructure steps command line/script to stand up or tear down cloud infrastructure.

c) Move to build to right compute environment

d) Able to handle all the configuration generation process.

d) Pushing application components to their appropriate services, such as web servers, API services, and database services.

e) Executing any steps required to restarts services or call service endpoints that are needed for new code pushes.

f) Executing continuous tests and rollback environments if tests fail.

g) Providing log data and alerts on the state of the delivery.

Below there is a table which will help us better to understand what we need to put an effort and what we will gain if we enable CI/CD in place:

Devops Tools

Jenkins: Jenkins is an open source automation server which is used to automate the software build, and deliver or deploy the build. It can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed. In short, Jenkins enables continuous integration which helps to accelerate the development. There are ample of plugins available which enables integration for Various DevOps stages. For example Git, Maven 2 project, Amazon EC2, HTML publisher etc. Alert:

Pingdom: Pingdom is a platform which enables monitoring to check the availability, performance, transaction monitoring (website hyperlink) and incident collection of your websites, servers or web applications. Beauty is if you are using collaboration tool like slack or flock you can just integrate by using the webhook (Pretty much simple no code required )you can easily get notified at any time. Pingdom also provides API (pingdom.com/resources/api/2.1) so you can have your customized dashboard (Recently started) and the documentation is having enough details and self-explanatory.

Nagios: It’s an Open Source Monitoring Tool to monitor the computer network. We can monitor server, applications, incident manager etc and you can certainly configure email, SMS, Slack notifications and phone calls even. Nagios is licensed under GNU GPLv2. Listing some major components which can be monitored with Nagios:

=> Once we install Nagios we get a dashboard to monitor network services like SMTP, HTTP, SNMP, FTP, SSH, POP, etc and can view current network status, problem history, log files, notifications that have been triggered by the system, etc.

=> We can monitor Servers resources like disk drives, memory, processor, server load usage, system logs, etc.

Stackdriver: Stackdriver is again a Monitoring tool to get the visibility of performance, uptime, and overall health for cloud-powered applications. Stackdriver monitoring collects events and metadata from Google Cloud Platform, Amazon Web Services (AWS). Stackdriver consumes data and generates insights via dashboards, charts, and alerts. And for alerting we can integrate to collaboration tools like Slack, PagerDuty, HipChat, Campfire, and more.

Grafana: It is an open source visualization tool and can be used on top of different data stores like InfluxDB,Elasticsearch and Logz.io.We can create comprehensive charts with smart axis formats (such as lines and points) as a result of Grafana’s fast, client-side rendering — even over long ranges of time — that uses Flot as a default option. We can get the 3 different level of access, watcher, Editor and Admin, even we can enable G-Auth for having good access control. A detail information guide can be found here (docs.grafana.org/installation).

ELK: Elasticsearch: It's an open source realtime distributed, RESTful search and analytics engine. It collects unstructured data and stores in a cultivated format which is optimized and available for language based search. The beauty of Elastic is scalability, speed, document-oriented, schema-free. It scales horizontally to handle millions of events per second, while automatically managing how indices and queries are distributed across the cluster for smooth operations.

Logstash: Logstash is used to gather logging messages, convert them into JSON documents and store them in an ElasticSearch cluster.

reOptimize.io: Once we run ample of servers we usually end up with burning good amount not intentionally but of course because of not have a clear visualization. At reoptimize helps thereby providing a detailed insight about the cloud expense the integration can be done with 3-4 simple steps but before that you might need to look into the request which can be accessed here. Just a heads up that they only need a read access for all these and permission docs can be found here .

DevOps vs Agile

What are a DevOps engineer & roles and responsibilities of it?

Don’t want to be specific but I should say when they build or code they keep the business logic in mind. DevOps engineer should also seek for why this but why not this. They add value when they communicate infra which help business to scale. DevOps team should have control on data matrix collection which could be business driven and the automation pipeline would be Dev driven. As current demand of the cloud is on the pick they should also have a control over the scalability.

DevOps engineer are expert collaborators and also they are often known as a mentor for a software developer as they help them to build a scalable architecture. At the same time, they join IT and security team to ensure that there is enough work has been done to push quality code and to avoid data breach.

DevOps engineer salary

As per the current market, trend salary doesn’t seem to be a bar for right candidate. The basic expectation from companies are, the candidate should be a good learner and at the same time, he or she should be able to implement that expertise to resolve the real-time problem. And for cracking good no’s you must have the ability to build an automated pipeline of DevOps

Above table is a kind of current trending no’s which we recently heard from candidates, companies and also the analytical firms who all participate in this (Hiring Firms). Nowadays there is we can see there is a demand for DevOps engineer in almost every organization. And also I must say Startup always beat MNC in salary though they also beat in work, which I believe it is always a good sign to see a growth in your skill and no’s.