Just Enough Agile History
Just over thirty years ago, software projects were run very similar to engineering projects using the waterfall method – the concept was fully fleshed out before moving on to the design, followed by completing a design before moving on to development, followed by a fully developed product before moving on to testing. The recurring problem was the planned solution was out of date by the time the software got to market. Technology changes as rapidly as the needs of the business. A large software project might take two years. In that time the concept might need to be slightly adjusted to meet the current market needs of an industry, and consider the number of new software options that have been invented over the past two years. Throughout the 1990s, developers worked on several methods of addressing this issue. Throughout the early 2000s, the future of software development changed forever and the Manifesto for Agile Software Development (a.k.a. the Agile Manifesto) was born.
Just Enough DevOps History
There were frustrations with the processes of development teams often clashing with the processes of operations teams. This was especially frustrating for QA teams that were testing for both development and infrastructure projects. In 2008, an idea was presented called Agile Infrastructure, bridging the gap between Development and Operations. It wasn’t very popular until a presentation at a conference in 2009 about the relationship between Dev and Ops at Flickr that facilitates 10+ deployments per day. In Autumn 2009, a conference called DevOpsDays was held to further brainstorm on the Dev and Ops relationship. It drew some innovators across multiple facets of the Information Technology structure; to further brainstorm on the Dev and Ops relationship. The term DevOps to describe this structure was born.
Agile Software Development
An Agile project starts with a minimum viable product (MVP) and then grows through an iterative process of software development. An example most people can relate to is Teams. Did you notice that the first implementation of Teams was only a little bit helpful and then gradually over time new features were added and it continues to become increasingly more robust? Teams is an Agile project. Microsoft released a MVP. As users suggest features and submit or upvote requirements on Microsoft’s forums. Microsoft prioritizes those requirements into releases, and Teams is continuously developed with new features added (and bugs fixed) with each release. As features become irrelevant due to the evolution of technology, they are removed during the release iterations. Eventually, technology will outgrow the usefulness of Teams, and Teams will reach the end of the product lifecycle, which is called End of Life.
At its most basic implementation, Agile consists of four values and twelve principles.
Twelve Principles
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
DevOps
On a larger scale, DevOps involves an entire organization, but this page will remain focused on DevOps team processes. The image of the DevOps team structure represents what an ideal DevOps team would look like.
Operations: All aspects of Infrastructure and Cyber-Security, including the infrastructure required to run the software the development team is engineering.
Development: All aspects of software engineering, including software that increases efficiencies of the operations teams.
Quality Assurance: All aspects of project planning and Implementation testing for all development and operations projects.
Continuous Delivery
A core value of DevOps is continuous delivery, which necessitates the breakdown of silos. Working on meshing DevOps processes as a whole will reduce the lag time of waiting on another team in all three areas of DevOps.
Before DevOps: (a fictional scenario) A new application is being deployed using technologies that have never been included in a company’s infrastructure. There isn’t really a project plan, so software engineers create a great solution that looks beautiful in the emulator, but when they go to deploy it they realize that none of the protocols being used are set up on the web server. The testers were supposed to be testing this tomorrow, but now we have a lag time of two weeks for operations to have the bandwidth to set up the web server, so the testers have nothing to do this week but will be slammed in two weeks when this hits at the same time as another project coming from another development team. The project is delayed a month because it was developed in a siloed environment.
After DevOps: (a fictional scenario) A project manager from the QA team is overseeing the planning, requirements gathering, and testing activities of a DevOps project. While they are in a planning meeting with Dev, it is mentioned that a specific protocol will be used. They shoot a quick Teams message to Ops to ask if they see any issue with using this protocol. Ops replies that it isn’t really an issue, but it will need to be set up. The PM adds a task into the Ops backlog with a due date two weeks before the estimated dev deployment of that release. The web server is set up with everything it needs to deploy the project before the developers are ready to deploy. The deployment is successful and the testers start their testing on schedule.
Agile DevOps at Anvil
Two important concepts that make Agile and DevOps pair well are the Continuous Improvement aspects of Agile and the Continuous Delivery aspects of DevOps. These are the processes the Anvil DevOps team is improving every week, or rather every sprint.
Sprint: Agile is an iterative process, and work is planned in two-week sprints based on a prioritization process and continuous review of the team’s backlog. A biweekly sprint planning meeting occurs after checking with managers to determine the most urgent priorities.
Scrum: This is a term Agile adopted from the word used for a rugby huddle at the start of a play. We don’t use this Agile term much, but we do have a daily standup to check in on three things – what each person completed yesterday, what each person is working on today, what roadblocks need to be removed. Dev and Ops have separate standup meetings followed by a meeting where the Scrum Masters (another term we don’t use but are roles being filled by the Dev supervisor and Ops manager) meet with the DevOps manager to discuss all crossover items that came up during the Scrum meetings.
Work comes in three different ways – A company initiative, a user enters a Redmine, continuous development/improvement opportunities. Work is assigned to Dev or Ops depending on which area of responsibility it falls under.
QA: Though the DevOps group does not have a QA team, we are developing some QA processes. The Dev team has started to scratch the surface of QA by implementing software testing and creating test processes that will be followed on future projects. The Ops team has set up environments to facilitate software testing. Software moves from a Dev environment, where Devs verify their changes, to a Test environment, where different Devs have prepared test cases to run. When the software has passed the tests and is ready for a production release, it will move to a Production environment. There are many QA discussions happening, and we will continue to grow these processes.
The Future State
Agile is a multi-iterative process that continues until the end of the product lifecycle.
Bottom: This is the project. A project is made up of multiple releases that continue until the product goes end of life. A release is typically done each quarter.
Middle: Each release is made up of iterations that repeat until a release has been completed.
Top: Each iteration contains the two-week sprints where we plan, design, develop, test, and deploy the work until that iteration is completed.
Continuous Planning → Continuous Development → Continuous Improvement (retrospective)
