{"id":6849,"date":"2020-10-20T12:43:03","date_gmt":"2020-10-20T10:43:03","guid":{"rendered":"https:\/\/www.synthesis.co.za\/?p=6849"},"modified":"2020-10-20T12:43:03","modified_gmt":"2020-10-20T10:43:03","slug":"achieving-operational-excellence-with-terraform","status":"publish","type":"post","link":"https:\/\/www.synthesis.co.za\/achieving-operational-excellence-with-terraform\/","title":{"rendered":"Achieving Operational Excellence with Terraform"},"content":{"rendered":"

20 October 2020<\/strong><\/p>\n

By: Jared Naude, Cloud Architect – Synthesis Software Technologies\u00a0<\/em><\/p>\n

As companies go to cloud, they\u00a0<\/span>must<\/span>\u00a0adopt a new operating model to effectively\u00a0<\/span>build and run their environment<\/span>.\u00a0<\/span>In this blog post, which is the first part of a\u00a0<\/span>two<\/span>-part series,\u00a0<\/span>we\u00a0<\/span>will explore a few key aspects\u00a0<\/span>of\u00a0<\/span>unlocking the cloud operating model and achieving operational excellence during the cloud journey.\u00a0<\/span>\u00a0<\/span><\/p>\n

In the era of rapid technological change, the cloud is key for future-proofing businesses and maintaining a competitive edge. Businesses that\u00a0<\/span>are migrating<\/span>\u00a0<\/span>their applications and infrastructure into the cloud can benefit from the latest technology and<\/span>\u00a0<\/span>move<\/span>\u00a0away from their aging on-premises infrastructure, putting them at the forefront of innovation.<\/span>\u00a0<\/span><\/p>\n

The Cloud Journey<\/span><\/span>\u00a0and Cloud Operating Model<\/span><\/span><\/h2>\n

The question facing enterprises today is not should we go to cloud but how do we go to cloud?\u00a0<\/span>As Africa\u2019s first AWS Advanced Consulting partner, we know that the journey to the cloud is not always easy.\u00a0<\/span>Organi<\/span>z<\/span>ations<\/span>\u00a0may experience uneasiness during the journey especially the fear of the unknown and the large number of decisions that they need to make.<\/span>\u00a0<\/span><\/p>\n

It is\u00a0<\/span>critical that an operating model is formed as organi<\/span>z<\/span>ations go through this journey. It may be tempting to take existing on-premises procedures and\u00a0<\/span>processes into the cloud. However, doing so will lead to failure as the cloud is significantly different from\u00a0<\/span>on-premise<\/span>s<\/span>\u00a0infrastructure. Some of the\u00a0<\/span>main\u00a0<\/span>drivers to adopt cloud technology include\u00a0<\/span>minimizing<\/span>\u00a0operational costs, workforce productivity, cost avoidance, operational resilience, and business agility. By not re-examining the operating model, organi<\/span>z<\/span>ations will not unlock the true benefits of cloud.<\/span>\u00a0<\/span><\/p>\n

Organi<\/span>z<\/span>ations must consider what new techniques and processes can be put in place to help with the provisioning, securing, connecting and management of cloud environments. One of these techniques is the adoption of\u00a0<\/span>DevOps\u00a0<\/span>in the organi<\/span>z<\/span>ation, it enables applications to be delivered faster while ensuring quality control and minimi<\/span>z<\/span>ing operational overhead.<\/span>\u00a0To maximize efficiency and to ensure that cloud practices are adopted, organi<\/span>z<\/span>ations need to create a virtual highway for teams to get to the cloud, this includes onboarding material, clear governance processes and technical enablement which may consist of both appropriate tooling and training.\u00a0<\/span><\/p>\n

Provisioning<\/span><\/span>\u00a0\u2013\u00a0<\/span><\/span>Reproducible\u00a0<\/span><\/span>Infrastructure as Code<\/span><\/span>\u00a0<\/span><\/h2>\n

T<\/span>he first key challenges organi<\/span>z<\/span>ations must address is how infrastructure will be provisioned and managed.\u00a0<\/span>In today<\/span>‘<\/span>s\u00a0<\/span>on-premise<\/span>\u00a0<\/span>environments, infrastructure is manually\u00a0<\/span>setup\u00a0<\/span>which<\/span>\u00a0relies on a\u00a0<\/span>ticketing and queuing system<\/span>\u00a0that can often attract lengthy delays<\/span>. Using this workflow, no quality control exists, and it is not possible to reproduce infrastructure configuration to enable debugging or testing before rolling out configurations to production.\u00a0<\/span>\u00a0<\/span><\/p>\n

The adoption of\u00a0<\/span>Infrastructure as\u00a0<\/span>C<\/span>ode<\/span>\u00a0in the organi<\/span>z<\/span>ation<\/span>\u00a0<\/span>is essential to address this challenge. Infrastructure as Code\u00a0<\/span>is the\u00a0<\/span>practice of using code to describe infrastructure and services along with their respective configurations<\/span>,<\/span>\u00a0which is\u00a0<\/span>both\u00a0<\/span>human readable and machine consumable.\u00a0<\/span>Our recommended tool for this problem is\u00a0<\/span>HashiCorp<\/span>\u00a0<\/span>Terraform<\/span>.\u00a0<\/span>You describe the infrastructure and configuration that you want and\u00a0<\/span>Terraform will determine the order of the steps that need to be applied.<\/span>\u00a0<\/span>The desired state is expressed in a simple human readable language, after which Terraform can determine the sequence in which the infrastructure needs to be created.<\/span>\u00a0<\/span>\u00a0<\/span><\/p>\n

We love Terraform for its simplicity and how easy it is to learn and use, in addition to the fact that Terraform supports<\/span>\u00a0over\u00a0<\/span>200\u00a0<\/span>providers including<\/span>\u00a0hardware, PaaS, IaaS and SaaS<\/span>\u00a0services.\u00a0<\/span>This means that Terraform\u00a0<\/span>can provision to more than just the cloud.\u00a0<\/span>It can<\/span>\u00a0provision and configure\u00a0<\/span>on-premise<\/span>\u00a0tools like VMWare, Venafi and\u00a0<\/span>InfoBlox<\/span>\u00a0as well as network security appliances like F5,\u00a0<\/span>FortiOS<\/span>, Cisco, Palo Alto and Check Point devices.\u00a0<\/span>This extensible provider support allows complex infrastructure to be setup from a single Terraform run.\u00a0<\/span>\u00a0<\/span><\/p>\n

Incorporating\u00a0<\/span>I<\/span>nfrastructure as\u00a0<\/span>C<\/span>ode into the development lifecycle means that tried and tested software engineering processes can be adopted. This includes managing source code in version control and using branching strategies to allow for locali<\/span>z<\/span>ed testing before pushing configuration to a production environment.<\/span>\u00a0Using pull requests before merging code can allow teams to review changes\u00a0<\/span>before<\/span>\u00a0they are<\/span>\u00a0committed to upstream branches.<\/span>\u00a0<\/span>The configuration of infrastructure is also reproducible and allows a dev environment to be replicated into production.<\/span>\u00a0<\/span><\/p>\n

With infrastructure configuration\u00a0<\/span>codified, it can be managed using well-known and well-tested software engineering practices such as version control, branching, pull requests and CI\/CD tooling. Code can be reviewed by the necessary teams before it is committed to upstream branches. A<\/span>n example<\/span>\u00a0of this<\/span>\u00a0would be that a developer needs to adjust a permissions policy<\/span>.\u00a0<\/span>T<\/span>he developer can make the change and commit it into source control<\/span>\u00a0after<\/span>\u00a0<\/span>which\u00a0<\/span>t<\/span>he developer can then create a pull-request to merge the new permissions into the policy. A predefined set of people can then<\/span>\u00a0review and approve the change or ask the developer to make revisions as appropriate.<\/span><\/p>\n

\"\"<\/span><\/p>\n

Building Security into the\u00a0<\/span><\/span>development lifecycle<\/span><\/span><\/h2>\n

Working in the financial services industry<\/span>\u00a0for over 20 years<\/span>,\u00a0<\/span>we know that security is one of the top concerns<\/span>\u00a0<\/span>our clients have<\/span>\u00a0during the cloud journey. There are many stories of data getting exposed from misconfigured S3 buckets, Elastic Search instances, databases and SIEMs to name a few.<\/span>\u00a0Developers should not need to be AWS S3 experts to use the service in a secure manner that complies with the organi<\/span>z<\/span>ation\u2019s security and compliance standards.<\/span>\u00a0<\/span><\/p>\n

To solve this challenge,\u00a0<\/span>Terraform<\/span>\u00a0has the capability to consume modules. Modules are reusable and adaptable building blocks for cloud infrastructure and services. They are a set of predefined and prebuilt terraform code which describe common architecture and patterns that the organi<\/span>z<\/span>ation uses. Modules are created by experts in the organi<\/span>z<\/span>ation<\/span>\u00a0in consultation with security teams<\/span>\u00a0so that they adhere to policies, frameworks, and standards of the organi<\/span>z<\/span>ation.\u00a0<\/span>\u00a0<\/span><\/p>\n

Once these modules have been created, they can be uploaded to source control and module registries where they can be consumed by the organi<\/span>z<\/span>ation. By using the modules, developers and IT professionals know that the<\/span>\u00a0infrastructure they are spinning up adheres to company policy even if they are not experts in the infrastructure that they are creating. This enables developers to focus on solving business problems\u00a0<\/span>rather than focusing on\u00a0<\/span>undifferentiated\u00a0<\/span>heavy lifting of setting up infrastructure.<\/span>\u00a0<\/span><\/p>\n

Security teams can\u00a0<\/span>enforce policies on the type of infrastructure that can be created, how it is used, and which teams get to use it.\u00a0<\/span>HashiCorp\u2019s<\/span>\u00a0Sentinel\u00a0<\/span>P<\/span>olicy as\u00a0<\/span>C<\/span>ode\u00a0<\/span>F<\/span>ramework, can also be used to provide compliance and governance without requiring a shift in the overall team workflow<\/span>\u00a0and is defined as code too, enabling collaboration and comprehension for\u00a0<\/span>DevSecOps<\/span>\u00a0teams.\u00a0<\/span>\u00a0<\/span><\/p>\n

By adopting this way of working, organi<\/span>z<\/span>ations allow developers to build infrastructure that complies\u00a0<\/span>with\u00a0<\/span>policies and moves away from ticket-based systems for infrastructure to be created. These systems can often result in developers waiting weeks or longer to provision infrastructure.\u00a0<\/span><\/p>\n

\"\"<\/p>\n

Terraform Cloud & Terraform Enterprise<\/span><\/span>\u00a0<\/span><\/h2>\n

Terraform comes in\u00a0<\/span>three<\/span>\u00a0flavors: open source, cloud, and enterprise. Terraform open source is distributed as a single binary and is available using a CLI. When working in a significant team, state management and resource runs may become an issue. Terraform Cloud solves this problem and is the\u00a0<\/span>fastest way to adopt Terraform<\/span>. Terraform Cloud is offered as\u00a0<\/span>a service<\/span>\u00a0where\u00a0<\/span>practitioners, teams, and global businesses\u00a0<\/span>can\u00a0<\/span>collaborate on infrastructure and manage risks for security, compliance, and operational constraints.<\/span>\u00a0Terraform Enterprise is deployed on infrastructure in your environment and can facilitate\u00a0<\/span>collaboration, governance, and self-service workflows on top of the\u00a0<\/span>I<\/span>nfrastructure as\u00a0<\/span>C<\/span>ode provisioning from open source.\u00a0<\/span>\u00a0<\/span><\/p>\n

Both Terraform and E<\/span>nterprise provide<\/span>\u00a0workspaces, modules, and other powerful constructs for teams working together to build infrastructure.\u00a0<\/span>Modules can be uploaded to registries which will e<\/span>nabl<\/span>e<\/span>\u00a0developers to quickly provision<\/span>\u00a0infrastructure<\/span>\u00a0in a self-service fashion.\u00a0<\/span>In addition, both platforms support SSO via an identity provider, logging and monitoring and dedicated support from\u00a0<\/span>HashiCorp<\/span>. Both platforms also support\u00a0<\/span>Sentinel<\/span>\u00a0which<\/span>\u00a0is an embeddable policy as code framework to enable fine-grained, logic-based policy decisions that can be<\/span>\u00a0used by teams to limit certain actions, configurations or deployments.<\/span>\u00a0<\/span><\/p>\n