A Primer for Emerging Developers on DevOps, CDNs and More
July 28, 2015 | Esteban Borges
Flash forward to today and companies are beginning to merge their development and operations teams under a single umbrella known as DevOps.
Under DevOps, the lines are a bit more blurred and developers need to have at least a working knowledge of the world outside of code. Full stack developers, or developers who know all areas of the web stack, are practically a necessity. For new developers, this might seem daunting. But I recommend looking at it as an exciting challenge, an opportunity to become more capable and versed than your peers of yesteryear.
In this post we’ll discuss the impact of DevOps on new developers and on web development as a whole.
Web Development Isn’t Just About HTML
When we talk about the web stack, we’re referring to the combined technologies that drive a website or web service. The HTML sent to the user’s browser is only the end result of the stack. How that HTML is sent – more importantly, how quickly it’s sent – is determined by the technologies that lie in the middle.
Look at the LAMP stack, one of the world’s most popular web application platforms.
LAMP consists of a Linux-based operating system, the Apache Web server, the MySQL database server, and the PHP programming language. Traditionally, these technologies would need a systems administrator, a web administrator, a database administrator, and a PHP developer. Each requires a distinct knowledge set, but they ultimately work together to provide a single unified service.
Rather than hire four specialized employees, companies have been investing in developers who are experienced in all four areas. The idea is that developers who understand the full stack are more likely to write optimized code, better understand bugs, and perform thorough testing and troubleshooting. The benefit for companies is clear: Why spend time and money passing software across multiple teams when a single dev can do it faster?
The Shifting Landscape of Web Development
Web development has seen a huge shift in its core technologies. Languages, protocols, and content have undergone massive changes since the early days of the web, and many of the best practices that drove development during the last decade are no longer relevant. There are four areas in particular that we’ll focus on: platform, security, content distribution and analytics.
At the heart of every web service lies a web platform. LAMP is one of the most popular platforms, but it’s gradually losing favor. For instance, LEMP (Linux, Nginx, MySQL, and PHP) is quickly becoming the preferred stack for thousands of service providers around the world. In fact, Nginx is so robust our team at MaxCDN relies on it to deliver content from our CDN.
Tip: Be sure to keep an eye on the up-and-coming platform MEAN (MongoDB, ExpressJS, AngularJS, NodeJS).
As a developer, knowing your platform lets you take advantage of the underlying software for optimization. For instance, the IETF recently approved the HTTP/2 specification. If you know your platform supports HTTP/2, you can start modifying your web pages to take advantage of HTTP/2’s features. As another example, if your web service stores content in a database, you can optimize your database around the type of content that’s being stored.
Knowing your platform gives you the freedom to tailor your service to your platform. And with DevOps, it helps you communicate your requirements with other team members.
The security of a web service is only as strong as its weakest link. With so many links in web development, it’s no wonder that the number and severity of attacks on web services has been steadily increasing over the past decade. Web attacks typically take two approaches: attacking the frontend (the HTML and scripts rendered by the browser) or the backend (the hosting environment).
Many of the more damaging exploits take place on the backend. SQL injection, for instance, lets an attacker execute commands on a database simply by using an unsecured input field on a web page. You don’t have to be a full-fledged database admin, but you should have a basic idea of how databases work to understand how SQL injection is made possible.
Attacks can go beyond the web stack as well. In 2014, thousands of Linux servers were exposed to a remote command execution vulnerability in the commonly-used Bash shell. The exploit relied on a web server passing an HTTP header to the shell.
For instance, if the web server used the user-agent header as a variable in a script, an attacker could append code to the header that would be executed by the script engine. This code could retrieve sensitive data, trigger a denial-of-service attacker, or download and run malicious software. The bug has since been patched, but it goes to show how sensitive web development is to security exploits.
CDN Adds a Layer of Security
Content delivery networks (CDNs) are gateways to millions of content providers around the world, and as a result undergo a constant barrage of attacks. CDNs often implement their own security standards to mitigate the impact on content providers. But with some CDNs like our own that uses Nginx, you can easily implement your own security in the form of a web application firewall (WAF).
As we’ll touch on in the next section, many developers are now an integral part of the content delivery strategy, so knowing how to add security to it is equally important. Fortunately, Nginx and new WAFs make this relatively easy.
WAFs apply a set of rules to all incoming and outgoing HTTP connections that identify and block common attacks including XSS, SQL injection, and distributed denial of service (DDoS). A CDN, combined with a WAF, can help secure your servers from a wide variety of attacks.
Some WAFs that integrate well with Nginx and CDNs include:
Content distribution has changed since the early days of the web. It used to be that content providers operated a server (or cluster of servers) that users connected to directly. If there was a problem with the server, or if a link between the user and the server was offline, then the website was unavailable. Companies built distributed data centers to increase their availability, but then had to worry about synchronizing data between each location.
The old client-server model got a much-needed revision with content delivery networks. CDNs addressed a lot of underlying issues by acting as an intermediary between websites and guests. This reduced some of the pressure from service providers, letting them serve more users with fewer resources or even go temporarily offline.
Image: CDN vs. single origin
Customers expect fast, 24/7/365 access to their favorite websites and CDNs help ensure that availability.
For developers, this means being aware of how CDNs integrate with their existing systems. For example, the links in a web application will need to point to the CDN rather than the local server. Certain parts of a page may need to be configured for caching, whereas some of the dynamic components may need to be fetched from the main server. Traditional developers might not have to worry about this, but full stack developers definitely need to understand how it can impact their application.
At one point, analytics was the realm of systems administrators and management. Using visual tools like Graphite and Cacti, admins could glimpse at network performance and server load. Troubleshooting an actual event still meant digging through server logs, searching debugging output from a scripting language, or plain old trial-and-error.
With DevOps shortening the gap between developers and sysadmins, analytics is becoming part of the developer’s workflow. Developers can get constant, automatic feedback on website performance to better manage tasks.
Analytics suites like New Relic automatically collect data on all areas of the stack from uptime to database performance and resource usage. Many suites even provide their own search syntax to help developers quickly search for errors, instances of downtime, unusual rendering times and more. Outside of analytic services, non-analytics Internet services (including MaxCDN) provide their own platforms.
Image: Screenshot of one of many reporting dashboards in the MaxCDN control panel
Platforms like this are giving emerging developers tasked with new projects the ability to easily monitor performance they’re responsible for. In the case of MaxCDN, developers can easily watch websites in real time and view information such as requests per second and bandwidth usage.
The Next 10 Years
Like it or not, developers have to know more than just code.
Compared to the early days of the web, we have more tools at our disposal, more functionality to provide, and more risks to look out for. The needs of web companies are changing, and many are counting on new developers to pave the way for changes in platform optimization, software management, security, analytics and content distribution.
If you’re a veteran dev, how have you seen the responsibilities of web developers change over the past 10 years, and where do you think we’ll be in the next 10? If you’re just coming into web development, are you faced with more responsibilities than you anticipated?
Let us know in the comment section below.