Netflix’s smart methods behind the streaming madness
A dedicated content delivery network, localized hardware to cache content, smart algorithms and managing buffer free streaming, is just of the work that goes into making your Netflix streaming experience enjoyable
It was at the beginning of this year that the previously predominantly US-centric streaming service Netflix expanded to another 130 countries, including India. And that has changed the entire video streaming scenario in the country. Movie downloads ruled for most users at the time. But in the present scenario, habits have changed, and streaming movies and TV shows is very much on the radar for a lot of users. What you still need is a fast enough home broadband connection and also 3G/4G subscriptions with enough data to support watching the latest episode of Narcos on the way home from work, in the evening. And the thing is, a greater demographic of users is now willing to spend on good internet, and pay for accessing streaming content.
What sort of sorcery this is, you may wonder, that millions of users now in as many as 190 countries can simultaneously watch perhaps even the same piece of content, without any interruptions? And that will include resolutions in Standard Definition (SD), bandwidth intensive High Definition (HD), the really heavy data consuming 4K resolutions and now even content in 4K HDR (High-dynamic-range) which requires a really fast broadband connection.
This is where Netflix’s own content delivery network (CDN) assumes greater importance. The company had realized as far back as in 2012, when they first started rolling this out in certain regions. It is called Open Connect, and what a CDN essentially does is that it runs alongside the ‘internet’ as we know it, but all the content being streamed by millions of users is actually on a parallel pipeline, if we may. Netflix says that this network handles ‘tens of terabits per second of simultaneous peak traffic’, amounting to as much as 125 million hours of viewing every day, around the world.
Perhaps the biggest aspect of this CDN is localization, at least some semblance of it. And for that, Netflix needs to tie up with Internet service providers (ISPs) and provides them with something known as Open Connect Appliances (OCAs). These are used to deliver Netflix streaming traffic to a certain geolocation. The biggest advantage is that the content is cached much closer to the destination it is being streamed to, so that it doesn’t have to be transferred completely over the network whenever it’s requested. In fact, to keep costs down, Netflix is building this hardware on its own. And there are specific components installed in each OCA, including Chassis(Sanmina), motherboard (Supermicro), processor (Intel) and so on.
For users in India, the content that we stream here does not get shared all the way from the servers in the US, but instead from one of the OCAs in Singapore. If necessary, some of the traffic can be shifted to OCAs in Hong Kong or Japan as well. At present, Netflix does not have any OCA tie-up with ISPs in India, though that might change in the future, depending on the volume of streaming traffic and localization of content.
Netflix says that the Open Connect CDN has smart algorithms working all the time, to actively monitor what shows are more popular in which geolocation, and caches that content automatically at the local level, to improve the streaming and viewing experience—negates the buffering issue, which ruins so many video streaming experiences.
Apart from the CDN, Netflix has also shifted to per title encoding, from the standard bitrate encodes for all the content on the platform. This meant there was a certain amount of bandwidth being wasted by encoding certain content at a much higher quality than what it otherwise needs. Fast paced action TV shows and movies, with fine details, explosions, fast moving visuals and dynamic visuals, need better encoding than a documentary, a stand-up comedy show or even an animated video. Which is why Netflix now does per-title encoding, which saved bandwidth, makes streaming smoother, improves detailing of content where needed and there is lesser data to cache as well.
The final, and perhaps the most relevant aspect of this entire streaming chain is the adaptive streaming quality. Depending on the state of the internet connection at your home or on the mobile device at the time, the streaming quality gets retuned automatically. If Netflix detects slow speeds, even temporarily, the quality will automatically get lowered for a certain duration at least, to ensure buffer free viewing as far as possible.
With the sheer volume of content that Netflix is handling globally, a robust streaming solution at the back-end becomes extremely critical.
In May, popular peer-to-peer (P2P) service BitTorrent announced their Live platform, which would continue to use the P2P model to stream content to users. The P2P method is designed in such a way that it spreads the burden of bandwidth equally across all downloaders of any particular file—a file is split into smaller blocks, and those who are downloading also have the responsibility of uploading it for other users connected on the network. However, it is important to note that BitTorrent does not have the variety and volume of content that Netflix has, which is why it doesn’t need its own CDN set-up, just yet.