AI

Meet the Emmy-winning engineer whose algorithms are behind your Netflix binge

Charles Ferrer — Tue, 21 Apr 2026 14:32:51 +0000

Meet the Emmy-winning engineer whose algorithms are behind your Netflix binge Charles Ferrer Tue, 04/21/2026 - 08:32

Charles Ferrer

Photo Credit: Jesse Petersen

Every time you hit play on a video, chances are you have Al Bovik to thank for its visual quality.

Bovik, professor and Provost’s Chair in Engineering in the Department of Electrical, Computer and Energy Engineering, has spent decades developing algorithms that now influence nearly 80% of internet and social media content.

At the center is digital visual perception, or using the neuroscience of human vision to make streamed video look as sharp and natural as possible. His work is used by familiar brands like Netflix, Amazon and YouTube.

Understanding not just how cameras capture patterns of light, Bovik explains, but how the brain interprets it is an important element that drives his research.

“The question that really gripped me over time was: can we model mathematically how we see?” Bovik says. “That’s a very different and much harder problem.”

His achievements in visual perception processing has landed him two Emmys: a Primetime Emmy Engineering Award for Outstanding Achievement in Engineering Development and a Technology and Engineering Emmy from the Academies of Television Arts and Sciences. They also earned him the IEEE Edison Medal, which he shares with Alexander Graham Bell, Nikola Tesla and Ray Dolby.

We sat down with Bovik to discuss his career, the neuroscience hiding behind your favorite TV or movie and why his proudest achievement isn’t just theories and algorithms.

For someone outside the field, how would you describe what digital processing is?

At its simplest, image processing is about manipulating visual information using computations. Digital processing involves inventing theories and algorithms to help make television and movies more efficient, faster and higher quality. What I do is more than that. It is modeling the visual parts of the brain mathematically, then using those models to create algorithms for better photography, TV shows and movies.

What first drew you toward the field of digital processing?

I’m a deeply visual person. Whenever I travel, the first place I go is an art museum. If I go a week without seeing a movie, I go into withdrawal. I’m a visual, spatial thinker and suddenly here was a field that lived at the intersection of mathematics and how we see the world. Then I took an image processing class from Thomas Huang, one of the inventors of image compression, and everything changed overnight. I knew immediately: This is what I want to do. I’ve never looked back.

What does the science of human vision reveal about how we see digital content?

We know that image processing happens in various brain centers, including the primary visual cortex — the very back of the brain. Vision requires processing an enormous amount of raw information, compressing it into concise, efficient representations that the brain can use to recognize a car on the highway or track a bird in flight. We can model that mathematically and start exploring questions like why do we look where we look, or where does your gaze land when you’re driving? The same holds true in videos — your eyes are directed to certain areas when viewing a particular scene.

What are the acclaimed algorithms that you innovated, ones people don’t necessarily notice?

We created a variety of algorithms used throughout the streaming and social media industries. These algorithms use mathematical models of how visual distortions are perceived in the human brain, using them to predict how a human will rate the visual quality of a picture or video. For example, they are widely used to control the compression of television and movies streamed worldwide. Compression is necessary since videos are huge and would not be practically streamable otherwise. One of them, called structural similarity (SSIM), allows the big streamers and social media platforms to compress content as much as possible to the point just before noticeable distortions appear. Engineers at companies like Netflix, Meta Platforms, Amazon and YouTube use this technology.

Can you walk us through what’s happening technically when someone presses play on Netflix?

Let’s say you’re watching Stranger Things. The moment you start a scene, up in the cloud, approximately 20 different versions of that scene have already been prepared, each compressed a different amount, each perceptually optimized using our algorithms. Some are also spatially downsampled: A 4K video might have versions encoded at 2K or even lower resolution.

Your device, whether it’s on your phone or TV, measures the available bandwidth, which changes constantly, especially if someone is on the move in a city with tall buildings and requests whichever of those 20 versions best fits your current conditions. This happens scene by scene, continuously.

Here’s the part that surprises most people: You might think you’re watching 4K, but if your bandwidth is constrained, you might actually be receiving a heavily compressed 2K version that’s been decompressed and upsampled back to 4K on your TV. Visually, you can’t tell the difference because of our video quality algorithms.

Your algorithms also help determine how much video can be compressed before viewers notice a difference. How does that work?

Another algorithm we developed, called visual information fidelity, or VIF, predicts how a person will perceive the quality of a video after it has been compressed. It tells the Netflix video quality system the point where distortions may be visible. Netflix’s video streaming is built on these neuroscience principles and sometimes I say that they have now become a visual neuroscience company.

Professor Al Bovik and two former PhD students at the 2015 Primetime Emmy Engineering Awards ceremony.

How did your first successful model, structural similarity, come about?

Almost by accident, honestly. My students and I were working on video compression, and we ran into a fundamental problem: How do you even measure whether your results are good? How does a human perceive the quality of a picture? Nobody had really solved that, and most researchers thought it was unsolvable. So we built our own model. We were amazed when the entire television industry noticed and adopted it. The streaming world discovered it early while they were wrestling with the question of how much to compress video before it starts looking distorted to a viewer. This was especially important since the new wireless/smartphone systems had very limited bandwidth. SSIM gave them a way to find that compression point and deliver perceptually compressed videos to everyone. Every photo uploaded to Facebook, Instagram, WhatsApp or Reels is now optimized using a model rooted in visual neuroscience. We had successfully introduced the principles of visual neuroscience throughout the internet.

You’ve also worked with Meta for nearly a decade on virtual and augmented reality. What does that world look like?

It’s one of the most exciting problems I’ve worked on. Imagine wearing advanced AR glasses here in Colorado, while your colleague is wearing a similar pair in Paris. You can see each other in 3D, in real time, as if you’re in the same room. The challenge is that the display is an inch from your eye, so you need a far denser resolution, perhaps 8K or 16K, which means vastly more data to compress and transmit. Our approach is the avatar model: rather than sending a live 3D video feed, you build a photo realistic 3D model of the person which is stored on your friend’s AR glasses, and only transmit their facial movements determined by cameras and image processing in your own glasses, which requires far less bandwidth. The 3D avatar is animated in real time on the receiving end.

What are you most proud of during your teaching career and working partnering with some of the largest digital giants?

I ask myself, “Am I giving my students the best possible opportunities?” My students are not just programmers, and they’re not just video engineers. They’re also trained as visual psychologists and neuroscientists. The thing I’m most proud of is the successes of my students. The Netflix video team is largely composed of students from our Laboratory for Image and Video Engineering (LIVE). What matters most to me are the people who came through this lab and went on to shape an industry. No less than six of my students have Emmy statuettes on their shelves at work or home. If I were to ask myself why I’m here at CU Boulder, that’s the answer, along with living in the Colorado mountains!

What’s an aspect that people may not realize about your work in image processing?

The internet now accounts for nearly 10% of global carbon emissions, and that’s growing fast. Our algorithms help reduce internet video data volume, which is 80% of internet traffic, by nearly 25%. By reducing the amount of data moving through global networks, we are shaving off a meaningful fraction of that footprint, and the ecological impact is real.

Burning question: Do you have a favorite movie and show that has used your algorithm?

Pretty much any movie or TV show I watch will be processed by these algorithms. These would include British Mystery shows like Broadchurch, Grace and Prime Suspect, which my wife and I watch all the time, and movies with great acting, cinematography and directing, like The Godfather, 2001: A Space Odyssey, Blade Runner, Spartacus, Gladiator and many more. This year, I especially liked Sinners and One Battle After Another.

Two-time Emmy‑winning electrical engineer Al Bovik shares how his algorithms shape the visual quality of nearly 80% of streamed video worldwide. By combining neuroscience with engineering, his work impacts some of the largest digital platforms behind your TV or movie binge.

Off

Traditional

White

Scientists harness AI to reveal forces behind glacier surges

Charles Ferrer — Thu, 05 Mar 2026 22:12:42 +0000

Scientists harness AI to reveal forces behind glacier surges Charles Ferrer Thu, 03/05/2026 - 15:12

Charles Ferrer

Ute Herzfeld (PI), Harald Sandal (pilot), Gustav Svanstroem (helicopter technician) and Matthew Lawson (research assistant) during the Negribreen Glacier System Airborne Geophysical campaign (Photo Credit: Thomas Trantow).

Glaciers are constantly changing and reshaping the Earth’s surface.

CU Boulder researchers have developed a new machine learning tool to better understand how Arctic glaciers suddenly accelerate or “surge”.

The team, led by Ute Herzfeld, a research professor in the Department of Electrical, Computer and Energy Engineering, created an open-source cyberinfrastructure called GEOCLASS-image, designed to decode the physical processes behind glacier motion using high-resolution satellite imagery and machine learning.

Glacier surges are sudden bursts of movement in otherwise slow-flowing ice.

Normally, glaciers move at a steady pace, but during a rare “surge”, that rate can accelerate up to 200 times faster than usual. The ice fractures into deep crevasses and pushes large volumes of ice toward the ocean. These dramatic events provide scientists with new insight into the unpredictable drivers of sea-level rise.

“Most deep machine learning systems don’t know what to look for in images,” said Herzfeld, who is also the director of the Geomathematics, Remote Sensing and Cryospheric Sciences Laboratory. “We have built a system that understands the physics of ice deformation, so the classifications actually mean something.”

Understanding how a glacier surges

Unlike traditional artificial intelligence systems that often struggle to interpret complex natural phenomena, the team created a new neural network approach—VarioCNN—to better understand glacial acceleration.

“Surging glaciers are one of the deep uncertainties in sea-level rise projections,” Herzfeld said. “They can move much faster than normal and current earth system models do not yet have the ability to account for them.”

To tackle this problem, Herzfeld and her team merged two powerful approaches: a deep convolutional neural network (CNN), common in the field of computer science and remote sensing and a physics-informed neural network model that captures how crevasses in the ice form, widen and intersect during motion.

“Think of neural networks as Lego blocks,” Herzfeld said. “We’ve taken some from physically informed models, some from deep learning and built a new kind of AI that’s meaningful.”

Putting AI to the test

The team tested their approach on a real-world event: the unexpected 2016 surge of Negribreen, a glacier located in the Arctic archipelago of Svalbard a 1,000 km south of the North Pole.

This isn’t just another AI model but one that understands the physics of glacial acceleration.
~Ute Herzfeld

Using Maxar WorldView satellite imagery collected in 2016-2018, the researchers tracked subtle changes across the glacier’s surface with remarkable detail.

They discovered that crevasse patterns, which change dramatically during a surge, hold information about surge dynamics that can be retrieved using their neural network approach.

One-dimensional crevasses appeared at the leading edge of the surge, while deeper within the surge area, complex patterns tell the story of the transformation and deformation of the ice, which can be of use in numerical modeling of the glacial acceleration.

Shear, a type of deformation that plays a key role in glacial acceleration, is easily misclassified in deep learning, but correctly identified using VarioCNN.

With their new VarioCNN model, they classified different types of crevasses from satellite images and used those patterns to interpret how the glacier moved and changed.

Results of the classification were then used to understand how the surge expanded and affected the entire Negribreen glacier system. Ultimately, ice mass equivalent to 1% of global annual sea-level rise transferred to the ocean.

Published in Remote Sensing, their results demonstrated how integrating physical knowledge into a neural network model, carried out at the computational level, can advance machine learning and glaciological understanding of glacier surges. The paper was selected as the cover story of Remote Sensing receiving record downloads during the first two weeks after publication.

Student Connor Meyers setting up a GPS station at the edge of Negribreen (Photo Credit: Ute Herzfeld).

“The problem of task-oriented machine learning is especially intriguing to me,” said Silas Twickler (Phys’25) who was a research assistant on the project. “While simply applying pre-existing neural networks may be sufficient for certain applications, the augmentation of these networks can allow for a drastic improvement in machine learning.”

AI for the geosciences

A major hurdle in applying machine learning to studying glaciers is the limited amount of labeled data. To overcome this, Herzfeld’s team developed a way that allows scientists to gradually refine the model using a relatively small number of hand-labeled satellite images.

VarioCNN was trained on just a few thousand of examples, far fewer than the 100,000 images than typical deep learning models require. Due to its modular design, the GEOCLASS cyberinfrastructure can be adapted to study other geophysical processes and potentially surfaces of other planets.

“Our tool is not just for glaciologists, but for anyone working with remote sensing and physical systems,” Herzfeld said. “Ultimately, we hope to give scientists better tools to understand how the Earth is changing.”

This research was funded by the National Science Foundation Office of Advanced Cyberinfrastructure and NASA Earth Sciences Division.

Off

Traditional

White

Negribreen glacier during an ice surge in 2017 (Credit: Ute Herzfeld).