Mathematics’ Nikolaos Mitsakos Applies Deep Learning to Image Analysis

“Don’t be afraid to take challenging classes in graduate school,” said Nikolaos Mitsakos, a Ph.D. student in mathematics, who works on improving algorithms for automated image analysis.

For Mitsakos, whose research is advised by Manos Papadakis, a professor of mathematics in the College of Natural Sciences and Mathematics, graduate school has been about learning new skills, as well as finding ways to challenge himself.

This has meant learning a solid foundation of theoretical mathematics, while also finding ways to apply it to the problem of image analysis. Along the way, this has led Mitsakos to apply machine learning to his research, knowledge which has come in handy for his post-graduation plans of working in industry.

“The general set of skills that you learn from trial and error become very useful,” Mitsakos said.

Image Analysis Used in a Broad Range of Contexts

Image analysis is used to extract meaningful information from photos or video feeds. Uses range from identifying faces in security footage to classifying objects in photographs to aiding in medical diagnoses. In the context of automation, image analysis is a particularly challenging problem.

At its most basic, a digital image is a collection of pixels, tiny dots that form an image. The unique configuration of pixels is one layer of meaning. The objects these pixels portray is another layer of meaning, one that the human mind takes for granted, but, in the context of a computing task, becomes enormously complex.

“A computer understands an image as a matrix, with each pixel corresponding to one intensity value,” Mitsakos said.

For a computer to analyze an image, it needs to segment these pixels into identifiable objects, which can then be classified and analyzed.

Harmonic Analysis: A Branch of Mathematics Useful for Image Processing

Mitsakos’ area of expertise is in harmonic analysis, a branch of mathematics which represents signals as a superposition of basic waves, with applications ranging from number theory and signal processing to quantum mechanics.

For example, every time a musician plays a note, the resulting sound is made up of a primary frequency, as well as a distinctive set of higher frequencies. The main frequency is what defines the specific note, while the higher frequencies act as a sort of fingerprint, giving each instrument its own unique sound. In some cases, the individual fingerprint of these higher frequencies can make analysis of the primary frequencies harder.

Extracting a Clearer Signal for Image Processing

A similar principle is at play when it comes to images.

“For example, let’s think of an image that is corrupted by high-frequency noise,” Mitsakos said. “By decomposing the signal, processing the high-frequency components and then putting everything back together, one can filter out the high-frequency noise, obtaining a clearer image to work with. Of course, modern harmonic analysis offers tools that allow us to achieve more complicated goals, along the lines of extracting useful information from images.”

In his research, Mitsakos incorporates this filtering into deep learning, which is a subset of machine learning. Deep learning, which is modeled after the neural networks in our brain, is patterned in a series of layers, with each successive layer processing output from the previous one.

Mitsakos, who has been working with machine learning since his time as a master’s student, saw applying deep learning to his research as an opportunity.

“During the process of graduate school, you gain important training and knowledge,” Mitsakos said.

- Rachel Fairbank, College of Natural Sciences and Mathematics