Achieving Super-Resolution

One application of this research breakthrough is to overcome physical camera lens limitations in mobile phones

Can you take high-resolution, high-quality images on your mobile phone without being limited by the physical lens? Are you able to remaster classic films and video games for today’s 4K world?

One answer to these questions lies in the ability to upscale a low-resolution image to super-resolution by restoring the missing high frequency components. And this is an area that Nanyang Associate Professor Loy Chen Change, from Nanyang Technological University’s School of Computer Science and Engineering, has been focusing on within the field of computer vision.

Unlike conventional interpolation techniques, image super-resolution aims to provide sharper edges and textures for a more pleasing and vivid viewing experience. But it is a very tough nut to crack. As Prof Loy explained, “This is mathematically difficult because there are far too many high-resolution possibilities for a low-solution pixel.”

Prof Loy’s team has been investigating novel deep learning-based algorithms to solve this problem and invented the first deep convolutional network for single image super-resolution in 2014 (Learning a Deep Convolutional Network for Image Super-Resolution, ECCV, 2014).

This seminal work has inspired a new wave of technologies that make use of AI upscaling in mobile phone photography. Such technologies, which can be seen in various models of Xiaomi and Vivo mobile phones, typically exploit redundant spatio-temporal information from several photos taken consecutively to generate a single sharp and vivid image. This enables users to capture high-resolution and high-quality images without being limited by the physical lens in their mobile phones.

Image super-resolution can also be applied to re-digitise and restore film stock as well as to remaster classic video games. “The widespread presence of 4K and more recently 8K televisions is driving demand for high-quality versions of existing videos, as existing videos are limited by the resolution they were originally acquired in,” said Prof Loy. “Image super-resolution technology is able to enhance the quality much better, compared with conventional methods such as bilinear or bicubic interpolation that often give rise to blurry edges and textures.”

The technology also helps users to save considerable costs in bandwidth when they send images over the network. This is achieved by transmitting low-resolution images and upscaling them at the receiving end-user device.

These developments, however, are “only the tip of the iceberg”. “Given enough time and resources, computer vision could be integrated into most things that the human eye can perceive,” said Prof Loy.

He pointed out that by observing objects and how they move, a human being is capable of grasping the structure and some information on the object, and then generalising the knowledge to unseen samples.

In contrast, modern deep learning systems rely heavily on massive amounts of annotated data for learning effective representations. This means hundreds of thousands of hours spent on manual labelling for each percentage gain in accuracy.

“Can deep models learn meaningful visual representation without labelled data?”

His team is trying to answer this by developing new learning approaches for deep learning so that it can learn from a massive number of images and videos in an unsupervised manner, namely without explicit annotations.

As he continues on his research journey, Prof Loy feels that he has been most fortunate to be able to do research in a field that he is truly passionate in and to work with the right people. “I have been very lucky to have met the right people at the right time and at the right place (my collaborators, my postdocs, and my students). I am still working closely with many of these people, and I look forward to an even more exciting research expedition ahead.”

Software engineer Lim Xing Yi and chemical engineering graduate Ng Jian Ming seemed destined for different career paths when artificial intelligence (AI) beckoned, and the AI Apprenticeship Programme (AIAP®) gave them the opportunity to pivot. 

Lim Xing Yi was working as a digital business technologist with a local telco when he received an email from his university about the launch of the AIAP. His interest was stoked, but he had an upcoming internship placement in Shanghai and had already secured a job with the telco.

“I was keen on acquiring skills in data science and in particular AI, but I had other commitments at that point of time, so I had to forgo the opportunity,” he recalled.

But the thought persisted and in 2019, he decided to act on it. “I guess it was the strong and constant digital marketing presence from AIAP that prompted me to just go ahead and made the move,” he said.  

For Ng Jian Ming, his interest in AI was piqued when he was exposed to non-core modules such as Python Programming and Business Analytics as an undergraduate and had the chance to work with analytics during his internship. After graduating with an honours degree in Chemical Engineering in 2018, he decided to pursue a different path and signed up for AIAP.

Taking a deep dive into AI
Launched by AI Singapore (AISG) in August 2018, AIAP is a full-time structured training programme that aims to groom local AI talent and enhance their career opportunities in AI-related roles.

The nine-month programme equipped Jian Ming and Xing Yi with the technical and soft skills that they needed to embark on a career in AI.

For Jian Ming, this meant taking a deep dive into software engineering and best practices, and honing his communications skills to be able to translate technical AI jargon into terms that made it easier for laymen to understand.

For Xing Yi, important takeaways from the programme included time management and organisational skills which are crucial in a data science project.

“Building the machine learning pipeline requires logical flows with careful planning, otherwise more time and effort will be needed to rebuild the pipeline,” he said. “You are dealing not only with model training but also with data acquisition, model artefacts, the code repository, and documentations. These need to be well-organised for efficient development work and iterative training. These skills helped to save me time searching around for the things I needed.

Weekly sprints under the AIAP also gave him a sense of the typical AI project timeline and stakeholder management, and equipped him with knowledge to handle an AI project.

Working on the 100E project
As part of the 7-month on the job training on a real world AI problem, Xing Yi and Jian Ming worked on a 100 Experiments (100E) project with a medical technology start-up called EM2AI (part of Q&M Dental Group). 100E is the flagship programme by AISG to help companies develop solutions to AI problems where no commercial off-the-shelf solution exists.

The project undertaken by Xing Yi and Jian Ming was focused on helping dentists understand the dental health of the patient in a more efficient manner by detecting tooth pathologies such as decay and gum disease through the use of deep learning on dental X-ray images.

Over the course of the project, the apprentices encountered challenges such as pathologies that were less obvious and difficult to spot with the human eye, or those that were very similar in nature. With the increasing number of pathologies to detect, the speed of AI inferencing was also an issue that had to be addressed.

Our role is to ensure that the trained AI object detection models are comparable to less-experienced dentists in detecting the most common dental pathologies, and possibly many other complex and uncommon ones in the near future,” said Xing Yi.

After their apprenticeship, EM2AI went on to offer Xing Yi and Jian Ming full-time roles as AI engineers and they joined the company in June 2020 upon graduation to continue to build out the solution they were developing during the AIAP. This is indeed one of the “ideal marriage” outcomes of AIAP where well-trained Singaporean AI Engineers join their Singapore-based project sponsors to build out their AI capabilities.

A challenging and rewarding career
Five months into the “real AI world”, the work continues to be both challenging and rewarding for both graduates. “As new start-up, there was a lot of ground work to be done to set up the data science workflow, software engineering practices and agile methodologies,” said Xing Yi.

Both are working on the “upgraded version” of the 100E project as well as other AI initiatives in the company. “Given the full picture of the company’s system and future goals, the challenge now is not only to improve our AI system’s scalability but also to ensure an end-to-end AI system that is aligned with other systems in the clinic,” said Jian Ming.

Despite the new challenges, he finds the work rewarding and enjoyable. Citing the 100E project as an example, he said, “It is not only an innovative solution from a market perspective, but is also meaningful in that it has the potential to improve the quality of life for both patients and dentists.”

Both are grateful to AIAP for opening up these opportunities for them. “The AIAP is a good stepping stone to AI because it provides the chance to work on real-world problems through the 100E project,” said Xing Yi. “The opportunity is very hard to come by for someone with little experience in AI, and the experience helps convince people that we can implement and deploy an AI project.”

Echoing his view, Jian Ming said, “It would have been very difficult to enter this field without the apprenticeship programme. It was the key to help me make the transition and begin my career in the tech industry.”

For more details on the AIAP, please visit

If you are keen to know how to get into the AIAP, do check out this AIAP Field Guide