Natural language processing (NLP) has been around for more than 50 years but has become more well-known recently, thanks to personal assistant applications such as Apple’s Siri and Amazon’s Alexa. It is also the driving force behind language translation applications such as Google Translate.

It has been no easy feat coming this far. Enabling computers to understand how humans naturally speak or type continues to be a complex undertaking.

Using machine learning algorithms, NLP extracts, analyses and infers useful information from large amounts of text. This information can then be used to make predictions, find hidden relations within data and detect anomalies.

Understanding the connections
There are many interesting problems in the field of NLP and historically various models have been proposed. But what is missing is a systematic, unified view of the algorithms that have been developed. And this is what Lu Wei, an Associate Professor with the Information Systems Technology and Design Pillar at SUTD, is focusing on in his research.

“A unified framework will allow us to understand the connections between many classic models in NLP, and enable researchers (such as my students) to build new models on top of it once they have mastered it,” said Prof Lu.

For example, the framework can be used to build models that can identify word spans in texts that refer to public figures or issues or products, and to detect the sentiments associated with them.  This can be used to analyse social media posts to find out what people feel about a certain issue or if they have an affinity towards a particular organisation.

For organisations and governments, this is one way to get a better understanding of the multifaceted needs of the community or market. 

Another AI model developed based on the framework leverages NLP techniques to teach primary school students how to solve arithmetic word problems. It takes questions which are written in natural language and parses them into mathematical formulas.

Industry collaboration
Prof Lu’s team has also partnered industry players such as Alibaba, Boeing and ByteDance to build new models for a wide spectrum of applications using the unified framework.

In one of the projects, the team collaborated with Alibaba on the task of Chinese address parsing. Unlike English addresses, Chinese addresses are typically written as a series of Chinese characters, often intermixed with digits and English letters. The Chinese address parsing model is able to encode regular patterns within chunks that appear at the beginning of a Chinese address, while flexibly capturing the irregular patterns and rich dependencies within chunks of different test types that appear towards the end of the address. This is achieved by designing a novel structured representation integrating both a linear structure and a latent-variable tree structure.

In a field now largely dominated by empirical deep learning approaches, Prof Lu strongly believes that it is important to understand the fundamental mathematical principles behind the various models proposed for NLP. “I hope the framework is one useful step towards what I would like to achieve,” he said.

For more information on Prof Lu and his research work, please click here.

Article has been adapted from ASPIRE, an SUTD research publication.