We’re often asked about how Rainbird can work alongside Natural Language Processing technologies.  In this article we’ll answer some of the top questions.

What is Natural Language Processing?

Natural Language Processing (NLP), and its cousins Natural Language Understanding (NLU) and Natural Language Generation (NLG), are sets of technologies and techniques that allow computers to process and interact with natural languages.  When we talk about natural languages, we mean the human languages that we use every day to communicate with one another. 

Natural languages are typically fluid, expressive, and very ambiguous. Consider the English sentence “John saw Mary with the telescope”.  Although that’s a perfectly valid sentence it’s not at all clear who has the telescope.

Computer languages on the other hand are rigid, well defined, and unambiguous. They allow us to be explicit in how we set tasks, request information, and process data.

NLP technologies provide a bridge between the ambiguous world of human interactions and the formal definitions and structures computers need.  It’s important to recognise that NLP is no one thing, rather it’s a set of techniques and technologies we might apply to helping bridge that gap.

While many NLP technologies are now largely commoditised, NLP remains a very active research area in the AI comunity with new advances being made regularly.  The techniques involved in NLP vary widely, from structured text analysis to statistics and Machine Learning.  Some approaches to NLP may be familiar to you if you’ve used Rainbird.  This is because, although Rainbird is not an NLP technology, the way we structure expertise in knowledge maps has part of its basis in linguistic theory.

Some examples of NLP techniques include:

  • Named Entity Recognition - a technique to find and classify entities in a sentence. In the sentence “I live in London” we might identify the word London and classify it as a city.
  • Part of Speech tagging – marking a word in a sentence to identify its role, taking into account its position and context.  This might be as simple as finding verbs in a sentence, or more complex such as discovering the subject and objects of the text.
  • Intent recognition – classifying a sentence as one of a set predefined actions we expect the user to be requesting.
  • Sentiment Analysis – analysing text to discover its emotional context. For instance, we might want to know that “Rainbird is fantastic” is a positive statement.

How is NLP used?

There are many examples of NLP being used at large scale. Alongside enterprise applications you’re likely to come across examples of NLP being used frequently in your everyday life. 

  • When you use a search engine, or search in an online store, NLP techniques are used to help find relevant articles or products.
  • Most of us will be used to using automated customer service chat bots, and digital assistants that make our interactions with enterprise organisation smoother and quicker.
  • NLP can be used to apply structure to legal documents, which we can then ask questions of and reason about.
  • NLP techniques allow us to process unstructured text at a volume that would be impossible for humans alone to perform.  For instance, NLP is used by organisations to screen the huge volume of media output online for negative press coverage.

How can Rainbird and NLP technologies work together?

At its heart NLP is about applying structure and meaning to unstructured data. Rainbird allows us to build models that reason over structured data.  What’s more Rainbird can be used to create intelligent conversations with human users without the need to pre-script the interaction.  As a result, NLP technologies and Rainbird together are a powerful combination. There are several use cases where these technologies can intersect well. 

NLP techniques can allow us to build solutions that start with a human describing their problem in natural language, then have Rainbird reason and ask questions to deliver a solution. 

For example, imagine a banking application that supports customers on account queries.  The customer might enter: “Yesterday I setup a recurring transaction, although no payments have been taken, I’d like to cancel it”.  We can use some NLP technology to recognise the intent as “cancel a recurring transaction”, we can also know the timeline was yesterday and there are no payments.  By injecting these discovered facts into Rainbird and forming the appropriate query Rainbird will come back with the perfect and most relevant questions, before proving the right answer.  This creates an extremely satisfactory user journey.

Beyond improving user interactions NLP can be used to provide structured data sources to Rainbird from unstructured data.  If you’ve used Rainbird you may have come across our “Hello World” demo.  In that knowledge map we need to know about countries and their national languages.  We could point some NLP technology at the Wikipedia article on countries and their languages, and use the results as a data source for Rainbird.