Project Description

Bamboo Pipe is an intelligent NLP pipeline interfaced by an elegant platform, developed for the purpose of analysing large volumes of articles to find the information you need. Not only does it increase the research efficiency, you’ll enjoy yourself the entire time too! This solution has been developed for the analysts at Datarama.

Company Background

Datarama is a business intelligence company specializing in regulatory compliance. Combining advanced technology and expert human analysis, analysts at Datarama conduct due diligence analysis on companies and provide clients with information about whether they conform to rules, laws, and standards in the business world.

Problem Statement

Datarama’s reliance on NexisDiligence as a news provider creates an undesirable dependency on external sources. NexisDiligence displays false positive results 50% of the time. Hence, analysts need to conduct additional manual checks that take up time that could have been better spent on more value-added tasks.

Additionally, articles currently stored in their database are unstructured and analysts face a great difficulty finding past articles to refer to. The process of entering information individually into their internal Ingestion Portal is repetitive and time-consuming. These issues make up the bulk of the difficulty that analysts face in the daily routine and we aim to solve these issues by streamlining the time consuming workflow.

How is it integrated

When analysts interact with the React frontend web application, it will send API requests to the backend server. The backend server will contact different services such as the MariaDB database, AWS S3 bucket, the Google scraper, as well as RabbitMQ message broker server. The message broker acts as the communication bridge between our NLP pipeline and the backend server. Documents are sent to the pipeline to be processed through the NLP models in parallel and results are collected and then sent back to the server.

Here is our user interface

Bamboo Pipe is a web application that streamlines the evaluation process of textual data and eases Datarama’s information ingestion process.

1. Login

Users can login using designated username and password

2. Upload & Search

Users can upload a CSV of URLs, or conduct Google search, or manually key in URL. Users can then define entity-sentiment pairs and topics they are interested in

3. Filter

Users can view the filtering progress on the dashboard. Finished results will be shown. A "tick" means a match with users' search parameters.

4. Filter Finished

Users can see the filtering process has finished. They can then proceed to view the output results.

5. Results

Users can see the highlighted entities with sentiment scores. Users can view the top-3 topics per paragraph identified. Paragraphs will be highlighted accordingly when users hover on the topic results.

6. Download

Users can download the processed results into CSV and download the articles uploaded in a ZIP file or individual PDF.

Advanced NLP Technology

Two Natural Language Processing (NLP) modules were developed to ease the article evaluation process.

Entity-Sentiment Analysis

Our Entity-Sentiment Analysis model identifies the named entity present in an article and detects the sentiment with respect to it. This analysis is conducted on a sentence level to detect news with negative sentiment surrounding the entity and hence relevant for the analysts.

Topic Modelling

Our Topic Modelling model identifies the topics present in an article so that analysts can easily understand the gist of an article.

Bamboo Pipe