BIORBIT LLC

Data Processing

Our approach to data structure and analytics are different than traditional information architectures.
A traditional data warehouse approach expects the data to undergo standardized ETL processes and eventually map into pre-defined schema, also known as “schema on write”.
A criticism of the traditional approach is the lengthy process to make changes to the pre-defined schema. One aspect of the appeal of Big Data is that the data can be captured without requiring a ‘defined’ data structure. Rather, the structure will be derived either from the data itself or through other algorithmic process, also known as “schema on read”.

A Data Lake is a storage repository that holds a vast amount of raw data in its native format until it is needed. While a hierarchical data warehouse stores data in files or folders, a Data Lake uses a flat architecture to store data.
Each data element in a lake is assigned a unique identifier and tagged with a set of extended metadata tags. When a business question arises, the data lake can be queried for relevant data, and that smaller set of data can then be analyzed to help answer the question.
The term Data Lake is often associated with Hadoop oriented object storage. In such a scenario, an organization’s data is first loaded into the Hadoop platform, and then business analytics and data mining tools are applied to the data where it resides on Hadoop’s cluster nodes of commodity computers.

TIBCO’s SpotFire is an analytics and BI platform for analysis of data predictive and complex statistics.
It allows us to create, manipulate, and deploy rich analytical visualizations, learning about our data and letting us get to insights quicker – regardless of whether we are an experienced analyst or a new user to visual analytics. The Big Data figure above represents the implementation of Big Data with the combination of Data Lake, Hadoop and SpotFire. It uses NoSQL columnar database to store the data.

As part of introducing the big data platform for few of our clients, We recommended taking iterative approach to initiate this modern data platform.
Set up Big Data platform in cloud environment  Execute the Big Data POC for a specific financial use case Potential use case to considered for doing the big data POC is for Audit Analytics.

Audit analytics is an analytical process by which insights are extracted from operational, financial, and other forms of electronic data internal or external to the organization.
These insights can be historical, real-time, or predictive and can also be risk-focused (e.g., controls effectiveness, fraud, waste, abuse, policy/regulatory noncompliance) or performance-focused (e.g., increased sales, decreased costs, improved profitability) and frequently provide the “how?” and “why?” answers.

Predictive analytics is a form of advanced analytics that uses both new and historical data to forecast activity, behavior and trends. Big data platform enables building effective predictive analytics solution by analyzing all required data from a single data lake repository.
It involves applying statistical analysis techniques, analytical queries and automated machine learning algorithms to data sets to create predictive models that place a numerical value, or score, on the likelihood of a particular event happening.