Big Data

Big data can be stored, acquired, processed, and analyzed in many ways. Every big data source has different characteristics, including the frequency, volume, velocity, type, and veracity of the data. When big data is processed and stored, additional dimensions come into play, such as governance, security, and policies. Choosing an architecture and building an appropriate big data solution is challenging because so many factors have to be considered.

This “Big data architecture and patterns” series presents a structured and pattern-based approach to simplify the task of defining an overall big data architecture. Because it is important to assess whether a business scenario is a big data problem, we include pointers to help determine which business problems are good candidates for big data solutions.

If you’ve spent any time investigating big data solutions, you know it’s no simple task. This series takes you through the major steps involved in finding the big data solution that meets your needs.

We begin by looking at types of data described by the term “big data.” To simplify the complexity of big data types, we classify big data according to various parameters and provide a logical architecture for the layers and high-level components involved in any big data solution. Next, we propose a structure for classifying big data business problems by defining atomic and composite classification patterns. These patterns help determine the appropriate solution pattern to apply. We include sample business problems from various industries. And finally, for every component and pattern, we present the products that offer the relevant function.

Part 1 explains how to classify big data. Additional articles in this series cover the following topics:

  • Defining a logical architecture of the layers and components of a big data solution
  • Understanding atomic patterns for big data solutions
  • Understanding composite (or mixed) patterns to use for big data solutions
  • Choosing a solution pattern for a big data solution
  • Determining the viability of a business problem for a big data solution
  • Selecting the right products to implement a big data solution

Big data business problems by type

Business problem Big data type Description
Utilities: Predict power consumption Machine-generated data Utility companies have rolled out smart meters to measure the consumption of water, gas, and electricity at regular intervals of one hour or less. These smart meters generate huge volumes of interval data that needs to be analyzed.Utilities also run big, expensive, and complicated systems to generate power. Each grid includes sophisticated sensors that monitor voltage, current, frequency, and?other important operating characteristics.

To gain operating efficiency, the company must monitor the data delivered by the sensor. A big data solution can analyze power generation (supply) and power consumption (demand) data using smart meters.

Telecommunications: Customer churn analytics Web and social dataTransaction data Telecommunications operators need to build detailed customer churn models that include social media and transaction data, such as CDRs, to keep up with the competition.The value of the churn models depends on the quality of customer attributes (customer master data such as date of birth, gender, location, and income) and the social behavior of customers.

Telecommunications providers who implement a predictive analytics strategy can manage and predict churn by analyzing the calling patterns of subscribers.

Marketing: Sentiment analysis Web and social data Marketing departments use Twitter feeds to conduct sentiment analysis to determine what users are saying about the company and its products or services, especially after a new product or release is launched.Customer sentiment must be integrated with customer profile data to derive meaningful results. Customer feedback may vary according to customer demographics.
Customer service: Call monitoring Human-generated IT departments are turning to big data solutions to analyze application logs to gain insight that can improve system performance. Log files from various application vendors are in different formats; they must be standardized before IT departments can use them.
Retail: Personalized messaging based on facial recognition and social media Web and social dataBiometrics Retailers can use facial recognition technology in combination with a photo from social media to make personalized offers to customers based on buying behavior and location.This capability could have a tremendous impact on retailers? loyalty programs, but it has serious privacy ramifications. Retailers would need to make the appropriate privacy disclosures before implementing these applications.
Retail and marketing: Mobile data and location-based targeting Machine-generated dataTransaction data Retailers can target customers with specific promotions and coupons based location data. Solutions are typically designed to detect a user’s location upon entry to a store or through GPS.Location data combined with customer preference data from social networks enable retailers to target online and in-store marketing campaigns based on buying history. Notifications are delivered through mobile applications, SMS, and email.
FSS, Healthcare: Fraud detection Machine-generated dataTransaction data


Fraud management predicts the likelihood that a given transaction or customer account is experiencing fraud. Solutions analyze transactions in real time and generate recommendations for immediate action, which is critical to stopping third-party fraud, first-party fraud, and deliberate misuse of account privileges.Solutions are typically designed to detect and prevent myriad fraud and risk types across multiple industries, including:

  • Credit and debit payment card fraud
  • Deposit account fraud
  • Technical fraud
  • Bad debt
  • Healthcare fraud
  • Medicaid and Medicare fraud
  • Property and casualty insurance fraud
  • Worker compensation fraud
  • Insurance fraud
  • Telecommunications fraud

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s