The Brain Is Not Probabilistic


The Double-Sided Nature of Big Data in Health Care

big dataThe explosion of big data in the health care industry is both a boon and concern. The race to quickly recover actionable insights from such data can lead to innovation and pitfalls, yet big data has the potential to reap rewards in assisting patients and improve operational and clinical processes.

When data assets are transformed into data insights, everyone wins — leading to lower health care costs, healthier patients, improved consumer and staff satisfaction and greater understanding and transparency in processes for all.

Big data’s actionable insights have the ability to create a culture of meaningful health care, yet there are many roadblocks to overcome on the way there. Provider organizers must take a closer look at how they collect, store, analyze and present data to patients, staff and business partners to avoid a data mess without any beneficial or relevant insights.

What is the most meaningful way to offer these insights? Here are the benefits and challenges providers face when it comes to these big data issues, with approaches to resolve these obstacles to cultivate a culture of meaningful health care and success in the long haul.

The Larger Impact of Data Collection

The gathering of population data typically drives big data statistics for medical research, yet medical care data is driven by information gathered on an individual level. Individual-level statistics must be made available in a way that is meaningful and accessible for all, particularly non-scientists.

Algorithms and interfaces must be created as graphical representations that offer an intuitive understanding, allowing people to make sense of the details of big data without needing irrelevant commands to access the information.

Information technology and big data have the potential to transform medicine. For example, medicine is becoming more elegant and precise to smaller groups and individuals, tailoring diagnoses and tests built on their unique genetic makeups with respect for personal autonomy through ethical analysis.

Artificial intelligence has the potential to fill in the gaps of data humans cannot collect and process in a timely manner. Predictive modeling optimizes how health care resources will be allocated to the community to meet the greatest needs.

Data Collection, Storage and Analysis Must Be Transparent

Data collection quickly becomes expensive, and providers must weigh the risks with the potential rewards of insightful innovation. In people’s everyday lives, there is a greater call for transparency on where and how medicines are sourced and why particular choices are made.

healthcare-big-dataThe danger of data collection to the public and provider comes when internal processes are not transparent — how a provider obtains, stores and analyzes its data must be clear to the consumer and staff. Otherwise, patient privacy is jeopardized and a provider faces security risks. Many hospitals, much like nonprofits, now share their data policies, findings and actions directly on their websites to be more accessible.

Health care providers should update IT infrastructures however possible. Unfortunately, electronic health care records are constructed to optimize how providers bill, not how they provide care. One of the biggest challenges providers face is streamlining data across infrastructures to work together and provide meaningful analysis.

Providers must ask themselves: “Are we delivering the right data?” Ideally, big data will be able to express the state of a patient in a meaningful way that shows more than what they owe, which is also a major source of frustration for patients. The variability of procedural total costs differ, sometimes drastically, and to fill the gap, many employers collaborate with third-party administrators or vendors to supply more transparent health care cost information to their staff.

Those in the front lines of health care cannot gain relevant information from patient claims, which are often months or even years outdated. Claims address billing, not vital clinical details that detail the care process. Here is an example of an opportunity to provide relevant and meaningful information that solves many frustrations for patients and clinicians.

Helpful Big Data Streamlining Solutions

Clinicians are understandably particular about operating room and clinic cleanliness, but what about the integrity of their data?

Most data cleaning takes place manually, but some third-party IT vendors provide automated tools that use logic rules to scrub data: comparing, contrasting and correcting large datasets. Increasingly, these tools are becoming more precise and sophisticated, ensuring accuracy and cleanliness in data collection.

One of the most powerful open-source data analysis platforms for big data is Hadoop. Originally developed to aggregate web search indexes and other routine functions, Hadoop has the potential to process large data amounts as an organizer and analyzer tool. Multiple vendors, such as Cloudera and AWS, distribute open-source software, and propriety options such as HBase and Cassandra are also available as database components.

Such cloud-based programs and open-source platforms help keep big data costs low for providers. However, many of these open-source tools need programming, and the issues of privacy and security must be addressed. The payoff comes in meaningful data that may be used to develop case protocols and care protocols, while gaining greater insight into the delivery of health care operations.

According to a report by McKinsey, if U.S. health care providers used big data creatively to drive quality and efficiency, it could generate more than $300 billion in value in the public sector. Two-thirds of this could reduce U.S. health care spending by roughly 8 percent. A retailer that used big data to its maximum potential could, in theory, boost its operating margin by more than 60 percent.

Using big data to revolutionize the health care industry is not about tossing technology at the problem and offering various numbers to consumers and relevant parties. Improving and integrating datasets can also benefit how medical researchers gather and use population data. For example, it will be easier to see which patients have missed vital screenings or are taking numerous medications with contraindications.

Big data has the potential to transform medicine in a meaningful way, and to do so, data must be shared and displayed in a way that is intuitive and meaningful for all, ultimately transforming confusion into understanding and knowledge that empowers, to the benefit of patients, staff and providers.

The post is by Nathan Sykes, a business writer with a passion for tech and IT. To see more posts by Nathan, you can read his blog, Finding An Outlet, or follow him on Twitter.

The post The Double-Sided Nature of Big Data in Health Care appeared first on Big Data Analytics News.

via Big Data Analytics News http://ift.tt/2BIyHTi

How AI Learns What You’re Willing to Pay

By Ted Gaubert.

Why are we paying different prices? Is it ‘price personalization’ or a form of 'price discrimination'? The answer isn't so simple.

The world of Artificial Intelligence (AI) dynamic pricing engines is rapidly progressing and changing the competitive landscape. This article provides an overview of a few areas that influence how an AI pricing engine decides the price to show you:

  1. Predicting market demand & micro-segmentation
  2. Accumulating asymmetric information
  3. Estimating ‘Willingness to Pay’
  4. Shaping demand

1. Predicting Market Demand and Micro-Segmentation

AI micro-segmentation uses many customer attributes and behaviors to bucket customers by estimated willingness to pay. To explain in simple terms, let’s assume we have three buckets. We segment our customers by (A) high paying customers, (B) medium paying customers and (C) low paying customers. One strategy to maximize profits would be to first sell only to the high paying customer group A. Then any remaining seats could be sold to medium paying group B. Lastly, any leftovers could be sold to low paying group C.

Wait a second! We all know airline prices go up as the booking date gets closer to the departure date. Everyone knows to get a great price on an airline ticket you should book early!

So, it’s not as easy as first selling to high paying group A, then medium paying group B and giving leftovers to low paying group C. In most cases the pricing and selling happens in the reverse order – Low paying group C, then medium group B and lastly high paying group A.

How then are sales to the high-paying group A maximized if selling to group C happens first? What prevents all the seats from being sold to low paying group C with no seats being left to sell to the high paying group A?

Part of the answer is predicting the number of potential buyers and how much each of them is willing to pay weeks in advance. Often before most of the customers have even decided to travel!

2. Accumulating Asymmetric Information

A big part of the AI pricing game is having AI learn everything about what is happening in the market. The goal is to have better information than competitors in order to make better decisions. This information advantage is sometimes referred to as asymmetric information.

In terms of demand prediction, asymmetric information allows the AI pricing engine to achieve a more accurate demand prediction than competitors. Ultimately, this advantage results in greater confidence by the pricing engine to hold a price or move it up or down to maximize profit in response to what is happening in the market.

To see how this works, let’s take a hypothetical airline market with 3 airlines serving a destination like Porcupi, Montana. In the airline industry, Porcupi is a small town that sits in the ‘long tail’. This means it is just one of many towns and cities served where each generates only a small amount of revenue. However, like the classic example of Amazon.com, the sum of everything in the ‘long tail’ adds up to be a massive revenue number.

Suppose you are an airline operator and you know a big festival will soon take place in Porcupi. You know that significantly more people will be going to Porcupi than the number of available airplane seats across all the competitors. If you are the only airline operator who knows about the big festival, then the pricing strategy is easy. Hold the price high until all the competitors have sold all their seats. Then, travelers will have to pay a high price for a seat on the last plane into Porcupi.

However, most of the 'long tail' markets don't generate enough revenue to economically justify hiring a human to monitor what's happening in a small city and then make micro-adjustments to prices. Similar to how Amazon leverages technology to autonomously make demand prediction and pricing decisions across half a billion products, the same is happening across many other industries.

AI can learn about local events that are happening in real time on a global basis far more economically than what could ever be achieved by a group of humans. This enables a company leveraging AI to gather asymmetric information which enables better demand predictions and strategic pricing decisions.

3. Estimating Willingness To Pay

Data is constantly being collected about your customer behavior such as:

  • What type of items did you look at?
  • How long did you spend on each web page?
  • What items did you put in your basket?
  • What items did you purchase?
  • What do people pay that look and behave like you?

All this data and more gets fed into an AI engine that translates your behavior into a persona and tries to predict things about you, one of them being estimating the ‘maximum price’ you are willing to pay.

Keep in mind, this doesn't imply you will receive a 'personalized price' even though it is technically possible. The practice of 'personalized pricing' is highly debated for numerous reasons including ethical, brand, loyalty and legal concerns.

However, 'willingness to pay' can be used to determine how likely you will purchase an item at the current market price. This likelihood gets incorporated into demand predictions by micro-segment and, ultimately, the price. Consequently, the AI engine can control sales velocity by knowing how much to sell at what price.

4. Shaping Demand Dynamically

Predicting competitive response + demand + micro-segmentation + ‘maximum price willing to pay’ are all based on probabilities. There will always be some level of ‘error’ in the predictions. In other words, things may sell a little faster or slower than expected. AI pricing engines use dynamic demand shaping to change the shape of the demand curve by adjusting prices. This could be based on real-time inventory or any other myriad of factors. In the airline and hotel pricing world, demand shaping can be used to optimize profit and minimize overselling or underselling airline seats and hotel rooms.


Every industry is selling a product or service to a customer at some price. The topics discussed in this article are broadly applicable to a wide range of industries and business scenarios outside just dynamic pricing of airline seats and hotel rooms. Manufacturers estimate demand to know which products to produce in what quantities. Distributors manage pricing and demand forecasts to optimize inventory and distribution logistics. Marketers use demand estimations to make decisions of promotions, targeting and marketing spend.  Advanced algorithmic techniques and AI engines are quietly transforming how organizations compete.

The result of all the complex algorithmic interplay determines the price we are quoted, the advertisements we are shown and the product mix we find when shopping.


Opinions are my own. My articles are written for a broad global audience to share and to inspire others to explore new ideas.  I simplify for clarity and welcome your comments, corrections and critique in a way that enriches the knowledge and lives of our professional LinkedIn community. Thanks! 😊

Original. Reposted with permission.


via KDnuggets http://ift.tt/2E6IfZ4