Visualizing Advanced Analytics

Advanced Analytics with Aster

I recently stumbled upon Teradata’s Aster and I’m pretty fired up. It turns out there is an entire community dedicated to helping data visualization people like myself learn how to implement advanced analytic functions. The site includes a link to download Aster Express free of charge and includes a slew of great training videos.

Click here to see the Teradata Aster Community

I can almost hear the Data Scientists reading this post laughing at me for just discovering that. Meanwhile all of the Data Visualization people stopped reading and have already clicked the link and started downloading.

Visualization with Qlik Sense

Well if you Data Scientists are so cool did you know that there is likewise an online community site dedicated to helping you learn how to visualize your super cool analytic results? Well did you? The Qlik Sense Community offers similar free downloads for the product as well a slew of great training videos.

Click here to see the Qlik Sense Community

Guess me and the other Data Visualization peeps get the last laugh after all.

Kidding, and sharing of links aside, this is a serious post about how Data Science and Data Visualization can be married through the partnership of Qlik Sense and Teradata Aster. They are an easy and natural fit. Why?

Because Aster uses an SQL’ish syntax they call SQL-MR. Qlik Sense can easily fire any native SQL-MR directly against Aster, retrieve the results and then visualize them. No need to build out views. No need to save the results into tables. Simply fire the SQL-MR queries directly as written.

By offering a complete set of Open API’s Qlik Sense provides developers around the world the ability to construct visualizations to enhance what is available natively in the product. Like what you ask? Well a Sankey for one thing so you can visualize paths. Network/Graphing objects for another so you can visualize networks. Like … oh go see for yourself at:

Click here to see the Qlik Sense Community for Extensions

For your viewing pleasure

I could write and write and write and bore you to tears … or … I could take advantage of this chance to show of my cool new Qlik Dork video stinger and demonstrate the functionality … visually.

In a mere 3:57  I take the pure NPath SQL-MR query that John Thuma demonstrated in the Aster training video series for bank web clicks data and I implement it inside of Qlik Sense. I then take the results and display them in the raw form and using a Sankey.

Wowed yet? Don’t be that’s just me getting warmed up. In a paltry 3:05 this second video demonstrates how you can modify the NPath query so that the results aren’t aggregated. Why wouldn’t I allow it to aggregate the million plus paths? So that I can tie the raw paths together with customer demographics information. Allowing you to then discover the paths for selected cohorts. No way!!!

Yes way. C’mon I’m the Qlik Dork of course I would go the extra step for you. I even utilize a mapping object to select customers from selected states. All while the Sankey diagram is being updated to show the paths that were returned from Aster based on the selections.

But wait! There’s more.

I know you are now fired up and you want more. Don’t worry my friends I’m just getting started down this path of marrying Data Science and Data Visualization. What can you expect next? Keep it a secret but given my background in healthcare it may just have something to do with utilizing an NPath SQL-MR query in Aster to analyze the events for surgical patients but you didn’t hear it from me. After all it’s not like I’m trying to actually help people do real world stuff like that.

SurgicalSankey

Posted in Visualization | Tagged , , , | 2 Comments

To achieve, or not to achieve action

Portrait of William Shakespeare

Portrait of William Shakespeare

That is the question.

At least it’s the question that we in the business intelligence community should be focusing on. Why weave my title so closely to one of the most famous lines by William Shakespeare?

Simple. Our ability to drive actionable intelligence relies heavily on our ability to weave a story around the data insights that we have discovered.

Discovering that we have 10 serious issues in our company and having $5 in your pocket will get you a cup of coffee at Starbucks. But being able to share the information about even 1 of those issues in a way that leads to actual change will put such a spring in your step that coffee will be unneeded.

In her fantastic book “Storytelling with Data” author Cole Nussbaumer Knaflic introduces two great phrases which really brought about great clarification to me. Exploratory Analysis vs Explanatory Analysis.

Exploratory Analysis are the actions that we take to do data discovery. It’s the drilling around. Poking under the hood. Using our human intuition to question the data. And the lights that dawn as a result.

Explanatory Analysis on the other hand is the art of being able to use the data to communicate a story that helps induce actions from those that have the power to make them. It involves our ability to use one of the oldest forms of human communication, storytelling, that has sadly become a lost art.

Emotional Call to Action

Storytelling can involve some very in your face kind of messages as a way to ensure that leadership has a call to action. For example imagine that we’ve spent a few days consuming clinical and financial data using a dashboard similar to the following that has multiple linked screens that we utilized to find an issue with a particular set of selections.

Dashboard

We could hold a meeting and put leadership to sleep showing them how cool our ability to navigate is or we can simply lead with a slide like the following that grabs attention.

BabySlide

You probably don’t want to use humorous sarcasm in your presentation to point the finger at a group but I think it works for this post as you kind of expect it from me. The slide includes enough details to insight some action and by all means include the actions you want to see taken. Of course you may have to prove your details and that’s exactly why the Storytelling feature in Qlik Sense is so valuable you can jump in and out of your story to do demonstrate the exploratory analysis you have done to support the explanatory analysis you are using in the meeting.

Narration

Perhaps your data doesn’t really require such an emotional tug to ensure action is taken. Perhaps all you are trying to do is provide some narration to help draw attention to help explain the data.

Consider the following chart before and after adding a few narrative elements are added to help the audience focus on the important things:

ChartWithNoNarration

NarrativePage

 

Openness

As I share on my About page I am far from an expert on any of the things I write about. I’m reading. Learning. Growing. Every single day just like you with the help of many others in the industry. Data is my thing and I own that. But I will be honest and tell you that providing narration for my stories is not something that comes naturally to me.

In fact the key points above … yeah I stole them. Well not actually stole them so much as I copied them to the clipboard and pasted them into my storyboard from what I think is one of the coolest new elements of technology that I’ve seen in a long time. It’s a narration extension for Qlik Sense that you simply tell which chart you want it to consider and it does the narration for you. That is a serious help to someone like me who is trying to learn how to help my audience understand the data that I’m presenting to them.

The fact that Qlik chose to construct it’s architecture using an Open API and the fact that anyone who can code can gain access to the patented Qlik technology while adding value through their secret sauce is what makes it possible for a group like Narrative Science  who is blazing trails in the field of natural language to build such an awesome extension.

The following video will let you see the narrative science extension in action. If you are a Qlik customer you will get all of the instructions you need and can download this exciting new object from this download location that includes instructions on how to install and has it’s own video that demonstrates it’s powerful capabilities. .

To achieve, or not to achieve action

There was a day when all we had to do in our field was surface data. Yeah those days have long since passed. Our jobs now entail not only finding the needles in the data hay stacks but helping our leadership teams understand them so that they can take action. I challenge you today to grow not only in the field of Exploratory Analysis but also in the emerging field of Explanatory Analysis.

Become a storyteller.

Add narration to your charts rather than just pasting them into presentations because you think they look pretty.

Use your newly developed skills to “incite action” and effect real change in your organizations.

Finally quit being selfish and keeping my tips to yourselves. For crying out loud start sharing these pages with others.

Posted in Uncategorized | Tagged , , , , , | 2 Comments

A Bunch of Whiny Brats

Ever have one of those days where you feel like you are surrounded by a bunch of whiny brats?

No I’m not talking about your children (or grand children in my case.) I’m talking about your leadership team.

You beat your head against the wall to surface data from a cocktail napkin and merge it with 147 other data sources from database systems, Excel sheets and external data sources on the web and you make it work. You put all of the data into an amazing analytical application that is truly Functional Art that even Alberto Cairo would give you two thumbs up for. But without even so much as a pat on the back for the great job the first response is “We want something simpler. We already have Executive Portal can’t you just embed those charts into the site we are already have a link to?”

A bunch of whiny brats right. It’s just one more link to save to your favorites. It’s just one more application to learn. But noooooo they want to press the easy button because unlike you that has to learn 189 things per day to stay current they don’t want to change their delicate little processes.

Embedded Analytics

Well don’t be dismayed my friend there are whiny brats like that all over the world and the Qlik platform enables you to support them. I’m not joking. The Qlik API’s enable you to take the gorgeous work you’ve done and embed the KPI’s or charts directly into your existing portal and this quick 6 and a half minute video I show you exactly how to do that.

Ok now how could anyone could complain about this right? You can embed your genius analytical solutions right into the portal they use every day. You can embed Finance related data right into their Sharepoint page and it relates and allows interaction.

C’mon even your leadership team has to stand back in awe. Amazed at your skill and the innovation of Qlik’s platform to support that kind of functionality. Right?

Wrong! These are whiny little brats you are dealing with. Their first reactions are “That’s pretty nice but I don’t want to see the same 5 charts that Bob sees. I need to control my own dashboard because I’m the center of my universe.”

Are you kidding me??? They have access to key information on their mobile device from their executive portal and that isn’t enough?

No it’s not enough.

The reality is that your leadership team aren’t whiny little brats they are saavy business people who need to constantly push the threshold. They need access to the company data that has been kept from them for years. For crying out loud their mothers use Pinetrest everyday to “pin” recipes and come back to them whenever they want. Yet there you stand telling them that every time they want something added/removed from the portal they have to fill out a ticket request and wait for you to be the bottleneck in their accessing the information they need to do their job?

Self Service Dashboards

C’mon this is Qlik we are talking about. A company named by Forbes as one of the Top 10 Innovative Growth companies. Of course they can provide Self Service Dashboard capabilities. What do you think they are doing just helping you visualize data on your own workstation?

How simple can they make it? You know that Pinetrest site that has had “pins” pressed over 50 Billion times … yeah … they’ve made it that simple and in this short 4 minute video I’ve made it that simple for you to see how.

An Innovative Platform

“There are no dreams to large, no innovation unimaginable and no frontier’s beyond our reach.” – John S Herrington.

“There’s wa way to do it better – find it.” – Thomas Edison

Unless your leaders can consume it your companies data is not an asset it is a very expensive liability. Qlik is providing you a platform that allows only your mind to limit how you surface it. You have right now at your disposal the tools to surface your data via embedded analytics on your existing portals as well as allowing your staff to surface only the data they are actually interested in via their own personalized dashboards by simply “pinning” objects.

Just building data visualizations isn’t the answer. Presenting Actionable Intelligence in a way that can be consumed and acted upon is the goal. Now that you know what’s available it’s just a matter of whether you want to innovate the way data is consumed within your company or not.

Posted in Self Service, User Adoption | Tagged , | 4 Comments

Visualizing How to Improve

Besides helping customers by day and being an all around Qlik Dork at other times I happen to have a very strong passion for helping fastpitch softball players elevate their game. When I say elevate the game I mean getting over their greatest fears so that they can play the game like they OWN IT.

I have zero interest in spending hours of my life working with players on how to improve the minutia of their game (foot work for a double play, where to go to receive a cutoff, etc), that’s where their coaches and hours and hours and hours of practice come in. What I teach them to do is dive. Head first. All out. No fear. Diving aggressively with no fear. The change in every aspect of their game is so astronomically improved once they overcome that fear the rest of their game falls into place.  Click this link and watch the intro to one of my instructional videos to see what all out speed and a lack of fear looks like exploding through the air

You are still reading because you know me well enough by now to realize that there is a solid point to why I brought up what I do in softball. If you are going to set goals to improve it seems only reasonable that you figure out how to make the biggest impact with your time. Whether it is in the lives of young ball players, whether it is with your own actions or whether you are trying to improve quality at a health facility to help improve the health of your patients.

Clinical Quality Measurements

I recCompliancePercentageently had the pleasure of working with a large health system who wanted to focus on analysis of their Clinical Quality Measurement data. To set the stage they had 62.5 million quality measurement records covering 35 different measures across 8 systems and involving 511 practice groups and covering 2,241 providers.

Naturally we needed to illustrate some “dashboardy” type deal to reflect their starting point. They happened to be at 56.01%. Is that good or bad?

That my friends is a trick question. Starting points are neither good nor bad they are simply starting points. So as you consider your improvement efforts don’t judge yourself based on some myth in your head of where you should have been simply measure and report where you are. Then we look at going forward.

The next logical step of course is to begin analyzing the data. In a traditional report driven world we would ask for some details based on the different Compliancequality measures. So we did that. We created a very simple chart that showed the name of the quality measure, the # of members involved (patients), the number of quality measurements taken for the measure as well as the % of the measurements that were compliant.

You know the typical stuff that emulates what you could get out of any $9.95 report writing tool. Then we sorted the chart in order of the % of records that were compliant. It’s where they were.

Naturally we also added the ability to change the dimension (Measure Name) to System, Practice or Provider so we could see the same details as we drilled in.

Visualizing How to Improve

The purpose of the project was to improve their compliance percentages as an organization. So here is where my opening point comes in. What should they spend their time on? Who should they speak to?

The natural inclination for folks is to start with the worst on the chart and go from there. The compliance for “Seizure – New Onset” was at 30.18%. Again neither good nor bad, just where it was. I said the natural inclination is to start there, but that would be wrong.  My friends the biggest bang for the buck isn’t to charge down the halls trying to improve the compliance of a measurement that only has 45,255 out of the 62.5 Million overall records. So if we’re going to help them determine how to best spend the time of their valuable human resources I better create a visualization that actually does that.

Visualizing what Isn’t Right

The visualization that I believes help most with where to spend time is a Pareto Chart. Instead of looking at compliance percentages a Pareto Chart does the opposite it looks at what isn’t compliant. More to the point a Pareto Chart looks at each Quality Measure (or any dimension) and looks into how many non-compliant measurements it has versus all of the non-compliant measurements in the entire data set.

It also visualizes the cumulative effect as you proceed through the list. Sometimes graphics in posts are simply to add a little excitement like the girl diving, but in this case a picture is needed to really understand the tremendous impact of what a Pareto Chart can do for you.

In the image below you will see that “Colorectal Cancer Screening” by itself as a measure makes up 37% of all of the non compliant measurements. Why? If you look at the detail chart above closely it has over 15 million measurements for it’s members. It’s the biggest piece of the pie by a long shot. Followed by “Cervical Cancer Screening” and then “Diabetes Management.” The red line indicates the cumulative affect and you’ll see that if you focused your time on simply the top 6 of the 35 measurements you would be effecting a cumulative 80% of all measurements.

Pareto

How did you do that?

That Pareto dealio is pretty powerful isn’t it? Kind of illustrates how to improve in a slick and easy method so I know you want to jump right into your systems and add it so the question you may be asking is “How did you do that?”

I start be creating a combo chart. The bars simply represent the total number of items that are not compliant divided by the overall total of non compliant items which is handled using simple SUM functions and the wonderful key word “TOTAL” that tells the system to ignore the dimension that the current row may represent. (The IsNotCompliant field is simply a bit field with a 0 or 1 value indicating if the measure was not compliant or not)

 SUM(IsNotCompliant)  / SUM(TOTAL IsNotCompliant)

The cumulative line is simply the exact same expression and the clicking of the radio button that says “Full Accumulation.”

IFullAccumulationt’s really that easy.

It’s really that powerful.

The question now is simply “Where can you use a Pareto Chart to help your organizations ”

Meaningful Use? Absolutely!

CPOE? Absolutely!

What I love about this profession is that we have the tools that make visualizing how to improve so easy. What would take weeks in an old fashioned report writing and hours and hours of old fashioned human analytic skills can be created in a single chart that instantly identifies where people need to spend their time if they want to improve the numbers and not just measure the numbers.

Posted in Visualization | 1 Comment

Avoiding a Data Tornado

tornadoYou know I love to go out on a limb using data metaphors. Sometimes they are my own and sometimes I flat out steal them from others. (Imitation is the sincerest form of flattery you know.) I’ve wanted to continue my series on The Data Consumption Continuum for a few weeks now. But just writing my thoughts? That’s crazy. I’ve had to show great patience in waiting for just the right metaphor to come along to catch your attention and draw you in.  The “what in the world is Qlik Dork up to now” kind of lead. Recently inspiration struck as I came across this beautiful data metaphor “Data Tornado” from Tyler Bell.

In his post “Big Data: An opportunity in search of a metaphor” he introduces the concept as one of the major thought processes that surrounds data consumption in this great big data world we now find ourselves. He frames data as a problem of near-biblical scale, with subtle undertones of assured disaster if proper and timely preparations are not considered. (Don’t worry it’s not all doom and gloom he also introduces several positive metaphors but hey read those on your own time I’m trying to make a point here.)

We are at an age in the history of information where many analysts and businesses are begging for Self Service. Screaming if you will at IT “Just give me access to the data it belongs to the company I’m tired of waiting for you to write a report.” They are savvy and they know full well that the data is just sitting in a database or on a file share somewhere so why can’t they have access to it?

So why doesn’t IT want to just turn over the data and stop listening to the griping? Because the IT leadership team is worried about the Data Tornado that will ensue from all of these yahoos just randomly grabbing data and reporting 18 versions of the truth. You wondered where I was going with it didn’t you? And who can really blame them. You immediately understood the term “18 versions of the truth” because you’ve been burned by it in the past … multiple times.

DataFluencyYou can’t get any more succinct than Zach and Chris Gemignani in their book “Data Fluency” — “You can’t dump data into an organization and expect it to be useful. Creating value from data is a complex puzzle; one that few organizations have solved.” The answer to why not is found partly in another of their excerpts “The goal of a data fluent culture, in part, is to ensure that everyone knows what is meant by a term like customer satisfaction. A data fluency culture breaks down when people spend more time debating terminology, calculations, and validity of data sources rather than discussing what action to take based on the results.”

Enter Governed Self Service

Rest easy my friend. My post isn’t about the wide spread panic currently surrounding “self service” and that terms association with a “data tornado.” It’s about how to AVOID it. It’s about a new phrase you should repeat to yourself in the mirror a few dozen times until you begin believing your own facial expressions when you say it “Governed Self Service.”

The word “governed” seems to have negative connotations by many and those thoughts need to change. It doesn’t (have to) mean that IT is restricting you from accessing data. It can and should mean that IT is adding value to the data to ensure that the right data is used by the right people at the right times. They don’t want to be storm chasers or fire fighters dealing with the carnage after a data tornado has struck. Data Governance is a way for them to prevent the tornado in the first place by ensuring that you fully understand what you are surfacing.

Enter Qlik Sense

Self Service is a technology agnostic term. Many high quality tools are in the market that allow you to display data. Qlik Sense goes beyond the ability to display data and allows you to build in the governance that is so desperately needed to avoid data tornadoes and satisfy the well phrased concerns needed for a truly data fluent organization through the use of pre-defined Dimensions and Measures.

Imagine that we have a set of data that surrounds customers and the analyst needs to display a count of the customers. Easy enough … after we define what the term “customer count” means. If we are just looking at table that has customer demographics the count is obvious. But what if we are looking at a table of data that is all of the customer orders. Is the count the literal count that 100 customer (orders) were placed or should we display the unique count of customers so that we know we only had 76 different customers that placed those 100 orders?

Dimensions and Measures allow IT to build a framework of understanding to help analysts surface data in a way that avoids confusion. This screen shot illustrates how much metadata IT can add to a measure that can be used by an analyst in a way that ensures they use something as simple as a count correctly. You will see that the measure can contain a name, it shows the expression, it contains a description and holy cow it can even have tags associated that analysts can search for desired measures in a world where there might be thousands.

Measure

Enter Architeqt

As I’ve literally crisscrossed the country this year presenting to potential (and existing) Qlik customers they love this concept. But many in IT have begged for even more governance. “Dalton that’s great but Dimensions and Measures are only defined within single applications. What happens if we make changes? How can we apply changes across all of the applications? What if we need to add more as we develop more sources of data? After 30 years in the IT trenches I can do nothing but whole heartedly agree with them because maintenance is one of those things that IT considers but many analysts don’t.

No problem because that’s where Architeqt comes in. Architeqt is the framework for providing serious data governance across all of your Qlik Sense applications and is the brain child of Alexander Karlsson. It provides you the ability to create what he calls “Blueprints” which are the dimensions/measures/visuals that you need to share across all of your applications and then … oh this is so cool … use those blueprints in any of your Qlik Sense applications. And keep them in synch when you make changes.

Architeqt_Sync

There are many very small incremental steps that I’ve seen in my career. But my hat goes off to him because Architeqt isn’t one of those things. To me what Alexander has created provides the infrastructure that IT has been clamoring for. It provides them the assurance that they can maintain all of those vital formulas across all of the applications while still allowing analysts to freely access data. Combined with the ease of use of Qlik Sense provides to analysts to grab data and go forth with consuming data it finally provides a framework for … say it with me … Governed Self Service.

Exit Stage Right

While I would love to go and on with lots of additional information I know this is the right time for me to step off the stage and allow you to dig into Architeqt for yourself. Simply click this link and it will take you directly to this phenomenal new extension. The site will contain all of the information you need to download and configure this Qlik Sense Extension as well as a nifty You Tube video where you can see it all in action.

Posted in Self Service | Tagged , , , , | Leave a comment

Flipping Homes or just Flipping Out?

I’ve enjoyed my career in Business Intelligence but after seeing the following visualization which shows the amazing potential for earning profit in the home flipping business I think it’s time I became a real estate mogul.

FlipChartUnless you’ve been under a rock or you are probably aware of the blitz of television shows dedicated entirely to showing us how easy it is. The underlying needs for house flipping is the startup capital to make purchases with, and the keen eye of a designer to help you choose the right colors to slap on the walls. I’ve got like $12 saved up which is probably more than enough to get started and fortunately I’m blessed with a wife that has a great eye for design. If you aren’t as fortunate as me you may need to find a business partner and a designer who you will more than likely have to pay.

Getting started

As business intelligence professionals I think it’s only good common sense for us to get started by playing to our strengths … use analytics to help us make our home purchases. After all as advocates of actionable intelligence certainly we would trust our own life savings in our analytical hands. Right???

The first thing we would want to do is figure out what aspects of a home are most responsible for attracting the highest price. Those data science types call what we are trying to do a “multiple regression.” In real estate mogul language it means – “Hey dingbat before throwing all $12 down on the table to buy a home you probably need to know whether it’s the homes square footage or the lot size or the number of bathrooms or the number of bedrooms or the amount of taxes or the proximity to schools that has the most impact on the sales price.”

Multiple Regression

Not too hard to understand the importance that knowledge would have on our ability to turn a profit. But how does that data science multiple regression stuff? It’s simple you fire up R, load your data, you run the LM function and let it give you the answers.

Seriously it’s that easy. Here is how we would load our previous home sales data:

Housing = read.table(“C:/RealEstateMogul/housing.txt”, header=TRUE)

Then if we want R to tell us what the correlation is between the Price of the home and the Size (of home) and the Lot (size) we simply type the following

Results = lm(Price ~ Size + Lot, data=Housing)

Iterating combinations

R very well may tell you that there is a really strong correlation between the home size, lot size and the price. But unless you are lazy you would probably also want to know if there is an even stronger correlation. In other words is the size of the home and the number of bathrooms more important? Or perhaps lot size and number of bedrooms? In our case all we would have to do is go through every possibility of 2 variables. Then all combinations of 3 variables. Then all combinations of 4 variables. Then all combinations of … you get the idea.

As you can imagine it’s this manual coding of all of the combinations, this grunt work, that those data scientists don’t really enjoy. Fortunately as a budding tycoon I’m also a Qlik Dork and I have full intentions of using QlikView as well as R.

QlikView and R Integration

You see this is kind of the perfect use case for the QlikView and R integration. Not only do I want to be able to simply check whatever combination of variables I want to use, I also want to be able to filter the data and choose what is passed to R. That way I can verify the best combination of variables as well as confirm that the correlation holds true across time periods, across zip code ranges etc. Or I may determine the variables that are best suited to 30542 versus 90210.

QlikViewScreenShot

Behind the scenes there are only a handful of lines of vbscript code behind the button that says “Run in R.” Basically it outputs the data from a table so that whatever you have filtered is put into a CSV type file. Then it calls R tells it to read the file it just output, then tells it to run the LM function using the variables you’ve checked and asks it to output the results to a file and then reads that data back in to QlikView so you can see the results. Including a scatter plot output showing relationship between all of the variables.

ROutput

Closing

Some aren’t even aware that QlikView integrates with R. Others that do know figure “I’m going to do the modeling in R anyway and figure there really isn’t much that the QlikView integration can do for them.” Hopefully both types of people end up stumbling on this post. Feel free to nudge them by passing on the link. You see the beauty isn’t just that QlikView can call R. It isn’t just that you can check variables on a screen. You are more than free to write additional code that would literally iterate through every potential combination, and instruct R to write the results to filenames that match the combinations so that in 1 button press you get all of the results for all combinations.

So what? So what!!! The “so what” here is that so many of you out there are thinking “data scientists are seriously expensive and we can’t afford them in our company.” You are so right. You can’t afford to pay a data scientist full time to sit and iterate through every combination of your data. After all housing variables are mere child’s play compared to the massive amount of variables in healthcare for instance.

But you can afford to consult with one. You could have them build a model and then you simply use QlikView to iterate through all of the variables and then send them the output to review. Or what about that grad student in data science who has a few days in which to get some “real world experience” would QlikView’s integration to R allow you to take advantage of them?

Predictive Analytics is an important part of the overall data consumption continuum. The integration and what QlikView offers you sitting on top of R may be just what your organization needs to jump start your ability to reap huge rewards that predictive analytics offers.

As for me, it was fun using house flipping as a great use case to help me convey how to use predictive analytics. As you guessed though it turns out that $12 isn’t even enough to buy a gallon of paint to slap on walls. So I guess I’ll just have to continue doing what I love … helping others consume data.

Resources for those hungry for more

You know how this blogging stuff works. If I write to much then I lose my audience. But in this case I know that flipping homes is really on the minds of a lot of you. So I’ve tried to predict some of your questions and provide you with links to more detailed answers and opportunities because that’s just the kind of dork I am.

“I want to see more so I can get a better idea of just how cool this stuff is”

The following You Tube video is a Qlik Dork exclusive and will probably not go viral so you shouldn’t have any problems at all viewing it. https://youtu.be/jwZ1K6invPI

“No fair having all of the fun yourself. I want to be a house flipping phenom as well. How do I get my hands on this stuff?”

Great question. You can download the QlikView application used, as well as an implementation guide to help you configure R on your machine by clicking this link when prompted the password is “PredictiveAnalytics”

“I am somewhat familiar with R and I really do have an interest in house flipping. How can I get more information about the subjec?”

I’m not a data scientist, I don’t play one on TV and I haven’t even stayed at a Holiday Inn Express recently. However, the following links will give you all of the information you need about how to do a multiple regression on home sales data and how to read the results. They are from the serious data science minds at Columbia.

Summary version to wet your whistle

Really complex document that will blow your mind if you aren’t really into statistics

 

Posted in Predictive Analytics | Tagged , , , | Leave a comment

The Power of Now

MatchstickFishOne of my favorite activities as a child was doing mental puzzles. Matchstick puzzles like this were among my favorites. Easy to understand wording like “Move 2 matchsticks and turn this fish that is swimming to the left into a guitar playing crocodile swimming to the right.”

I may have taken a tiny bit of liberty with the wording to see if you were paying attention but I’m sure you remember the concepts. Sometimes they were absurdly obvious. More often than not though they forced you to think out of the box and ignore the obvious solution that would lead you to failure.

In the spirit of this classic puzzle concept I need you to shift 1 thing and turn my least favorite 4 letter word “NEXT” into my favorite word “NOW.”

NextMatchsticks

I needed this light hearted approach because the word “NEXT” simply makes my skin crawl. You are sitting in a meeting trying to discuss important business issues and a question arises that can make or break the ability to make a decision and someone blurts out “yadda yadda yadda NEXT blah blah blah.”

If you don’t believe me just pay attention to your next meeting. Someone surfaces static data on a slide and a leader asks “Why did that happen last month?” or “Why are those 2 providers under-performing the others” or “Why did our expenses go up?” The answer is more often than not “That’s a great question. I will research that and get you the answer NEXT _____.” I don’t care if the blank is “Week” “Month” “Meeting” or the phrase “Time the committee meets.” NEXT is still a 4 letter word that simply means “We are never going to be able to make a decision.” Because “next” never gets here.

Qlik Dork’s 1’st Law of Analytics

Questions always, always, always give birth to additional questions. It’s Newtons’ 4’th Law of motion.

Ok maybe it’s actually Qlik Dorks 1’st Law of Analytics, rather than Netwon’s 4’th Law of Motion but you get the point and your experiences confirm this natural law. We’ve got 0’s and 1’s filling every nook and cranny of our buildings. We’ve got really great minds sitting in offices and cubicles. Yet when it’s time to perform. Time to answer questions. Time to present “Actionable Intelligence” what do we get? The dreaded word “next.”

So what is going to turn the word “NEXT” into the word “NOW?” As in “Oh let me get you that answer right NOW so that we can make a decision right NOW while everyone is already together.”

Mobile is Key

I suggest that the answer lies in the mobile devices you already have in your hands. Yes the one you are more than likely reading this post on is what’s going to turn the word “NEXT” into “The Power of NOW.”

We use our mobile devices for almost everything in our lives outside of the office but we seem to put them into airplane mode once we step into our places of work. Who can really be blamed for that though? Our business problems are much more difficult to solve than making reservations for a trip while standing in an elevator, and then seeing a 3D satellite view of that city when we arrive while we get turn by turn directions that include live information about traffic patterns in front of us.

We all use analytics on our desktop computers but it’s not like they would really work on our mobile devices. Would they? Take the following screen for example that represents the common look and feel of a typical “dashboard.” It represents the number of messages on Instagram from users around the world regarding ice-cream. It’s critical when I travel that I know what the “locals” are saying. Should I consider “gelato” in Genoa? If so which vendor(s) should I go to and which should I avoid?

WhoScreamsForIceCream

Responsive Design

Kind of proves the point that it’s far too big to fit on a pad or a phone. Well …. It used to be. But enter the magical world of “Responsive Design.” Responsive Design is far more than marketing terms to describe the process of shrinking a dashboard like into 10 pixels wide by 10 high so that we need a magnifying glass to view it. Responsive Design involves applying serious intelligence behind the scenes that actually “moves” and “alters” the objects as needed to fit whatever the size and orientation of your device happens to be.

Instead of elaborating or pages and pages I will point you to an excellent article that describes responsive design in more depth by someone far better equipped and a lot more succinct than I:

Click here for a more complete write up including pretty pictures to understand Responsive Design

Cool stuff right. Wait this post is about to get a whole lot better. The following link is for a Qlik Sense Demo site and will take you to the dashboard I’ve shown above. You will be able to not only see, but interact with the data right on your mobile device. Just for fun see if you can figure out the most talked about term in the United States and what month the most posts occurring?

Click here to interact with a dashboard right on your mobile device

Another Visualization Exercise

In the past I’ve helped you “visualize” lots of types of data, and even a concept like the Data Consumption Continuum. But now I’ve got a visualization task that is for you and you alone to solve:

Visualize yourself interacting with your company data instead of popsicle posts.

Visualize yourself utilizing mobile devices during your meetings and turning the word “next” into the phrase of “let me get you that answer right now.”

Visualize the leap forward in your company’s decision making when you begin taking advantage of the “power of now” that responsive design can bring you.

Embracing responsive design will allow you to switch the right “matchsticks” to convert the word next into the word now. But hey … no reason to rush. You can always transform your business NEXT year. After all it’s not like your competition is doing it right NOW.

Posted in User Adoption, Visualization | Tagged , , | Leave a comment

Visualizing the Data Consumption Continuum

As this is intended to be a blog about visualization it seemed only fitting that I devote this post to visualizing the concept I introduced last week that I called a Data Consumption Continuum.

It all started with …

A few years ago my good friend and colleague at the time at Northeast Georgia Health System, Zach Ivie, helped me solidify a visual very close to the following.

DataConsumptionContinuum_Image1

Despite the fact that this is a rather crude version of the work of art that Zach, a graphic guru, helped produce, I believe the message will come across loud and clear anyway. That message is that as you move through the Data Consumption Continuum there is a higher level of skill demanded of both the data consumer and the data producer. However, the benefit to the company of making those investments is clearly that there is a higher level of return on investment as well as a higher level of process improvement that can be achieved.

Not very hard concepts to grasp, but we found it very useful in trying to help data consumers understand that we weren’t trying to replace their roles. We needed them to understand we were trying to help them become even more valuable to the company. Having a visual that they could readily grasp was very effective in our efforts to establish trust that we saw them as teammates and provide the incentive to do the hard work that needed to be done to progress through the continuum. Obviously the same image helps build support through management as well. (In hindsight I would add “Time needed to produce” along the bottom as an annotation as well to help management understand each step in the continuum takes longer than the prior steps.)

Much appreciated input came along …

I presented the graphic when I spoke at the 2014 Qlik World Users Conference in Orlando.

DataConsumptionContinuum_Image2Among othersI met at the time, two gentlemen from Lee Memorial Hospital named Marcello Zottolo and Roger Chen stood out. Over the course of a few weeks following the conference we spent a lot of time chatting and growing our friendship. During which time these Process Improvement gurus asked why the image was like steps instead of being a straight line up the continuum. I indicated that I wanted to illustrate very clearly that you didn’t move up the continuum by accident. That there had to be clear and pronounced commitment to doing so. But I asked how they it. They were gracious enough to share an image with me that was something like the following, and you know I love visuals:

I liked the implications that their suggestion implied. Clearly we should never be standing still and that even the stages of the Data Consumption Continuum are in fact a process. That the hard work I indicated could be visualized as a ball rolling forward, and that there should be documentation supporting the standard work after advances are made. So I adjusted my thinking to include their great thinking and suggestions along with my basic premise that the different stages required a knowing decisions.

Combining the thoughts provided room for annotations to note exactly what I believe those commitments are: Ever increasing levels of trust. There are clearly static reports that are simply needed as “checklists” for workflows. However, there are also thousands more  developed simply so that business data consumers can avoid having to accept any responsibility. It’s easy to say “I took off the report that Dalton built for me. Clearly he did it wrong.” Which is also ok with me because I can say “Clearly they gave me bad requirements.” Thus the proliferation of 18 versions of the truth continues. Moving to Guided Analytics forces those data consumers to accept the responsibility for making selections filtering the data they are then responsible for the choices they make. Which only makes sense as they know their business better than IT and a lot is missed in writing up requirements documents. As you are probably well aware from experience getting others to accept responsibility is easier said than done.

Moving along the continuum from Guided to Self Service Analytics forces a department wide level of trust on IT’s part that if they construct governed data sources they can trust the business units to build the applications instead of IT building them. The next stage requires an enterprise wide level of trust that the investments of the amount of time, salary and technology required will pay off. (Clearly there are a lot more complexities than just “trust” but the symmetry in the visual was to hard to pass up.)

DataConsumptionContinuum_Image3

One of the great pleasures in my search for more effective ways to communicate and be the catalyst for change is the identification of others in different fields who are also working tirelessly to train up. Others who continually reiterate and show examples to help crack my think skull.

Never stop consuming knowledge …

One such field that I’m trying to hone my skills in is Storytelling. Cole Nussbaumer is one of those afore mentioned people that I love learning from on an ongoing basis. I’ve always been a self-professed data nerd and was always of the mind “For crying out loud the data speaks for itself.” Cole has helped me realize that’s not even close to reality in the field of Business Intelligence. Her blog conveniently named for people like me Storytelling With Data is a resource I’ve referred to many times. She backs up her pontificating through examples to show the flow through projects. For me seeing a visual as a starting point then seeing it progress forward using her techniques step by step has really helped me understand the value of storytelling and more importantly how to do it.

But how does this relate to this post? Well if you give me a second I’m getting to that part of the “story.” I’ve begun realizing that Actionable Intelligence requires not only the data and visuals but storytelling to explain things so that executives fully understand the ramifications. Thusly I present my current approach of Visualizing the Data Consumption Continuum.

DataConsumptionContinuum_Image4

Disclaimer: Clearly I’m no graphic artist. Hopefully my crude crayon type drawings aid in your ability to consume the concept of the Data Consumption Continuum. 😉

 

 

 

Posted in Training, User Adoption, Visualization | Leave a comment

The Data Consumption Continuum

Consum Mass QuantitiesWe humans love to consume.

Food.

Water.

Fuels.

And my personal favorite Chocolate.

You name it we want to consume it. Health warnings have little effect at deterring us from consuming mass quantities of the wrongs things. Yet sadly there is one thing that we need to survive and it seems that no matter how hard we try we simply can’t consume enough of it … DATA.

We need to consume data to survive in our own lives, as well as within our occupations. Yet try as we may we seem to be starving for it. I believe part of the problem is that unlike eating chocolate there is no “right” way to consume data. We humans are all over what I call the Data Consumption Continuum and can’t figure out how to accommodate one another’s ferocious appetite for this particular commodity.

Static Data

Reports are the most basic form of data consumption. Static reports have been around much longer than we have had computers and or a specialty field called Business Intelligence. They are a wonderful thing in situations where the subject matter doesn’t change. Whether it be workflow reports, ie lists that I mentioned in my last post we love to consume or whether it is truly static data like reports of what our Accounts Receivable was as of a particular date.

So there are good reasons to have the 3,209 static data reports your company has. But it simply doesn’t make sense to have business meetings end with the conclusion “Yep our company is losing money. Let’s all have a great week and enjoy the last days before we are unemployed.” Someone in the meeting is inevitably going to ask the “Who, what, when, where and why” questions. Thus report 3,210 ends up being born over the course of the next several hours, days, weeks or months.

Guided Analytics

Many including yourself may think of guided analytics as the polar opposite of static reporting. I’m not one of those people.

I believe guided analytics are a wonderful way to speed up the process of asking questions and getting answers. They allow you to “drill, drill, drill” until your heart is content. My posts have praised and demonstrated the many different ways that visualizations enable us as human beings to more rapidly perceive and consume massive quantities of this wonderful commodity we call data.

For some guided analytical applications are like a sudden speed reading course. The applications allow them to consume the data from 3,209 static reports in 1 sitting. For others it’s like handing their companies 0’s and 1’s to Michelangelo and suddenly a beautiful portrait of their company appears.

The definition of the word continuum is “a continuous sequence in which adjacent elements are not perceptibly different from each other, although the extremes are quite distinct.”

For all of the wonderful things guided analytics offer I see them as a mere single step away from static reporting for one reason. They only allow you to answer the questions that you already had in mind when the applications are built. If you think you will want to see how surgical costs relate to length of stay you build your application to contain that data. It’s great that we have the technology now that enables us to consume that many 0’s and 1’s at the same time, in really slick visual ways. But is it really so magical that we are able to answer the questions we had when we asked for the application to be built?

Self-Service Analytics

Hopefully you now understand why I’ve used the phrase “data consumption continuum” and why I believe guided analytics are but a single step away from static reporting. If you don’t then you stopped reading already, so those that are still tracking with me are ready to take another step forward on the human consumption continuum to self-service analytics.

I’m honestly baffled by the concept that self-service analytics are a way to allow end users to quickly visualize data from some a single data source. Woo-hoo look at the pretty pictures I can create for you from your XLS file or a single SQL query. Really??? That’s a leap forward in technology? I’ve never known a version of Excel that didn’t offer charting right inside the tool itself. If all you want are pretty graphics from a single data set just use the tool your data is in which is likely Excel.

I believe self-service analytics enables us to answer the questions that we didn’t have when we built our guided analytics solutions. It enables us to consume all of the data we built into our application and then begin consuming more data. Data that perhaps wasn’t even available when our guided analytics applications were constructed.

The important thing in my mind is that self-service analytics must offer the ability to consume new data as well as our existing data sources. When I get to the end of the road and can’t get answers just looking at my cost and length of stay data I need the ability to now consume readmission data along with that data not instead of it. To consume patient vital information along with it. To consume patient satisfaction data along with it. Whatever the new data may be self-service analytics should be an additive process. A step forward on a continuum of our consumption of mass quantities of data. One that allows us to grab the data and move on rather than having to go through a long requirements and prioritization exercise with IT.

Predictive Analytics

For many data science is very hard to understand. It seems that they think Data Scientists go into a room with their magic potions and terabytes of data and emerge with all of the answers to the company’s problems. That’s simply not the case.

Data scientists simply apply age old statistical formulas to data. The same data that we display in static reports. The same data that we display in guided analytics applications. The same data that we consume in self-service analytics. But they do so in a slightly advanced and more scientific approach. You or I as mere mortals say “My spidey senses are tingling. I think there may be a relationship between our profit and the patient’s length of stay.” We ask for a report, we use guided analytics or perhaps self-service analytics and if we see even a minor trend we immediately jump to a conclusion there is a cause and effect, not just a relationship and we say “Aha I’m a genius! Quick change everything in our company I’ve found the problem.”

Data scientists say “give me the data for every variable we have and I will help you find the BEST correlations. The ones that statistically have the highest probabilities. The factors that actually lead to patients being readmitted for instance.”

That is a great thing. But it’s not like they simply run a magic statistical formula and come to the answers because that isn’t how statistics works. They methodically run formulas on different combinations of the variables. What about A, B, C, D and E? Nothing there so let me try A, B, D, E and F. Nothing there. Let me try this. Let me try that. Let me try the other thing. And they churn and churn and churn very methodically until months later they provide an answer. An answer that in the past could have been achieved with more people and more time. So my assertion is that like the steps forward from static reporting to guided analytics and then to self-service analytics predictive analytics is a step forward simply in that it enables us to do things faster.

Each phase I’ve discussed allows us to take a step forward. A step that speeds things up. A step that allows us to consume more data. But the incremental steps are obvious to follow from one to the other. Yet if you look at how far apart static data is from self-service analytics you see that those extremes really are quite different. Static data reports identify the historical data we already had in our systems. While the result of predictive analytics is even more data that we can use and when combined with our current data can identify alternative actions we can take. Prompting us to take action rather than simply reporting to us. In other words a Data Consumption Continuum.

I have two reasons for this post:

  • To help you realize that wherever you may happen to be along the continuum in your ability to either produce or consume data you should consider the fact that the others in your company who may be at a different stage aren’t “dinosaurs” and they aren’t just “resistant to change”  and aren’t really that different from you. If you change your focus and understand the key concepts that differentiate the stages you will be better equipped to communicate with them so that you can both move forward as a team.
  • To help you realize that instead of looking at each phase as something completely different that requires it’s own tools you should consider thinking of implementing a data consumption “platform” rather than implementing new tools for each stage you progress.  A platform that enables you to surface your valuable data one time and reuse it over and over along the various stages.

Like what you are reading be part of the conversation and share your thoughts. Also consider showing me some love and following @QlikDork on Twitter.

 

Posted in Random thoughts, Self Service | 2 Comments

Crawl. Walk. Run. Fly?

CrawlingIn 30 years as a parent I can tell you I’ve had many memorable moments with my 2 amazing daughters. As humans, most of our memorable moments around babies involve movement. Let’s face it as cute as they are it gets old just watching them lay on their backs and smile. So when they first get the desire to start moving and can make themselves roll over we get excited. When they can finally control the movement of their hands and their legs and can crawl to us our hearts jump for joy. When they can take those first Frankensteinish steps to our waiting, open arms we get overwhelmed. Once they’ve mastered balance and movement and begin running we get to play our first games of chase and tag with them.

Crawling. Walking. Running. Memorable moments indeed. Those movements enable us accomplish so much. How much more could we accomplish if we ever learned how to fly. Don’t look at this post like I’m crazy … we’ve all jumped off the porch and flapped our arms wildly. Some of you jumped from heights higher than a porch attempting to fly and broke some bones in the process. You know who you are.

Yes there is a point to all of this. As the parent of 0’s and 1’s (data) for the past 30 years I’ve seen it crawl, walk and run and I’m beginning to see it fly. No I’m not talking about the Cloud, I’m talking about Self Service Data Visualization. I suppose I should back up a bit.

Those who work in IT, specifically those who work in data administration type roles work with data at least 8 hours a day. For most of us that is more time than we spend with our own kids. We have principles, best practices we follow to protect the data. In many ways we are more highly trained to protect the data than we are to protect our own children. So it should be of no surprise to you in business roles when you deal with IT folks that are a little over protective of their data children.

Crawl.

I just love checking off items in lists. I think we all do. There is a certain sense of accomplishment each time we check an item off. Which is why I think so much of Business Intelligence surrounds simply generating lists and static reports. Don’t believe me? The Checklist Manifesto: How to ChecklistManifestoGet Things Right by Atul Gawande, is a best selling book that shows the power that simple checklists have in the complexity of our lives and how they can help us avoid failures. Kind of like allowing our “data babies” to crawl. We control how much of it goes out and to whom it is allowed to “crawl.” Don’t get me wrong there is a huge part of the workforce in many companies who rely on those lists to do their job processes. IT management also loves lists and static reports. They can easily be used to justify more head counts. If each report takes 1 week to build and you want 52 reports … boom 1 person year totally justified. Wait you want thousands of reports … “woo hoo” an entire data services team springs into being. You get to see your data. IT data workers still control the data. IT management grows its organization. It’s a win-win-win.

Walk.

Sooner or later though executives start asking for more. They want KPI’s, Dashboards and Scorecards. For some these are the Holy Grail in a matter of speaking in the field. But when you really think about all of these, they are simply the next incremental step in growing up. All the check marks, arrows and circles simply tell them the answers to the questions that they previously asked. Unlike crawling though, I think this is kind of like taking the first actual steps because at least the pretty pictures allow them to consume the data faster than they could if they had to read 317 separate static reports. IT is still central in the process, they get more staff, they get more resources and now there is direct interface with executives. It’s an ever bigger win-win-win. Our data babies are growing up indeed.

Run.

More than once in my posts I’ve shared the concept that Analytics allows the end user to answer tDataConsumptionhe question that they had in their head when they started, but it also allows them to ask the next question. Analytics allow business users to not only see issues but drill into them. Find the roots of successes or problems. So you probably aren’t surprised that I consider Analytics to be like running in terms of consuming data and I consider it a thing of beauty. Kind of like watching my granddaughter sprint across the soccer field and score a goal. It’s also a little scarier for those data parents in IT, because Analytics can’t be done without a lot of input from the actual business users and without a lot of learning about the business processes by the IT staff constructing the analytics applications. Our data babies sure have grown, and very rapidly indeed. Where has the time gone?

Fly?

If you consider how much we can accomplish simply by running just imagine what we as humans could accomplish by flying. Similarly Self-Service Data Visualization has that same great potential compared to just crawling with lists/static reports, walking with dashboards or running with analytics applications. Unfortunately for those of us in IT this goes against everything we’ve been trained to do. Everything we’ve spent our entire career perfecting … controlling the data. Self Service Data Visualization means trusting others to do the right thing with the right data. Trust me when I say “that is as hard for data parents to swallow as it was for me giving my daughter’s hands in marriage.” But can you really blame IT for fearing that? For 30 years I’ve seen business users combine data from cocktail napkins, flat files, spread sheets and personal hunches then deliver numbers in a meeting that directly conflict with those before them. So have executives. Which is exactly why so many companies are stuck watching their data crawl, walk or run … but never get to see it fly.

Flying

So what’s the answer? In my humble opinion it involves a marriage between IT and the business community. We’ve all seen that in human marriages opposites attract. So why do we allow them to repel and work against each other in offices? IT has the staff to properly govern and protect the data to ensure a single source of truth. That has to be respected. The business community on the other hand has the knowledge about their processes that in most cases IT completely lacks. That also has to be respected.

Companies can continue to allow rebels do self-service from untrusted sources and continue to plummet to the ground as a result. Companies can continue to allow IT to completely control all access and enforce that all data requests have to be resolved by them and continue to plummet to the ground as a result. Or they can arrange a marriage between the two. One in which IT is trusted to provide a single source of truth data libraries where the business users can then serve themselves. One in which we see our “data babies” leave the nest and fly.

But hey what do I know, I think I’m the parent of a couple of trillion 0’s and 1’s.

 

Posted in Self Service | Tagged , , , | Leave a comment