Prescriptive — How can I use all these things to take business decisions/actions.
With that overview, let’s look at Descriptive analysis a little bit more. This is usually the first step for any organization to start getting value out of all their data. They should be able to answer questions like: What were my sales last quarter? How about same quarter last year? Then compare them to see if they made progress. They can also report on sales (Actual vs goals) for last n months and see if they are trending in the right direction. Things like this! Once you have a good process and platform to get this right then the organization is ready to advance to next step which is diagnostic and this is where you start analyzing the key drivers and underlying reasons to figure “why” it’s happening. But you need start at Descriptive!
“Machine Learning” is a subset of “Data Analysis” — it’s just one of the activities that you could apply to solve a data analysis problem, you just need to find a problem that can use machine learning wizardry! What kind of activities?, you say — well, to answer that we will need to step back and categorize what problems could be solved by Data Analysis. There are broadly three kinds of problems:
“What” Problems. Few example: What are my sales number for last quarter? Can we compare it to same quarter last year? Now, can we break it down by Regions and Product Categories? — you see all these questions could be answered by a querying your data stores or by your Business Intelligence platform. Yo do NOT need machine learning for this. Moving on…
“Why” Problems: Few example: Why did the customer cancel their contract? Why is the profit in region A declining Quarter over Quarter? You see this is little bit more challenging than “what” questions — you will need to structure the problem and pull data from multiple sources. Why did customer cancel? You may want to look at internal (e.g. customer complaints) and external (e.g. bankruptcy) data. Usually you won’t need to apply Machine Learning here — you might benefit in some cases where you “cluster” all churned customers and see if you can find some patterns but again Machine learning is not you primary tool here. Moving on…
“What’s next” problems: This what you have been waiting for — this is where Machine learning could be applied. Example: Which customer accounts will cancel their account this fiscal year? — This is where you train a machine learning algorithm to predict which customers will churn this year. Note that the work you did for “why” problems where you identified some characteristics of churned customers will still be applicable here — and that brings me to: Most organizations don’t usually jump from “What” to “What’s next” stage — every organization is at a different stage depending on their maturity and you can’t apply machine learning to every data analysis problem. Also, with more and more companies using “data” to gain competitive edge, if you are not using machine learning then chances are high that your competitor is and they may out-compete you and that’s why it’s important to continuously invest and reach the highest level — more and more companies and executives are realizing this and it’s a great thing for the data community!
To conclude: Depending on the analytics maturity of your organization and the business problem at hand, you might have to use Machine learning to solve a data analyis problem…And it never hurts to pick up Machine learning basics along with other data analysis skills that you might have.
Why are there so many analytics startups in the past 2 years? With Google Analytics getting better every year (for FREE!), what is the value proposition? I understand the need to augment with some new perspectives such as Clicktale, but I’m not sure I understand the value prop of KissMetrics, SpringMetrics, etc?
Answer:
There are two main reasons:
Features gap between google analytics (free) and google analytics (premium aka 360 now!) — there are a lot companies (esp. with multi-million customers) that want to use premium features but still cant justify the ROI of GA premium. So there are analytics startups out there that try to cater to these “gaps”. Even though GA is improving, there will always be some feature gap(s).
Access to venture capital for these startups — so these startups found a market and they went for it. They also had access to venture capital (easier two years back then it is today!) and it also helped them that “big data” and “data science” was (and still is!) a highly discussed tech topic.
I believe we will see some consolidation in next few years.
Based on how you are framing your question, it seems that you currently don’t have “Data Analysis” Background but want to build a career in this field. Here are three things you could do:
Learn Tech Skills:You will need technical knowledge to be successful at analyzing data. SQL and Excel are a good starting point. You could do a lot with these tools — then depending on the bandwidth that you might have you could explore R. How do you learn this? Here’s a learning pathway: Learn #Data Analysis online – free curriculum ; Also search for free courses on Coursera or other platforms.
Learn Soft/Business Skills: This is as important as tech skills (if not more!) when it comes to Data Analysis. Finding Insights from your data is half the battle, you will need to put the insights in a context/story and influence business decisions and sometimes influence business change. we know change is always hard! So your soft/business skills will be very important. Also, you will benefit a lot from learning about how to break down problems, communicate your solution by using “business” language vs tech-speak.
Apply them (and keep improving):Now that you have picked up some tech and soft/biz skills, apply them! Get an internship, Help out a non-profit in your free time (Data Kind, Statistics Without borders, Volunteer Match are good resources to find a non-profit) and start applying your skills! It would also help you get some “Real” world experience and applying what you have learned while “learning-on-the-job” is arguably the BEST way to pick something up!
A machine learning algorithm in Azure ML has few parameter settings that you can set — in this post, we will talk about 1) why you should NOT stick with default settings 2) How can “Tune model hyperparameters” module help you do so?
So first up, why you should not be using the default setting? The parameter settings that are applied to a model impacts the accuracy (or call it predictive power) of the algorithm…sometimes it may be significant and sometimes not but either case, you won’t know until you try changing the default values. In other words, by tuning the hyperparameters you could significantly boost your model’s performance!
Now, how do you go about setting the parameters such that it gives optimal performance? Let’s say that there are 3 parameters then that is 27 different combinations! How do you know which one is the best? You could dig a little deeper into the mechanism of how algorithms works and narrow down your list but that would still take some time. So, there should be a better way, right? Luckily there is: This where “Tune Model Hyperparameters” comes in! You can use it with any algorithm in Azure ML. This module helps you tune the hyper-parameters. There’s some things that you still need do like decide: Do you want the module to just try random n combinations? OR Do you want the module to try all combinations (fyi: this is compute-intensive operation!)? … AND you will have to decide what are you are optimizing for? Depending on the algorithm it would let you pick the evaluation metric that you want to optimize.
Now, there are some good articles already written that walks you through how to get about doing this so I am going to share these links with you:
Running the “Entire Grid” mode will slow down the training time for the model. You might want to make sure that it’s acceptable and the cost (longer training time) to benefit (better accuracy) is worth it for your case
When you are comparing algorithms to decide the best one that fits your problem, instead of comparing “model with default parameter settings” with each other, try comparing the “model with tuned hyper-parameters”
