Why you should use “Tune Model Hyper parameters” module in Azure Machine Learning?

Standard

A machinAuzre Machine Learninge learning algorithm in Azure ML has few parameter settings that you can set — in this post, we will talk about 1) why you should NOT stick with default settings 2) How can “Tune model hyperparameters” module help you do so?

So first up, why you should not be using the default setting? The parameter settings that are applied to a model impacts the accuracy (or call it predictive power) of the algorithm…sometimes it may be significant and sometimes not but either case, you won’t know until you try changing the default values. In other words, by tuning the hyperparameters you could significantly boost your model’s performance!

Now, how do you go about setting the parameters such that it gives optimal performance? Let’s say that there are 3 parameters then that is 27 different combinations! How do you know which one is the best? You could dig a little deeper into the mechanism of how algorithms works and narrow down your list but that would still take some time. So, there should be a better way, right? Luckily there is: This where “Tune Model Hyperparameters” comes in! You can use it with any algorithm in Azure ML. This module helps you tune the hyper-parameters. There’s some things that you still need do like decide: Do you want the module to just try random n combinations? OR Do you want the module to try all combinations (fyi: this is compute-intensive operation!)? … AND you will have to decide what are you are optimizing for? Depending on the algorithm it would let you pick the evaluation metric that you want to optimize.

Now, there are some good articles already written that walks you through how to get about doing this so I am going to share these links with you:

  1. Tune Hyper Parameters (MSDN)
  2. Understanding sweep parameters module in Azure ML

I want to conclude by sharing few tips:

 Notes from the field:

  1. Running the “Entire Grid” mode will slow down the training time for the model. You might want to make sure that it’s acceptable and the cost (longer training time) to benefit (better accuracy) is worth it for your case
  2. When you are comparing algorithms to decide the best one that fits your problem, instead of comparing “model with default parameter settings” with each other, try comparing the “model with tuned hyper-parameters”

PowerPivot Model: Why am I not seeing “Month Names” in correct logical order?

Standard

This blog post is for people who have seen the reports built on PowerPivot model where the Month Names are not in correct logical order. So instead of  “January, February, March, April …” (which is correct logical order), the order in the report would be displayed as “April, February, January, March..” (which is NOT correct).

This is what I am talking about:

powerpivot model month name not sorted correctly

Here, Month names are not sorted correctly, right? So how do we solve this? Let’s see this in this blog post!

Understanding the Sample Data-set

Optional: Download a sample data-set to practice what’s described in this blog-post: Download – Paras Doshi Blog’s sample data set

Now the data looks like this:

DateDaily New numberMonthMonth Name
1/1/2012 0:0001January
1/2/2012 0:0001January
1/3/2012 0:0001January
1/4/2012 0:0001January
1/5/2012 0:0001January
1/6/2012 0:0001January
1/7/2012 0:0001January
…..………………

Note that “Daily new number” is used for the demo purpose. I had to anonymize the data before I could share it with you all!

Originally, the table had just two columns but since I wanted to add “time Intelligence” to the model. I added two columns Month and Month Name.

Now here’s the DAX behind these two columns:

ColumnDAX
MonthMONTH(‘Sample Table'[Date])
Month NameRELATED(Month[MonthName])

There’s a relationship between Month Column of “Sample Table” and Month column of “Month” table

And here’s what Month Table looks like:

MonthMonthName
1January
2February
3March
4April
5May
6June
7July
8August
9September
10October
11November
12December

Creating a Report on Top of Sample Data-set.

Now if you create the report of Month Name vs Daily New Number, it will look like:

powerpivot model month name not sorted correctly

Oops, Problem detected: Month name are not in correct order.

So now let’s solve it.

Solution to sort month name correctly

Let’s solve this issue in our PowerPivot Model. Here’s what you do:

1. Go to PowerPivot model, Select “Month Name” column from the sample table

2 Toolbar > Home > Sort By column > click on “Sort by column”

3. And set the properties in the dialog box as:

powerpivot model sort by column dialog box

Now, go back to the report and refresh the connection to the model.

Done!

powerpivot model month name are sorted correctly

This is so because by default since the month name is “Text” – it was sorted in A to Z format. But you saw how we can use the sort by column property in PowerPivot model to fix this issue.

That’s about it for the post! your feedback is welcome!

Want to Read More? Here are few links: