End to End Demo: Hadoop (HDInsight) + Hive + Excel + Power View + Azure Data Market

Standard

A great end to end demo shown by Microsoft at Strata conference 2012:

Description about Demo:

Scenario: Analyze web logs of an online bike store.

Tools demonstrated:

Hadoop (Get started with HDInsight)

Hive.

Excel 2013

Power View

Azure Data Market.

A Dashboard in Power View showing co-relation between discount campaigns and the traffic:

This is mash-up of data from Hadoop (traffic) and data from SQL Server (discount campaigns)

end to end demo microsoft hadoop hdinsight

Conclusion

In this blog-post, I shared an awesome demo about HDInsight. check it out!

Getting started with HDInsight (a.k.a Microsoft’s Big Data hadoop Platform) on local Windows Machine!

Standard

Recently Microsoft announced HDInsight on Windows server! and so it’s good to get a chance to play with its public preview! Currently there two ways you can run HDInsight: 1) Via Windows Azure 2) On your local Windows machine.

In this blog-post, I would show you step by step to install a HDInsight on a local Windows Machine. For the purpose of this blog-post, I am going to show it on Windows 7 but it also supported on Windows Server 2008 R2.

download hadoop on windows machine hdinsight

Note that the ideal audience for this blog-post would be a developer who wants to kick tires of Hadoop on windows machine to see what it can do! If I had wanted to target it to Hadoop administrators then I would have shown how to do it on Windows Server and also how to manage the Hadoop cluster with system center. But for this blog-post, I am going to target developers so that they can get started playing with Hadoop on a windows machine! With that, here are the steps to install Hadoop (HDInight) on Windows 7:

1) open Web Platform Installer. Download and install it if you haven’t yet.

2) search for Hadoop

install hadoop windows via web platform installer

3) Install it!

4) You should get a message saying that it successfully installed it!

5) Do you see a Microsoft HDInsight Dashboard ICON on your Desktop? Yes? Great! Open it!

windows hadoop big data dashboard6) And here’s the IIS manager showing the site that hosts the above Dashboard. Just wanted to show this to folks who might not see the Dashboard at http://localhost:8085/

IIS windows hadoop local host site port 8085

7) That’s about it for his post. If you want to continue learning, check out the “documentation” link at the bottom on the Hadoop Dashboard which is: http://social.technet.microsoft.com/wiki/contents/articles/6204.hadoop-based-services-for-windows-en-us.aspx

Conclusion:

In this blog-post, we saw how to install HDInsight (Microsoft’s Hadoop) on local windows machine.

Related Articles:

Who on earth is creating “Big data”?

Want to learn about BigData? read Oreilly’s Book “Planning for BigData”

How to Install Microsoft HDInsight Server Hadoop on Windows 8 Professional

Five quick reasons why I use virtual machines on my personal computer:

Standard

I was talking to a friend yesterday about virtual machines.  The topic got started because I had about  4 virtual machines and I had to explain why I had such a setup – why just not “dual-boot” – and so I thought I document the reasons that I gave out. Note that I am using Virtual Machines on my personal computer and this blog post falls into the class of “personal technology” and I’ll not touch upon why businesses use virtual machines. Before we begin, a quick note about what are virtual machines? Think of virtual machines as “software application” that can run “operating systems” in them. Example: You have Windows X on your machine and there’s an application on your machine that says “LinuxOS.xyz” – if you click on it, it would open Linux operating system as an application without leaving your windows machine. This is not technically correct definition but from a personal technology standpoint, all you need to know is that “virtual machines” lets you run operating systems like linux, windows, etc like an application on your main operating system. Here’s my current configuration: “I have windows 7 as my main operating system. and I have a couple of virtual machines running Windows Server 2008 R2, one virtual machine having a windows 7 environment & one machine to kick tires of Hadoop”. with that, here are the reasons that I use virtual machines:

1) I can have multiple flavors of operating systems running as application on top of my “main operating systems”. And I do not have to worry about the hassles of dual/multiple boot.

2) I can COPY a virtual machine and PASTE it on a different machine. Basically share “OS along w/ app installed” with others or open them up using a different computer

3) I can “Save” a state of a virtual machine. For example I can save the state of my virtual machine today and if something happens tomorrow then I can just “restore” it to the previously saved state. Think of it like “system restore”

4) When I am on a virtual machine, it gives me the freedom to play around with “do not touch” and “not recommended” configuration. I can experiment things I want to without worrying about “breaking” my main operation system

5) Do you have software’s on your OS that you installed for one-off purpose and forgot to uninstall it later? I usually install applications that I rarely use on a separate virtual machine. This helps me keep my main operating system cleaner.

Those were the quick five reasons I use virtual machines on my computer, if you want to get started you can check out: Microsoft virtual PC or Oracle’s Virtual Box.

Question: Do you use virtual machines on your personal computer? Yes? What is your “why”?  why not share that in the comments section?

SQL Azure: Is there a “per transaction” cost for Windows Azue SQL Database?

Standard

Question: Is there a “per transaction” cost for Windows Azue SQL Database (SQL Azure)?

Short Answer: No

I recently answered the question on MSDN forum where the question was about Transactions and the associated cost in SQL Azure. As of now, There is no “per transaction” cost associated with SQL Azure. There are two parameters that affect your SQL Azure Bill: 1) Database Size 2) Outbound Data Transfer and an example of an outbound transfer would be data access by an application hosted outside of your Azure DB’s data-center.

If you want to read more about SQL Azure pricing, here’s the official resource:

https://www.windowsazure.com/en-us/pricing/details/#header-4

And here are some of the blog-posts that I’ve written on the same topic:

SQL Azure got a new pricing model!

SQL Azure: Indexes are very helpful but they cost $Money$

Getting started with SQL Azure – Part 3: Provisioning and Billing Model of SQL Azure << Paras Doshi

For my Archives: Few questions answered on Windows Azure & SQL Azure MSDN forums

Standard

I normally Blog about the answers that I give out on MSDN forums. The answer on MSDN forum is generally brief and to the point and in the blog post – I expand it to cover related areas. Here are the questions for which I didn’t choose to write a blog. So I am just going to archive them for now:

Why would I not bother “REFRESHing” my Desktop ever again..

Standard

Out of the blue, a question popped in my head: “why do I click on refresh button when I am staring at my Desktop?” – I didn’t know the “Technical” reason and since curiosity got better off me – I read this, this and this and I learned that: It does nothing! Neah, it does something – it redraws the icons on desktop. What does it mean? Have you ever changed the view of the desktop and messed up the icons? I remember I have and If you “refresh” your desktop at that moment – it would redraw the icons on the desktop. other than this, it does nothing!

I just learned that and so I thought I would share that with you!

anyhow, I also wanted to see the “negative impact on performance” of “keeping the F5 (refresh) on hold while on Desktop”. See what It did to my machine:

Before:  10-25% CPU Usage

performance before refersh the desktop

With other things constant (Ceteris Paribus), I kept the F5 key (refresh) on Desktop on hold. Result? 50-65% CPU Usage. 

performance after refersh the desktop

Conclusion:

So unless I want to redraw the icons on my desktop, I would not bother refreshing my Desktop ever again..

 

How to Disable password expiration for Windows Server 2008 R2 (domain controller)?

Standard

I have written about how to disable password expiration for Windows Server 2008 R2 if it is NOT a domain controller. You can Find that post here: http://parasdoshi.com/2012/04/19/how-to-disable-the-password-expiration-policy-in-windows-server-2008-r2-demo-machine/

Now, if you are looking to disable the password for the Windows Server 2008 R2 dev. machine which is also a Domain Controller then follow these steps:

1) If you go to “Local security policy- you’ll see the options but it is not going to allow you to change the setting even if you are logged in as domain administrator.

windows server 2008 r2 disable password expiration local security policy

2. So we need an alternate path to edit the password expiration policy.

Go to Start > Administrative Tools > Group Policy Management

3. Here click on “edit” for the default domain policy for the domain of your choice:

windows server 2008 r2 disable password expiration group policy management

4. Go To Policies > Windows Settings > Security Settings > Account Policies > Password Policy

windows server 2008 r2 disable password expiration group policy management editor

5. Change the Password Policy!

Note that changing your password policy to disable password expiration is a security vulnerability. It’s applicable for your Demo Machine only. Or your Dev Machine. The reason I am documenting it that I do not want to change the password of Windows Server on which I have my Sharepoint BI dev environment Setup. It’s MY Dev Environment and I am NOT sharing it with other folks PLUS I do not anything sensitive on it, So I can afford disabling the password expiration policy.

That’s about it for this post. Happy Tweaking!

New Azure portal is ALL HTML 5!

Standard

New Azure portal is HTML 5 – so what? it just means that portal would be accessible from all devices! Do not get me wrong, I am not against Silver-light but it’s just it was little limiting because the portal was not accessible from say iPad. So from the accessibility stand-point, I am happy!

Let me share a conversation I had with @krisherpi few months back where he was not able to access Azure portal from a tablet that he had just bought – At that time, I had commented that I wish the portal was build using HTML 5 so that we could have more device options to connect to Azure portal – well, seems like Azure team was already working on that!

So I just wanted to point this out. And this is just one of the many awesome features that were discussed at Meet Windows Azure event (7 June 2012)

HTML 5 Powered Azure portal and it’s metro-styled !

new azure portal html 5

Did you knew: You can change the Location of “Desktop” folder

Standard

Did you knew that we can change the location the “Desktop” folder. Yes! The default location where files/folders are stored is “c:users<xyz>desktop” and you can change this.

But why do you want to do this?

1.FUN!

2. It’s a good practice to separate USER data and SYSTEM data. So if you have a D (Or E?) drive in your computer, you can change the location of the Desktop from C to D (or E).

So how do you do that?

1. Go to c:users<xyz>desktop

2. Right Click on the Desktop Folder and select “Properties”

dekstop folder properties 3. In the properties dialog box. Select “Location” and after entering the NEW LOCATION – click on MOVE. 

location property of the desktop folder

Click on Apply. You’re done!

I tested it on Windows 7 and Windows Server 2008 R2 and it works. It does not seem to work with Windows XP. If you know of any windows OS’s where it works – let us know in the comments!

I thought of writing this as a blog-post after this tip that I submitted was well-received. Any-who that’s about it for this post, I hope it was helpful!

How to enable RDP (Remote Desktop) for a Windows Azure Web Role?

Standard

In this blog post, we would see how to enable RDP (Remote Desktop) for a Windows Azure web role. You would see that it’s pretty straightforward. Just note that this blog post is not about How to deploy a Windows Azure web role and it’s also not about How to develop an application that can be ported to Windows Azure. Nope, This blog post is just about enabling Remote Desktop for an ‘Hello world’ application (MVC3 web app) deployed as a Windows Azure web role. And yes, I  would be using the Azure SDK 1.6 (Nov 2011). I mention this because Windows Azure is growing rapidly and growing for the better and since Microsoft folks are trying to give us the best developer experience – you may find that the exact steps may differ in future.

image

I created an app by following steps: File, New, Project, Installed templates, Visual c#, cloud, Windows Azure project, ASP.Net MVC3 web role, Internet application. Clicked OK

Then I edited the default text in the app and so now I have this nice little MVC3 web app running Locally. Just a Hello world app.

Courtesy: A default ASP.NET MVC 3 project with an account controller that uses forms authentication.

image

Now in solution explorer, right-click the project and click on “publish

3

Now, In the Publish Windows Azure application, choose the subscription. If you are doing it for the first time, click on “Sign in to download credentials”. Login with the Live ID and download a file like:

5

And click the import button in the Publish Windows Azure application wizard and point to this file and open it.

This is how it looks:

image

Click on Next >

Now if your subscription does not have any hosted services, you would be prompted to create one. Provide the Name and Location.

image

Now once you do that – In the common settings, To enable Remote Desktop you’ll need to select the check box Enable Remote Desktop for all roles. Also notice that since I just want to the app to be in staging environment, I selected the Environment as staging. To enable Remote Desktop to a web role, you can set the environment as production too. Either way, you can enable Remote Desktop.

7

Now when to click on the check box, a window would pop up and here, you will need to specify the username and password that you will use to RDP into the Web role. Provide a strong password which is important, and by default it asks you for a password that is at least 6 characters in length with upper, lower, digits and symbols in it.

image

Click Ok when done.

And press next >

The next step shows the summary:

image

Click on Publish

You need to wait for a 3-4 minutes while it is deploying and you can see the progress in the Windows Azure Activity Log. And when it completes, you can see a the Website URL:10

Now to RDP into Azure web role, Go to Azure management portal, Hosted services storage accounts and CDN, Hosted services.

Now here select your subscription, and select the instance:

11

Now on the Top Right section of the portal, In the Remote Access section, Click on Connect

image

save the .rdp file.

Then, open the file. And click on connect and provide the password which you had set earlier for RDP earlier. click on OK.

13

It’ll attempt to connect.

image

On successful connection, you would see:

Windows Azure management portal RDPConclusion:

In this blog-post, we saw how to enable remote desktop for a Windows Azure web role.