Sentiment Analysis using LingPipe on windows 7:

Standard

In this post, I’ll point you to the resource using which you can perform sentiment analysis using LingPipe on a windows OS. Along with that I’ll share couple of issues that I ran into when I was trying to run this demo on a Windows 7:

So first up, here’s the resource:

http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.html

Now here are a couple of issues that I had:

1. Error: could not find or load the main class PolarityBasic

lingpipe could not find or load main class polaritybasic

To solve this error, you’ll need to build the files given under the C:lingpipe-4.1.0demostutorialsentiment – we use ANT for this. Let’s see how to do that:

2. Building sentiment.jar using ant jar

After successfully downloading ant on windows and setting the ANT_HOME variable to c:apache-ant-1.8.4 – I was still getting the error that ant is not a recognized command.

So I ran following commands:

C:>set ANT_HOME=C:apache-ant-1.8.1
C:>set JAVA_HOME=C:jdk1.6.0_24
C:>set PATH=%ANT_HOME%bin;%JAVA_HOME%bin
C:>ant -version
// it worked!

Thanks: http://stackoverflow.com/questions/5607664/installing-ant-ant-home-is-set-incorrectly-on-windows-7

Now I ran the following command:

build sentiment.jar ant lingpipe

3. In the tutorial they used POLARITY_DIR – I didn’t use that, Instead I just inputted c:review_polarity because that’s where I unzipped the movie review dataset:

movie review sentiment analysis polarity

Here’s the screenshot about the command that does basic polarity analysis:

sentiment analysis lingpipe windows

And Thanks: http://stackoverflow.com/questions/15010184/lingpipe-and-sentiment-analysis/15011482

Resource: A great tutorial for Hadoop on local windows and Azure.

Standard

Here’s the resource: http://gettingstarted.hadooponazure.com/gettingStarted.html > “HDInsight Jumpstart”

The Tutorial will teach you how to analyze log files using Hadoop Tools like MapReduce, Hive, SQooP – check it out! It works with both HDInsight for local windows as well as Hadoop on Azure:

HDInsight hadoop on windows starting guide tutorial

Conclusion:

I hope this resource helps you get started on building an end-to-end solution with Hadoop on Windows/Azure.

Presentation Tip: Change the Font Size of the Windows Command Prompt

Standard

I have researched about Presentation Tips over past few weeks, one Tip that I read again and again is that “Make Fonts larger for readability”. To that end, I just changed the font size of the command line prompt so that when I am presenting, the audience is able to see what I am typing. So If you’ve have to present to audience something via command prompt, this should be of helps:

1. Open command prompt > Right click near the Title bar > select properties

command prompt change font size

2. switch to font tab > select the font and the size. you can also change the color, layout among other things here.

command prompt change font size font color layout

3. see how it looks:

font size changed command prompt

Conclusion:

In this post, we saw how to change the Font size of the windows command prompt.

your comments are very welcome!

How to load some data to Hadoop on Windows to get started?

Standard

In this post, I want to point out that HDInsight (Hadoop on Windows) comes with a sample datasets (log files) that you can load using the command:

1. Hadoop command Line > Navigate to c:HadoopGettingStarted

2. Execute the following command:

powershell -ExecutionPolicy unrestricted –F importdata.ps1 w3c

import data to hadoop on windows file system

After you have successfully executed the command, you can sample files in /w3c/input folder:

w3c log files iis hadoop on windows

Conclusion: In this post, we saw how to load some data to Hadoop on Windows file system to get started. Your comments are very welcome.

Official Resource: http://gettingstarted.hadooponazure.com/loadingData.html

Hadoop on Windows: How to Browse the Hadoop Filesystem?

Standard

This Blog post applies to Microsoft® HDInsight Preview for a windows machine. In this Blog Post, we’ll see how you can browse the HDFS (Hadoop Filesystem)?

1. I am assuming Hadoop Services are working without issues on your machine.

2. Now, Can you see the Hadoop Name Node Status Icon on your desktop? Yes? Great! Open it (via Browser)

3. Here’s what you’ll see:

Hadoop File System Browse

4. Can you see the “Browse the filesystem” link? click on it. You’ll see:

hadoop file system name node status windows

5. I’ve used the /user/data lately, so Let me browse to see what’s inside this directory:

user data hadoop sqoop hive mapreduce

6. You can also type in the location in the check box that says Goto

7. If you’re on command line, you can do so via the command:

hadoop fs -ls /

hadoop command line list all files system

And if you want to browse files inside a particular directory:

hadoop command line sqoop mapreduce hdfs file system

Official Resource:

HDFS File System Shell Guide

Conclusion

In this post, we saw how to browse Hadoop File system via Hadoop Command Line & Hadoop Name Node Status

Related Articles:

Microsoft® HDInsight Preview for Windows: How to create a directory in Hadoop File System?

Standard

In this post, we’ll see how to create a directory in the Hadoop File System for HDInsight’s windows version.

Here are the steps:

1. You have the Microsoft® HDInsight Preview for Windows Installed on your machine. Here’s a tutorial: Installing HDInsight (Microsoft’s Hadoop) on windows 7

2. Make sure that the Cluster is up & running! To check this, I click on the “Microsoft HDInsight Dashboard” or open http://localhost:8085/ on my machine

Did you get any “wait for cluster to start..” message? No? Great! Hopefully, all your services are working perfectly and you are good to go now!

3. Let’s start the Hadoop Command Line (can you see the Icon on the Desktop? Yes? Great! Open that!)

4. Here the command to create a directory looks like:

hadoop fs -mkdir /user/data/input

The above command creates /user/data/input

5. Let’s verify that the input directory was created under /user/data

hadoop fs -ls /user/data

hadoop file system list files in a directory create directory

Conclusion:
In this post, we saw how to create a directory in Hadoop (on windows) file system and also we saw how to list files/directory using the -ls command.

Related Articles:

 

Azure PASS VC Next meeting: Kung Fu Migration to Windows Azure SQL Database

Standard

Azure PASS VC’s next meeting:

Kung Fu Migration to Windows Azure SQL Database

Speaker: Scott Klein, Technical Evangelist Microsoft

Summary: As cloud computing becomes more popular and cloud-based solutions the norm rather than the fringe, the need to efficiently migrate your database is crucial. This demo-filled session will discuss the tips and tricks, methods and strategies for migrating your on-premises SQL Server databases to Windows Azure SQL Database, AKA SQL Azure. Focusing primarily on SQL Server Data Tools and the DAC Framework, this session will focus on how these tools can make you a kung-fu migration master.

About Scott: Scott Klein is a Corporate Technical Evangelist for Microsoft focusing on Windows Azure SQL Database (AKA SQL Azure) and related cloud-ready data services. His entire career has been built around SQL Server, working with SQL Server since the 4.2 days. Prior to Microsoft he was a SQL Server MVP for several years, then followed that up by being one of the first 4 SQL Azure MVPs. Scott is the author of over ½ dozen books for both WROX and APress, including Pro SQL Azure. He can be found talking about Windows Azure SQL Database and database scalability and performance at events large and small wherever he can get people to listen, such as SQL Saturday events, local SQL Server user groups, and TechEd.

Details at http://azure.sqlpass.org/

Download the calendar file: http://www.sqlpass.org/iCal.aspx?EventID=245

How to Join Azure PASS VC’s?

If you want to stay updated on meeting announcements, please consider registering on PASS’s website and Joining our VC:

If you do not have a SQLPASS account:

a. Go to http://www.sqlpass.org/RegisterforSQLPASS.aspx

b. Fill up the required information and register

Now, After successful login/registration – Go to http://www.sqlpass.org/MYPASS.aspx

a. switch to MyChapters section

b. Now under virtual chapters, you would see a list of virtual chapters. Join the one’s you are interested in!

my PASS my Chapter Azure VC

I look forward to seeing you at next Azure PASS VC’s meeting!

Windows 7: How do you stop programs from automatically running when Windows Starts?

Standard

In this blog-post, we would see how you can stop programs from automatically running when windows start. We’ll see how you do that using MSConfig. Before we begin, let’s discuss why would you want to stop few programs from automatically running when windows starts? Well, programs consume memory and thus lowers overall system performance. So by switching unwanted (or infrequently used) programs – you are increasing system performance. So ready? here are the steps:

1) Start > type “msconfig.exe” > open msconfig.exe

2) In MSConfig.exe > switch to “startup” tab

3) Disable the programs by unchecking the check-box.

Note: Before Disabling, please verify what a program is. You do not want to disable programs like Microsoft Security Essentials or third part firewall. That would be bad!

4) So I unchecked check boxes for Google Toolbar, Skype and Spotify on my netbook – and so these programs would not run automatically when my net-book starts.

disable startup programs msconfig windows 7

5) When you’re done – Click OK > And You can choose to restart your computer – changes would be applied only after the restart.

That’s about it.

Note that here are other methods too like one’s listed here, research and use the method that suits you – But do NOT forget to switch off the unwanted programs from running automatically when windows starts.

Hadoop on Azure’s Javascript Interactive Console has basic graphing functions:

Standard

The Hadoop on Azure’s Javascript console has basic graphing functions: Bar, Line & Chart. I think this is great becuase it gives an opportunity to visualize data that’s in HDFS directly from the Interactive Javascript Console! Here’s a screenshot:

hadoop on azure bar and line graph javascript

In the console, I ran the help(“graph”) command to see how I can use this function:
Draw a graph of data
graph.bar(data, options) Bar graph
graph.line(data, options) Line graph
graph.pie(data, options) Pie chart

Parameters
data (array) Array of data objects
options (object) Options object, with
x (string) Property to use for x-axis values
y (string) Property to use for y-axis values
title (string) Graph title
orientation (number) x-axis label orientation in degrees
tickInterval (number) x-axis tick interval

Conclusion:

In this blog-post, I posted that Hadoop on Azure’s Javascript Interactive Console has basic graphing functions.

Related articles:

Visualizing MapReduce Algorithm with WordCount Example:

Standard

In this blog-post, we would visualize how MapReduce Algorithms operates to perform a Word Count on a Text Input:

First of all, for all programmers out there, Here is the code (Javascript):

[sourcecode language=”javascript”]
var map = function (key, value, context) {
var words = value.split(/[^a-zA-Z]/);
for (var i = 0; i < words.length; i++) {
if (words[i] !== "") {
context.write(words[i].toLowerCase(), 1);
}
}
};
var reduce = function (key, values, context) {
var sum = 0;
while (values.hasNext()) {
sum += parseInt(values.next());
}
context.write(key, sum);
};
[/sourcecode]

Courtesy: Microsoft Hadoop on Azure Samples

Now, let’s visualize this using an example.

Suppose the Text is “Hadoop on Azure sample Hadoop is on Windows Azure Hadoop is on Windows server” – Then this is how you can think of what happens to your input when it is processed first by Map function and then by Reduce function:

INPUTMAPREDUCE

Hadoop on Azure sample

Hadoop is on Windows Azure

Hadoop is on Windows server

Hadoop1Hadoop3
On1
Azure1on3
Sample1
Hadoop1Azure2
Is1
On1Sample1
Windows1
Azure1Is2
Hadoop1
Is1Windows2
On1
Windows1Server1
Server1

Conclusion:

In this blog post, we visualized how MapReduce Algorithm operates for a WordCount Example.