The role of Sentiment Analysis in Social Media Monitoring:

I’ve posted tutorial/resources about the Technical Side of Sentiment Analysis on this Blog. Here are the Links, if you need them:

LingPipe (Java Based) | Python | R language resource | Microsoft’s Tool “Social Analytics”

Apart from this, I’ve used other Tools per project requirements and It’s been fun designing and developing projects on “Sentiment Analysis” primarily using Social Media Monitoring. Having worked with clients on projects that use “Sentiment Analysis” – I reflected about the role of Sentiment Analysis in Social Media Monitoring. And in this blog post, I am sharing these reflections:

What is Social Media Monitoring?

Social Media Monitoring is a process of “monitoring” conversations happening on social media channels about your brand/company.

Is it NEW? Not really. The idea of monitoring or gathering data about what is being talked about the brand/company is not new. Earlier, it was newspapers and magazine-articles and now, it’s the social media channels including online news, forums and blogs and thus the name given to this process is “Social Media Monitoring”

What is Sentiment Analysis?

Analyzing data to categorize it under a “sentiment” (emotion).

Example. Is this review saying positive, negative or neutral thing about our product.

side-note: Sentiment analysis is often categorized under “Big Data Analytics”.

What’s the Role of Sentiment Analysis in Social Media Monitoring?

We’ve seen that in social media monitoring, we gather all online conversations about a brand/product/company. Now wouldn’t it be great to take the data that we have and bucket it under “Positive”, “Negative” or “Neutral” categories for further analysis?

So few questions that can be answered after we have results from sentiment analysis:

1) Are people happy or sad about our product?

2) What do they like about our product?

3) What do they hate about our service?

4) Is there a trend or seasonality in sentiment data?

Among other business insights that may be not be easily answerable with just plain text data.

Thus sentiment analysis is one of the step in social media monitoring that assists in analyzing sentiment of all the conversations happening on the social web about a brand/product.

That’s about this for this post. Here’s a related post: Three Data Collection Tips for Social Media Analytics

your comments are very welcome!

Beginner’s Guide: Sentiment Analysis using Python on Windows

This is beginner’s guide to sentiment analysis using Python NLTK on windows. We’ll start w/ installing Python and NLTK and then see how to perform sentiment analysis.

Step 1: Install Python & NLTK

I followed the steps listed on http://nltk.org/install.html

1. Search for python 2.7.3 for windows and install it.

2. Search for Python setup Tools for Windows and install it.

3. Install PIP (for win 64 bit), NLTK and PyYAML.

4. Test installation: Start>All Programs>Python27>IDLE, then type import nltk

Now,

5. Also type:

>>> Import random

6. And also install movie_reviews corpus by typing:

>>>nltk.download()

in the new window that opens, install the movie_reviews corpus.

Step 2: Sentiment Analysis

I followed the code explained in the NLTK book in the section “document classification” in ch 6 learning to classify text. Here is the section: http://nltk.org/book/ch06.html#document-classification

Using the code I was able to run the Naive Bayes Classifier to categorize text:

Conclusion:

In this post, we learned how to perform sentiment analysis using Python on windwos platform. NLTK supports classifiers other than Naive Bayes, and also there are resources that will help you increase the accuracy of the classifier. And I hope that this post acts as a starting guide for you!

Sentiment Analysis using LingPipe on windows 7: (parasdoshi.com)
Three Data Visualizations I liked this week: (parasdoshi.com)
Sentiment Analysis in R w/ Twitter data feeds (parasdoshi.com)
Second Try: Sentiment Analysis in Python : Andy Bromberg (andybromberg.com)
Sentiment Symposium Tutorial (http://sentiment.christopherpotts.net/)
What’s “Naive” about Naive Bayes Machine Learning Algorithm? (parasdoshi.com)

Three Data Visualizations I liked this week:

I have been working on creating Dashboards for one of my projects. As a part of the research, I looked at few Dashboards out their on the inter-webs. Here are three of them that I liked:

1. Social Media & Sentiment Analysis:

What I like about this Dashboard is the creative use of Data via Sentiment Analysis:

2. Microsoft Research’s Viral Search Project:

What a creative way to visualize viral content!

3. Social Media analytic’s Dashboard:

Nice one page social dashbaord!

Do you see the bottom right part of the report that shows you engagement levels by post type, if you want to compute it – here’s my blog post on that: Social Media Analytics. Facebook Page Smackdown: Status updates vs Images?

Sentiment Analysis using LingPipe on windows 7: (parasdoshi.com)

Recapping my social media activities during Jan 1 – Feb 24 2013:

Recapping my social media activities during Jan 1 – Feb 20 2013:

Interesting resource covering topics Opinion Mining, Sentiment Analysis, and Opinion Spam Detection: http://bit.ly/NGlNYD
SQL Server Technical Article: Leveraging a Hadoop cluster from SQL Server Integration Services (SSIS) http://bit.ly/13aZWhx
querying Hive using SQL-like commands: http://on.fb.me/TDQujY
[Resource] Getting Started with Microsoft HDInsight http://bit.ly/Wru0zv #Hadoop #Microsoft
Reading: “Next Generation Data Warehouse Platforms: #TDWI 2009 Q4 Best Practices Report”; Interesting stats on the total data volume that organizations manage: http://on.fb.me/ZnQ0O1
Presentation Tips: When you move, they look at you. When you stop, they look at the screen – http://bit.ly/uQ5Amf
“In pioneer days they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. We shouldn’t be trying for bigger computers, but for more systems of computers”
– Grace Hopper
Download Slides about Dr Dewitt’s PASS Talks [2008-2011] http://bit.ly/XrD2xa
WOW! Nice data visualization: interactive map Foursquare published last week? Check out Dallas: http://4sq.com/W3EIjo
[resource] Great answer on Quora on What is random forests? http://b.qr.ae/UErw4f
The marketing landscape has truly changed! Nice slide deck: http://slidesha.re/WGp3aj http://on.fb.me/VvZLs0
Learning about Google Analytics in Web Analytic’s course: http://on.fb.me/Uq8i4Z
World heat map showing Network Readiness Index 2012: http://bit.ly/12w5aqK http://on.fb.me/WPsvAf
Does High Speed Broadband Increase Economic Growth? Here’s the URL, consider thinking about it before reading it: http://onforb.es/OldnpW
Exploring Weka, a popular machine learning tool. http://on.fb.me/WXLsAM

That’s about it for this post.

If we want to read related past posts, here they are:

Let’s connect and converse on any of these people networks!

Sentiment Analysis using LingPipe on windows 7:

In this post, I’ll point you to the resource using which you can perform sentiment analysis using LingPipe on a windows OS. Along with that I’ll share couple of issues that I ran into when I was trying to run this demo on a Windows 7:

So first up, here’s the resource:

http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.html

Now here are a couple of issues that I had:

1. Error: could not find or load the main class PolarityBasic

To solve this error, you’ll need to build the files given under the C:lingpipe-4.1.0demostutorialsentiment – we use ANT for this. Let’s see how to do that:

2. Building sentiment.jar using ant jar

After successfully downloading ant on windows and setting the ANT_HOME variable to c:apache-ant-1.8.4 – I was still getting the error that ant is not a recognized command.

So I ran following commands:

C:>set ANT_HOME=C:apache-ant-1.8.1
C:>set JAVA_HOME=C:jdk1.6.0_24
C:>set PATH=%ANT_HOME%bin;%JAVA_HOME%bin
C:>ant -version
// it worked!

Thanks: http://stackoverflow.com/questions/5607664/installing-ant-ant-home-is-set-incorrectly-on-windows-7

Now I ran the following command:

3. In the tutorial they used POLARITY_DIR – I didn’t use that, Instead I just inputted c:review_polarity because that’s where I unzipped the movie review dataset: