This is a Quick Post, Just want to share a command to upload local data to HDFS using Hadoop Command Line.
The command looks like:
> hadoop fs -copyFromLocal input.txt input/SqrtJob/input.txt
HDFS and MapReduce inner workings in a nutshell.
Click on the image to view larger sized image
This Blog post applies to Microsoft® HDInsight Preview for a windows machine. In this Blog Post, we’ll see how you can browse the HDFS (Hadoop Filesystem)?
1. I am assuming Hadoop Services are working without issues on your machine.
2. Now, Can you see the Hadoop Name Node Status Icon on your desktop? Yes? Great! Open it (via Browser)
3. Here’s what you’ll see:
4. Can you see the “Browse the filesystem” link? click on it. You’ll see:
5. I’ve used the /user/data lately, so Let me browse to see what’s inside this directory:
6. You can also type in the location in the check box that says Goto
7. If you’re on command line, you can do so via the command:
hadoop fs -ls /
And if you want to browse files inside a particular directory:
HDFS File System Shell Guide
In this post, we saw how to browse Hadoop File system via Hadoop Command Line & Hadoop Name Node Status
In this post, we’ll see how to create a directory in the Hadoop File System for HDInsight’s windows version.
Here are the steps:
1. You have the Microsoft® HDInsight Preview for Windows Installed on your machine. Here’s a tutorial: Installing HDInsight (Microsoft’s Hadoop) on windows 7
2. Make sure that the Cluster is up & running! To check this, I click on the “Microsoft HDInsight Dashboard” or open http://localhost:8085/ on my machine
Did you get any “wait for cluster to start..” message? No? Great! Hopefully, all your services are working perfectly and you are good to go now!
3. Let’s start the Hadoop Command Line (can you see the Icon on the Desktop? Yes? Great! Open that!)
4. Here the command to create a directory looks like:
hadoop fs -mkdir /user/data/input
The above command creates /user/data/input
5. Let’s verify that the input directory was created under /user/data
hadoop fs -ls /user/data
In this post, we saw how to create a directory in Hadoop (on windows) file system and also we saw how to list files/directory using the -ls command.