Hadoop

Since I am a senior I have been looking into companies and have found several that are looking for “experience with Hadoop”, and since this is an application dealing with “big data” I thought it would be relevant to share with the class and see if anyone had any personal experience with it and would like to share with us 🙂

hadoop

Apache Hadoop is a framework for running applications on large cluster built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce, where the application is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, it provides a distributed file system (HDFS) that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both MapReduce and the Hadoop Distributed File System are designed so that node failures are automatically handled by the framework.

Advertisements

About caprince11

Bloom
Image | This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Hadoop

  1. cyberprofgus says:

    Nice to see your post on hadoop, Chloe. Traci has some experience with hadoop, so I would recommend having a chat with her when she visits class next Thursday. There are some researchers on campus using it, but it does not have a large presence. One downside is the requirement of a dedicated (HDFS formatted) filesystem, which may mean that you’ll need to spend energy collecting and arranging your data from elsewhere.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s