Big Data with Python & Hadoop

Big Data - these two words are heard so often nowadays. But what exactly is Big Data ? Can we, Pythonistas, enter the wonder world of Big Data ? The answer is definitely “Yes”.

This talk is an introduction to the big data processing using Apache Hadoop and Python. We’ll talk about Apache Hadoop, it’s concepts, infrastructure and how one can use Python with it. We’ll compare the speed of Python jobs under different Python implementations, including CPython, PyPy and Jython and also discuss what Python libraries are available out there to work with Apache Hadoop.

This talk is intended for beginners who want to know about Hadoop and Python or those who are already working with Hadoop but are wondering how to use it with Python or how to speed up their Python jobs.

Slides for this talk are available here and code used for benchmarks can be found here.