Hadoop and Spark Fundamentals: Unit 2
By External Provider
About this course
This course introduces the fundamentals of modern data processing for data engineers, analysts, and IT professionals. You will learn the basics of Hadoop MapReduce, including how it works, how to compile and run Java MapReduce programs, and how to debug and extend them using other languages. The course includes practical exercises such as word counts across multiple files, log file analysis, and large-scale text processing with datasets like Wikipedia. You will also cover advanced MapReduce features and use tools like Yarn and the Job Browser. The course then covers higher-level tools such as Apache Pig and Hive QL for managing data workflows and running SQL-like queries. Finally, you will work with Apache Spark and PySpark to gain experience with modern data analytics platforms. By the end of the course, you will have practical skills to work with big data in various environments.
What you will learn
- Understand and implement Hadoop MapReduce for distributed data processing, including compiling, running, and debugging applications.
- Apply advanced MapReduce techniques to real-world scenarios such as log analysis and large-scale text processing.
- Utilize higher-level tools like Apache Pig and Hive QL to streamline data workflows and perform complex queries.
- Gain hands-on experience with Apache Spark and PySpark for modern, scalable data analytics.