This repo contains implementations of PySpark for real-world use cases for batch data processing, streaming data processing sourced from Kafka, sockets, etc., spark optimizations, business specific bigdata processing scenario solutions, and machine learning use cases.

Simple app to test out pyspark streaming from Kafka.

twitter real-time sentiment analysis

An integration of Debezium PostgreSQL connectors with Kafka and Pyspark.

Nifi - Kafka - Pyspark merupakan sarana belajar saya untuk mengeksplorasi lebih dalam terkait penggunaan tools tersebut

Worked on Pyspark file streaming

Repository for practicing data manipulation and transformation using PySpark. Contains sample scripts for data pipelining, showcasing various techniques and best practices for handling and processing large datasets efficiently.

end-to-end image search app