What are Different Data Compression Methods in Hadoop
Compare gzip, bzip2, Snappy, LZ4, Zstandard and LZO in Hadoop, understand why splittability matters, and choose a codec for MapReduce, Parquet and ORC.
#hdfs#open-source
Tanishk · 7 min
Topic
Compare gzip, bzip2, Snappy, LZ4, Zstandard and LZO in Hadoop, understand why splittability matters, and choose a codec for MapReduce, Parquet and ORC.
#hdfs#open-source
Load files, database tables and event streams into HDFS with the HDFS shell, WebHDFS, DistCp, Spark JDBC and Kafka, now that Sqoop is retired and Flume is dormant.
#big-data#hdfs