How to calculate and apply Z-score by group in Apache Spark: step by step
Learn how to calculate and apply Z-score by group in Apache Spark to normalize variables by category. Practical PySpark…
5 min
19 Sep 2026
How to detect and remove outliers in Apache Spark: step by step
Detect and remove outliers in Apache Spark to clean data and improve models. Practical example with IQR and Z-score in …
6 min
03 Sep 2026
How to calculate rolling averages in Apache Spark: step by step
How to calculate rolling averages in Apache Spark for time series: practical example with Window, performance and tips …
4 min
16 Aug 2026
How to detect and fix skew in joins in Apache Spark: step by step
How to detect and fix skew in joins in Apache Spark to speed up large joins. Practical techniques: analysis, broadcast,…
4 min
30 Jul 2026
How to calculate percentiles and median in Apache Spark (example)
How to calculate percentiles and median in Apache Spark: practical guide with examples in PySpark using approxQuantile …
5 min
18 Jul 2026
How to Pivot in PySpark: Turn Rows into Columns
Learn how to pivot in PySpark with groupBy, pivot and agg: turn rows into columns, handle nulls, and reverse the pivot …
5 min
13 Jul 2026
How to Read and Write Parquet in PySpark: Step by Step
Learn how to read and write Parquet files in PySpark step by step: partitionBy, write modes, compression, and how to av…
4 min
11 Jul 2026
How to use explode in PySpark: expand arrays into rows
Learn to use the explode function in PySpark to turn arrays into several rows, with a practical example and how to avoi…
4 min
09 Jul 2026
How to Handle Null Values in PySpark: Step by Step
Learn to handle null values (NULL) in PySpark with na.drop, na.fill and coalesce. A step-by-step guide with practical e…
4 min
07 Jul 2026
How to Use Window Functions in PySpark: Step by Step
Learn to use Window functions in PySpark with row_number, running sum and lag. A hands-on, step-by-step example to buil…
4 min
06 Jul 2026
How to Filter Rows in PySpark with filter and where
Learn how to filter rows in PySpark with filter() and where(): simple and combined conditions plus handy filters like i…
4 min
05 Jul 2026
How to add columns with withColumn in PySpark
Learn how to add columns to a DataFrame with withColumn in PySpark: calculated, constant and conditional columns plus t…
4 min
04 Jul 2026