PySpark – GroupBy and sort DataFrame in descending order

In this article, we will discuss how to groupby PySpark DataFrame and then sort it in descending order.

Methods Used

• groupBy(): The groupBy() function in pyspark is used for identical grouping data on DataFrame while performing an aggregate function on the grouped data.

Syntax: DataFrame.groupBy(*cols)

Parameters:

• cols→ Columns by which we need to group data

• sort(): The sort() function is used to sort one or more columns. By default, it sorts by ascending order.

Syntax: sort(*cols, ascending=True)

Parameters:

• cols→ Columns by which sorting is needed to be performed.

• PySpark DataFrame also provides orderBy() function that sorts one or more columns. By default, it orders by ascending.

Syntax: orderBy(*cols, ascending=True)

Parameters:

• cols→ Columns by which sorting is needed to be performed.
• ascending→ Boolean value to say that sorting is to be done in ascending order

Example 1: In this example, we are going to group the dataframe by name and aggregate marks. We will sort the table using the sort() function in which we will access the column using the col() function and desc() function to sort it in descending order.

Output:

Example 2: In this example, we are going to group the dataframe by name and aggregate marks. We will sort the table using the sort() function in which we will access the column within the desc() function to sort it in descending order.

Output:

Example 3: In this example, we are going to group the dataframe by name and aggregate marks. We will sort the table using the orderBy() function in which we will pass ascending parameter as False to sort the data in descending order.

Output: