tree-optimization/100981 - fix SLP patterns involving reductions

Message ID 9522qnpr-8691-9o4n-43or-o57s10qr7378@fhfr.qr
State New
Headers show
Series
  • tree-optimization/100981 - fix SLP patterns involving reductions
Related show

Commit Message

Richard Biener June 9, 2021, 12:52 p.m.
The following fixes the SLP FMA patterns to preserve reduction
info and the reduction vectorization to consider internal function
call defs for the reduction stmt.

Bootstrap & regtest running on x86_64-unknown-linux-gnu, Andre
verified we're not turning an ICE into a wrong-code bug
(.COMPLEX_MUL now appears in the reduction chain).

Note there's a testcase for the ICE which adds -march=armv8.3-a
and a testcase for correctness which doesn't since I didn't find
any dg effective target verifying armv8.3-a code can run.

2021-06-09  Richard Biener  <rguenther@suse.de>

	PR tree-optimization/100981
	* tree-vect-loop.c (vect_create_epilog_for_reduction): Use
	gimple_get_lhs to also handle calls.
	* tree-vect-slp-patterns.c (complex_pattern::build): Transfer
	reduction info.

	* gfortran.dg/vect/pr100981-1.f90: New testcase.
	* gfortran.dg/vect/pr100981-2.f90: Likewise.
---
 gcc/tree-vect-loop.c         | 2 +-
 gcc/tree-vect-slp-patterns.c | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

-- 
2.26.2

Comments

Aaron Sawdey via Gcc-patches June 9, 2021, 3:26 p.m. | #1
Hi Richi,

> -----Original Message-----

> From: Gcc-patches <gcc-patches-

> bounces+tamar.christina=arm.com@gcc.gnu.org> On Behalf Of Richard

> Biener

> Sent: Wednesday, June 9, 2021 1:53 PM

> To: gcc-patches@gcc.gnu.org

> Cc: Richard Sandiford <Richard.Sandiford@arm.com>

> Subject: [PATCH] tree-optimization/100981 - fix SLP patterns involving

> reductions

> 

> The following fixes the SLP FMA patterns to preserve reduction info and the

> reduction vectorization to consider internal function call defs for the

> reduction stmt.

> 

> Bootstrap & regtest running on x86_64-unknown-linux-gnu, Andre verified

> we're not turning an ICE into a wrong-code bug (.COMPLEX_MUL now

> appears in the reduction chain).

> 

> Note there's a testcase for the ICE which adds -march=armv8.3-a and a

> testcase for correctness which doesn't since I didn't find any dg effective

> target verifying armv8.3-a code can run.


It's called arm_v8_3a_complex_neon_hw

Regards,
Tamar

> 

> 2021-06-09  Richard Biener  <rguenther@suse.de>

> 

> 	PR tree-optimization/100981

> 	* tree-vect-loop.c (vect_create_epilog_for_reduction): Use

> 	gimple_get_lhs to also handle calls.

> 	* tree-vect-slp-patterns.c (complex_pattern::build): Transfer

> 	reduction info.

> 

> 	* gfortran.dg/vect/pr100981-1.f90: New testcase.

> 	* gfortran.dg/vect/pr100981-2.f90: Likewise.

> ---

>  gcc/tree-vect-loop.c         | 2 +-

>  gcc/tree-vect-slp-patterns.c | 5 ++++-

>  2 files changed, 5 insertions(+), 2 deletions(-)

> 

> diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index

> ba36348b835..ee79808472c 100644

> --- a/gcc/tree-vect-loop.c

> +++ b/gcc/tree-vect-loop.c

> @@ -5247,7 +5247,7 @@ vect_create_epilog_for_reduction (loop_vec_info

> loop_vinfo,

>        gcc_assert (STMT_VINFO_RELATED_STMT (orig_stmt_info) == stmt_info);

>      }

> 

> -  scalar_dest = gimple_assign_lhs (orig_stmt_info->stmt);

> +  scalar_dest = gimple_get_lhs (orig_stmt_info->stmt);

>    scalar_type = TREE_TYPE (scalar_dest);

>    scalar_results.create (group_size);

>    new_scalar_dest = vect_create_destination_var (scalar_dest, NULL); diff --

> git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c index

> b25655c9876..2ed49cd9edc 100644

> --- a/gcc/tree-vect-slp-patterns.c

> +++ b/gcc/tree-vect-slp-patterns.c

> @@ -544,6 +544,8 @@ complex_pattern::build (vec_info *vinfo)

>      {

>        /* Calculate the location of the statement in NODE to replace.  */

>        stmt_info = SLP_TREE_REPRESENTATIVE (node);

> +      stmt_vec_info reduc_def

> +	= STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info));

>        gimple* old_stmt = STMT_VINFO_STMT (stmt_info);

>        tree lhs_old_stmt = gimple_get_lhs (old_stmt);

>        tree type = TREE_TYPE (lhs_old_stmt); @@ -568,9 +570,10 @@

> complex_pattern::build (vec_info *vinfo)

>  	= vinfo->add_pattern_stmt (call_stmt, stmt_info);

> 

>        /* Make sure to mark the representative statement pure_slp and

> -	 relevant. */

> +	 relevant and transfer reduction info. */

>        STMT_VINFO_RELEVANT (call_stmt_info) = vect_used_in_scope;

>        STMT_SLP_TYPE (call_stmt_info) = pure_slp;

> +      STMT_VINFO_REDUC_DEF (call_stmt_info) = reduc_def;

> 

>        gimple_set_bb (call_stmt, gimple_bb (stmt_info->stmt));

>        STMT_VINFO_VECTYPE (call_stmt_info) = SLP_TREE_VECTYPE (node);

> --

> 2.26.2

Patch

diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index ba36348b835..ee79808472c 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -5247,7 +5247,7 @@  vect_create_epilog_for_reduction (loop_vec_info loop_vinfo,
       gcc_assert (STMT_VINFO_RELATED_STMT (orig_stmt_info) == stmt_info);
     }
   
-  scalar_dest = gimple_assign_lhs (orig_stmt_info->stmt);
+  scalar_dest = gimple_get_lhs (orig_stmt_info->stmt);
   scalar_type = TREE_TYPE (scalar_dest);
   scalar_results.create (group_size); 
   new_scalar_dest = vect_create_destination_var (scalar_dest, NULL);
diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c
index b25655c9876..2ed49cd9edc 100644
--- a/gcc/tree-vect-slp-patterns.c
+++ b/gcc/tree-vect-slp-patterns.c
@@ -544,6 +544,8 @@  complex_pattern::build (vec_info *vinfo)
     {
       /* Calculate the location of the statement in NODE to replace.  */
       stmt_info = SLP_TREE_REPRESENTATIVE (node);
+      stmt_vec_info reduc_def
+	= STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info));
       gimple* old_stmt = STMT_VINFO_STMT (stmt_info);
       tree lhs_old_stmt = gimple_get_lhs (old_stmt);
       tree type = TREE_TYPE (lhs_old_stmt);
@@ -568,9 +570,10 @@  complex_pattern::build (vec_info *vinfo)
 	= vinfo->add_pattern_stmt (call_stmt, stmt_info);
 
       /* Make sure to mark the representative statement pure_slp and
-	 relevant. */
+	 relevant and transfer reduction info. */
       STMT_VINFO_RELEVANT (call_stmt_info) = vect_used_in_scope;
       STMT_SLP_TYPE (call_stmt_info) = pure_slp;
+      STMT_VINFO_REDUC_DEF (call_stmt_info) = reduc_def;
 
       gimple_set_bb (call_stmt, gimple_bb (stmt_info->stmt));
       STMT_VINFO_VECTYPE (call_stmt_info) = SLP_TREE_VECTYPE (node);