testsuite: clarify scan-dump file globbing behavior

Message ID xu8f5zcxsd45.fsf@harwath.name
State New
Headers show
Series
  • testsuite: clarify scan-dump file globbing behavior
Related show

Commit Message

Frederik Harwath May 15, 2020, 10:31 a.m.
Hi,

The test commands for scanning optimization dump files
perform globbing on the argument that specifies the suffix
of the dump files to be scanned. This behavior is currently
undocumented. Furthermore, the current implementation of
"scan-dump" and related procedures yields an error whenever
the file name globbing matches more than one file (due to an
attempt to call "open" on multiple files) while a failure to
match any file at all results in an unresolved test.

This patch documents the globbing behavior. The dump
scanning procedures are changed to make the test unresolved
if globbing matches more than one file.

The procedures in scandump.exp all perform the file name expansion in
essentially the same way and I have extracted this into a new
procedure. But there is one very minor exception:

> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

> -     verbose -log "dump file: $dumpbase.$suf"


"scan-dump" is the only procedure that prints the "dump file: ..." line.
Should this be kept or is it ok to remove this as I have done in the
patch? $dumpbase.$suf does not emit the correct file name anyway
(a random example from my testing: "dump file: stdatomic-init.c.dce*")
and the name of the files can be inferred from the test name easily.

I have tested the changes by running "make check" (with a
--enable-languages=C only build, but this covers lots of uses
of the affected test procedures) and observed no regressions.

Ok to commit this to master?

Best regards,
Frederik

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter

Comments

Thomas Schwinge May 15, 2020, 1:34 p.m. | #1
Hi Frederik!

(We had internally discussed this.)

I can't formally approve testsuite patches, but did a review anyway:

On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:
> The test commands for scanning optimization dump files

> perform globbing on the argument that specifies the suffix

> of the dump files to be scanned. This behavior is currently

> undocumented. Furthermore, the current implementation of

> "scan-dump" and related procedures yields an error whenever

> the file name globbing matches more than one file (due to an

> attempt to call "open" on multiple files) while a failure to

> match any file at all results in an unresolved test.

>

> This patch documents the globbing behavior.


ACK.

> The dump

> scanning procedures are changed to make the test unresolved

> if globbing matches more than one file.


(The code changes look good, but I have not tested that specific aspect.)

> The procedures in scandump.exp all perform the file name expansion in

> essentially the same way and I have extracted this into a new

> procedure.


Thanks, that looks like I thought it should.

> But there is one very minor exception:

>

>> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>>      set dumpbase [dump-base $src [lindex $args 3]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 2]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -     verbose -log "$testcase: dump file does not exist"

>> -     verbose -log "dump file: $dumpbase.$suf"

>

> "scan-dump" is the only procedure that prints the "dump file: ..." line.

> Should this be kept or is it ok to remove this as I have done in the

> patch? $dumpbase.$suf does not emit the correct file name anyway

> (a random example from my testing: "dump file: stdatomic-init.c.dce*")

> and the name of the files can be inferred from the test name easily.


OK to remove, as far as I'm concerned.

> I have tested the changes by running "make check" (with a

> --enable-languages=C only build, but this covers lots of uses

> of the affected test procedures) and observed no regressions.


I did a full 'make check' on powerpc64le-unknown-linux-gnu -- no issues
found.

> Ok to commit this to master?


As I said, not an approval, and minor comments (see below), but still:

    Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>


Do we have to similarly also audit/alter other testsuite infrastructure
files, anything that uses '[glob [...]]'?  (..., and then generalize
'glob-dump-file' into 'glob-one-file', or similar.)  That can be done
incrementally, as far as I'm concerned.

> From 6912e03d51d360dbbcf7eb1dc8d77d08c2a6e54c Mon Sep 17 00:00:00 2001

> From: Frederik Harwath <frederik@codesourcery.com>

> Date: Fri, 15 May 2020 10:35:48 +0200

> Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

>

> The test commands for scanning optimization dump files

> perform globbing on the argument that specifies the suffix

> of the dump files to be scanned.  This behavior is currently

> undocumented.  Furthermore, the current implementation of

> "scan-dump" and similar procedures yields an error whenever

> the file name globbing matches more than one file (due to an

> attempt to call "open" on multiple files) while a failure to

> match any file results in an unresolved test.

>

> This commit documents the globbing behavior.  The dump

> scanning procedures are changed to make the test unresolved

> if globbing matches more than one file.

>

> gcc/ChangeLog:

>

> 2020-05-15  Frederik Harwath  <frederik@codesourcery.com>

>

>       * doc/sourcebuild.texi: Describe globbing of the

>       dump file scanning commands "suffix" argument.

>

> gcc/testsuite/ChangeLog:

>

> 2020-05-15  Frederik Harwath  <frederik@codesourcery.com>

>

>       * lib/scandump.exp (glob-dump-file): New proc.

>       (scan-dump): Use glob-dump-file for file name expansion.

>       (scan-dump-times): Likewise.

>       (scan-dump-dem): Likewise.

>       (scan-dump-dem-not): Likewise.

> ---

>  gcc/doc/sourcebuild.texi       |  4 ++-

>  gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------

>  2 files changed, 46 insertions(+), 12 deletions(-)

>

> diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi

> index 240d6e4b08e..b6c5a21cb71 100644

> --- a/gcc/doc/sourcebuild.texi

> +++ b/gcc/doc/sourcebuild.texi

> @@ -2888,7 +2888,9 @@ stands for zero or more unmatched lines; the whitespace after

>

>  These commands are available for @var{kind} of @code{tree}, @code{ltrans-tree},

>  @code{offload-tree}, @code{rtl}, @code{offload-rtl}, @code{ipa}, and

> -@code{wpa-ipa}.

> +@code{wpa-ipa}.  The @var{suffix} argument which describes the dump file


(You can add your new sentence addition on a new line.)

> +to be scanned may contain a glob pattern that must expand to exactly one

> +file name.


May also make this more useful/explicit:

    This is useful if, for example, if a pass has several static
    instances [correct terminology?], and depending on torture testing
    command-line flags, a different instance executes and produces a dump
    file, and so in the test case you can use a generic [put example
    here] to scan the varying dump files names.

(Or similar.)


Grüße
 Thomas


> diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp

> index d6ba350acc8..f3a991b590a 100644

> --- a/gcc/testsuite/lib/scandump.exp

> +++ b/gcc/testsuite/lib/scandump.exp

> @@ -39,6 +39,34 @@ proc dump-base { args } {

>      return $dumpbase

>  }

>

> +# Expand dump file name pattern to exactly one file.

> +# Return a single dump file name or an empty string

> +# if the pattern matches no file or more than one file.

> +#

> +# Argument 0 is the testcase name

> +# Argument 1 is the dump file glob pattern

> +proc glob-dump-file { args } {

> +

> +    set pattern [lindex $args 1]

> +    set dump_file "[glob -nocomplain $pattern]"

> +    set num_files [llength $dump_file]

> +

> +    if { $num_files != 1 } {

> +     set testcase [lindex $args 0]

> +     if { $num_files == 0 } {

> +         verbose -log "$testcase: dump file does not exist"

> +     }

> +

> +     if { $num_files > 1 } {

> +         verbose -log "$testcase: multiple dump files found"

> +     }

> +

> +     return

> +    }

> +

> +    return $dump_file

> +}

> +

>  # Utility for scanning compiler result, invoked via dg-final.

>  # Call pass if pattern is present, otherwise fail.

>  #

> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

> -     verbose -log "dump file: $dumpbase.$suf"

>       unresolved "$testname"

>       return

>      }

> @@ -113,9 +141,10 @@ proc scan-dump-times { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 4]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"

> +

> +    set pattern "$dumpbase.[lindex $args 3]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -159,9 +188,10 @@ proc scan-dump-not { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -216,9 +246,10 @@ proc scan-dump-dem { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -272,9 +303,10 @@ proc scan-dump-dem-not { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
Frederik Harwath May 19, 2020, 8:35 a.m. | #2
Hi Thomas,

Thomas Schwinge <thomas@codesourcery.com> writes:

> I can't formally approve testsuite patches, but did a review anyway:


Thanks for the review!

> On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:


>> The dump

>> scanning procedures are changed to make the test unresolved

>> if globbing matches more than one file.

>

> (The code changes look good, but I have not tested that specific aspect.)


We do not have automated tests for the testsuite commands :-), but I
have of course tested this manually.

> As I said, not an approval, and minor comments (see below), but still:

>

>     Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>

> Do we have to similarly also audit/alter other testsuite infrastructure

> files, anything that uses '[glob [...]]'?  (..., and then generalize

> 'glob-dump-file' into 'glob-one-file', or similar.)  That can be done

> incrementally, as far as I'm concerned.


I also think it would make sense to adapt similar test commands as well.

> May also make this more useful/explicit:

>

>     This is useful if, for example, if a pass has several static

>     instances [correct terminology?], and depending on torture testing

>     command-line flags, a different instance executes and produces a dump

>     file, and so in the test case you can use a generic [put example

>     here] to scan the varying dump files names.

>

> (Or similar.)


I have moved the explanation below the description of the individual
commands and added an example. See the attached revised patch.

Best regards,
Frederik

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
From 2a17749d6dbcac690d698323240438722d6119ef Mon Sep 17 00:00:00 2001
From: Frederik Harwath <frederik@codesourcery.com>

Date: Fri, 15 May 2020 10:35:48 +0200
Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

The test commands for scanning optimization dump files
perform globbing on the argument that specifies the suffix
of the dump files to be scanned.  This behavior is currently
undocumented.  Furthermore, the current implementation of
"scan-dump" and similar procedures yields an error whenever
the file name globbing matches more than one file (due to an
attempt to call "open" on multiple files) while a failure to
match any file results in an unresolved test.

This commit documents the globbing behavior.  The dump
scanning procedures are changed to make the test unresolved
if globbing matches more than one file.

gcc/ChangeLog:

2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

	* doc/sourcebuild.texi: Describe globbing of the
	dump file scanning commands "suffix" argument.

gcc/testsuite/ChangeLog:

2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

	* lib/scandump.exp (glob-dump-file): New proc.
	(scan-dump): Use glob-dump-file for file name expansion.
	(scan-dump-times): Likewise.
	(scan-dump-dem): Likewise.
	(scan-dump-dem-not): Likewise.

Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

---
 gcc/doc/sourcebuild.texi       | 13 ++++++++
 gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------
 2 files changed, 56 insertions(+), 11 deletions(-)

diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi
index 240d6e4b08e..9df4b06d460 100644
--- a/gcc/doc/sourcebuild.texi
+++ b/gcc/doc/sourcebuild.texi
@@ -2911,6 +2911,19 @@ Passes if @var{regex} does not match demangled text in the dump file with
 suffix @var{suffix}.
 @end table
 
+The @var{suffix} argument which describes the dump file to be scanned
+may contain a glob pattern that must expand to exactly one file
+name. This is useful if, e.g., different pass instances are executed
+depending on torture testing command-line flags, producing dump files
+whose names differ only in their pass instance number suffix.  For
+example, to scan instances 1, 2, 3 of a tree pass ``mypass'' for
+occurrences of the string ``code has been optimized'', use:
+@smallexample
+/* @{ dg-options "-fdump-tree-mypass" @} */
+/* @{ dg-final @{ scan-tree-dump "code has been optimized" "mypass\[1-3\]" @} @} */
+@end smallexample
+
+
 @subsubsection Check for output files
 
 @table @code
diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp
index d6ba350acc8..f3a991b590a 100644
--- a/gcc/testsuite/lib/scandump.exp
+++ b/gcc/testsuite/lib/scandump.exp
@@ -39,6 +39,34 @@ proc dump-base { args } {
     return $dumpbase
 }
 
+# Expand dump file name pattern to exactly one file.
+# Return a single dump file name or an empty string
+# if the pattern matches no file or more than one file.
+#
+# Argument 0 is the testcase name
+# Argument 1 is the dump file glob pattern
+proc glob-dump-file { args } {
+
+    set pattern [lindex $args 1]
+    set dump_file "[glob -nocomplain $pattern]"
+    set num_files [llength $dump_file]
+
+    if { $num_files != 1 } {
+	set testcase [lindex $args 0]
+	if { $num_files == 0 } {
+	    verbose -log "$testcase: dump file does not exist"
+	}
+
+	if { $num_files > 1 } {
+	    verbose -log "$testcase: multiple dump files found"
+	}
+
+	return
+    }
+
+    return $dump_file
+}
+
 # Utility for scanning compiler result, invoked via dg-final.
 # Call pass if pattern is present, otherwise fail.
 #
@@ -67,10 +95,10 @@ proc scan-dump { args } {
     set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
-	verbose -log "dump file: $dumpbase.$suf"
 	unresolved "$testname"
 	return
     }
@@ -113,9 +141,10 @@ proc scan-dump-times { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 4]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"
+
+    set pattern "$dumpbase.[lindex $args 3]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -159,9 +188,10 @@ proc scan-dump-not { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -216,9 +246,10 @@ proc scan-dump-dem { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -272,9 +303,10 @@ proc scan-dump-dem-not { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
-- 
2.17.1
Frederik Harwath May 25, 2020, 11:13 a.m. | #3
Frederik Harwath <frederik@codesourcery.com> writes:

Hi Rainer, hi Mike,
ping: https://gcc.gnu.org/pipermail/gcc-patches/2020-May/545803.html

Best regards,
Frederik

> Hi Thomas,

>

> Thomas Schwinge <thomas@codesourcery.com> writes:

>

>> I can't formally approve testsuite patches, but did a review anyway:

>

> Thanks for the review!

>

>> On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:

>

>>> The dump

>>> scanning procedures are changed to make the test unresolved

>>> if globbing matches more than one file.

>>

>> (The code changes look good, but I have not tested that specific aspect.)

>

> We do not have automated tests for the testsuite commands :-), but I

> have of course tested this manually.

>

>> As I said, not an approval, and minor comments (see below), but still:

>>

>>     Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>>

>> Do we have to similarly also audit/alter other testsuite infrastructure

>> files, anything that uses '[glob [...]]'?  (..., and then generalize

>> 'glob-dump-file' into 'glob-one-file', or similar.)  That can be done

>> incrementally, as far as I'm concerned.

>

> I also think it would make sense to adapt similar test commands as well.

>

>> May also make this more useful/explicit:

>>

>>     This is useful if, for example, if a pass has several static

>>     instances [correct terminology?], and depending on torture testing

>>     command-line flags, a different instance executes and produces a dump

>>     file, and so in the test case you can use a generic [put example

>>     here] to scan the varying dump files names.

>>

>> (Or similar.)

>

> I have moved the explanation below the description of the individual

> commands and added an example. See the attached revised patch.

>

> Best regards,

> Frederik

>

> From 2a17749d6dbcac690d698323240438722d6119ef Mon Sep 17 00:00:00 2001

> From: Frederik Harwath <frederik@codesourcery.com>

> Date: Fri, 15 May 2020 10:35:48 +0200

> Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

>

> The test commands for scanning optimization dump files

> perform globbing on the argument that specifies the suffix

> of the dump files to be scanned.  This behavior is currently

> undocumented.  Furthermore, the current implementation of

> "scan-dump" and similar procedures yields an error whenever

> the file name globbing matches more than one file (due to an

> attempt to call "open" on multiple files) while a failure to

> match any file results in an unresolved test.

>

> This commit documents the globbing behavior.  The dump

> scanning procedures are changed to make the test unresolved

> if globbing matches more than one file.

>

> gcc/ChangeLog:

>

> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>

>       * doc/sourcebuild.texi: Describe globbing of the

>       dump file scanning commands "suffix" argument.

>

> gcc/testsuite/ChangeLog:

>

> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>

>       * lib/scandump.exp (glob-dump-file): New proc.

>       (scan-dump): Use glob-dump-file for file name expansion.

>       (scan-dump-times): Likewise.

>       (scan-dump-dem): Likewise.

>       (scan-dump-dem-not): Likewise.

>

> Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

> ---

>  gcc/doc/sourcebuild.texi       | 13 ++++++++

>  gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------

>  2 files changed, 56 insertions(+), 11 deletions(-)

>

> diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi

> index 240d6e4b08e..9df4b06d460 100644

> --- a/gcc/doc/sourcebuild.texi

> +++ b/gcc/doc/sourcebuild.texi

> @@ -2911,6 +2911,19 @@ Passes if @var{regex} does not match demangled text in the dump file with

>  suffix @var{suffix}.

>  @end table

>

> +The @var{suffix} argument which describes the dump file to be scanned

> +may contain a glob pattern that must expand to exactly one file

> +name. This is useful if, e.g., different pass instances are executed

> +depending on torture testing command-line flags, producing dump files

> +whose names differ only in their pass instance number suffix.  For

> +example, to scan instances 1, 2, 3 of a tree pass ``mypass'' for

> +occurrences of the string ``code has been optimized'', use:

> +@smallexample

> +/* @{ dg-options "-fdump-tree-mypass" @} */

> +/* @{ dg-final @{ scan-tree-dump "code has been optimized" "mypass\[1-3\]" @} @} */

> +@end smallexample

> +

> +

>  @subsubsection Check for output files

>

>  @table @code

> diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp

> index d6ba350acc8..f3a991b590a 100644

> --- a/gcc/testsuite/lib/scandump.exp

> +++ b/gcc/testsuite/lib/scandump.exp

> @@ -39,6 +39,34 @@ proc dump-base { args } {

>      return $dumpbase

>  }

>

> +# Expand dump file name pattern to exactly one file.

> +# Return a single dump file name or an empty string

> +# if the pattern matches no file or more than one file.

> +#

> +# Argument 0 is the testcase name

> +# Argument 1 is the dump file glob pattern

> +proc glob-dump-file { args } {

> +

> +    set pattern [lindex $args 1]

> +    set dump_file "[glob -nocomplain $pattern]"

> +    set num_files [llength $dump_file]

> +

> +    if { $num_files != 1 } {

> +     set testcase [lindex $args 0]

> +     if { $num_files == 0 } {

> +         verbose -log "$testcase: dump file does not exist"

> +     }

> +

> +     if { $num_files > 1 } {

> +         verbose -log "$testcase: multiple dump files found"

> +     }

> +

> +     return

> +    }

> +

> +    return $dump_file

> +}

> +

>  # Utility for scanning compiler result, invoked via dg-final.

>  # Call pass if pattern is present, otherwise fail.

>  #

> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

> -     verbose -log "dump file: $dumpbase.$suf"

>       unresolved "$testname"

>       return

>      }

> @@ -113,9 +141,10 @@ proc scan-dump-times { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 4]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"

> +

> +    set pattern "$dumpbase.[lindex $args 3]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -159,9 +188,10 @@ proc scan-dump-not { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -216,9 +246,10 @@ proc scan-dump-dem { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> @@ -272,9 +303,10 @@ proc scan-dump-dem-not { args } {

>      set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""

>      set src [file tail $filename]

>      set dumpbase [dump-base $src [lindex $args 3]]

> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

> +

> +    set pattern "$dumpbase.[lindex $args 2]"

> +    set output_file "[glob-dump-file $testcase $pattern]"

>      if { $output_file == "" } {

> -     verbose -log "$testcase: dump file does not exist"

>       unresolved "$testname"

>       return

>      }

> --

> 2.17.1

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
Frederik Harwath June 3, 2020, 5:37 a.m. | #4
Frederik Harwath <frederik@codesourcery.com> writes:

ping :-)

> Frederik Harwath <frederik@codesourcery.com> writes:

>

> Hi Rainer, hi Mike,

> ping: https://gcc.gnu.org/pipermail/gcc-patches/2020-May/545803.html

>

> Best regards,

> Frederik

>

>> Hi Thomas,

>>

>> Thomas Schwinge <thomas@codesourcery.com> writes:

>>

>>> I can't formally approve testsuite patches, but did a review anyway:

>>

>> Thanks for the review!

>>

>>> On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:

>>

>>>> The dump

>>>> scanning procedures are changed to make the test unresolved

>>>> if globbing matches more than one file.

>>>

>>> (The code changes look good, but I have not tested that specific aspect.)

>>

>> We do not have automated tests for the testsuite commands :-), but I

>> have of course tested this manually.

>>

>>> As I said, not an approval, and minor comments (see below), but still:

>>>

>>>     Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>>>

>>> Do we have to similarly also audit/alter other testsuite infrastructure

>>> files, anything that uses '[glob [...]]'?  (..., and then generalize

>>> 'glob-dump-file' into 'glob-one-file', or similar.)  That can be done

>>> incrementally, as far as I'm concerned.

>>

>> I also think it would make sense to adapt similar test commands as well.

>>

>>> May also make this more useful/explicit:

>>>

>>>     This is useful if, for example, if a pass has several static

>>>     instances [correct terminology?], and depending on torture testing

>>>     command-line flags, a different instance executes and produces a dump

>>>     file, and so in the test case you can use a generic [put example

>>>     here] to scan the varying dump files names.

>>>

>>> (Or similar.)

>>

>> I have moved the explanation below the description of the individual

>> commands and added an example. See the attached revised patch.

>>

>> Best regards,

>> Frederik

>>

>> From 2a17749d6dbcac690d698323240438722d6119ef Mon Sep 17 00:00:00 2001

>> From: Frederik Harwath <frederik@codesourcery.com>

>> Date: Fri, 15 May 2020 10:35:48 +0200

>> Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

>>

>> The test commands for scanning optimization dump files

>> perform globbing on the argument that specifies the suffix

>> of the dump files to be scanned.  This behavior is currently

>> undocumented.  Furthermore, the current implementation of

>> "scan-dump" and similar procedures yields an error whenever

>> the file name globbing matches more than one file (due to an

>> attempt to call "open" on multiple files) while a failure to

>> match any file results in an unresolved test.

>>

>> This commit documents the globbing behavior.  The dump

>> scanning procedures are changed to make the test unresolved

>> if globbing matches more than one file.

>>

>> gcc/ChangeLog:

>>

>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>

>>      * doc/sourcebuild.texi: Describe globbing of the

>>      dump file scanning commands "suffix" argument.

>>

>> gcc/testsuite/ChangeLog:

>>

>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>

>>      * lib/scandump.exp (glob-dump-file): New proc.

>>      (scan-dump): Use glob-dump-file for file name expansion.

>>      (scan-dump-times): Likewise.

>>      (scan-dump-dem): Likewise.

>>      (scan-dump-dem-not): Likewise.

>>

>> Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>> ---

>>  gcc/doc/sourcebuild.texi       | 13 ++++++++

>>  gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------

>>  2 files changed, 56 insertions(+), 11 deletions(-)

>>

>> diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi

>> index 240d6e4b08e..9df4b06d460 100644

>> --- a/gcc/doc/sourcebuild.texi

>> +++ b/gcc/doc/sourcebuild.texi

>> @@ -2911,6 +2911,19 @@ Passes if @var{regex} does not match demangled text in the dump file with

>>  suffix @var{suffix}.

>>  @end table

>>

>> +The @var{suffix} argument which describes the dump file to be scanned

>> +may contain a glob pattern that must expand to exactly one file

>> +name. This is useful if, e.g., different pass instances are executed

>> +depending on torture testing command-line flags, producing dump files

>> +whose names differ only in their pass instance number suffix.  For

>> +example, to scan instances 1, 2, 3 of a tree pass ``mypass'' for

>> +occurrences of the string ``code has been optimized'', use:

>> +@smallexample

>> +/* @{ dg-options "-fdump-tree-mypass" @} */

>> +/* @{ dg-final @{ scan-tree-dump "code has been optimized" "mypass\[1-3\]" @} @} */

>> +@end smallexample

>> +

>> +

>>  @subsubsection Check for output files

>>

>>  @table @code

>> diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp

>> index d6ba350acc8..f3a991b590a 100644

>> --- a/gcc/testsuite/lib/scandump.exp

>> +++ b/gcc/testsuite/lib/scandump.exp

>> @@ -39,6 +39,34 @@ proc dump-base { args } {

>>      return $dumpbase

>>  }

>>

>> +# Expand dump file name pattern to exactly one file.

>> +# Return a single dump file name or an empty string

>> +# if the pattern matches no file or more than one file.

>> +#

>> +# Argument 0 is the testcase name

>> +# Argument 1 is the dump file glob pattern

>> +proc glob-dump-file { args } {

>> +

>> +    set pattern [lindex $args 1]

>> +    set dump_file "[glob -nocomplain $pattern]"

>> +    set num_files [llength $dump_file]

>> +

>> +    if { $num_files != 1 } {

>> +    set testcase [lindex $args 0]

>> +    if { $num_files == 0 } {

>> +        verbose -log "$testcase: dump file does not exist"

>> +    }

>> +

>> +    if { $num_files > 1 } {

>> +        verbose -log "$testcase: multiple dump files found"

>> +    }

>> +

>> +    return

>> +    }

>> +

>> +    return $dump_file

>> +}

>> +

>>  # Utility for scanning compiler result, invoked via dg-final.

>>  # Call pass if pattern is present, otherwise fail.

>>  #

>> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>>      set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""

>>      set src [file tail $filename]

>>      set dumpbase [dump-base $src [lindex $args 3]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 2]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -    verbose -log "$testcase: dump file does not exist"

>> -    verbose -log "dump file: $dumpbase.$suf"

>>      unresolved "$testname"

>>      return

>>      }

>> @@ -113,9 +141,10 @@ proc scan-dump-times { args } {

>>      set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"

>>      set src [file tail $filename]

>>      set dumpbase [dump-base $src [lindex $args 4]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 3]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -    verbose -log "$testcase: dump file does not exist"

>>      unresolved "$testname"

>>      return

>>      }

>> @@ -159,9 +188,10 @@ proc scan-dump-not { args } {

>>      set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""

>>      set src [file tail $filename]

>>      set dumpbase [dump-base $src [lindex $args 3]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 2]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -    verbose -log "$testcase: dump file does not exist"

>>      unresolved "$testname"

>>      return

>>      }

>> @@ -216,9 +246,10 @@ proc scan-dump-dem { args } {

>>      set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""

>>      set src [file tail $filename]

>>      set dumpbase [dump-base $src [lindex $args 3]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 2]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -    verbose -log "$testcase: dump file does not exist"

>>      unresolved "$testname"

>>      return

>>      }

>> @@ -272,9 +303,10 @@ proc scan-dump-dem-not { args } {

>>      set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""

>>      set src [file tail $filename]

>>      set dumpbase [dump-base $src [lindex $args 3]]

>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>> +

>> +    set pattern "$dumpbase.[lindex $args 2]"

>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>      if { $output_file == "" } {

>> -    verbose -log "$testcase: dump file does not exist"

>>      unresolved "$testname"

>>      return

>>      }

>> --

>> 2.17.1

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
Thomas Schwinge June 24, 2020, 10:04 a.m. | #5
Hi!

Given that nobody is available to review/approve this patch, and it
cannot really cause any harm, will my (old) review/"non-formal approval"
be sufficient for Frederik to push this?  Or, in other words: Frederik,
please push if nobody objects within the next week.


Grüße
 Thomas


On 2020-06-03T07:37:01+0200, Frederik Harwath <frederik@codesourcery.com> wrote:
> Frederik Harwath <frederik@codesourcery.com> writes:

>

> ping :-)

>

>> Frederik Harwath <frederik@codesourcery.com> writes:

>>

>> Hi Rainer, hi Mike,

>> ping: https://gcc.gnu.org/pipermail/gcc-patches/2020-May/545803.html

>>

>> Best regards,

>> Frederik

>>

>>> Hi Thomas,

>>>

>>> Thomas Schwinge <thomas@codesourcery.com> writes:

>>>

>>>> I can't formally approve testsuite patches, but did a review anyway:

>>>

>>> Thanks for the review!

>>>

>>>> On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:

>>>

>>>>> The dump

>>>>> scanning procedures are changed to make the test unresolved

>>>>> if globbing matches more than one file.

>>>>

>>>> (The code changes look good, but I have not tested that specific aspect.)

>>>

>>> We do not have automated tests for the testsuite commands :-), but I

>>> have of course tested this manually.

>>>

>>>> As I said, not an approval, and minor comments (see below), but still:

>>>>

>>>>     Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>>>>

>>>> Do we have to similarly also audit/alter other testsuite infrastructure

>>>> files, anything that uses '[glob [...]]'?  (..., and then generalize

>>>> 'glob-dump-file' into 'glob-one-file', or similar.)  That can be done

>>>> incrementally, as far as I'm concerned.

>>>

>>> I also think it would make sense to adapt similar test commands as well.

>>>

>>>> May also make this more useful/explicit:

>>>>

>>>>     This is useful if, for example, if a pass has several static

>>>>     instances [correct terminology?], and depending on torture testing

>>>>     command-line flags, a different instance executes and produces a dump

>>>>     file, and so in the test case you can use a generic [put example

>>>>     here] to scan the varying dump files names.

>>>>

>>>> (Or similar.)

>>>

>>> I have moved the explanation below the description of the individual

>>> commands and added an example. See the attached revised patch.

>>>

>>> Best regards,

>>> Frederik

>>>

>>> From 2a17749d6dbcac690d698323240438722d6119ef Mon Sep 17 00:00:00 2001

>>> From: Frederik Harwath <frederik@codesourcery.com>

>>> Date: Fri, 15 May 2020 10:35:48 +0200

>>> Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

>>>

>>> The test commands for scanning optimization dump files

>>> perform globbing on the argument that specifies the suffix

>>> of the dump files to be scanned.  This behavior is currently

>>> undocumented.  Furthermore, the current implementation of

>>> "scan-dump" and similar procedures yields an error whenever

>>> the file name globbing matches more than one file (due to an

>>> attempt to call "open" on multiple files) while a failure to

>>> match any file results in an unresolved test.

>>>

>>> This commit documents the globbing behavior.  The dump

>>> scanning procedures are changed to make the test unresolved

>>> if globbing matches more than one file.

>>>

>>> gcc/ChangeLog:

>>>

>>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>>

>>>     * doc/sourcebuild.texi: Describe globbing of the

>>>     dump file scanning commands "suffix" argument.

>>>

>>> gcc/testsuite/ChangeLog:

>>>

>>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>>

>>>     * lib/scandump.exp (glob-dump-file): New proc.

>>>     (scan-dump): Use glob-dump-file for file name expansion.

>>>     (scan-dump-times): Likewise.

>>>     (scan-dump-dem): Likewise.

>>>     (scan-dump-dem-not): Likewise.

>>>

>>> Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>>> ---

>>>  gcc/doc/sourcebuild.texi       | 13 ++++++++

>>>  gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------

>>>  2 files changed, 56 insertions(+), 11 deletions(-)

>>>

>>> diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi

>>> index 240d6e4b08e..9df4b06d460 100644

>>> --- a/gcc/doc/sourcebuild.texi

>>> +++ b/gcc/doc/sourcebuild.texi

>>> @@ -2911,6 +2911,19 @@ Passes if @var{regex} does not match demangled text in the dump file with

>>>  suffix @var{suffix}.

>>>  @end table

>>>

>>> +The @var{suffix} argument which describes the dump file to be scanned

>>> +may contain a glob pattern that must expand to exactly one file

>>> +name. This is useful if, e.g., different pass instances are executed

>>> +depending on torture testing command-line flags, producing dump files

>>> +whose names differ only in their pass instance number suffix.  For

>>> +example, to scan instances 1, 2, 3 of a tree pass ``mypass'' for

>>> +occurrences of the string ``code has been optimized'', use:

>>> +@smallexample

>>> +/* @{ dg-options "-fdump-tree-mypass" @} */

>>> +/* @{ dg-final @{ scan-tree-dump "code has been optimized" "mypass\[1-3\]" @} @} */

>>> +@end smallexample

>>> +

>>> +

>>>  @subsubsection Check for output files

>>>

>>>  @table @code

>>> diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp

>>> index d6ba350acc8..f3a991b590a 100644

>>> --- a/gcc/testsuite/lib/scandump.exp

>>> +++ b/gcc/testsuite/lib/scandump.exp

>>> @@ -39,6 +39,34 @@ proc dump-base { args } {

>>>      return $dumpbase

>>>  }

>>>

>>> +# Expand dump file name pattern to exactly one file.

>>> +# Return a single dump file name or an empty string

>>> +# if the pattern matches no file or more than one file.

>>> +#

>>> +# Argument 0 is the testcase name

>>> +# Argument 1 is the dump file glob pattern

>>> +proc glob-dump-file { args } {

>>> +

>>> +    set pattern [lindex $args 1]

>>> +    set dump_file "[glob -nocomplain $pattern]"

>>> +    set num_files [llength $dump_file]

>>> +

>>> +    if { $num_files != 1 } {

>>> +   set testcase [lindex $args 0]

>>> +   if { $num_files == 0 } {

>>> +       verbose -log "$testcase: dump file does not exist"

>>> +   }

>>> +

>>> +   if { $num_files > 1 } {

>>> +       verbose -log "$testcase: multiple dump files found"

>>> +   }

>>> +

>>> +   return

>>> +    }

>>> +

>>> +    return $dump_file

>>> +}

>>> +

>>>  # Utility for scanning compiler result, invoked via dg-final.

>>>  # Call pass if pattern is present, otherwise fail.

>>>  #

>>> @@ -67,10 +95,10 @@ proc scan-dump { args } {

>>>      set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""

>>>      set src [file tail $filename]

>>>      set dumpbase [dump-base $src [lindex $args 3]]

>>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>>> +

>>> +    set pattern "$dumpbase.[lindex $args 2]"

>>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>>      if { $output_file == "" } {

>>> -   verbose -log "$testcase: dump file does not exist"

>>> -   verbose -log "dump file: $dumpbase.$suf"

>>>     unresolved "$testname"

>>>     return

>>>      }

>>> @@ -113,9 +141,10 @@ proc scan-dump-times { args } {

>>>      set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"

>>>      set src [file tail $filename]

>>>      set dumpbase [dump-base $src [lindex $args 4]]

>>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"

>>> +

>>> +    set pattern "$dumpbase.[lindex $args 3]"

>>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>>      if { $output_file == "" } {

>>> -   verbose -log "$testcase: dump file does not exist"

>>>     unresolved "$testname"

>>>     return

>>>      }

>>> @@ -159,9 +188,10 @@ proc scan-dump-not { args } {

>>>      set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""

>>>      set src [file tail $filename]

>>>      set dumpbase [dump-base $src [lindex $args 3]]

>>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>>> +

>>> +    set pattern "$dumpbase.[lindex $args 2]"

>>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>>      if { $output_file == "" } {

>>> -   verbose -log "$testcase: dump file does not exist"

>>>     unresolved "$testname"

>>>     return

>>>      }

>>> @@ -216,9 +246,10 @@ proc scan-dump-dem { args } {

>>>      set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""

>>>      set src [file tail $filename]

>>>      set dumpbase [dump-base $src [lindex $args 3]]

>>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>>> +

>>> +    set pattern "$dumpbase.[lindex $args 2]"

>>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>>      if { $output_file == "" } {

>>> -   verbose -log "$testcase: dump file does not exist"

>>>     unresolved "$testname"

>>>     return

>>>      }

>>> @@ -272,9 +303,10 @@ proc scan-dump-dem-not { args } {

>>>      set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""

>>>      set src [file tail $filename]

>>>      set dumpbase [dump-base $src [lindex $args 3]]

>>> -    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"

>>> +

>>> +    set pattern "$dumpbase.[lindex $args 2]"

>>> +    set output_file "[glob-dump-file $testcase $pattern]"

>>>      if { $output_file == "" } {

>>> -   verbose -log "$testcase: dump file does not exist"

>>>     unresolved "$testname"

>>>     return

>>>      }

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
Andreas Krebbel via Gcc-patches June 26, 2020, 5:14 p.m. | #6
On Jun 2, 2020, at 10:37 PM, Frederik Harwath <frederik@codesourcery.com> wrote:
> 

> Frederik Harwath <frederik@codesourcery.com> writes:

> 

> ping :-)


Ok.

>> Frederik Harwath <frederik@codesourcery.com> writes:

>> 

>> Hi Rainer, hi Mike,

>> ping: https://gcc.gnu.org/pipermail/gcc-patches/2020-May/545803.html

>> 

>> Best regards,

>> Frederik

>> 

>>> Hi Thomas,

>>> 

>>> Thomas Schwinge <thomas@codesourcery.com> writes:

>>> 

>>>> I can't formally approve testsuite patches, but did a review anyway:

>>> 

>>> Thanks for the review!

>>> 

>>>> On 2020-05-15T12:31:54+0200, Frederik Harwath <frederik@codesourcery.com> wrote:

>>> 

>>>>> The dump

>>>>> scanning procedures are changed to make the test unresolved

>>>>> if globbing matches more than one file.

>>>> 

>>>> (The code changes look good, but I have not tested that specific aspect.)

>>> 

>>> We do not have automated tests for the testsuite commands :-), but I

>>> have of course tested this manually.

>>> 

>>>> As I said, not an approval, and minor comments (see below), but still:

>>>> 

>>>>    Reviewed-by: Thomas Schwinge <thomas@codesourcery.com>

>>>> 

>>>> Do we have to similarly also audit/alter other testsuite infrastructure

>>>> files, anything that uses '[glob [...]]'?  (..., and then generalize

>>>> 'glob-dump-file' into 'glob-one-file', or similar.)  That can be done

>>>> incrementally, as far as I'm concerned.

>>> 

>>> I also think it would make sense to adapt similar test commands as well.

>>> 

>>>> May also make this more useful/explicit:

>>>> 

>>>>    This is useful if, for example, if a pass has several static

>>>>    instances [correct terminology?], and depending on torture testing

>>>>    command-line flags, a different instance executes and produces a dump

>>>>    file, and so in the test case you can use a generic [put example

>>>>    here] to scan the varying dump files names.

>>>> 

>>>> (Or similar.)

>>> 

>>> I have moved the explanation below the description of the individual

>>> commands and added an example. See the attached revised patch.

>>> 

>>> Best regards,

>>> Frederik

>>> 

>>> From 2a17749d6dbcac690d698323240438722d6119ef Mon Sep 17 00:00:00 2001

>>> From: Frederik Harwath <frederik@codesourcery.com>

>>> Date: Fri, 15 May 2020 10:35:48 +0200

>>> Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

>>> 

>>> The test commands for scanning optimization dump files

>>> perform globbing on the argument that specifies the suffix

>>> of the dump files to be scanned.  This behavior is currently

>>> undocumented.  Furthermore, the current implementation of

>>> "scan-dump" and similar procedures yields an error whenever

>>> the file name globbing matches more than one file (due to an

>>> attempt to call "open" on multiple files) while a failure to

>>> match any file results in an unresolved test.

>>> 

>>> This commit documents the globbing behavior.  The dump

>>> scanning procedures are changed to make the test unresolved

>>> if globbing matches more than one file.

>>> 

>>> gcc/ChangeLog:

>>> 

>>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>> 

>>>     * doc/sourcebuild.texi: Describe globbing of the

>>>     dump file scanning commands "suffix" argument.

>>> 

>>> gcc/testsuite/ChangeLog:

>>> 

>>> 2020-05-19  Frederik Harwath  <frederik@codesourcery.com>

>>> 

>>>     * lib/scandump.exp (glob-dump-file): New proc.

>>>     (scan-dump): Use glob-dump-file for file name expansion.

>>>     (scan-dump-times): Likewise.

>>>     (scan-dump-dem): Likewise.

>>>     (scan-dump-dem-not): Likewise.

Patch

From 6912e03d51d360dbbcf7eb1dc8d77d08c2a6e54c Mon Sep 17 00:00:00 2001
From: Frederik Harwath <frederik@codesourcery.com>
Date: Fri, 15 May 2020 10:35:48 +0200
Subject: [PATCH] testsuite: clarify scan-dump file globbing behavior

The test commands for scanning optimization dump files
perform globbing on the argument that specifies the suffix
of the dump files to be scanned.  This behavior is currently
undocumented.  Furthermore, the current implementation of
"scan-dump" and similar procedures yields an error whenever
the file name globbing matches more than one file (due to an
attempt to call "open" on multiple files) while a failure to
match any file results in an unresolved test.

This commit documents the globbing behavior.  The dump
scanning procedures are changed to make the test unresolved
if globbing matches more than one file.

gcc/ChangeLog:

2020-05-15  Frederik Harwath  <frederik@codesourcery.com>

	* doc/sourcebuild.texi: Describe globbing of the
	dump file scanning commands "suffix" argument.

gcc/testsuite/ChangeLog:

2020-05-15  Frederik Harwath  <frederik@codesourcery.com>

	* lib/scandump.exp (glob-dump-file): New proc.
	(scan-dump): Use glob-dump-file for file name expansion.
	(scan-dump-times): Likewise.
	(scan-dump-dem): Likewise.
	(scan-dump-dem-not): Likewise.
---
 gcc/doc/sourcebuild.texi       |  4 ++-
 gcc/testsuite/lib/scandump.exp | 54 +++++++++++++++++++++++++++-------
 2 files changed, 46 insertions(+), 12 deletions(-)

diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi
index 240d6e4b08e..b6c5a21cb71 100644
--- a/gcc/doc/sourcebuild.texi
+++ b/gcc/doc/sourcebuild.texi
@@ -2888,7 +2888,9 @@  stands for zero or more unmatched lines; the whitespace after
 
 These commands are available for @var{kind} of @code{tree}, @code{ltrans-tree},
 @code{offload-tree}, @code{rtl}, @code{offload-rtl}, @code{ipa}, and
-@code{wpa-ipa}.
+@code{wpa-ipa}.  The @var{suffix} argument which describes the dump file
+to be scanned may contain a glob pattern that must expand to exactly one
+file name.
 
 @table @code
 @item scan-@var{kind}-dump @var{regex} @var{suffix} [@{ target/xfail @var{selector} @}]
diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp
index d6ba350acc8..f3a991b590a 100644
--- a/gcc/testsuite/lib/scandump.exp
+++ b/gcc/testsuite/lib/scandump.exp
@@ -39,6 +39,34 @@  proc dump-base { args } {
     return $dumpbase
 }
 
+# Expand dump file name pattern to exactly one file.
+# Return a single dump file name or an empty string
+# if the pattern matches no file or more than one file.
+#
+# Argument 0 is the testcase name
+# Argument 1 is the dump file glob pattern
+proc glob-dump-file { args } {
+
+    set pattern [lindex $args 1]
+    set dump_file "[glob -nocomplain $pattern]"
+    set num_files [llength $dump_file]
+
+    if { $num_files != 1 } {
+	set testcase [lindex $args 0]
+	if { $num_files == 0 } {
+	    verbose -log "$testcase: dump file does not exist"
+	}
+
+	if { $num_files > 1 } {
+	    verbose -log "$testcase: multiple dump files found"
+	}
+
+	return
+    }
+
+    return $dump_file
+}
+
 # Utility for scanning compiler result, invoked via dg-final.
 # Call pass if pattern is present, otherwise fail.
 #
@@ -67,10 +95,10 @@  proc scan-dump { args } {
     set testname "$testcase scan-[lindex $args 0]-dump $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
-	verbose -log "dump file: $dumpbase.$suf"
 	unresolved "$testname"
 	return
     }
@@ -113,9 +141,10 @@  proc scan-dump-times { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-times $suf \"$printable_pattern\" [lindex $args 2]"
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 4]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 3]]"
+
+    set pattern "$dumpbase.[lindex $args 3]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -159,9 +188,10 @@  proc scan-dump-not { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-not $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -216,9 +246,10 @@  proc scan-dump-dem { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-dem $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
@@ -272,9 +303,10 @@  proc scan-dump-dem-not { args } {
     set testname "$testcase scan-[lindex $args 0]-dump-dem-not $suf \"$printable_pattern\""
     set src [file tail $filename]
     set dumpbase [dump-base $src [lindex $args 3]]
-    set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
+
+    set pattern "$dumpbase.[lindex $args 2]"
+    set output_file "[glob-dump-file $testcase $pattern]"
     if { $output_file == "" } {
-	verbose -log "$testcase: dump file does not exist"
 	unresolved "$testname"
 	return
     }
-- 
2.17.1