[v2,2/2] testsuite: Fix run-time tracking down of `libgcc_s'

Message ID alpine.LFD.2.21.1911290526340.13542@redsun52.ssa.fujisawa.hgst.com
State New
Headers show
Series
  • Fix run-time handling of `libgcc_s' in testing
Related show

Commit Message

Maciej W. Rozycki Nov. 29, 2019, 9:09 a.m.
Fix a catastrophic libgo testsuite failure in cross-compilation where 
the shared `libgcc_s' library cannot be found by the loader at run time 
in build-tree testing and consequently all test cases fail the execution 
stage, giving output (here with the `x86_64-linux-gnu' host and the 
`riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation 
mode as the target board) like:

spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe
./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
FAIL: archive/tar

To do so rework `gcc-set-multilib-library-path' so as not to rely on the 
`rootme' TCL variable to have been preset in testsuite invocation, which 
only works for the GCC test suites and not for library test suites, and 
also use `remote_exec host' rather than `exec' to invoke the compiler in 
determination of `libgcc_s' locations, so that the solution works in 
remote testing as well while also avoiding the hardcoded limit of the 
executable's path length imposed by `exec'.

This is based on an observation that the multilib root directory can be 
determined by stripping out the multilib directory in effect as printed 
with the `-print-multi-directory' option from the path produced by the 
`-print-file-name=' option.  And then individual full multilib paths can 
be assembled for the other multilibs by appending their respective 
multilib directories to the multilib root directory.

Unlike with the old solution the full multilib paths are not checked for 
the presence of the shared `libgcc_s' library there, but that is 
supposed to be harmless.  Also the full multilib path for the multilib 
used with the compiler used for testing will now come first, which 
should reduce run-time processing in the usual case.

With this change in place test output instead looks like:

spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs:..././gcc/lib64/lp64d:..././gcc/.:..././gcc/lib32/ilp32:..././gcc/lib32/ilp32d:..././gcc/lib64/lp64 ./a.exe
PASS
PASS: archive/tar

No summary comparison, because the libgo testsuite does not provide one 
in this configuration for some reason, however this change improves 
overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs.

	gcc/testsuite/
	* lib/gcc-defs.exp (gcc-set-multilib-library-path): Use 
	`-print-file-name=' to determine the multilib root directory.  
	Use `remote_exec host' rather than `exec' to invoke the 
	compiler.
---
Hi,

 As PR testsuite/40699, PR testsuite/40707 and PR testsuite/40709 and 
resulting r149508, a revert of r149113 ("Tidy up testsuite handling of 
LD_LIBRARY_PATH"), 
<https://gcc.gnu.org/ml/gcc-patches/2009-06/msg00151.html> and some other 
commits, have indicated non-selected multilib directories have to be 
included in the dynamic loader's library path for some targets for some 
reason.  So I have decided to preserve the approach, even though it 
appears odd to me.

  Maciej

Changes from v1:

- Resolve the issue globally in `gcc-set-multilib-library-path' in 
  gcc-defs.exp rather than for libgo only in `go_link_flags' in go.exp.
---
 gcc/testsuite/lib/gcc-defs.exp |   43 +++++++++++++++++++++++++++--------------
 1 file changed, 29 insertions(+), 14 deletions(-)

Comments

Ian Lance Taylor Dec. 2, 2019, 11:39 p.m. | #1
On Fri, Nov 29, 2019 at 1:09 AM Maciej W. Rozycki <macro@wdc.com> wrote:
>

> Fix a catastrophic libgo testsuite failure in cross-compilation where

> the shared `libgcc_s' library cannot be found by the loader at run time

> in build-tree testing and consequently all test cases fail the execution

> stage, giving output (here with the `x86_64-linux-gnu' host and the

> `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation

> mode as the target board) like:

>

> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe

> ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory

> FAIL: archive/tar

>

> To do so rework `gcc-set-multilib-library-path' so as not to rely on the

> `rootme' TCL variable to have been preset in testsuite invocation, which

> only works for the GCC test suites and not for library test suites, and

> also use `remote_exec host' rather than `exec' to invoke the compiler in

> determination of `libgcc_s' locations, so that the solution works in

> remote testing as well while also avoiding the hardcoded limit of the

> executable's path length imposed by `exec'.

>

> This is based on an observation that the multilib root directory can be

> determined by stripping out the multilib directory in effect as printed

> with the `-print-multi-directory' option from the path produced by the

> `-print-file-name=' option.  And then individual full multilib paths can

> be assembled for the other multilibs by appending their respective

> multilib directories to the multilib root directory.

>

> Unlike with the old solution the full multilib paths are not checked for

> the presence of the shared `libgcc_s' library there, but that is

> supposed to be harmless.  Also the full multilib path for the multilib

> used with the compiler used for testing will now come first, which

> should reduce run-time processing in the usual case.

>

> With this change in place test output instead looks like:

>

> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs:..././gcc/lib64/lp64d:..././gcc/.:..././gcc/lib32/ilp32:..././gcc/lib32/ilp32d:..././gcc/lib64/lp64 ./a.exe

> PASS

> PASS: archive/tar

>

> No summary comparison, because the libgo testsuite does not provide one

> in this configuration for some reason, however this change improves

> overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs.

>

>         gcc/testsuite/

>         * lib/gcc-defs.exp (gcc-set-multilib-library-path): Use

>         `-print-file-name=' to determine the multilib root directory.

>         Use `remote_exec host' rather than `exec' to invoke the

>         compiler.


This patch is for better Go support but the patch is to generic code.
The patch is fine with me but should ideally be reviewed by a
testsuite maintainer.

Ian
Maciej W. Rozycki Dec. 3, 2019, 6:01 p.m. | #2
On Mon, 2 Dec 2019, Ian Lance Taylor wrote:

> >         gcc/testsuite/

> >         * lib/gcc-defs.exp (gcc-set-multilib-library-path): Use

> >         `-print-file-name=' to determine the multilib root directory.

> >         Use `remote_exec host' rather than `exec' to invoke the

> >         compiler.

> 

> This patch is for better Go support but the patch is to generic code.

> The patch is fine with me but should ideally be reviewed by a

> testsuite maintainer.


 I agree.  I have cc-ed you to keep you in the circle and I have added the 
testsuite people now too, but ultimately I think it's Richard who might be 
the best person to look into it as he's been highly skilled in this area 
and he has also dealt with this particular piece in the past.

 Richard, would you be able to find a few cycles and be kind enough to 
look into it?

 I'm somewhat surprised it's only libgo that suffers from the lack of 
run-time library paths in my setup where a random copy of `libgcc_s' is 
not already present in the environment, but maybe no other target library 
test suite pulls `libgcc_s' with any of its tests.

  Maciej

Patch

Index: gcc/gcc/testsuite/lib/gcc-defs.exp
===================================================================
--- gcc.orig/gcc/testsuite/lib/gcc-defs.exp
+++ gcc/gcc/testsuite/lib/gcc-defs.exp
@@ -324,29 +324,44 @@  proc dg-additional-files-options { optio
 # for COMPILER, including multilib directories.
 
 proc gcc-set-multilib-library-path { compiler } {
-    global rootme
+    set shlib_ext [get_shlib_extension]
+    set options [lrange $compiler 1 end]
+    set compiler [lindex $compiler 0]
 
-    # ??? rootme will not be set when testing an installed compiler.
-    # In that case, we should perhaps use some other method to find
-    # libraries.
-    if {![info exists rootme]} {
+    set libgcc_s_x [remote_exec host "$compiler" \
+		    "$options -print-file-name=libgcc_s.${shlib_ext}"]
+    if { [lindex $libgcc_s_x 0] == 0 \
+	 && [set libgcc_s_dir [file dirname [lindex $libgcc_s_x 1]]] != "" } {
+	set libpath ":${libgcc_s_dir}"
+    } else {
 	return ""
     }
 
-    set libpath ":${rootme}"
-    set options [lrange $compiler 1 end]
-    set compiler [lindex $compiler 0]
-    if { [is_remote host] == 0 && [which $compiler] != 0 } {
-	foreach i "[eval exec $compiler $options --print-multi-lib]" {
+    set multi_dir_x [remote_exec host "$compiler" \
+		     "$options -print-multi-directory"]
+    set multi_lib_x [remote_exec host "$compiler" \
+		     "$options -print-multi-lib"]
+    if { [lindex $multi_dir_x 0] == 0 && [lindex $multi_lib_x 0] == 0 } {
+	set multi_dir [string trim [lindex $multi_dir_x 1]]
+	set multi_lib [string trim [lindex $multi_lib_x 1]]
+	if { "$multi_dir" == "." } {
+	    set multi_root "$libgcc_s_dir"
+	} else {
+	    set multi_match [string last "/$multi_dir" "$libgcc_s_dir"]
+	    if { "$multi_match" < 0 } {
+		return $libpath
+	    }
+	    set multi_root [string range "$libgcc_s_dir" \
+			    0 [expr $multi_match - 1]]
+	}
+	foreach i "$multi_lib" {
 	    set mldir ""
 	    regexp -- "\[a-z0-9=_/\.-\]*;" $i mldir
 	    set mldir [string trimright $mldir "\;@"]
-	    if { "$mldir" == "." } {
+	    if { "$mldir" == "$multi_dir" } {
 		continue
 	    }
-	    if { [llength [glob -nocomplain ${rootme}/${mldir}/libgcc_s*.so.*]] >= 1 } {
-		append libpath ":${rootme}/${mldir}"
-	    }
+	    append libpath ":${multi_root}/${mldir}"
 	}
     }