Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
neko_coreutils-8.12 in beta - sort is broken
#11
Re: neko_coreutils-8.12 in beta - sort is broken
MrThinly!

mrthinlysliced post_id=1192 time=1529996400 user_id=83 Wrote:If it's any use, the patch I'm using for core utils is here (sorry, SGI patch style):

https://github.com/danielhams/didbs/blob...rsgi.patch
Good stuff, thanks for putting it there.
Quote:(1) Use of math.h in seq.c -> The macro tests for inf and nan fail, so I included math.h to use the library versions - this stops make check running forever on the misc/seq test
Very nice, thanks. There was a comment about not using math.h because of adding -lm to the library list. As if that is a big issue :-) so i added this to my patch.

Quote:(2) A tweak to numfmt.c so it doesn't try to use fpsetprec if it isn't defined (irix seems to have the ieee header but doesn't define fpsetprec?)
fpsetprec is used, among other things, for the so-called setting of denormals or subnormals floating point numbers. These numbers have less precision than ordinary IEEE754 floating point single and double numbers but have a range which allows expression of floating point numbers closer to zero than normal singles or doubles.
IRIX does have the ability to set denormalized floats, but not in the usual way. I have researched this for Python 3.1.1 and have a patch which does this for Python 3.x. Let me find that one... Ah, an excerpt from neko_python-3.1.1_irix.patch:
Code:
--- Modules/python.c.save       Fri Sep 25 22:58:16 2009
+++ Modules/python.c    Fri Sep 25 23:00:14 2009
@@ -4,8 +4,11 @@
#include <locale.h>

#ifdef __FreeBSD__
-#include <floatingpoint.h>
+# include <floatingpoint.h>
#endif
+#ifdef __sgi
+# include <sys/fpu.h>
+#endif

#ifdef MS_WINDOWS
int
@@ -135,6 +138,12 @@

        m = fpgetmask();
        fpsetmask(m & ~FP_X_OFL);
+#endif
+#ifdef __sgi
+       unsigned int m;
+
+       m = get_fpc_csr();
+       set_fpc_csr(m & ~FPCSR_FLUSH_ZERO);
#endif
        if (!argv_copy || !argv_copy2) {
                fprintf(stderr, "out of memory\n");
So we may be able to use get_fpr_csr() and set_fpr_csr() instead of fpsetprec(), but this needs a bit of research.
Quote:The package still fails a number of tests:

https://github.com/danielhams/didbs/blob.../knownbugs
I am taking a look at these but they take time.
A few of them are related to strtod, see below...
Quote:Still don't know how to get past the "argument list to long" failure later in make check. (systune ncargs is 50000, fwiw).
I have:
Code:
orcus:/> systune ncargs
        ncargs = 131072 (0x20000)
...and no problems finishing the 'gmake check' tests. Try increasing it.
Quote:Current one: seq seems to support 0xF23EF style parameters on linux - on irix this test fails.
This is a bug in IRIX strtod, well, an omission from the c99 standards. This issue has been raised by Brent Canavan here
There is still quite some stuff to do, but this seems to be converging. Thanks Thinly for pressing this on.
dexter1 Offline
Global Moderator
******

Posts: 130
Threads: 8
Joined: May 2018
Find Reply
06-26-2018, 09:30 AM
#12
Re: neko_coreutils-8.12 in beta - sort is broken
Hey Dexter,

Roger that for the systune, that's got it.

RE: math.h & isinf isnan - I didn't have to add the math library and a quick glance at the headers indicates these are front end compiler implementations so no link to m necessary.

RE: irix platform fpsetprec - Aha! Nice, you're way ahead of me there :-)

I'm still bumping myself against the strtod issues. I wrote myself a little test:

Code:
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <stdbool.h>

int main(int argc, char **argv )
{
  const char* teststrs[] = {
    "-0",
    "0",
    "0xF1235",
    "1xF1235",
    "xF1235",
    "-xF1235",
    "-0.1356",
    "0.1356",
    " -0.1356",
    " 0.1356",
    "nan",
    "inf",
    "1e1",
    "-1e1",
    "1e-1",
    "-1e-1",
  };
  int numteststrs = sizeof(teststrs) / sizeof(const char *);

  for( int i = 0 ; i < numteststrs ; ++i )
    {
      const char *onestr = teststrs[i];
      char *endptr = NULL;
      double testVal =  strtod( onestr, &endptr);

      bool isStartChar = endptr == onestr;
      bool isEndChar = *endptr == '\0';

      int distFromStart = (!isStartChar ? endptr - onestr : -1);

      printf("isc(%d)\tiec(%d)\tdist(%d) \tSource(%s)\tstrtod(%lf)\n",
         isStartChar, isEndChar, distFromStart,
         onestr, testVal
         );
    }
  return 0;
}

Output on Linux:
Code:
[[email protected] mycomps]$ ./test_c
isc(0)    iec(1)    dist(2)     Source(-0)    strtod(-0.000000)
isc(0)    iec(1)    dist(1)     Source(0)    strtod(0.000000)
isc(0)    iec(1)    dist(7)     Source(0xF1235)    strtod(987701.000000)
isc(0)    iec(0)    dist(1)     Source(1xF1235)    strtod(1.000000)
isc(1)    iec(0)    dist(-1)     Source(xF1235)    strtod(0.000000)
isc(1)    iec(0)    dist(-1)     Source(-xF1235)    strtod(0.000000)
isc(0)    iec(1)    dist(7)     Source(-0.1356)    strtod(-0.135600)
isc(0)    iec(1)    dist(6)     Source(0.1356)    strtod(0.135600)
isc(0)    iec(1)    dist(8)     Source( -0.1356)    strtod(-0.135600)
isc(0)    iec(1)    dist(7)     Source( 0.1356)    strtod(0.135600)
isc(0)    iec(1)    dist(3)     Source(nan)    strtod(nan)
isc(0)    iec(1)    dist(3)     Source(inf)    strtod(inf)
isc(0)    iec(1)    dist(3)     Source(1e1)    strtod(10.000000)
isc(0)    iec(1)    dist(4)     Source(-1e1)    strtod(-10.000000)
isc(0)    iec(1)    dist(4)     Source(1e-1)    strtod(0.100000)
isc(0)    iec(1)    dist(5)     Source(-1e-1)    strtod(-0.100000)

Output on Irix:
Code:
[[email protected] TestProj]$ ./test_c
isc(0)    iec(1)    dist(2)     Source(-0)    strtod(0.000000)
isc(0)    iec(1)    dist(1)     Source(0)    strtod(0.000000)
isc(1)    iec(0)    dist(-1)     Source(0xF1235)    strtod(0.000000)
isc(0)    iec(0)    dist(1)     Source(1xF1235)    strtod(1.000000)
isc(1)    iec(0)    dist(-1)     Source(xF1235)    strtod(0.000000)
isc(1)    iec(0)    dist(-1)     Source(-xF1235)    strtod(0.000000)
isc(0)    iec(1)    dist(7)     Source(-0.1356)    strtod(-0.135600)
isc(0)    iec(1)    dist(6)     Source(0.1356)    strtod(0.135600)
isc(0)    iec(1)    dist(8)     Source( -0.1356)    strtod(-0.135600)
isc(0)    iec(1)    dist(7)     Source( 0.1356)    strtod(0.135600)
isc(0)    iec(1)    dist(3)     Source(nan)    strtod(nan0x10000000)
isc(0)    iec(1)    dist(3)     Source(inf)    strtod(inf)
isc(0)    iec(1)    dist(3)     Source(1e1)    strtod(10.000000)
isc(0)    iec(1)    dist(4)     Source(-1e1)    strtod(-10.000000)
isc(0)    iec(1)    dist(4)     Source(1e-1)    strtod(0.100000)
isc(0)    iec(1)    dist(5)     Source(-1e-1)    strtod(-0.100000)

So there's a bug in the Irix printf of nan too...

I did notice that coreutils allows setting "REPLACE_STRTOD" - but for the moment I've not found if/when that is being used.

First I'll try and see why the replace isn't doing what it should - once it's consistently calling the replacement function I can look at maybe some copy-pasta from glibc strtod. Then perhaps a wrapper/replacement for the printf pieces is needed too.

Smells like these (strtod, printf) are going to be issues elsewhere (would certainly explain some failures from the gawk + sed make check runs I've done).

Kr,

Mr TS
mrthinlysliced Offline
Octane

Posts: 71
Threads: 5
Joined: May 2018
Find Reply
06-29-2018, 10:03 AM
#13
Re: neko_coreutils-8.12 in beta - sort is broken
Couple more little tweaks here:

https://github.com/danielhams/didbs/blob...rsgi.patch
A bit of detail:

[*] coreutils-8.29/src/numfmt.c - make use of dexters suggestion of get_fpc_csr and set_fpc_csr to disable flush to zero of denormalised floats (helps reduce failures of a test)
[*] coreutils-8.29/src/sleep.c - introduce an extra "isnan" test where for some reason (0<=nan) isn't what I'd expect. This possibly can be better done with fp setup and/or flags, but it's strange to me - since I'm guessing this should be ieee conformance related..

I also looked into why the misc/shred-remove test was failing and it seems that the irix version is doing more renames before removal that a linux version. Possibly related to differences between real glibc renameat2 and the stub implementation inside coreutils.

Summary:
Total: 603
Pass: 420
Skip: 175
Fail: 7 (really 6, the misc/shred-remove failure looks benign)
Error: 1
mrthinlysliced Offline
Octane

Posts: 71
Threads: 5
Joined: May 2018
Find Reply
07-17-2018, 09:57 PM
#14
Re: neko_coreutils-8.12 in beta - sort is broken
Usual place: https://github.com/danielhams/didbs/blob...rsgi.patch
For 8.29 I'm now down to two consistent failures and one intermittent failure. The intermittent failure is annoying + doesn't happen all the time - it looks like sometimes USR1 signals aren't being properly picked up by a child process (I tried manually sending with kill, no go) and it will happily sit there forever... (it's the dd/stats test about 80% of the way through all the tests).

Built with 7.4.4m Mipspro on mips4.

Env:

Code:
export LD_LIBRARYN32_PATH=/usualstuff/lib
export LC_ALL=C
export SHELL=/usualstuff/bin/bash
export CONFIG_SHELL=/usualstuff/bin/bash
export PERL=/usualstuff/bin/perl
export CC=c99
export CFLAGS="-mips4 -O1"
export LDFLAGS="-mips4"

Launching Configuration:

Code:
gl_cv_func_strtod_works=no gl_cv_func_c99_strtold=no utils_cv_xlc=no ./configure --prefix=/usualstuff/ --disable-silent-rules

Note: I find the version of bash in nekoware has frequent "pauses" when it should be getting on with things - this looks like it's waiting on reaping "defunct" child processes. Going back to pre-4.X bash resolves this issue (bash 3.2.57 works fine for me).
mrthinlysliced Offline
Octane

Posts: 71
Threads: 5
Joined: May 2018
Find Reply
07-24-2018, 08:37 AM


Forum Jump:


Users browsing this thread: 2 Guest(s)