Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

locale.t ends with panic "attempting to unlock already unlocked locale" on Mac OS X Tiger #23159

Open
sevan opened this issue Mar 26, 2025 · 18 comments · May be fixed by #23172
Open

locale.t ends with panic "attempting to unlock already unlocked locale" on Mac OS X Tiger #23159

sevan opened this issue Mar 26, 2025 · 18 comments · May be fixed by #23172

Comments

@sevan
Copy link
Contributor

sevan commented Mar 26, 2025

On Mac OS X Tiger/i386 (darwin 8) when running the test suite 2 failures are observed. One in run/locale.t and one in ../lib/locale.t.

Test Summary Report
-------------------
run/locale.t                                                       (Wstat: 65280 (exited 255) Tests: 41 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
run/todo.t                                                         (Wstat: 0 Tests: 6 Failed: 0)
  TODO passed:   3-4
../lib/locale.t                                                    (Wstat: 65280 (exited 255) Tests: 380 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
Files=2908, Tests=1180577, 2206 wallclock secs (170.38 usr 33.76 sys + 1608.72 cusr 206.82 csys = 2019.68 CPU)
Result: FAIL

for run/locale.t

panic: locale.c: 4715: attempting to unlock already unlocked locale; depth was 0                                                            
Dubious, test returned 255 (wstat 65280, 0xff00)                                                                                            
All 41 subtests passed                                                                                                                      
(less 3 skipped subtests: 38 okay)

for ../lib/locale.t

panic: locale.c: 4715: attempting to unlock already unlocked locale; depth was 0
Dubious, test returned 255 (wstat 65280, 0xff00)
All 380 subtests passed

The issue is not observed on Tiger/powerpc but I'm not sure if that's just because all the PowerPC systems I have to test on are uniprocessor and this is a bug observed on multiprocessor systems or the issue is actually exclusive to the Intel build as there are version differences with components shipped with the OS (they're newer on the Intel build)

I built and ran the test suite using the following steps
./Configure -Duseshrplib -Duselargefiles -Dusethreads -des -Dusedevel && make && make test
@jkeenan
Copy link
Contributor

jkeenan commented Mar 27, 2025

Can you provide information on the version of perl you were testing? For example, can you post the output of these commands:

$ perl -v | head -2 | tail -1
$ perl -V

Thanks.

@sevan
Copy link
Contributor Author

sevan commented Mar 27, 2025

This is perl 5, version 41, subversion 11 (v5.41.11) built for darwin-thread-multi-2level
built from blead, repo at commit 3e734cb

Summary of my perl5 (revision 5 version 41 subversion 11) configuration:
  Derived from: 
  Platform:
    osname=darwin
    osvers=8.11.1
    archname=darwin-thread-multi-2level
    uname='darwin smes.local 8.11.1 darwin kernel version 8.11.1: wed oct 10 18:23:28 pdt 2007; root:xnu-792.25.20~1release_i386 i386 i386 '
    config_args='-Duseshrplib -Duselargefiles -Dusethreads -des -Dusedevel -Dprefix=/Users/sme/myperl'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-std=gnu99 -fno-common -DPERL_DARWIN -DNO_THREAD_SAFE_QUERYLOCALE -DNO_POSIX_2008_LOCALE -DHAS_BROKEN_LANGINFO_CODESET -fno-strict-aliasing -pipe -D_FORTIFY_SOURCE=2'
    optimize='-O3'
    cppflags='-std=gnu99 -fno-common -DPERL_DARWIN -DNO_THREAD_SAFE_QUERYLOCALE -DNO_POSIX_2008_LOCALE -DHAS_BROKEN_LANGINFO_CODESET -fno-strict-aliasing -pipe'
    ccversion=''
    gccversion='4.0.1 (Apple Computer, Inc. build 5370)'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'
    ldflags =''
    libpth=/usr/lib
    libs=-lpthread -ldbm -ldl -lm -lc
    perllibs=-lpthread -ldl -lm -lc
    libc=
    so=dylib
    useshrplib=true
    libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -bundle -undefined dynamic_lookup'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_ZAPHOD32
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Locally applied patches:
    uncommitted-changes
  Built under darwin
  Compiled at Mar 26 2025 21:47:04
  @INC:
    /Users/sme/myperl/lib/site_perl/5.41.11/darwin-thread-multi-2level
    /Users/sme/myperl/lib/site_perl/5.41.11
    /Users/sme/myperl/lib/5.41.11/darwin-thread-multi-2level
    /Users/sme/myperl/lib/5.41.11

@jkeenan
Copy link
Contributor

jkeenan commented Mar 27, 2025

I don't have a Mac, particularly not an older one like that. So the following questions fall into the category of "grasping at straws". I searched for that particular error message on DDG, Google and Stack Exchange and did not find anything that looked relevant.

  1. Do you have any other C compiler you could try to run ./Configure with (e.g., -Dcc=clang)?
  2. Once ./Configure completes, please run make test_prep and then cd t; TEST_JOBS=1 ./perl harness -v run/locale.t ../lib/locale.t; cd - and post any interesting results.
  3. Were you previously able to build ./perl on this machine with your configuration arguments? (Say, perl-5.40.0; perl-5.41.9; etc.)

@khwilliamson, any thoughts on this?

@sevan
Copy link
Contributor Author

sevan commented Mar 27, 2025 •
edited
Loading

I don't have a Mac, particularly not an older one like that. So the following questions fall into the category of "grasping at straws". I searched for that particular error message on DDG, Google and Stack Exchange and did not find anything that looked relevant.

  1. Do you have any other C compiler you could try to run ./Configure with (e.g., -Dcc=clang)?

Yep, I have GCC 14.2 built and installed too. Happy to try with that and report back but do you really expect a compiler issue for 2 test cases which seem to indicate a race, if I've understood the panic message correctly?

  1. Once ./Configure completes, please run make test_prep and then cd t; TEST_JOBS=1 ./perl harness -v run/locale.t ../lib/locale.t; cd - and post any interesting results.

ack.

  1. Were you previously able to build ./perl on this machine with your configuration arguments? (Say, perl-5.40.0; perl-5.41.9; etc.)>

I've been building the perl releases on Tiger/powerpc and run the test suite there, there hasn't been such failures. That's why I mentioned that my powerpc hardware is all uniprocessor, whereas the intel build is on a dual core machine.
But take that with a pinch of salt, I may be jumping to conclusions.

@khwilliamson, any thoughts on this?

@Leont
Copy link
Contributor

Leont commented Mar 29, 2025

Could you produce a C stacktrace for that error? Without that it is very difficult to actually to pinpoint where things go wrong.

@khwilliamson
Copy link
Contributor

khwilliamson commented Mar 30, 2025 via email

@sevan
Copy link
Contributor Author

sevan commented Mar 30, 2025 •
edited
Loading

Thanks for the hints, folks.
As per INSTALL doc, I configured with -DEBUGGING=both specified
./Configure -Duseshrplib -Duselargefiles -Dusethreads -des -Dusedevel -DEBUGGING=both

In the Configure output I see

cflags.SH: ccflags  = -std=gnu99 -fno-common -DPERL_DARWIN -DNO_THREAD_SAFE_QUERYLOCALE -DNO_POSIX_2008_LOCALE -DHAS_BROKEN_LANGINFO_CODESET -fno-strict-aliasing -pipe -D_FORTIFY_SOURCE=2 -DEBUGGING=both
cflags.SH: stdflags = 
cflags.SH: optimize = -O3 -g

When I run the run/locale.t tests I noticed that it still complains about not building with -DDEBUGGING
Recompile perl with -DDEBUGGING to use -D switch (did you mean -d ?)
when I run ./perl -DLv run/locale.t in the t directory.

Is that expected?
Will try again later tonight (some hours away) with just -DDEBUGGING specified and report back.

output from t/run/locale.t at present

Recompile perl with -DDEBUGGING to use -D switch (did you mean -d ?)
# locales available: C POSIX af_ZA af_ZA.ISO8859-1 af_ZA.ISO8859-15 af_ZA.UTF-8 am_ET am_ET.UTF-8 be_BY be_BY.CP1131 be_BY.CP1251 be_BY.ISO8859-5 be_BY.UTF-8 bg_BG bg_BG.CP1251 bg_BG.UTF-8 ca_ES ca_ES.ISO8859-1 ca_ES.ISO8859-15 ca_ES.UTF-8 cs_CZ cs_CZ.ISO8859-2 cs_CZ.UTF-8 da_DK da_DK.ISO8859-1 da_DK.ISO8859-15 da_DK.UTF-8 de_AT de_AT.ISO8859-1 de_AT.ISO8859-15 de_AT.UTF-8 de_CH de_CH.ISO8859-1 de_CH.ISO8859-15 de_CH.UTF-8 de_DE de_DE.ISO8859-1 de_DE.ISO8859-15 de_DE.UTF-8 el_GR el_GR.ISO8859-7 el_GR.UTF-8 en_AU en_AU.ISO8859-1 en_AU.ISO8859-15 en_AU.US-ASCII en_AU.UTF-8 en_CA en_CA.ISO8859-1 en_CA.ISO8859-15 en_CA.US-ASCII en_CA.UTF-8 en_GB en_GB.ISO8859-1 en_GB.ISO8859-15 en_GB.US-ASCII en_GB.UTF-8 en_IE en_IE.UTF-8 en_NZ en_NZ.ISO8859-1 en_NZ.ISO8859-15 en_NZ.US-ASCII en_NZ.UTF-8 en_US en_US.ISO8859-1 en_US.ISO8859-15 en_US.US-ASCII en_US.UTF-8 es_ES es_ES.ISO8859-1 es_ES.ISO8859-15 es_ES.UTF-8 et_EE et_EE.ISO8859-15 et_EE.UTF-8 eu_ES eu_ES.ISO8859-1 eu_ES.ISO8859-15 eu_ES.UTF-8 fi_FI fi_FI.ISO8859-1 fi_FI.ISO8859-15 fi_FI.UTF-8 fr_BE fr_BE.ISO8859-1 fr_BE.ISO8859-15 fr_BE.UTF-8 fr_CA fr_CA.ISO8859-1 fr_CA.ISO8859-15 fr_CA.UTF-8 fr_CH fr_CH.ISO8859-1 fr_CH.ISO8859-15 fr_CH.UTF-8 fr_FR fr_FR.ISO8859-1 fr_FR.ISO8859-15 fr_FR.UTF-8 he_IL he_IL.UTF-8 hi_IN.ISCII-DEV hr_HR hr_HR.ISO8859-2 hr_HR.UTF-8 hu_HU hu_HU.ISO8859-2 hu_HU.UTF-8 hy_AM hy_AM.ARMSCII-8 hy_AM.UTF-8 is_IS is_IS.ISO8859-1 is_IS.ISO8859-15 is_IS.UTF-8 it_CH it_CH.ISO8859-1 it_CH.ISO8859-15 it_CH.UTF-8 it_IT it_IT.ISO8859-1 it_IT.ISO8859-15 it_IT.UTF-8 ja_JP ja_JP.UTF-8 kk_KZ kk_KZ.PT154 kk_KZ.UTF-8 ko_KR ko_KR.UTF-8 lt_LT lt_LT.ISO8859-13 lt_LT.ISO8859-4 lt_LT.UTF-8 nl_BE nl_BE.ISO8859-1 nl_BE.ISO8859-15 nl_BE.UTF-8 nl_NL nl_NL.ISO8859-1 nl_NL.ISO8859-15 nl_NL.UTF-8 no_NO no_NO.ISO8859-1 no_NO.ISO8859-15 no_NO.UTF-8 pl_PL pl_PL.ISO8859-2 pl_PL.UTF-8 pt_BR pt_BR.ISO8859-1 pt_BR.UTF-8 pt_PT pt_PT.ISO8859-1 pt_PT.ISO8859-15 pt_PT.UTF-8 ro_RO ro_RO.ISO8859-2 ro_RO.UTF-8 ru_RU ru_RU.CP1251 ru_RU.CP866 ru_RU.ISO8859-5 ru_RU.KOI8-R ru_RU.UTF-8 sk_SK sk_SK.ISO8859-2 sk_SK.UTF-8 sl_SI sl_SI.ISO8859-2 sl_SI.UTF-8 sr_YU sr_YU.ISO8859-2 sr_YU.ISO8859-5 sr_YU.UTF-8 sv_SE sv_SE.ISO8859-1 sv_SE.ISO8859-15 sv_SE.UTF-8 tr_TR tr_TR.ISO8859-9 tr_TR.UTF-8 uk_UA uk_UA.ISO8859-5 uk_UA.KOI8-U uk_UA.UTF-8 zh_CN zh_CN.UTF-8 zh_HK zh_HK.UTF-8 zh_TW zh_TW.UTF-8
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails individ char
ok 2 - /l matching of [bracketed] doesn't skip non-first POSIX class
# using non-C locale 'af_ZA'
ok 3 - retrieving current non-C LC_NUMERIC doesn't give 'C'
ok 4 - retrieving current non-C LC_ALL doesn't give 'C'
ok 5 - no locales where LC_NUMERIC breaks
ok 6 - LC_NUMERIC without environment nor setlocale() has no effect in any locale
ok 7 # skip no UTF-8 locale available where LC_NUMERIC radix isn't ASCII
# using the 'af_ZA' locale for LC_NUMERIC tests
ok 8 - format() does not look at LC_NUMERIC without 'use locale'
ok 9 - format() looks at LC_NUMERIC with 'use locale'
ok 10 - localeconv() looks at LC_NUMERIC with and without 'use locale'
ok 11 - format() does not look at LC_NUMERIC with 'use locale qw(:collate :characters :collate :ctype :monetary :time)'
ok 12 - format() looks at LC_NUMERIC with 'use locale'
ok 13 - format() looks at LC_NUMERIC with 'use locale ":!collate"'
ok 14 - format() looks at LC_NUMERIC with 'use locale ":not_collate"'
ok 15 - format() looks at LC_NUMERIC with 'use locale ":!characters"'
ok 16 - format() looks at LC_NUMERIC with 'use locale ":not_characters"'
ok 17 - format() looks at LC_NUMERIC with 'use locale ":!collate"'
ok 18 - format() looks at LC_NUMERIC with 'use locale ":not_collate"'
ok 19 - format() looks at LC_NUMERIC with 'use locale ":!ctype"'
ok 20 - format() looks at LC_NUMERIC with 'use locale ":not_ctype"'
ok 21 - format() looks at LC_NUMERIC with 'use locale ":!monetary"'
ok 22 - format() looks at LC_NUMERIC with 'use locale ":not_monetary"'
ok 23 - format() looks at LC_NUMERIC with 'use locale ":!time"'
ok 24 - format() looks at LC_NUMERIC with 'use locale ":not_time"'
ok 25 - format() looks at LC_NUMERIC with 'use locale ":numeric"'
ok 26 - too late to look at the locale at write() time
ok 27 - too late to ignore the locale at write() time
ok 28 - version does not clobber version
ok 29 - version does not clobber version (via eval)
ok 30 - sprintf() and printf() look at LC_NUMERIC regardless of constant folding
ok 31 - Uses the above test to verify that on Windows the system default locale has lower priority than LC_NUMERIC
ok 32 - LANG is used if LC_ALL, LC_NUMERIC are invalid
ok 33 - C locale is used if LC_ALL, LC_NUMERIC, LANG are invalid
ok 34 - No compile error on v-strings when setting the locale to non-dot radix at compile time when default environment has non-dot radix
ok 35 - Radix print properly in locale scope, and without
ok 36 - Can do math when radix is a comma
ok 37 - Thread join doesn't disrupt calling thread
ok 38 - POSIX::strtod() uses underlying locale
ok 39 - Properly toggles to radix dot locale
ok 40 # skip Test only valid when LC_ALL syntax is name=value pairs
ok 41 # skip Test only valid when LC_ALL syntax is name=value pairs
panic: locale.c: 4715: attempting to unlock already unlocked locale; depth was 0

@mauke
Copy link
Contributor

mauke commented Mar 30, 2025

That -DEBUGGING=both in ccflags looks wrong. It should say -DDEBUGGING for a debug build. (Configure arguments are not ccflags.)

@sevan
Copy link
Contributor Author

sevan commented Mar 30, 2025

That -DEBUGGING=both in ccflags looks wrong. It should say -DDEBUGGING for a debug build. (Configure arguments are not ccflags.)

It is definitely the code in Configure that's wrong and not a documentation bug, right?

@Leont
Copy link
Contributor

Leont commented Mar 30, 2025

It is definitely the code in Configure that's wrong and not a documentation bug, right?

Except I can't reproduce the issue. Can you do a git clean or make clean before trying it again?

@sevan
Copy link
Contributor Author

sevan commented Mar 30, 2025

It is definitely the code in Configure that's wrong and not a documentation bug, right?

Except I can't reproduce the issue. Can you do a git clean or make clean before trying it again?

Ok, I'll do that next

@sevan
Copy link
Contributor Author

sevan commented Mar 30, 2025

Worked when I ran make clean and removed config.sh manually before running ./Configure -Duseshrplib -Duselargefiles -Dusethreads -des -Dusedevel -DDEBUGGING

Output from running the 2 failing tests with a debugging build:

run_locale_t-output.txt

lib_locale_t-output.txt

@sevan
Copy link
Contributor Author

sevan commented Mar 30, 2025

Except I can't reproduce the issue. Can you do a git clean or make clean before trying it again?

Initially, I made a mistake and configured with -Accflags="-DEBUGGING=both"
When I realised it didn't work, I ran make clean and tried -DEBUGGING=both but the config.sh remained in between attempts and the cflags value is retained.
I had to remove config.sh before -DEBUGGING=both set ccflags and cppflags correctly in config.sh.

@khwilliamson
Copy link
Contributor

khwilliamson commented Apr 1, 2025 via email

@sevan
Copy link
Contributor Author

sevan commented Apr 1, 2025

I will look further and get you a proper patch in the next few days

Thank you. Happy to take the patch for a test run & report back whenever it is available.
In the meantime I'll try the workaround you mentioned and see how it goes.

@sevan
Copy link
Contributor Author

sevan commented Apr 3, 2025

I looked briefly at the logs so far. I'm unsure what your goal is. If you just need a workaround for the short term, I believe this will get you going Add -Accflags='-DNO_LOCALE_COLLATE'" to your Configure parameters. What looks to be happening is that a locale for Australia is defective on this box, and the code does an early return as a result, and doesn't get the backing out exactly right. That is, it returns before it did the locale lock, and doesn't realize that it never got locked. I will look further and get you a proper patch in the next few days

Confirmed, configuring with -Accflags='-DNO_LOCALE_COLLATE' sidesteps the issue

All tests successful.
Elapsed: 2644 sec
u=22.78  s=11.56  cu=2076.99  cs=221.14  scripts=2748  tests=1180959

@khwilliamson khwilliamson linked a pull request Apr 6, 2025 that will close this issue
Open
@khwilliamson
Copy link
Contributor

I put up a patch for this several days ago but typed this issue number incorrectly, so that github did not notify anyone subscribed to just this ticket.

@sevan
Copy link
Contributor Author

sevan commented Apr 6, 2025

I cleaned my source directory, applied the patch, removed config.sh + Policy.sh and configured with
./Configure -Duseshrplib -Duselargefiles -Dusethreads -des -Dusedevel -DDEBUGGING

Test suite passed:

All tests successful.
Elapsed: 2660 sec
u=22.73  s=11.66  cu=2088.93  cs=219.98  scripts=2748  tests=1180795

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

locale.c: Don't do asymmetric back out on failure Perl/perl5
5 participants
@Leont @jkeenan @khwilliamson @sevan @mauke