-
-
Notifications
You must be signed in to change notification settings - Fork 135
/
Copy pathNEWS
2828 lines (1975 loc) · 109 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
XZ Utils Release Notes
======================
5.7.2beta (2025-03-08)
* On the man pages, mark the following LZMA Utils script aliases as
deprecated: lzcmp, lzdiff, lzless, lzmore, lzgrep, lzegrep, and
lzfgrep. The commands that start with xz* instead of lz* have
identical behavior.
The LZMA Utils aliases lzma, unlzma, and lzcat aren't deprecated
because some of these aliases are still in common use. lzmadec
and lzmainfo aren't deprecated either.
* xz: In the ENVIRONMENT section of the man page, warn about
problems that some uses of XZ_DEFAULTS and XZ_OPT may create.
* Windows (native builds, not Cygwin): In xz, xzdec, and lzmadec,
avoid an error message on broken pipe.
* Autotools: Fix out-of-tree builds when using the bundled
getopt_long.
* Translations:
- Updated: Chinese (traditional), Croatian, Finnish, Georgian,
German, Korean, Polish, Romanian, Serbian, Spanish, Swedish,
Turkish, and Ukrainian
- Added: Dutch
* Man page translations:
- Updated: German, Korean, Romanian, and Ukrainian
- Added: Italian and Serbian
5.7.1alpha (2025-01-23)
* All fixes from 5.6.4.
* liblzma:
- Improve LZMA/LZMA2 encoder speed on 64-bit PowerPC (both
endiannesses) and those 64-bit RISC-V processors that
support fast unaligned access.
- x86/x86-64/E2K CLMUL CRC code was rewritten. It's faster and
doesn't cause false positives from sanitizers. Attributes
like __attribute__((__no_sanitize_address__)) are no longer
present.
- On 32-bit x86, CLMUL CRC and the old (but still good)
assembly versions now co-exist with runtime detection.
Both Autotools and CMake build systems handle this
automatically now.
- Use the CRC32 instructions on LoongArch to make CRC32
calculation faster.
- Add low-level APIs for RISC-V, ARM64, and x86 BCJ filters
to lzma/bcj.h. These are primarily for erofs-utils.
- Minor tweaks to ARM64 CRC32 code and BCJ filters were made.
* xz:
- Synchronize the output file and its directory before deleting
the input file using fsync(). This reduces the probability of
data loss after a system crash. However, it can be a major
performance hit if processing many small files.
NOTE: No syncing is done when xz isn't going to delete
the input file.
- Add a new option --no-sync to disable the sync-before-delete
behavior. It's useful when compressing many small files and
one doesn't worry about data loss in case of a system crash.
- Make --single-stream imply --keep.
- Use automatic word wrapping for the text in --help and
similar situations to hopefully make the strings easier for
majority of translators (no need to count spaces anymore).
* xz, xzdec, lzmainfo: When printing messages, replace
non-printable characters with question marks. This way
malicious filenames cannot be used to send escape sequences
to a terminal. This change is also applied to filenames shown
in "xz --robot --list".
* xz and xzdec on Linux: Add support for Landlock ABI versions 5
and 6.
* CMake updates:
- Increase the minimum required CMake version to 3.20.
- Revise the configuration variables and some of their options.
Document them in the file INSTALL.
- Attempt to produce liblzma.pc so that the paths are based on
${prefix}, which makes it simpler to override the paths
if the liblzma files have been moved.
- To enable translations, gettext-tools is now required. The
CMake build no longer supports installing pre-compiled
message catalog binary files (po/*.gmo).
- Apple: Use Mach-O shared library versioning that is
compatible with GNU Libtool. This should make it easier to
switch between the build systems on Apple OSes that enforce
the correct compatibility_version (macOS >= 12 doesn't?).
This change is backward compatible: binaries linked against
old CMake-built liblzma will run with liblzma that uses
Libtool style versioning.
- Windows (not Cygwin): Document CMAKE_DLL_NAME_WITH_SOVERSION
(CMake >= 3.27) in the file INSTALL. This option should
usually be left to its default value (OFF). It can be set
to ON if the liblzma DLL filename must be compatible with
the versioned filename produced by GNU Libtool. For example,
binaries distributed in MSYS2 use a versioned DLL filename.
- CMake support is no longer experimental. (It was already
not experimental when building for native Windows.)
* Windows: Building liblzma with Visual Studio 2013 is no longer
supported. Visual Studio 2015 or later (with CMake) can be used
to build liblzma and the command line tools.
* Add preliminary Georgian translation. This already contains
translations of most of the strings that are now automatically
word wrapped.
* Add build-aux/license-check.sh. Without arguments, it checks that
no license information has been forgotten. With the -v argument,
it shows the license info (or the lack of it) for each file.
If the .git directory is available, only the files in the
repository are checked. Without the .git directory, a clean tree
from an extracted release tarball is expected.
5.6.4 (2025-01-23)
* liblzma: Fix LZMA/LZMA2 encoder on big endian ARM64.
* xz:
- Fix --filters= and --filters1= ... --filters9= options
parsing. They require an argument, thus "xz --filters lzma2"
should work in addition to "xz --filters=lzma2".
- On the man page, note in the --compress and --decompress
options that the default behavior is to delete the input
file unless writing to standard output. It was already
documented in the DESCRIPTION section but new users in
a hurry might miss it.
* Windows (native builds, not Cygwin): Fix regressions introduced
in XZ Utils 5.6.3 which caused non-ASCII characters to display
incorrectly. Only builds with translation support were affected
(--enable-nls or ENABLE_NLS=ON). The following changes affect
builds that have translations enabled:
- Require UCRT because MSVCRT doesn't support UTF-8
locales and thus translations won't be readable on
Windows 10 version 1903 and later. (MSVCRT builds
are still possible with --disable-nls or ENABLE_NLS=OFF.)
- Require gettext-runtime >= 0.23.1 because older versions
don't autodetect the use of the UTF-8 code page. This
resulted in garbled non-ASCII characters even with UCRT.
- Partially fix alignment issues in xz --verbose --list
with translated messages. Chinese (simplified),
Chinese (traditional), and Korean column headings
are misaligned still because Windows and MinGW-w64
don't provide wcwidth() and XZ Utils doesn't include
a replacement function either.
* CMake: Explicitly disable unity builds. This prevents build
failures when another project uses XZ Utils via CMake's
FetchContent module, and that project enables unity builds.
* Update Chinese (traditional) and Serbian translations.
5.6.3 (2024-10-01)
IMPORTANT: This includes a Windows-specific security fix to
the command line tools (CVE-2024-47611). liblzma isn't affected
by this issue.
* liblzma:
- Fix x86-64 inline assembly compatibility with GNU Binutils
older than 2.27.
- Fix the build with GCC 4.2 on OpenBSD/sparc64.
* xzdec: Display an error instead of failing silently if the
unsupported option -M is specified.
* lzmainfo: Fix integer overflows when rounding the dictionary and
uncompressed sizes to the nearest mebibyte.
* Windows (except Cygwin and MSYS2): Add an application manifest to
xz, xzdec, lzmadec, and lzmainfo executables:
- Declare them compatible with Vista/7/8/8.1/10/11. This way
the programs won't needlessly use Operating System Context
of Vista when running on later Windows versions. This setting
doesn't mean that the executables cannot run on even older
versions if otherwise built that way.
- Declare them as UAC-compliant. MSVC added this by default
already but it wasn't done with MinGW-w64, at least not
with all toolchain variants.
- Declare them long path aware. This makes long path names
work on Windows 10 and 11 if the feature has been enabled
in the Windows registry.
- Use the UTF-8 code page on Windows 10 version 1903 and later.
* Now command line tools can access files whose names
contain characters that don't exist in the current
legacy code page.
* The options --files and --files0 now expect file lists
to be in UTF-8 instead of the legacy code page.
* This fixes a security issue: If a command line contains
Unicode characters (for example, filenames) that don't
exist in the current legacy code page, the characters are
converted to similar-looking characters with best-fit
mapping. Some best-fit mappings result in ASCII
characters that change the meaning of the command line,
which can be exploited with malicious filenames to do
argument injection or directory traversal attacks.
UTF-8 avoids best-fit mappings and thus fixes the issue.
(CVE-2024-47611)
Forcing the process code page to UTF-8 is possible only
on Windows 10 version 1903 and later. The command line
tools remain vulnerable if used on an old older
version of Windows.
This issue was discovered by Orange Tsai and splitline
from DEVCORE Research Team.
A related smaller issue remains: Windows filenames may
contain unpaired surrogates (invalid UTF-16). These are
converted to the replacement character U+FFFD in the
UTF-8 code page. Thus, filenames with different unpaired
surrogates appear identical and aren't distinguishable
from filenames that contain the actual replacement
character U+FFFD.
* When building with MinGW-w64, it is recommended to use
UCRT version instead of the old MSVCRT. For example,
non-ASCII characters from filenames won't print
correctly in messages to console with MSVCRT with
the UTF-8 code page (a cosmetic issue). liblzma-only
builds are still fine with MSVCRT.
- Cygwin and MSYS2 process command line options differently and
the above issues don't exist. There is no need to replace the
default application manifest on Cygwin and MSYS2.
* Autotools-based build:
- Fix feature checks with link-time optimization (-flto).
- Solaris: Fix a compatibility issue in version.sh. It matters
if one wants to regenerate configure by running autoconf.
* CMake:
- Use paths relative to ${prefix} in liblzma.pc when possible.
This is done only with CMake >= 3.20.
- MSVC: Install liblzma.pc as it can be useful with MSVC too.
- Windows: Fix liblzma filename prefix, for example:
* Cygwin: The DLL was incorrectly named liblzma-5.dll.
Now it is cyglzma-5.dll.
* MSVC: Rename import library from liblzma.lib to lzma.lib
while keeping liblzma.dll name as is. This helps with
"pkgconf --msvc-syntax --libs liblzma" because it mungles
"-llzma" in liblzma.pc to "lzma.lib".
* MinGW-w64: No changes.
- Windows: Use the correct resource file for lzmadec.exe.
Previously the resource file for xzdec.exe was used for both.
Autotools-based build isn't affected.
- Prefer a C11 compiler over a C99 compiler but accept both.
- Link Threads::Threads against liblzma using PRIVATE so that
-pthread and such flags won't unnecessarily get included in
the usage requirements of shared liblzma. That is,
target_link_libraries(foo PRIVATE liblzma::liblzma) no
longer adds -pthread if using POSIX threads and linking
against shared liblzma. The threading flags are still added
if linking against static liblzma.
* Updated translations: Catalan, Chinese (simplified), and
Brazilian Portuguese.
5.6.2 (2024-05-29)
* Remove the backdoor (CVE-2024-3094).
* Not changed: Memory sanitizer (MSAN) has a false positive
in the CRC CLMUL code which also makes OSS Fuzz unhappy.
Valgrind is smarter and doesn't complain.
A revision to the CLMUL code is coming anyway and this issue
will be cleaned up as part of it. It won't be backported to
5.6.x or 5.4.x because the old code isn't wrong. There is
no reason to risk introducing regressions in old branches
just to silence a false positive.
* liblzma:
- lzma_index_decoder() and lzma_index_buffer_decode(): Fix
a missing output pointer initialization (*i = NULL) if the
functions are called with invalid arguments. The API docs
say that such an initialization is always done. In practice
this matters very little because the problem can only occur
if the calling application has a bug and these functions
return LZMA_PROG_ERROR.
- lzma_str_to_filters(): Fix a missing output pointer
initialization (*error_pos = 0). This is very similar
to the fix above.
- Fix C standard conformance with function pointer types.
- Remove GNU indirect function (IFUNC) support. This is *NOT*
done for security reasons even though the backdoor relied on
this code. The performance benefits of IFUNC are too tiny in
this project to make the extra complexity worth it.
- FreeBSD on ARM64: Add error checking to CRC32 instruction
support detection.
- Fix building with NVIDIA HPC SDK.
* xz:
- Fix a C standard conformance issue in --block-list parsing
(arithmetic on a null pointer).
- Fix a warning from GNU groff when processing the man page:
"warning: cannot select font 'CW'"
* xzdec: Add support for Linux Landlock ABI version 4. xz already
had the v3-to-v4 change but it had been forgotten from xzdec.
* Autotools-based build system (configure):
- Symbol versioning variant can now be overridden with
--enable-symbol-versions. Documentation in INSTALL was
updated to match.
- Add new configure option --enable-doxygen to enable
generation and installation of the liblzma API documentation
using Doxygen. Documentation in INSTALL and PACKAGERS was
updated to match.
CMake:
- Fix detection of Linux Landlock support. The detection code
in CMakeLists.txt had been sabotaged.
- Disable symbol versioning on non-glibc Linux to match what
the Autotools build does. For example, symbol versioning
isn't enabled with musl.
- Symbol versioning variant can now be overridden by setting
SYMBOL_VERSIONING to "OFF", "generic", or "linux".
- Add support for all tests in typical build configurations.
Now the only difference to the tests coverage to Autotools
is that CMake-based build will skip more tests if features
are disabled. Such builds are only for special cases like
embedded systems.
- Separate the CMake code for the tests into tests/tests.cmake.
It is used conditionally, thus it is possible to
rm -rf tests
and the CMake-based build will still work normally except
that no tests are then available.
- Add a option ENABLE_DOXYGEN to enable generation and
installation of the liblzma API documentation using Doxygen.
* Documentation:
- Omit the Doxygen-generated liblzma API documentation from the
package. Instead, the generation and installation of the API
docs can be enabled with a configure or CMake option if
Doxygen is available.
- Remove the XZ logo which was used in the API documentation.
The logo has been retired and isn't used by the project
anymore. However, it's OK to use it in contexts that refer
to the backdoor incident.
- Remove the PDF versions of the man pages from the source
package. These existed primarily for users of operating
systems which don't come with tools to render man page
source files. The plain text versions are still included
in doc/man/txt. PDF files can still be generated to doc/man,
if the required tools are available, using "make pdf" after
running "configure".
- Update home page URLs back to their old locations on
tukaani.org.
- Update maintainer info.
* Tests:
- In tests/files/README, explain how to recreate the ARM64
test files.
- Remove two tests that used tiny x86 and SPARC object files
as the input files. The matching .c file was included but
the object files aren't easy to reproduce. The test cases
weren't great anyway; they were from the early days (2009)
of the project when the test suite had very few tests.
- Improve a few tests.
5.6.1 (2024-03-09)
IMPORTANT: This fixed bugs in the backdoor (CVE-2024-3094) (someone
had forgot to run Valgrind).
* liblzma: Fixed two bugs relating to GNU indirect function (IFUNC)
with GCC. The more serious bug caused a program linked with
liblzma to crash on start up if the flag -fprofile-generate was
used to build liblzma. The second bug caused liblzma to falsely
report an invalid write to Valgrind when loading liblzma.
* xz: Changed the messages for thread reduction due to memory
constraints to only appear under the highest verbosity level.
* Build:
- Fixed a build issue when the header file <linux/landlock.h>
was present on the system but the Landlock system calls were
not defined in <sys/syscall.h>.
- The CMake build now warns and disables NLS if both gettext
tools and pre-created .gmo files are missing. Previously,
this caused the CMake build to fail.
* Minor improvements to man pages.
* Minor improvements to tests.
5.6.0 (2024-02-24)
IMPORTANT: This added a backdoor (CVE-2024-3094). It's enabled only
in the release tarballs.
This bumps the minor version of liblzma because new features were
added. The API and ABI are still backward compatible with liblzma
5.4.x and 5.2.x and 5.0.x.
NOTE: As described in the NEWS for 5.5.2beta, the core components
are now under the BSD Zero Clause License (0BSD).
Since 5.5.2beta:
* liblzma:
- Disabled the branchless C variant in the LZMA decoder based
on the benchmark results from the community.
- Disabled x86-64 inline assembly on x32 to fix the build.
* Sandboxing support in xz:
- Landlock is now used even when xz needs to create files.
In this case the sandbox has to be more permissive than
when no files need to be created. A similar thing was
already in use with pledge(2) since 5.3.4alpha.
- Landlock and pledge(2) are now stricter when reading from
more than one input file and only writing to standard output.
- Added support for Landlock ABI version 4.
* CMake:
- Default to -O2 instead of -O3 with CMAKE_BUILD_TYPE=Release.
-O3 is not useful for speed and makes the code larger.
- Now builds lzmainfo and lzmadec.
- xzdiff, xzgrep, xzless, xzmore, and their symlinks are now
installed. The scripts are also tested during "make test".
- Added translation support for xz, lzmainfo, and the
man pages.
- Applied the symbol versioning workaround for MicroBlaze that
is used in the Autotools build.
- The general XZ Utils and liblzma API documentation is now
installed.
- The CMake component names were changed a little and several
were added. liblzma_Runtime and liblzma_Development are
unchanged.
- Minimum required CMake version is now 3.14. However,
translation support is disabled with CMake versions
older than 3.20.
- The CMake-based build is now close to feature parity with the
Autotools-based build. Most importantly a few tests aren't
run yet. Testing the CMake-based build on different operating
systems would be welcome now. See the comment at the top of
CMakeLists.txt.
* Fixed a bug in the Autotools feature test for ARM64 CRC32
instruction support for old versions of Clang. This did not
affect the CMake build.
* Windows:
- The build instructions in INSTALL and windows/INSTALL*.txt
were revised completely.
- windows/build-with-cmake.bat along with the instructions
in windows/INSTALL-MinGW-w64_with_CMake.txt should make
it very easy to build liblzma.dll and xz.exe on Windows
using CMake and MinGW-w64 with either GCC or Clang/LLVM.
- windows/build.bash was updated. It now works on MSYS2 and
on GNU/Linux (cross-compiling) to create a .zip and .7z
package for 32-bit and 64-bit x86 using GCC + MinGW-w64.
* The TODO file is no longer installed as part of the
documentation. The file is out of date and does not reflect
the actual tasks that will be completed in the future.
* Translations:
- Translated lzmainfo man pages are now installed. These
had been forgotten in earlier versions.
- Updated Croatian, Esperanto, German, Hungarian, Korean,
Polish, Romanian, Spanish, Swedish, Vietnamese, and Ukrainian
translations.
- Updated German, Korean, Romanian, and Ukrainian man page
translations.
* Added a few tests.
Summary of new features added in the 5.5.x development releases:
* liblzma:
- LZMA decoder: Speed optimizations to the C code and
added GCC & Clang compatible inline assembly for x86-64.
- Added lzma_mt_block_size() to recommend a Block size for
multithreaded encoding.
- Added CLMUL-based CRC32 on x86-64 and E2K with runtime
processor detection. Similar to CRC64, on 32-bit x86 it
isn't available unless --disable-assembler is used.
- Optimized the CRC32 calculation on ARM64 platforms using the
CRC32 instructions. Runtime detection for the instruction is
used on GNU/Linux, FreeBSD, Windows, and macOS. If the
compiler flags indicate unconditional CRC32 instruction
support (+crc) then the generic version is not built.
- Added definitions of mask values like
LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
* xz:
- Multithreaded mode is now the default. This improves
compression speed and creates .xz files that can be
decompressed in multithreaded mode. The downsides are
increased memory usage and slightly worse compression ratio.
- Added a new command line option --filters to set the filter
chain using the liblzma filter string syntax.
- Added new command line options --filters1 ... --filters9 to
set additional filter chains using the liblzma filter string
syntax. The --block-list option now allows specifying filter
chains that were set using these new options.
- Ported the command line tools to Windows MSVC.
Visual Studio 2015 or later is required.
* Added lz4 support to xzdiff/xzcmp and xzgrep.
5.5.2beta (2024-02-14)
* Licensing change: The core components are now under the
BSD Zero Clause License (0BSD). In XZ Utils 5.4.6 and older
and 5.5.1alpha these components are in the public domain and
obviously remain so; the change affects the new releases only.
0BSD is an extremely permissive license which doesn't require
retaining or reproducing copyright or license notices when
distributing the code, thus in practice there is extremely
little difference to public domain.
* liblzma
- Significant speed optimizations to the LZMA decoder were
made. There are now three variants that can be chosen at
build time:
* Basic C version: This is a few percent faster than
5.4.x due to some new optimizations.
* Branchless C: This is currently the default on platforms
for which there is no assembly code. This should be a few
percent faster than the basic C version.
* x86-64 inline assembly. This works with GCC and Clang.
The default choice can currently be overridden by setting
LZMA_RANGE_DECODER_CONFIG in CPPFLAGS: 0 means the basic
version and 3 means that branchless C version.
- Optimized the CRC32 calculation on ARM64 platforms using the
CRC32 instructions. The instructions are optional in ARMv8.0
and are required in ARMv8.1 and later. Runtime detection for
the instruction is used on GNU/Linux, FreeBSD, Windows, and
macOS. If the compiler flags indicate unconditional CRC32
instruction support (+crc) then the generic version is not
built.
* Added lz4 support to xzdiff/xzcmp and xzgrep.
* Man pages of xzdiff/xzcmp, xzgrep, and xzmore were rewritten
to simplify licensing of the man page translations.
* Translations:
- Updated Chinese (simplified), German, Korean, Polish,
Romanian, Spanish, Swedish, and Ukrainian translations.
- Updated German, Korean, Romanian, and Ukrainian man page
translations.
* Small improvements to the tests.
* Added doc/examples/11_file_info.c. It was added to the Git
repository in 2017 but forgotten to be added into distribution
tarballs.
* Removed doc/examples_old. These were from 2012.
* Removed the macos/build.sh script. It had not been updated
since 2013.
5.5.1alpha (2024-01-26)
* Added a new filter for RISC-V binaries. The filter can be used
for 32-bit and 64-bit binaries with either little or big
endianness. In liblzma, the Filter ID is LZMA_FILTER_RISCV (0x0B)
and the xz option is --riscv. liblzma filter string syntax
recognizes this filter as "riscv".
* liblzma:
- Added lzma_mt_block_size() to recommend a Block size for
multithreaded encoding
- Added CLMUL-based CRC32 on x86-64 and E2K with runtime
processor detection. Similar to CRC64, on 32-bit x86 it
isn't available unless --disable-assembler is used.
- Implemented GNU indirect function (IFUNC) as a runtime
function dispatching method for CRC32 and CRC64 fast
implementations on x86. Only GNU/Linux (glibc) and FreeBSD
builds will use IFUNC, unless --enable-ifunc is specified to
configure.
- Added definitions of mask values like
LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
- The XZ logo is now included in the Doxygen generated
documentation. It is licensed under Creative Commons
Attribution-ShareAlike 4.0.
* xz:
- Multithreaded mode is now the default. This improves
compression speed and creates .xz files that can be
decompressed multithreaded at the cost of increased memory
usage and slightly worse compression ratio.
- Added new command line option --filters to set the filter
chain using liblzma filter string syntax.
- Added new command line options --filters1 ... --filters9 to
set additional filter chains using liblzma filter string
syntax. The --block-list option now allows specifying filter
chains that were set using these new options.
- Added support for Linux Landlock as a sandboxing method.
- xzdec now supports pledge(2), Capsicum, and Linux Landlock as
sandboxing methods.
- Progress indicator time stats remain accurate after pausing
xz with SIGTSTP.
- Ported xz and xzdec to Windows MSVC. Visual Studio 2015 or
later is required.
* CMake Build:
- Supports pledge(2), Capsicum, and Linux Landlock sandboxing
methods.
- Replacement functions for getopt_long() are used on platforms
that do not have it.
* Enabled unaligned access by default on PowerPC64LE and on RISC-V
targets that define __riscv_misaligned_fast.
* Tests:
- Added two new fuzz targets to OSS-Fuzz.
- Implemented Continuous Integration (CI) testing using
GitHub Actions.
* Changed quoting style from `...' to '...' in all messages,
scripts, and documentation.
* Added basic Codespell support to help catch typo errors.
5.4.7 (2024-05-29)
* Not changed: Memory sanitizer (MSAN) has a false positive
in the CRC CLMUL code which also makes OSS Fuzz unhappy.
Valgrind is smarter and doesn't complain.
A revision to the CLMUL code is coming anyway and this issue
will be cleaned up as part of it. It won't be backported to
5.6.x or 5.4.x because the old code isn't wrong. There is
no reason to risk introducing regressions in old branches
just to silence a false positive.
* liblzma:
- lzma_index_decoder() and lzma_index_buffer_decode(): Fix
a missing output pointer initialization (*i = NULL) if the
functions are called with invalid arguments. The API docs
say that such an initialization is always done. In practice
this matters very little because the problem can only occur
if the calling application has a bug and these functions
return LZMA_PROG_ERROR.
- lzma_str_to_filters(): Fix a missing output pointer
initialization (*error_pos = 0). This is very similar
to the fix above.
- Fix C standard conformance with function pointer types.
This newly showed up with Clang 17 with -fsanitize=undefined.
There are no bug reports about this.
- Fix building with NVIDIA HPC SDK.
* xz:
- Fix a C standard conformance issue in --block-list parsing
(arithmetic on a null pointer).
- Fix a warning from GNU groff when processing the man page:
"warning: cannot select font 'CW'"
- Fix outdated threading related information on the man page.
* xzless:
- With "less" version 451 and later, use "||-" instead of "|-"
in the environment variable LESSOPEN. This way compressed
files that contain no uncompressed data are shown correctly
as empty.
- With "less" version 632 and later, use --show-preproc-errors
to make "less" show a warning on decompression errors.
* Autotools-based build system (configure):
- Symbol versioning variant can now be overridden with
--enable-symbol-versions. Documentation in INSTALL was
updated to match.
CMake:
- Linux on MicroBlaze is handled specially now. This matches
the changes made to the Autotools-based build in XZ Utils
5.4.2 and 5.2.11.
- Disable symbol versioning on non-glibc Linux to match what
the Autotools build does. For example, symbol versioning
isn't enabled with musl.
- Symbol versioning variant can now be overridden by setting
SYMBOL_VERSIONING to "OFF", "generic", or "linux".
* Documentation:
- Clarify the description of --disable-assembler in INSTALL.
The option only affects 32-bit x86 assembly usage.
- Add doc/examples/11_file_info.c. It was added to the
Git repository in 2017 but forgotten to be added into
distribution tarballs.
- Don't install the TODO file as part of the documentation.
The file is out of date.
- Update home page URLs back to their old locations on
tukaani.org.
- Update maintainer info.
5.4.6 (2024-01-26)
* Fixed a bug involving internal function pointers in liblzma not
being initialized to NULL. The bug can only be triggered if
lzma_filters_update() is called on a LZMA1 encoder, so it does
not affect xz or any application known to us that uses liblzma.
* xz:
- Fixed a regression introduced in 5.4.2 that caused encoding
in the raw format to unnecessarily fail if --suffix was not
used. For instance, the following command no longer reports
that --suffix must be used:
echo foo | xz --format=raw --lzma2 | wc -c
- Fixed an issue on MinGW-w64 builds that prevented reading
from or writing to non-terminal character devices like NUL.
* Added a new test.
5.4.5 (2023-11-01)
* liblzma:
- Use __attribute__((__no_sanitize_address__)) to avoid address
sanitization with CRC64 CLMUL. It uses 16-byte-aligned reads
which can extend past the bounds of the input buffer and
inherently trigger address sanitization errors. This isn't
a bug.
- Fixed an assertion failure that could be triggered by a large
unpadded_size argument. It was verified that there was no
other bug than the assertion failure.
- Fixed a bug that prevented building with Windows Vista
threading when __attribute__((__constructor__)) is not
supported.
* xz now properly handles special files such as "con" or "nul" on
Windows. Before this fix, the following wrote "foo" to the
console and deleted the input file "con_xz":
echo foo | xz > con_xz
xz --suffix=_xz --decompress con_xz
* Build systems:
- Allow builds with Windows win95 threading and small mode when
__attribute__((__constructor__)) is supported.
- Added a new line to liblzma.pc for MSYS2 (Windows):
Cflags.private: -DLZMA_API_STATIC
When compiling code that will link against static liblzma,
the LZMA_API_STATIC macro needs to be defined on Windows.
- CMake specific changes:
* Fixed a bug that allowed CLOCK_MONOTONIC to be used even
if the check for it failed.
* Fixed a bug where configuring CMake multiple times
resulted in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC
not being set.
* Fixed the build with MinGW-w64-based Clang/LLVM 17.
llvm-windres now has more accurate GNU windres emulation
so the GNU windres workaround from 5.4.1 is needed with
llvm-windres version 17 too.
* The import library on Windows is now properly named
"liblzma.dll.a" instead of "libliblzma.dll.a"
* Fixed a bug causing the Ninja Generator to fail on
UNIX-like systems. This bug was introduced in 5.4.0.
* Added a new option to disable CLMUL CRC64.
* A module-definition (.def) file is now created when
building liblzma.dll with MinGW-w64.
* The pkg-config liblzma.pc file is now installed on all
builds except when using MSVC on Windows.
* Added large file support by default for platforms that
need it to handle files larger than 2 GiB. This includes
MinGW-w64, even 64-bit builds.
* Small fixes and improvements to the tests.
* Updated translations: Chinese (simplified) and Esperanto.
5.4.4 (2023-08-02)
* liblzma and xzdec can now build against WASI SDK when threading
support is disabled. xz and tests don't build yet.
* CMake:
- Fixed a bug preventing other projects from including liblzma
multiple times using find_package().
- Don't create broken symlinks in Cygwin and MSYS2 unless
supported by the environment. This prevented building for the
default MSYS2 environment. The problem was introduced in
xz 5.4.0.
* Documentation:
- Small improvements to man pages.
- Small improvements and typo fixes for liblzma API
documentation.
* Tests:
- Added a new section to INSTALL to describe basic test usage
and address recent questions about building the tests when
cross compiling.
- Small fixes and improvements to the tests.
* Translations:
- Fixed a mistake that caused one of the error messages to not
be translated. This only affected versions 5.4.2 and 5.4.3.
- Updated the Chinese (simplified), Croatian, Esperanto, German,
Korean, Polish, Romanian, Spanish, Swedish, Ukrainian, and
Vietnamese translations.
- Updated the German, Korean, Romanian, and Ukrainian man page
translations.
5.4.3 (2023-05-04)
* All fixes from 5.2.12
* Features in the CMake build can now be disabled as CMake cache