-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathperl5260cdelta.html
3218 lines (2018 loc) · 163 KB
/
perl5260cdelta.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>perl5260cdelta - what is new for cperl v5.26.0</title>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<link rev="made" href="mailto:[email protected]" />
</head>
<body>
<ul id="index">
<li><a href="#NAME">NAME</a></li>
<li><a href="#DESCRIPTION">DESCRIPTION</a></li>
<li><a href="#Notice">Notice</a></li>
<li><a href="#Core-Enhancements">Core Enhancements</a>
<ul>
<li><a href="#No-magic-to-undef-yes-no-placeholder-SVs">No magic to undef/yes/no/placeholder SVs</a></li>
<li><a href="#Type-check-assignments">Type-check assignments</a></li>
<li><a href="#HvCLASS">HvCLASS</a></li>
<li><a href="#Type-infer-bless">Type-infer bless</a></li>
<li><a href="#Type-infer-subroutine-return-types">Type-infer subroutine return types</a></li>
<li><a href="#for-qw-is-legal-again">for qw() is legal again</a></li>
<li><a href="#Perl-can-now-do-default-collation-in-UTF-8-locales-on-platforms-that-support-it">Perl can now do default collation in UTF-8 locales on platforms that support it</a></li>
<li><a href="#Better-locale-collation-of-strings-containing-embedded-NUL-characters">Better locale collation of strings containing embedded NUL characters</a></li>
<li><a href="#Unescaped-literal-characters-in-regular-expression-patterns-are-no-longer-permissible">Unescaped literal "{" characters in regular expression patterns are no longer permissible</a></li>
<li><a href="#Literal-control-character-variable-names-are-no-longer-permissible">Literal control character variable names are no longer permissible</a></li>
<li><a href="#New-regular-expression-modifier-xx">New regular expression modifier /xx</a></li>
<li><a href="#NBSP-is-no-longer-permissible-in-N">NBSP is no longer permissible in \N{...}</a></li>
<li><a href="#CORE-subroutines-for-hash-and-array-functions-callable-via-reference">CORE subroutines for hash and array functions callable via reference</a></li>
<li><a href="#Unicode-9.0-is-now-supported">Unicode 9.0 is now supported</a></li>
<li><a href="#Use-of-p-script-uses-the-improved-Script_Extensions-property">Use of \p{script} uses the improved Script_Extensions property</a></li>
<li><a href="#Declaring-a-reference-to-a-variable">Declaring a reference to a variable</a></li>
<li><a href="#Indented-Here-documents">Indented Here-documents</a></li>
<li><a href="#and-INC">'.' and @INC</a></li>
<li><a href="#create-a-safer-utf8_hop-called-utf8_hop_safe">create a safer utf8_hop() called utf8_hop_safe()</a></li>
<li><a href="#CAPTURE-CAPTURE-and-CAPTURE_ALL">@{^CAPTURE}, %{^CAPTURE}, and %{^CAPTURE_ALL}</a></li>
<li><a href="#Improved-.pmc-loading">Improved .pmc loading</a></li>
<li><a href="#Added-SAFE_RX_-substrs-accessors">Added SAFE_RX_ substrs accessors</a></li>
</ul>
</li>
<li><a href="#Security">Security</a>
<ul>
<li><a href="#Storable-stack-overflows">Storable stack overflows</a></li>
<li><a href="#Escaped-colons-and-relative-paths-in-PATH">"Escaped" colons and relative paths in PATH</a></li>
<li><a href="#Unicode-identifiers:-Moderately-Restrictive-Level">Unicode identifiers: Moderately Restrictive Level</a></li>
<li><a href="#chdir-heap-buffer-overflow-on-the-perl-stack">chdir heap-buffer-overflow on the perl stack</a></li>
<li><a href="#Improved-Hash-DDoS-prevention">Improved Hash DDoS prevention</a></li>
<li><a href="#n-buffer-overflows">@{ \327 \n } buffer overflows</a></li>
<li><a href="#eval-q-.-chr-overlarge-stack-overflow">eval "q" . chr(overlarge) stack overflow</a></li>
<li><a href="#Protect-and-warn-on-hash-flood-DoS">Protect and warn on hash flood DoS</a></li>
<li><a href="#use-utf8-Script">use utf8 'Script'</a></li>
<li><a href="#Unicode-normalization-of-identifiers-names">Unicode normalization of identifiers/names</a></li>
<li><a href="#No-binary-symbols">No binary symbols</a></li>
<li><a href="#hash-seed-exposure">hash seed exposure</a></li>
<li><a href="#Warn-on-metasploit-CVE-2015-1592">Warn on metasploit CVE-2015-1592</a></li>
<li><a href="#Warn-on-metasploit-reverse-shells">Warn on metasploit reverse shells</a></li>
<li><a href="#syscalls-warnings-also-security">syscalls warnings also security</a></li>
</ul>
</li>
<li><a href="#Deprecations">Deprecations</a>
<ul>
<li><a href="#do_open-do_close-macros">do_open, do_close macros</a></li>
<li><a href="#Removed-as-package-seperator">Removed ' as package seperator</a></li>
<li><a href="#New-items-in-perldeprecation">New items in perldeprecation</a></li>
</ul>
</li>
<li><a href="#Incompatible-Changes">Incompatible Changes</a>
<ul>
<li><a href="#String-delimiters-that-arent-stand-alone-graphemes-are-illegal">String delimiters that aren't stand-alone graphemes are illegal</a></li>
<li><a href="#for-state-loops-still-illegal">for state loops still illegal</a></li>
<li><a href="#scalar-hash-return-value-changed">scalar(%hash) return value changed</a></li>
<li><a href="#keys-returned-from-an-lvalue-subroutine">keys returned from an lvalue subroutine</a></li>
</ul>
</li>
<li><a href="#Performance-Enhancements">Performance Enhancements</a></li>
<li><a href="#Modules-and-Pragmata">Modules and Pragmata</a>
<ul>
<li><a href="#New-Modules-and-Pragmata">New Modules and Pragmata</a></li>
<li><a href="#Updated-Modules-and-Pragmata">Updated Modules and Pragmata</a></li>
<li><a href="#Removed-Modules-and-Pragmata">Removed Modules and Pragmata</a></li>
</ul>
</li>
<li><a href="#Documentation">Documentation</a>
<ul>
<li><a href="#New-Documentation">New Documentation</a>
<ul>
<li><a href="#perldeprecation">perldeprecation</a></li>
</ul>
</li>
<li><a href="#Changes-to-Existing-Documentation">Changes to Existing Documentation</a>
<ul>
<li><a href="#perlobj">perlobj</a></li>
<li><a href="#perlop">perlop</a></li>
<li><a href="#perllocale">perllocale</a></li>
<li><a href="#perldiag">perldiag</a></li>
<li><a href="#perldtrace">perldtrace</a></li>
<li><a href="#perlguts">perlguts</a></li>
<li><a href="#perlvar">perlvar</a></li>
<li><a href="#perlootut">perlootut</a></li>
<li><a href="#perlhack">perlhack</a></li>
<li><a href="#perlre">perlre</a></li>
<li><a href="#perlinterp">perlinterp</a></li>
<li><a href="#perlcall">perlcall</a></li>
<li><a href="#perltie">perltie</a></li>
<li><a href="#perldata">perldata</a></li>
<li><a href="#perlexperiment-and-perlref">perlexperiment and perlref</a></li>
<li><a href="#perlfunc">perlfunc</a></li>
<li><a href="#perlunicode">perlunicode</a></li>
<li><a href="#perlvar1">perlvar</a></li>
<li><a href="#perlcommunity">perlcommunity</a></li>
<li><a href="#perldelta">perldelta</a></li>
<li><a href="#perllocale1">perllocale</a></li>
<li><a href="#perlmodinstall">perlmodinstall</a></li>
<li><a href="#perlmodlib">perlmodlib</a></li>
<li><a href="#perlnewmod">perlnewmod</a></li>
<li><a href="#perlintern-and-perlapi">perlintern and perlapi</a></li>
<li><a href="#perlsec">perlsec</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#Diagnostics">Diagnostics</a>
<ul>
<li><a href="#New-Diagnostics">New Diagnostics</a>
<ul>
<li><a href="#New-Errors">New Errors</a></li>
<li><a href="#New-Warnings">New Warnings</a></li>
</ul>
</li>
<li><a href="#Changes-to-Existing-Diagnostics">Changes to Existing Diagnostics</a></li>
</ul>
</li>
<li><a href="#Configuration-and-Compilation">Configuration and Compilation</a></li>
<li><a href="#Testing">Testing</a></li>
<li><a href="#Utility-Changes">Utility Changes</a>
<ul>
<li><a href="#c2ph">c2ph</a></li>
</ul>
</li>
<li><a href="#Platform-Support">Platform Support</a>
<ul>
<li><a href="#Platform-Specific-Notes">Platform-Specific Notes</a></li>
</ul>
</li>
<li><a href="#Internal-Changes">Internal Changes</a></li>
<li><a href="#Selected-Bug-Fixes">Selected Bug Fixes</a></li>
<li><a href="#Known-Problems">Known Problems</a></li>
<li><a href="#Errata-From-Previous-Releases">Errata From Previous Releases</a></li>
<li><a href="#Obituary">Obituary</a></li>
<li><a href="#Acknowledgements">Acknowledgements</a></li>
<li><a href="#Reporting-Bugs">Reporting Bugs</a></li>
<li><a href="#SEE-ALSO">SEE ALSO</a></li>
</ul>
<h1 id="NAME">NAME</h1>
<p>perl5260cdelta - what is new for cperl v5.26.0</p>
<h1 id="DESCRIPTION">DESCRIPTION</h1>
<p>This document describes the differences between the cperl 5.24.2 and the cperl 5.26.0 releases.</p>
<p>If you are upgrading from an earlier release such as v5.24.1c, first read the <a href="/cperl/perl5242cdelta.html">perl5242cdelta</a> documentation, which describes differences between v5.24.2c and v5.24.1c.</p>
<h1 id="Notice">Notice</h1>
<p>cperl v5.26.0c was merged with perl v5.26.0 (as all previous major cperl releases). The rejected upstream commits for the differences have been documented at the github issues <a href="https://github.com/perl11/cperl/issues/165">[cperl #165]</a> and <a href="https://github.com/perl11/cperl/issues/256">[cperl #256]</a>.</p>
<p><code>${^ENCODING}</code> and the encoding pragma was not removed, rather fixed instead.</p>
<p>User-type support is greatly enhanced. See <a href="#Type-check-assignments">"Type-check assignments"</a>, <a href="#HvCLASS">"HvCLASS"</a>, <a href="#Type-infer-bless">"Type-infer bless"</a>, and <a href="#Type-infer-subroutine-return-types">"Type-infer subroutine return types"</a>. Before just coretypes were properly checked, now <code>use types</code> adds warnings for all other types.</p>
<p>The still incomplete and slow implementation for the experimental subroutine signatures feature from 5.25.4 was not added, as cperl's signatures are over 50% faster for over a year already and have many more features. In detail the new <code>OP_ARGELEM</code>, <code>OP_ARGDEFELEM</code> and <code>OP_ARGCHECK</code> are not used, cperl still uses a single <code>OP_SIGNATURE</code> op and passes its arguments properly as in XS on the stack, not via <code>@_</code>.</p>
<p>cperl doesn't use the slow Siphash 1-3 as default on 64bit, and no hybrid hash function as introduced with 5.25.8. cperl rather uses a short and fast hash function and other typical hash table optimizations, while adding proper security in the collision resolution instead. A secure PRF (pseudo random function) can never ensure DoS safety for a hash table, contrary to the Siphash paper claims.</p>
<h1 id="Core-Enhancements">Core Enhancements</h1>
<h2 id="No-magic-to-undef-yes-no-placeholder-SVs">No magic to undef/yes/no/placeholder SVs</h2>
<p>cperl silently forbids attaching magic to the four major builtin SV sentinels undef, yes, no and placeholder, which are mostly compared to by pointer. Adding magic to them will break that comparison.</p>
<h2 id="Type-check-assignments">Type-check assignments</h2>
<p>Assignment type violations are now also warned, with <code>use warnings 'types'</code> enabled, previously only signature types were checked. Only signature type violations or <code>use types 'strict'</code> violations are fatal.</p>
<p>Note that the type system is still completely unsound. So far it is only there to catch the most common errors and enable coretype optimizations. cperl only.</p>
<h2 id="HvCLASS">HvCLASS</h2>
<p>With cperl <code>use base</code> or <code>use fields</code> now closes the <code>@ISA</code> and hereby enable compile-time checks and optimizations. The new <code>Internals::HvCLASS</code> function gets or sets the same type for base/field classes as with the upcoming class keyword. See <a href="https://github.com/perl11/cperl/issues/249">[cperl #249]</a>. cperl only.</p>
<h2 id="Type-infer-bless">Type-infer bless</h2>
<p>bless with a constant 2nd argument, the classname, infers this type to the enclosing sub if its the last statement in a body, or to the left-side assignment of a lexical variable. cperl only.</p>
<h2 id="Type-infer-subroutine-return-types">Type-infer subroutine return types</h2>
<p>Subroutine types, either declared or inferred, are now passed through to the type-checker at compile-time. cperl only.</p>
<h2 id="for-qw-is-legal-again">for qw() is legal again</h2>
<p>perl5.14 deprecated and 5.18 started disallowing a <a href="/cperl/perlsyn.html#Statement-Modifiers">for</a> loop with a <a href="/cperl/perlop.html#qw-STRING">qw()</a> list, "qw-as-parens".</p>
<p>The rationale to remove the handy <code>for qw()</code> syntax was technical and trivial to fix. cperl 5.25.3 re-instated it for <code>for</code> loops, but not for the rest. cperl does not insist on the backwards syntax to require <code>(qw( ... ))</code> around the <code>for</code> list.</p>
<pre><code> cperl5.25.3 -e'for qw(a b c) { print $_ }'
perl5.18 -e'for (qw(a b c)) { print $_ }'
perl5.14 -e'for $_ qw(a b c) { print $_ }'
=> Use of qw(...) as parentheses is deprecated at -e line 1
perl5.12 -e'for $_ qw(a b c) { print $_ }'</code></pre>
<p>The new additional cperl syntax is even easier to use than before. See <a href="https://github.com/perl11/cperl/issues/26">[cperl #26]</a>. cperl only.</p>
<h2 id="Perl-can-now-do-default-collation-in-UTF-8-locales-on-platforms-that-support-it">Perl can now do default collation in UTF-8 locales on platforms that support it</h2>
<p>Some platforms natively do a reasonable job of collating and sorting in UTF-8 locales. Perl now works with those. For portability and full control, <a href="/cperl/lib/Unicode/Collate.html">Unicode::Collate</a> is still recommended, but now you may not need to do anything special to get good-enough results, depending on your application. See <a href="/cperl/perllocale.html#Category-LC_COLLATE:-Collation:-Text-Comparisons-and-Sorting">"Category <code>LC_COLLATE</code>: Collation: Text Comparisons and Sorting" in perllocale</a></p>
<h2 id="Better-locale-collation-of-strings-containing-embedded-NUL-characters">Better locale collation of strings containing embedded <code>NUL</code> characters</h2>
<p>In locales that have multi-level character weights, these are now ignored at the higher priority ones. There are still some gotchas in some strings, though. See <a href="/cperl/perllocale.html#Collation-of-strings-containing-embedded-NUL-characters">"Collation of strings containing embedded <code>NUL</code> characters" in perllocale</a>.</p>
<h2 id="Unescaped-literal-characters-in-regular-expression-patterns-are-no-longer-permissible">Unescaped literal <code>"{"</code> characters in regular expression patterns are no longer permissible</h2>
<p>You have to now say something like <code>"\{"</code> or <code>"[{]"</code> to specify to match a LEFT CURLY BRACKET. This will allow future extensions to the language. This restriction is not enforced, nor are there current plans to enforce it, if the <code>"{"</code> is the first character in the pattern.</p>
<p>These have been deprecated since v5.16, with a deprecation message displayed starting in v5.22.</p>
<h2 id="Literal-control-character-variable-names-are-no-longer-permissible">Literal control character variable names are no longer permissible</h2>
<p>A variable name may no longer contain a literal control character under any circumstances. These previously were allowed in single-character names on ASCII platforms, but have been deprecated there since Perl v5.20. This affects things like <code>$<i>\cT</i></code>, where <i>\cT</i> is a literal control (such as a <code>NAK</code> or <code>NEGATIVE ACKNOWLEDGE</code> character) in the source code.</p>
<h2 id="New-regular-expression-modifier-xx">New regular expression modifier <code>/xx</code></h2>
<p>Specifying two <code>x</code> characters to modify a regular expression pattern does everything that a single one does, but additionally TAB and SPACE characters within a bracketed character class are generally ignored and can be added to improve readability, like <span style="white-space: nowrap;"><code>/[ ^ A-Z d-f p-x ]/xx</code></span>. Details are at <a href="/cperl/perlre.html#x-and-xx">"/x and /xx" in perlre</a>.</p>
<h2 id="NBSP-is-no-longer-permissible-in-N"><code>NBSP</code> is no longer permissible in <code>\N{...}</code></h2>
<p>The name of a character may no longer contain non-breaking spaces. It has been deprecated to do so since Perl v5.22.</p>
<h2 id="CORE-subroutines-for-hash-and-array-functions-callable-via-reference"><code>CORE</code> subroutines for hash and array functions callable via reference</h2>
<p>The hash and array functions in the <code>CORE</code> namespace--<code>keys</code>, <code>each</code>, <code>values</code>, <code>push</code>, <code>pop</code>, <code>shift</code>, <code>unshift</code> and <code>splice</code>--, can now be called with ampersand syntax (<code>&CORE::keys(\%hash</code>) and via reference (<code>my $k = \&CORE::keys; $k->(\%hash)</code>). Previously they could only be used when inlined.</p>
<h2 id="Unicode-9.0-is-now-supported">Unicode 9.0 is now supported</h2>
<p>A list of changes is at <a href="http://www.unicode.org/versions/Unicode9.0.0/">http://www.unicode.org/versions/Unicode9.0.0/</a>. Modules that are shipped with core Perl but not maintained by p5p do not necessarily support Unicode 9.0. <a href="/cperl/lib/Unicode/Normalize.html">Unicode::Normalize</a> does work on 9.0.</p>
<p>Note that some changed UCD database files in 9.0 stayed renamed to their shortened name in perl.</p>
<h2 id="Use-of-p-script-uses-the-improved-Script_Extensions-property">Use of <code>\p{<i>script</i>}</code> uses the improved Script_Extensions property</h2>
<p>Unicode 6.0 introduced an improved form of the Script (<code>sc</code>) property, and called it Script_Extensions (<code>scx</code>). As of now, Perl uses this improved version when a property is specified as just <code>\p{<i>script</i>}</code>. The meaning of compound forms, like <code>\p{sc=<i>script</i>}</code> are unchanged. This should make programs be more accurate when determining if a character is used in a given script, but there is a slight chance of breakage for programs that very specifically needed the old behavior. See <a href="/cperl/perlunicode.html#Scripts">"Scripts" in perlunicode</a>.</p>
<h2 id="Declaring-a-reference-to-a-variable">Declaring a reference to a variable</h2>
<p>As an experimental feature, Perl now allows the referencing operator to come after <a href="/cperl/perlfunc.html#my"><code>my()</code></a>, <a href="/cperl/perlfunc.html#state"><code>state()</code></a>, <a href="/cperl/perlfunc.html#our"><code>our()</code></a>, or <a href="/cperl/perlfunc.html#local"><code>local()</code></a>. This syntax must be enabled with <code>use feature 'declared_refs'</code>. It is experimental, and will warn by default unless <code>no warnings 'experimental::refaliasing'</code> is in effect. It is intended mainly for use in assignments to references. For example:</p>
<pre><code> use experimental 'refaliasing', 'declared_refs';
my \$a = \$b;</code></pre>
<p>See <a href="/cperl/perlref.html#Assigning-to-References">"Assigning to References" in perlref</a> for slightly more detail.</p>
<p>Note that this still looks much worse than the perl6 bind operator: my $a := $b;</p>
<h2 id="Indented-Here-documents">Indented Here-documents</h2>
<p>This adds a new modifier '~' to here-docs that tells the parser that it should look for /^\s*$DELIM\n/ as the closing delimiter.</p>
<p>These syntaxes are all supported:</p>
<pre><code> <<~EOF;
<<~\EOF;
<<~'EOF';
<<~"EOF";
<<~`EOF`;
<<~ 'EOF';
<<~ "EOF";
<<~ `EOF`;</code></pre>
<p>The '~' modifier will strip, from each line in the here-doc, the same whitespace that appears before the delimiter.</p>
<p>Newlines will be copied as is, and lines that don't include the proper beginning whitespace will cause perl to croak.</p>
<p>For example:</p>
<pre><code> if (1) {
print <<~EOF;
Hello there
EOF
}</code></pre>
<p>prints "Hello there\n" with no leading whitespace.</p>
<h2 id="and-INC">'.' and @INC</h2>
<p>The old cperl <code>-Dfortify_inc</code> security feature was now also introduced by perl5 and renamed to <code>-Ddefault_inc_excludes_dot</code>.</p>
<p>Because the testing and make process for perl modules does not work well with <code>.</code> missing from @INC, cperl and perl5 still support the environment variable <code>PERL_USE_UNSAFE_INC=1</code> which makes Perl behave as it previously did, returning <code>.</code> to @INC in all child processes.</p>
<h2 id="create-a-safer-utf8_hop-called-utf8_hop_safe">create a safer utf8_hop() called utf8_hop_safe()</h2>
<p>Unlike <code>utf8_hop()</code>, <code>utf8_hop_safe()</code> won't navigate before the beginning or after the end of the supplied buffer.</p>
<h2 id="CAPTURE-CAPTURE-and-CAPTURE_ALL">@{^CAPTURE}, %{^CAPTURE}, and %{^CAPTURE_ALL}</h2>
<p><code>@{^CAPTURE}</code> exposes the capture buffers of the last match as an array. So <code>$1</code> is <code>${^CAPTURE}[0]</code>.</p>
<p><code>%{^CAPTURE}</code> is the equivalent to <code>%+</code> (ie named captures)</p>
<p><code>%{^CAPTURE_ALL}</code> is the equivalent to <code>%-</code> (ie all named captures).</p>
<h2 id="Improved-.pmc-loading">Improved .pmc loading</h2>
<p>cperl now sets the correct <i>.pmc</i> filename for <code>__FILE__</code> and <code>CopFILE</code>, when it was loaded from it.</p>
<p>cperl also allows bypassing a <i>.pmc</i> if loaded explicitly via <a href="/cperl/perlfunc.html#do">do</a> and an absolute pathname.</p>
<p>This allows improved <i>.pmc</i> file caching of only selective parts of a module. Such as a method jit, which stores onlt some subs, but not the whole module in it's cache. Hence the Cache logic in the <i>.pmc</i> can now first load the parallel source <i>.pm</i> and then apply the <i>.pmc</i> optimizations. E.g. by loading a LLVM <i>.bc</i> file contents with only some subs.</p>
<p>The impact for existing code is low. If you loaded a .pmc via <code>do "/abspath/module.pm"</code> you need to add now a final "c" explictly: <code>do "/abspath/module.pmc"</code>.</p>
<p>With perl5 upstream those two longstanding PMC bugs made it impossible to use a partial Byte- or JitCache. It also makes it possible to re-instate the old python-like timestamp logic which was removed for pugs 2006 with commit <a href="https://github.com/perl11/cperl/commit/a91233bf4cf6a12df8935c3530a6ca900ca6ca2f">a91233bf4cf</a>.</p>
<p>See <a href="https://github.com/perl11/cperl/issues/244">[cperl #244]</a>. cperl only.</p>
<h2 id="Added-SAFE_RX_-substrs-accessors">Added SAFE_RX_ substrs accessors</h2>
<pre><code> SAFE_RX_CHECK_SUBSTR(rx)
SAFE_RX_ANCHORED_SUBSTR(rx)
SAFE_RX_ANCHORED_UTF8(rx)
SAFE_RX_FLOAT_SUBSTR(rx)
SAFE_RX_FLOAT_UTF8(rx)</code></pre>
<p>Other regex engines don't fill <code>rx->substrs->data[]</code>, so it is unsafe to access it. Only allow ext/re and Perl_core_reg_engine. Currently only used in <a href="/cperl/perlapi.html#op_dump">op_dump()</a>.</p>
<h1 id="Security">Security</h1>
<h2 id="Storable-stack-overflows">Storable stack overflows</h2>
<p>By reading malcrafted local Storable files or memory you could easily overwrite the local stack with controlled data. With bigger values you could cause an immediate exit, without backtrace or an exception being caught.</p>
<p>Another major stack-overflow fix is for <a href="https://rt.cpan.org/Ticket/Display.html?id=97526">[cpan #97526]</a>, limiting the maximal number of nested hash or arrays to 3000. Cpanel::JSON::XS has it at 512.</p>
<p>Note that p5p doesn't think that these are security issues. <a href="https://rt.perl.org/Public/Bug/Display.html?id=130635">[perl #130635]</a> (even if similar less severe attacks had a CVE and a metasploit module, which cperl detects).</p>
<p>cperl only so far. Uploaded to CPAN, but at this date still unauthorized.</p>
<h2 id="Escaped-colons-and-relative-paths-in-PATH">"Escaped" colons and relative paths in PATH</h2>
<p>On Unix systems, Perl treats any relative paths in the PATH environment variable as tainted when starting a new process. Previously, it was allowing a backslash to escape a colon (unlike the OS), consequently allowing relative paths to be considered safe if the PATH was set to something like <code>/\:.</code>. The check has been fixed to treat <code>.</code> as tainted in that example.</p>
<h2 id="Unicode-identifiers:-Moderately-Restrictive-Level">Unicode identifiers: Moderately Restrictive Level</h2>
<p>cperl as first dynamic scripting language follows the <b>General Security Profile</b> for identifiers in programming languages.</p>
<p><b>Moderately Restrictive</b>: Allow <code>Latin</code> with other Recommended or Aspirational scripts except <code>Cyrillic</code> and <code>Greek</code>. Otherwise, the same as <a href="http://www.unicode.org/reports/tr39/#Identifier_Characters">Highly Restrictive</a>, i.e. allow <code>:Japanese</code>, <code>:Korean</code> and <code>:Hanb</code>.</p>
<p><i>"Some characters are not in modern customary use, and thus implementations may want to exclude them from identifiers. These include characters in historic and obsolete scripts, scripts used mostly liturgically, and regional scripts used only in very small communities or with very limited current usage. The set of characters in Table 4, Candidate Characters for Exclusion from Identifiers provides candidates of these."</i></p>
<p>cperl honors the <a href="http://www.unicode.org/reports/tr31/#Table_Candidate_Characters_for_Exclusion_from_Identifiers">TR31 Candidate Characters for Exclusion from Identifiers </a></p>
<p>I.e. You may still declare those scripts as valid, but they are not automatically allowed, similar to the need to declare mixed scripts.</p>
<pre><code> use utf8;
my $ᭅ = 1; # \x{1b45} BALINESE LETTER KAF SASAK</code></pre>
<p>=> Invalid script Balinese in identifier ᭅ for U+1B45</p>
<pre><code> use utf8 'Balinese';
my $ᭅ = 1; # \x{1b45} BALINESE LETTER KAF SASAK
print "ok";</code></pre>
<p>=></p>
<pre><code> ok</code></pre>
<p>The scripts listed at "Table 6, Aspirational Use Scripts": <code>Canadian_Aboriginal</code>, <code>Miao</code>, <code>Mongolian</code>, <code>Tifinagh</code> and <code>Yi</code> are included, i.e. need not to be declared.</p>
<p>With this restriction cperl fulfills the Moderately Restrictive level for identifiers by default. See <a href="http://www.unicode.org/reports/tr39/#General_Security_Profile">http://www.unicode.org/reports/tr39/#General_Security_Profile</a> and <a href="http://www.unicode.org/reports/tr36/#Security_Levels_and_Alerts">http://www.unicode.org/reports/tr36/#Security_Levels_and_Alerts</a>.</p>
<p>Missing for more unicode security are warnings on single-, mixed and whole-script confusables, with a new utf8 warnings 'confusables' subcategory <a href="https://github.com/perl11/cperl/issues/265">[cperl #265]</a>.</p>
<p>With special declarations of the used scripts or turning off no warnings 'utf8', you can weaken the restriction level to <b>Minimally Restrictive</b>.</p>
<p>All utf8 encoded names are checked for wellformed-ness.</p>
<h2 id="chdir-heap-buffer-overflow-on-the-perl-stack"><code>chdir</code> heap-buffer-overflow on the perl stack</h2>
<p>When called without argument it overwrote subsequent stack entries with the easily controllable result. <a href="https://rt.perl.org/Public/Bug/Display.html?id=129130">[perl #129130]</a></p>
<h2 id="Improved-Hash-DDoS-prevention">Improved Hash DDoS prevention</h2>
<p>This is merely a theoretical problem, improving on the previous sleep solution against hash floods. Distributed hashflood attacks could lead to memory exhaustion and denial of service in threaded servers, which would bypass the original FAIL_DELAY-like intrusion detection and mitigation.</p>
<p>First sleep, but if >128 concurrent attacks are detected, exit hard. Use a global hash_slowdos counter. Note that this is also triggered by a 128*8*128 hash collision single source attack (=131072). This is still better, faster and smaller than the java solution to convert the linked list to a tree. We log the attackers and can block them. <a href="https://github.com/perl11/cperl/issues/246">[cperl #246]</a>. cperl only.</p>
<h2 id="n-buffer-overflows"><code>@{ \327 \n }</code> buffer overflows</h2>
<p>Fixed <code>@{ \327 \n }</code> tokenizer failures and heap buffer overflows in <code>sv_vcatpvfn_flags()</code> with wrong tracking of <code>PL_linestr</code>, the currently parsed line buffer. This can easily lead to security relevant exploits.</p>
<p><a href="https://rt.perl.org/Public/Bug/Display.html?id=128951">[perl #128951]</a></p>
<h2 id="eval-q-.-chr-overlarge-stack-overflow"><code>eval "q" . chr(overlarge)</code> stack overflow</h2>
<p>In <code>eval "q" . chr(100000000064)</code> generating the error message <code>Can't find string terminator "XXX"'</code> was overrunning a local buffer designed to hold a single utf8 char, since it wasn't allowing for the <code>\0</code> at the end.</p>
<p><a href="https://rt.perl.org/Public/Bug/Display.html?id=128952">[perl #128952]</a></p>
<h2 id="Protect-and-warn-on-hash-flood-DoS">Protect and warn on hash flood DoS</h2>
<p>If the collisions for a hash key lookup exceeds 128 tries (i.e. a linear search in a linked list), this qualifies as a malicious hash DoS (<i>Denial of Service</i>) attack. Generally maximal 8-10 collisions appear in normal hash table usage. Every 8th such hash flood attack performs a <code>sleep(2)</code> to limit the impact.</p>
<p>Detect and protect against it, also call the new <code>warn_security("Hash flood")</code>.</p>
<p>This security scheme is much easier and faster than trying to hide the random hash seed with randomized iterators and collisions lists, which cperl doesn't use.</p>
<p>See <a href="#New-Diagnostics">"New Diagnostics"</a>.</p>
<h2 id="use-utf8-Script">use utf8 'Script'</h2>
<p>In order to avoid TR39 confusable security hacks, we add the following unicode rules for identifiers and literals with <b>mixed script</b> properties:</p>
<ul>
<li><p>The 'Common', 'Latin' and 'Inherited' scripts are always allowed and don't need to be declared.</p>
</li>
<li><p>The first non-default unicode script for an identifier is the only allowed one. This qualifies as single-script. More scripts lead to parsers errors.</p>
</li>
<li><p>Additional unicode scripts can and should be declared via <b>use utf8 'Greek', 'script-name2'...</b>. This allows mixed scripts in identifiers. This can be scoped in blocks.</p>
</li>
<li><p>To fulfill the Moderately Restrictive Level for the Unicode <a href="http://www.unicode.org/reports/tr39/#Restriction_Level_Detection">General Security Profile </a> you may not mix Greek with Cyrillic identifier characters in the same program.</p>
</li>
</ul>
<p>See <a href="http://www.unicode.org/reports/tr39/#Mixed_Script_Detection">http://www.unicode.org/reports/tr39/#Mixed_Script_Detection</a> and <a href="https://github.com/perl11/cperl/issues/229">[cperl #229]</a></p>
<p>This holds for all identifiers (i.e. all names: package, gv, sub, variables) and literal numbers.</p>
<p>Currently there exist 131 scripts, see <a href="/cperl/lib/utf8.html#Valid-scripts">"Valid scripts" in utf8</a>.</p>
<h2 id="Unicode-normalization-of-identifiers-names">Unicode normalization of identifiers/names</h2>
<p>All stored utf8 names, identifiers and literals are parsed and stored as normalized NFC unicode, which prevents from various TR39 and TR36 unicode confusable and spoofing security problems.</p>
<p>However, dynamically created symbols via string refs are not normalized. <code>${"$decomposed"}</code> stays decomposed.</p>
<p>Note that even perl6 stores different names for confusables, which match each other due to their NFG rules on their string matchers. perl5 matches strictly binary, which leads to confusable and spoofing security problems.</p>
<p>See <a href="https://github.com/perl11/cperl/issues/228">[cperl #228]</a>, <a href="http://www.unicode.org/reports/tr36/">http://www.unicode.org/reports/tr36/</a>, <a href="http://www.unicode.org/reports/tr39">http://www.unicode.org/reports/tr39</a>, <a href="http://www.unicode.org/reports/tr31/">http://www.unicode.org/reports/tr31/</a> and the Python 3 discussion 2007 on PEP 3131 <a href="https://docs.python.org/3/reference/lexical_analysis.html#identifiers">https://docs.python.org/3/reference/lexical_analysis.html#identifiers</a>.</p>
<p>Python 3 normalizes to NFKC (Compatibility Decomposition, followed by Canonical Composition), cperl uses both canonical transformations. See <a href="http://unicode.org/reports/tr15/#Norm_Forms">http://unicode.org/reports/tr15/#Norm_Forms</a> for the difference. Basically NFKC transforms to shorter ligatures. NFC is recommended by TR15.</p>
<h2 id="No-binary-symbols">No binary symbols</h2>
<p>Fallback to the secure behvaiour as before v5.16 and strip symbol names of everything after the first \0 character. This protects from creating binary symbols as with <code>no strict 'refs'; ${"a\0\hidden"}</code>, which were especially problematic for package names, which were mapped 1:1 to filenames. With the default warning 'security' in effect, a warning is produced by the <a href="/cperl/perlapi.html#warn_security">"warn_security" in perlapi</a> API, same as for unsafe syscalls since 5.20.</p>
<p>See <a href="/cperl/perldiag.html#Invalid-0-character-in-string-for-SYMBOL:-s">"Invalid \0 character in string for SYMBOL: %s" in perldiag</a> and <a href="https://github.com/perl11/cperl/issues/233">[cperl #233]</a>.</p>
<h2 id="hash-seed-exposure">hash seed exposure</h2>
<p>cperl5.22.2 added a restraint to expose the internal hash secret seed via the environment variable PERL_HASH_SEED_DEBUG=1 to be hidden in taint mode. See <a href="https://github.com/perl11/cperl/issues/114">[cperl #114]</a> and <a href="/cperl/perl5222cdelta.html#Core-Enhancements">"Core Enhancements" in perl5222cdelta</a>.</p>
<pre><code> PERL_HASH_SEED_DEBUG=1 cperl5.22.2 -e1 =>
HASH_FUNCTION = FNV1A HASH_SEED = 0xecfb00eb PERTURB_KEYS = 0 (TOP)
PERL_HASH_SEED_DEBUG=1 cperl5.22.2 -t -e1 => empty</code></pre>
<p>But unfortunately not many perl services are actually protected with <code>-t</code>, even if cperl fixed taint mode to be actually secure. The seed exposure is only needed for a debugging perl, and actually is security relevant.</p>
<p>So <code>PERL_HASH_SEED_DEBUG=1</code> will now hide the seed value in non-DEBUGGING builds.</p>
<pre><code> PERL_HASH_SEED_DEBUG=1 cperl5.25.2 -e1 =>
HASH_FUNCTION = FNV1A HASH_SEED = <hidden> PERTURB_KEYS = 0 (TOP)</code></pre>
<p>Note that the seed is still trivially exposable via other means if a local script can be executed, as the seed value is readable from a fixed memory offset via unpack "P". That's why cperl fixed hash table security via proper means in the collision resolution, not via a slow hash function, and not via order hiding as perl5 believes in.</p>
<p>More discussion at <a href="https://github.com/google/highwayhash/issues/28">https://github.com/google/highwayhash/issues/28</a> and <a href="https://github.com/google/highwayhash/issues/29">https://github.com/google/highwayhash/issues/29</a>.</p>
<h2 id="Warn-on-metasploit-CVE-2015-1592">Warn on metasploit CVE-2015-1592</h2>
<p>Detection of the destructive attack against Movable-Type, the third vector only, which tries to delete <i>mt-config.cgi</i> was added to was added to <a href="/cperl/lib/Storable.html">Storable</a> 3.01c.</p>
<p>Calls <code>warn_security("Movable-Type CVE-2015-1592 Storable metasploit attack")</code>, but does not protect against it.</p>
<h2 id="Warn-on-metasploit-reverse-shells">Warn on metasploit reverse shells</h2>
<p>Detect the metasploit payload unix/reverse_perl and some existing variants. This is just a dumb match at startup against existing exploits in the wild, but not future variants. Calls <code>warn_security("metasploit reverse/bind shell payload")</code>, but do not protect against it. This warning is thrown even without <code>-w</code>.</p>
<p>Also detects the CVE-2012-1823 reverse/bind shell payload, which is widely exploited too. The security warning is called "CVE-2012-1823 reverse/bind shell payload".</p>
<h2 id="syscalls-warnings-also-security">syscalls warnings also security</h2>
<p>With a warnings 'syscalls' violation, i.e. detecting <code>\0</code> in arguments to C API syscalls, the new 'security' warnings category overrides the 'syscalls' category. I.e. the warning is produced by the <a href="/cperl/perlapi.html#warn_security">"warn_security" in perlapi</a> API, and to turn it off, you have to turn off both categories.</p>
<h1 id="Deprecations">Deprecations</h1>
<p>See the new <a href="/cperl/perldeprecation.html">perldeprecation</a> pod.</p>
<p>Many old deprecations got now a fixed final date, but several perl5 deprecations were undeprecated in cperl and rather fixed. <i>(as in previous cperl releases.)</i></p>
<h2 id="do_open-do_close-macros">do_open, do_close macros</h2>
<p>Those macros clash on darwin XTools with the system iostream <code>_OutputIterator</code> methods. We need to use the fullname <b>Perl_do_open</b> and <b>Perl_do_close</b> functions whenever perl needs to be embedded into C++ projects.</p>
<p>With the system C++ compiler on darwin <code>do_open</code>, <code>do_close</code> are now undefined. See <a href="https://github.com/perl11/cperl/issues/227">[cperl #227]</a></p>
<h2 id="Removed-as-package-seperator">Removed ' as package seperator</h2>
<p>Made something like <code>sub foo'bar;</code> a syntax error. <code>'</code> is not replaced by <code>::</code> anymore when used as package seperator. This was deprecated 10 years ago.</p>
<p>cperl fixed the <a href="#c2ph">"c2ph"</a> core utility using this last remaining perl4'ism, and removed the <code>isn't</code> method from <a href="/cperl/lib/Test/More.html">Test::More</a>. In a later versions <code>'</code> can be reenabled as proper IDContinue character for identifiers, e.g. for Test::More <code>isn't</code>.</p>
<p>See <a href="https://github.com/perl11/cperl/issues/217">[cperl #217]</a>.</p>
<h2 id="New-items-in-perldeprecation">New items in <a href="/cperl/perldeprecation.html">perldeprecation</a></h2>
<ul>
<li><p>Attribute "locked" is deprecated, and will disappear in Perl 5.28</p>
</li>
<li><p>Attribute "unique" is deprecated, and will disappear in Perl 5.28</p>
</li>
<li><p>Constants from lexical variables potentially modified elsewhere are deprecated. This will not be allowed in Perl 5.32</p>
</li>
<li><p>Deprecated use of my() in false conditional. This will be a fatal error in Perl 5.30</p>
</li>
<li><p>File::Glob::glob() will disappear in perl 5.30. Use File::Glob::bsd_glob() instead.</p>
</li>
<li><p>%s() is deprecated on :utf8 handles. This will be a fatal error in Perl 5.30</p>
</li>
<li><p>$* is no longer supported. Its use will be fatal in Perl 5.30</p>
</li>
<li><p>$* is no longer supported. Its use will be fatal in Perl 5.30</p>
</li>
<li><p>Opening dirhandle %s also as a file. This will be a fatal error in Perl 5.28</p>
</li>
<li><p>Opening filehandle %s also as a directory. This will be a fatal error in Perl 5.28</p>
</li>
<li><p>Setting $/ to a reference to %s as a form of slurp is deprecated, treating as undef. This will be fatal in Perl 5.28</p>
</li>
<li><p>Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <span style="white-space: nowrap;"><-- HERE</span> in m/%s/</p>
</li>
<li><p>Unknown charname '' is deprecated. Its use will be fatal in Perl 5.28</p>
</li>
<li><p>Use of bare << to mean <<"" is deprecated. Its use will be fatal in Perl 5.28</p>
</li>
<li><p>Use of code point 0x%s is deprecated; the permissible max is 0x%s. This will be fatal in Perl 5.28</p>
</li>
<li><p>Use of comma-less variable list is deprecated. Its use will be fatal in Perl 5.28</p>
</li>
<li><p>Use of inherited AUTOLOAD for non-method %s() is deprecated. This will be fatal in Perl 5.28</p>
</li>
<li><p>Use of strings with code points over 0xFF as arguments to %s operator is deprecated. This will be a fatal error in Perl 5.28</p>
</li>
</ul>
<h1 id="Incompatible-Changes">Incompatible Changes</h1>
<h2 id="String-delimiters-that-arent-stand-alone-graphemes-are-illegal">String delimiters that aren't stand-alone graphemes are illegal</h2>
<p>In order for Perl to eventually allow string delimiters to be Unicode grapheme clusters (which look like a single character, but may be a sequence of several ones), we stop allowing a single char delimiter that isn't a grapheme by itself. These are unlikely to exist in actual code, as they would typically display as attached to the character in front of them.</p>
<p>E.g. <code>qr ̂foobar̂;</code> is now an error, it is only deprecated with v5.25.9 upstream and will be illegal in perl5 v5.30. cperl only.</p>
<h2 id="for-state-loops-still-illegal">for state loops still illegal</h2>
<p>perl5.25.3 started allowing state variables in loops. cperl still disallows them.</p>
<pre><code> perl5.25.3 -E'use feature "declared_refs","refaliasing";
for state \$x (\$y) { print $x }'
=> warnings: Declaring references is experimental at -e line 1.
Aliasing via reference is experimental at -e line 1.
cperl5.25.3 -E'use feature "declared_refs","refaliasing";
for state \$x (\$y) { print $x }'
=> error: Missing $ on loop variable at -e line 1.</code></pre>
<p>and without declared_refs:</p>
<pre><code> perl5.25.3 -E'for state $x ($y) { print $x }'
cperl5.25.3 -E'for state $x ($y) { print $x }'
=> error: Missing $ on loop variable at -e line 1.</code></pre>
<h2 id="scalar-hash-return-value-changed"><code>scalar(%hash)</code> return value changed</h2>
<p>The value returned for <code>scalar(%hash)</code> will no longer show information about the buckets allocated in the hash. It will simply return the count of used keys. It is thus equivalent to <code>0+keys(%hash)</code>.</p>
<p>A form of backwards compatibility is provided via <code>Hash::Util::bucket_ratio()</code> which provides the same behavior as <code>scalar(%hash)</code> provided prior to Perl 5.25.</p>
<h2 id="keys-returned-from-an-lvalue-subroutine"><code>keys</code> returned from an lvalue subroutine</h2>
<p><code>keys</code> returned from an lvalue subroutine can no longer be assigned to in list context.</p>
<pre><code> sub foo : lvalue { keys(%INC) }
(foo) = 3; # death
sub bar : lvalue { keys(@_) }
(bar) = 3; # also an error</code></pre>
<p>This makes the lvalue sub case consistent with <code>(keys %hash) = ...</code> and <code>(keys @_) = ...</code>, which are also errors. <a href="https://rt.perl.org/Public/Bug/Display.html?id=128187">[perl #128187]</a></p>
<h1 id="Performance-Enhancements">Performance Enhancements</h1>
<dl>
<dt id="Faster-scalar-assignments">Faster scalar assignments</dt>
<dd>
<p>Seperate an unlikely codepath in scalar assignments (ASSIGN_CV_TO_GV) to another function, helping the CPU instruction cache. 10% faster on Intel.</p>
</dd>
<dt id="Bigger-lexer-buffers-allowing-faster-memcmp">Bigger lexer buffers allowing faster memcmp</dt>
<dd>
<p>Ensure that the lexer always sees large enough buffers to do fast wordwise memcmp comparisons, esp. with constant lengths.</p>
</dd>
<dt id="Less-initial-array-elements">Less initial array elements</dt>
<dd>
<p>The initial size of empty arrays went in cperl from 4 to 2, AvMAX = 1. Array speed is 2-15% faster on perlbench, overall speed the fastest of all so far. Memory win: <0.1%</p>
</dd>
<dt id="if-length-str-is-faster">if length($str) is faster</dt>
<dd>
<p>length in boolean context without get magic doesn't need to calculate the utf8 length, it only needs to check if SvCUR field is empty. And it doesn't need to allocate a new IV for the result, just use the existing sv_yes or sv_no. Analog to <a href="https://github.com/perl11/cperl/issues/245">[cperl #245]</a> for ref. cperl only.</p>
</dd>
<dt id="if-ref-is-faster">if ref() is faster</dt>
<dd>
<p>ref in boolean context doesn't need to allocate a string. 2-3x faster. See <a href="https://github.com/perl11/cperl/issues/245">[cperl #245]</a> and <a href="https://rt.perl.org/Public/Bug/Display.html?id=78288">[perl #78288]</a> cperl only. in perl5 announced for 5.28.</p>
</dd>
<dt id="readline-is-faster">readline is faster</dt>
<dd>
<p>Reading from a file line-by-line with <code>readline()</code> or <code><></code> should now typically be faster due to a better implementation of the code that searches for the next newline character.</p>
</dd>
<dt id="ref1-ref2-has-been-optimized"><code>$ref1 = $ref2</code> has been optimized.</dt>
<dd>
</dd>
<dt id="Array-and-hash-assignment-are-faster">Array and hash assignment are faster</dt>
<dd>
<p>e.g.</p>
<pre><code> (..., @a) = (...);
(..., %h) = (...);</code></pre>
<p>especially when the RHS is empty.</p>
<p>Note that perl5 hash assignment is still inferior to cperl hash assignment.</p>
</dd>
<dt id="Less-SvSCREAM">Less SvSCREAM</dt>
<dd>
<p>Reduce the number of odd special cases for the SvSCREAM flag.</p>
</dd>
<dt id="Better-do_vop">Better do_vop</dt>
<dd>
<p>Avoid <code>sv_catpvn()</code> in <code>do_vop()</code> when unneeded.</p>
</dd>
<dt id="Better-COW-in-Regex">Better COW in Regex</dt>
<dd>
<p>Enhancements in Regex concat COW implementation.</p>
</dd>
<dt id="Speed-up-AV-and-HV-clearing-undeffing">Speed up AV and HV clearing/undeffing.</dt>
<dd>
</dd>
<dt id="Converting-a-single-digit-string-to-a-number-is-now-substantially-faster">Converting a single-digit string to a number is now substantially faster.</dt>
<dd>
</dd>
<dt id="Simplified-split">Simplified split</dt>
<dd>
<p>The internal op implementing the <code>split</code> builtin has been simplified and sped up. Firstly, it no longer requires a subsidiary internal <code>pushre</code> op to do its work. Secondly, code of the form <code>my @x = split(...)</code> is now optimised in the same way as <code>@x = split(...)</code>, and is therefore a few percent faster. This required B::* compiler changes.</p>
</dd>
<dt id="Constant-fold-with-barewords">Constant fold with barewords</dt>
<dd>
<p>Bareword constant strings are now permitted to take part in constant folding. They were originally exempted from constant folding in August 1999, during the development of Perl 5.6, to ensure that <code>use strict "subs"</code> would still apply to bareword constants. That has now been accomplished a different way, so barewords, like other constants, now gain the performance benefits of constant folding.</p>
<p>This also means that void-context warnings on constant expressions of barewords now report the folded constant operand, rather than the operation; this matches the behaviour for non-bareword constants.</p>
</dd>
<dt id="Less-NULL-ops">Less NULL ops</dt>
<dd>
<p>Most NULL ops are now removed in the peephole optimizer. Check for <code>#if defined(PERL_REMOVE_OP_NULL)</code> in your XS module if you hardcoded any NULL-sensitive op-tree structure. See how many with <code>-Dk</code>.</p>
</dd>
<dt id="DPERL_FAKE_SIGNATURE">-DPERL_FAKE_SIGNATURE</dt>
<dd>
<p><code>-DPERL_FAKE_SIGNATURE</code> is now default, making most function calls 2x faster. See <a href="#fake_signatures">"fake_signatures"</a></p>
</dd>
<dt id="flto-support">-flto support</dt>
<dd>
<p>The new compiler option support allows generation of much faster code. I.e. clang-4.0 with -flto or zapcc produce ~20% faster code.</p>
</dd>
<dt id="for-loops">for loops</dt>
<dd>
<p>for loops got several enhancements:</p>
<p>new special <b>iter_ary</b> <code>for (@ary)</code> and <b>iter_lazyiv</b> <code>for (0..9)</code> ops to avoid a run-time switch in the generic iter op.</p>
<p>more aelem_u optimizations, less run-time out of bounds checks for shaped arrays in loops. E.g. in <code>my @a[5]; $a[$_] for (0..4);</code> the compilers knows that the max index for <code>@a</code> will be <code>4</code>, which is within the allowed shape of <code>@a</code>.</p>
</dd>
<dt id="omit-bounds-checks-for-multideref">omit bounds checks for multideref</dt>
<dd>
<p>The <code>multideref</code> OP has a new <code>MDEREF_INDEX_uoob</code> flag. This is used for unchecked out-of-bounds checks for arrays, to use the previous AvSHAPED array optimizations (aelem_u, aelemfast_lex_u) or loop out-of-bounds elimination with multideref OPs also. Such multideref ops appear pretty often even with single indices. E.g. in <code>my @b=(0..4); for (0..$#b) { $b[$_] = 0; }</code> <code>$b[$_]</code> is converted to a multideref, which previously was not optimized.</p>
<p>Those optimized indices are marked with a new " _u" suffix in the dumped multideref stringification.</p>
<p><code>MDEREF_MASK</code> changed to 0x10F, the <code>MDEREF_SHIFT</code> size from 7 to 8. The shift can also use faster intrinsics now.</p>
<p>The loop out-of-bounds elimination was fixed for simple lexical indices (e.g. <code>for my $i (0..$#a){ $a[$i] }</code>, which leads now to more aelem_u ops and subsequent mderef_u optimizations also.</p>
</dd>
<dt id="strEQc-strNEc">strEQc, strNEc</dt>
<dd>
<p>The new <code>strEQc</code>/<code>strNEc</code> macros are used instead of <code>strEQ(s,"constant")</code>. This enables word-wise comparison via memcpy, in opposite of byte-wise comparisons via strcmp with already known sizes. This is a 10% performance improvement under most optimization levels.</p>
<p>Use more <code>strEQc</code>, <code>strNEc</code> macros, when safe to use, i.e. the left buffer is big enough, now with Address Sanitizer fallbacks.</p>
<p>The new fast buffer comparison macros <code>strEQc</code> and <code>strNEc</code> compare a full string including the final <code>\0</code>, <code>memEQc</code> and <code>memNEc</code> just the start of a buffer, with constants strings. Note that valgrind and Address Sanitizer will complain about out of range access of the left side of the buffer. To access these buffers however is safe and will not lead to SIGBUS on stricter platforms. To prevent valgrind from warning on this, you may want to define <code>-DVALGRIND</code>, which uses a safe and slower fallback macro.</p>
</dd>
<dt id="padnames">padnames</dt>
<dd>
<p>Make all padnames not UTF8 per default, only the ones which are really UTF8. See <a href="#Internal-Changes">"Internal Changes"</a> and <a href="https://github.com/perl11/cperl/issues/208">[cperl #208]</a></p>
</dd>
<dt id="av_fetch">av_fetch</dt>
<dd>
<p>Improvements when reading from arrays have been imported from perl5. <code>av_fetch()</code> uses less branches reading from the end (negative indices), and a branch checking for freed <code>@_</code> elements has been removed,</p>
</dd>
<dt id="hv_common_magical">hv_common_magical</dt>
<dd>
<p>Extract <code>hv_common_magical()</code> to a seperate function. Extracts uncommon magical code in hot code to an extra static function to help keep the icache smaller. Only in rare cases this branch is taken. I.e filling ENV at startup, or using tied hashes.</p>
<p>Measured 2-15% faster with normal scripts, not using tied hashes.</p>
</dd>
<dt id="pre-allocated-hash-sizes-with-aassign">pre-allocated hash sizes with aassign</dt>
<dd>
<p>aassign: pre-allocate needed hash size with aassign, similar to arrays, avoiding run-time hash splits. e.g. <code>my %h = (.. =</code> .., .. => ..)></p>
<p>This version is 30% faster overall in the <a>Mail::SpamAssassin</a> testsuite than cperl-5.25.0.</p>
</dd>
<dt id="pre-allocate-more-hashes-and-stashes">pre-allocate more hashes and stashes</dt>
<dd>
<p>Pre-extend internal hashes and stashes to avoid unnecessary boot-time hash splits. <code>%warnings::</code>, <code>%Config::</code>, <code>%utf8::</code>, <code>%version::</code>.</p>
</dd>
<dt id="Faster-get_-sah-vs-API">Faster get_[sah]vs API</dt>
<dd>
<p>Added new <code>get_svs</code>, <code>get_avs</code>, <code>get_hvs</code> macros, and accompanied <code>get_[ash]vn_flags</code> API functions, to omit the run-time <code>strlen(name)</code> for constant names. (#191)</p>
</dd>
</dl>
<h1 id="Modules-and-Pragmata">Modules and Pragmata</h1>
<h2 id="New-Modules-and-Pragmata">New Modules and Pragmata</h2>
<dl>
<dt id="types-0.01">types 0.01</dt>
<dd>
<p>Controls the type-checker. See <a href="/cperl/lib/types.html">types</a> or <a href="/cperl/perltypes.html">perltypes</a>.</p>
</dd>
</dl>
<h2 id="Updated-Modules-and-Pragmata">Updated Modules and Pragmata</h2>
<dl>
<dt id="Archive-Tar-2.24">Archive-Tar 2.24</dt>
<dd>
<p>Better 09_roundtrip.t tests.</p>
<p>Handle tarballs compressed with pbzip2 (RT #119262)</p>
<p>Add missing strict/warnings pragma to Constants.pm</p>
<p>Check for gzip/bzip2 before round tripping gz/bz2 files in tests</p>
</dd>
<dt id="B-1.68_06">B 1.68_06</dt>
<dd>
<p>Use op_class API</p>
<p>Allow a 2nd optional CV argument for B::OP::aux_list, fixing B::Deparse and thereby Data::Dumper and Test2 is_deeply.</p>
<p>use the new get_svs, get_avs, get_hvs macros.</p>
</dd>
<dt id="B-C-1.55_02">B-C 1.55_02</dt>
<dd>
<p>Fixes for PERL_OP_PARENT: moresib, sibling, parent.</p>
<p>Fix hints/522_patched.pl dependency on C.so <a href="https://rt.cpan.org/Ticket/Display.html?id=120161">[cpan #120161]</a></p>
<p>PUSHRE replaced by SPLIT, no xpad_cop_seq, SVpbm_VALID</p>
<p>Improved dl_module_to_sofile without 2nd arg</p>
<p>Fixed IsCOW savepvn, store the last cowrefcnt.</p>
<p>Fixed wrong savepvn length, failing with asan.</p>
<p>Optimized mro_isa_changed_in initialization.</p>
<p>Better CopFILE_set, Fixup arenasize refcnt. Delay cvref to init2, properly set a SvRV to a XS sub. Optimize constpv for CvFILE (less constants to merge for gcc). Improve NV precision by one digit. Fix to compile in utf8_heavy.pl, abstract and set %INC. Fix generation of @B::C::Config::deps on Windows. Fix !C99 precedence bug (e.g. MSVC). Minor refactor to simplify save_hek. Use the new get_svs, get_avs, get_hvs macros. perlcc add --debug|-D Improve endav XSUB bump Abstract RITER_T and HVMAX_T for the various sizes, compat HEK_STATIC Defer REGCOMP for \P{} properties Change $sv->EXTFLAGS to compflags since 5.22 for CALLREGCOMP(). Turn off MGf_REFCOUNTED. global-buffer-overflow with dynamic COW strings, wrong savepvn args.</p>
</dd>
<dt id="B-Debug-1.24">B-Debug 1.24</dt>
<dd>
<p>Support 5.25.6 split optimization</p>
</dd>
<dt id="bignum-0.47c">bignum 0.47c</dt>
<dd>
<p>See <a href="https://github.com/rurban/bignum/commits/cperl">https://github.com/rurban/bignum/commits/cperl</a></p>
</dd>
<dt id="B::Terse-1.07">B::Terse 1.07</dt>
<dd>
<p>Update deprecation message</p>
</dd>
<dt id="Carp-1.42c">Carp 1.42c</dt>
<dd>
<p>Handle chunk errors phrases</p>
</dd>
<dt id="Config::Perl::V-0.27_01">Config::Perl::V 0.27_01</dt>
<dd>
</dd>
<dt id="Compress-Raw-Bzip2-2.074">Compress-Raw-Bzip2 2.074</dt>
<dd>
<p>Need Fix for Makefile.PL depending on . in @INC <a href="https://rt.cpan.org/Ticket/Display.html?id=120084">RT #120084</a></p>
</dd>
<dt id="Compress-Raw-Zlib-2.074">Compress-Raw-Zlib 2.074</dt>
<dd>
<p>Comment out unused variables & remove C++-ism. <a href="https://rt.cpan.org/Ticket/Display.html?id=120272">RT #120272</a></p>
</dd>
<dt id="CPAN::Meta-2.150010c">CPAN::Meta 2.150010c</dt>
<dd>
<p>And merge <i>cpan/Parse-CPAN-Meta</i> into it. <i>cpan/Parse-CPAN-Meta</i> is gone.</p>
<p>Parse-CPAN-Meta security: set $YAML::XS::DisableCode, $YAML::XS::DisableBlessed.</p>
<p>Add support for all known YAML and JSON modules: *::Syck, JSON::MaybeXS, Mojo::JSON. But JSON::Any is broken.</p>
<p>fixed UTF-8 issues, passes now all Test-CPAN-Meta tests.</p>
</dd>
<dt id="CPAN-2.17">CPAN 2.17</dt>
<dd>
<p>with full cperl support. reapply most of our patches. skip cperl builtin prereqs.</p>
<p>See <a href="https://github.com/andk/cpanpm/pull/109">https://github.com/andk/cpanpm/pull/109</a></p>
</dd>
<dt id="CPAN-Meta-Requirements">CPAN-Meta-Requirements</dt>
<dd>
<p>Moved from cpan to dist. <a href="https://github.com/perl11/cperl/issues/154">[cperl #154]</a>.</p>
</dd>
<dt id="Cpanel-JSON-XS-3.0231">Cpanel-JSON-XS 3.0231</dt>
<dd>
<p>- Fix need() overallocation (#84 Matthew Horsfall) and missing need() calls.</p>
<p>- Fix decode_prefix offset when the string was re-allocated. rather return the relative offset not the pointer to the old start.</p>
<p>- Fixes for g++-6, stricter -fpermissive and -Wc++11-compat.</p>
<p>- Added tests for ill-formed utf8 sequences from Encode.</p>
<p>- modfl() mingw 4.0 runtime bug [perl #125924]</p>
<p>- Tested with the comprehensive JSON decode spectests from http://seriot.ch/parsing_json.html. Not added to core. #72</p>
<p>- decode with BOM: UTF-8, UTF-16, or UTF-32.</p>
<p>- fixed detection of final \0 as illegal non-whitespace garbage. Fixes spectest 'n_number_then_00'. #72</p>
<p>- warn with unicode noncharacters as in core when not in relaxed mode. #74</p>
<p>- fail decode of non-unicode raw characters above U+10FFFF when not in relaxed mode.</p>
<p>- New stringify_infnan(3) infnan_mode.</p>
<p>- Fix inf/nan detection on HP-UX and others.</p>
<p>- Use faster strEQc macros.</p>
<p>- Prefer memEQ for systems without memcmp, to use bcmp there.</p>
<p>- Add more expect_false() to inf/nan branches.</p>
<p>- Fix av and hv length types: protect from security sensitive overflows, add HVMAX_T and RITER_T</p>
<p>- Add new "Hash key too large" error. perl5 silently truncates it, but we prefer errors.</p>
</dd>
<dt id="Config-6.22">Config 6.22</dt>
<dd>
<p>protect sv in END during global destruction, esp. with B::C. fixes for missing . in @INC (cperl or -Dfortify_inc).</p>
</dd>
<dt id="Cwd-4.65c">Cwd 4.65c</dt>
<dd>
<p>Fix -Wc++11-compat warnings</p>
</dd>
<dt id="Data-Dumper-2.163">Data-Dumper 2.163</dt>
<dd>
<p>Fix -Wc++11-compat warnings</p>