Merge pull request #3962 from oleibman/atsign

Excel Dynamic Arrays (Avoid Adding At-Signs to Formulas)

Author: oleibman

CHANGELOG.md

@@ -5,6 +5,32 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com)
 and this project adheres to [Semantic Versioning](https://semver.org).
 
+## TBD - 3.0.0
+
+### Dynamic Arrays
+
+- Support for Excel dynamic arrays is added. It is an opt-in feature, so our hope is that there will be no BC breaks, but it is a very large change. Full support is added for Xlsx. It is emulated as Ctrl-Shift-Enter arrays for Ods read and write and Excel2003 and Gnumeric read. Html/Pdf and Csv writers will populate cells on output if they are the result of array formulas. No support is added for Xls or Slk.
+
+### Added
+
+- Excel Dynamic Arrays. [Issue #3901](https://github.com/PHPOffice/PhpSpreadsheet/issues/3901) [Issue #3659](https://github.com/PHPOffice/PhpSpreadsheet/issues/3659) [Issue #1834](https://github.com/PHPOffice/PhpSpreadsheet/issues/1834) [PR #3962](https://github.com/PHPOffice/PhpSpreadsheet/pull/3962)
+
+### Changed
+
+- Nothing yet.
+
+### Deprecated
+
+- Nothing yet.
+
+### Moved
+
+- Nothing yet.
+
+### Fixed
+
+- Nothing yet.
+
 ## 2024-08-07 - 2.2.2
 
 ### Added

docs/references/features-cross-reference.md

@@ -365,15 +365,15 @@
         <td style="text-align: center; color: red;">✖</td>
     </tr>
     <tr>
-        <td style="padding-left: 1em;">Array</td>
-        <td style="text-align: center; color: red;">✖</td>
-        <td style="text-align: center; color: red;">✖</td>
-        <td style="text-align: center; color: red;">✖</td>
-        <td style="text-align: center; color: red;">✖</td>
-        <td style="text-align: center; color: red;">✖</td>
-        <td style="text-align: center; color: red;">✖</td>
+        <td style="padding-left: 1em;">Array Formula</td>
         <td style="text-align: center; color: red;">✖</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center;">N/A</td>
         <td style="text-align: center; color: red;">✖</td>
+        <td style="text-align: center;">N/A</td>
     </tr>
     <tr>
         <td style="padding-left: 1em;">Rich Text</td>
@@ -1005,6 +1005,7 @@
 5. <span id="footnote5">Xlsx macros can be read and written; their values can be retrieved and changed, but only in a binary form which is unlikely to be useful</span>
 6. <span id="footnote6">There is very limited support for reading styles from an Ods spreadsheet. Writing styles has better support, although Number Format is incomplete.</span>
 7. <span id="footnote7">In most cases, Html reader processes only inline styles; styles provided by Css classes may be ignored.</span>
+8. <span id="footnote8">Code must [opt in](../topics/recipes.md#array-formulas) to array output.</span>
 
 ## Writers
 
@@ -1175,6 +1176,15 @@
         <td style="text-align: center; color: red;">✖</td>
         <td style="text-align: center; color: red;">✖</td>
     </tr>
+    <tr>
+        <td style="padding-left: 0.5em;"><strong>Array Formula<a href="#footnote8"><sup>8</sup></a></strong></td>
+        <td style="text-align: center; color: red;">✖</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+        <td style="text-align: center; color: green;">✔</td>
+    </tr>
     <tr>
         <td style="padding-left: 0.5em;"><strong>Rows and Column Properties</strong></td>
         <td></td>

docs/references/function-list-by-category.md

@@ -529,7 +529,7 @@ CHAR                     | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Charac
 CLEAN                    | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Trim::nonPrintable
 CODE                     | \PhpOffice\PhpSpreadsheet\Calculation\TextData\CharacterConvert::code
 CONCAT                   | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::CONCATENATE
-CONCATENATE              | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::CONCATENATE
+CONCATENATE              | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::actualCONCATENATE
 DBCS                     | **Not yet Implemented**
 DOLLAR                   | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Format::DOLLAR
 EXACT                    | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Text::exact
@@ -586,5 +586,10 @@ WEBSERVICE               | \PhpOffice\PhpSpreadsheet\Calculation\Web\Service::we
 
 Excel Function           | PhpSpreadsheet Function
 -------------------------|--------------------------------------
-ANCHORARRAY              | **Not yet Implemented**
-SINGLE                   | **Not yet Implemented**
+
+## CATEGORY_MICROSOFT_INTERNAL
+
+Excel Function           | PhpSpreadsheet Function
+-------------------------|--------------------------------------
+ANCHORARRAY              | \PhpOffice\PhpSpreadsheet\Calculation\Internal\ExcelArrayPseudoFunctions::anchorArray
+SINGLE                   | \PhpOffice\PhpSpreadsheet\Calculation\Internal\ExcelArrayPseudoFunctions::single

docs/references/function-list-by-name.md

@@ -15,7 +15,7 @@ ADDRESS                  | CATEGORY_LOOKUP_AND_REFERENCE  | \PhpOffice\PhpSpread
 AGGREGATE                | CATEGORY_MATH_AND_TRIG         | **Not yet Implemented**
 AMORDEGRC                | CATEGORY_FINANCIAL             | \PhpOffice\PhpSpreadsheet\Calculation\Financial\Amortization::AMORDEGRC
 AMORLINC                 | CATEGORY_FINANCIAL             | \PhpOffice\PhpSpreadsheet\Calculation\Financial\Amortization::AMORLINC
-ANCHORARRAY              | CATEGORY_UNCATEGORISED         | **Not yet Implemented**
+ANCHORARRAY              | CATEGORY_MICROSOFT_INTERNAL    | \PhpOffice\PhpSpreadsheet\Calculation\Internal\ExcelArrayPseudoFunctions::anchorArray
 AND                      | CATEGORY_LOGICAL               | \PhpOffice\PhpSpreadsheet\Calculation\Logical\Operations::logicalAnd
 ARABIC                   | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpreadsheet\Calculation\MathTrig\Arabic::evaluate
 AREAS                    | CATEGORY_LOOKUP_AND_REFERENCE  | **Not yet Implemented**
@@ -89,7 +89,7 @@ COMBIN                   | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpread
 COMBINA                  | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpreadsheet\Calculation\MathTrig\Combinations::withRepetition
 COMPLEX                  | CATEGORY_ENGINEERING           | \PhpOffice\PhpSpreadsheet\Calculation\Engineering\Complex::COMPLEX
 CONCAT                   | CATEGORY_TEXT_AND_DATA         | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::CONCATENATE
-CONCATENATE              | CATEGORY_TEXT_AND_DATA         | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::CONCATENATE
+CONCATENATE              | CATEGORY_TEXT_AND_DATA         | \PhpOffice\PhpSpreadsheet\Calculation\TextData\Concatenate::actualCONCATENATE
 CONFIDENCE               | CATEGORY_STATISTICAL           | \PhpOffice\PhpSpreadsheet\Calculation\Statistical\Confidence::CONFIDENCE
 CONFIDENCE.NORM          | CATEGORY_STATISTICAL           | \PhpOffice\PhpSpreadsheet\Calculation\Statistical\Confidence::CONFIDENCE
 CONFIDENCE.T             | CATEGORY_STATISTICAL           | **Not yet Implemented**
@@ -510,7 +510,7 @@ SHEET                    | CATEGORY_INFORMATION           | **Not yet Implemente
 SHEETS                   | CATEGORY_INFORMATION           | **Not yet Implemented**
 SIGN                     | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpreadsheet\Calculation\MathTrig\Sign::evaluate
 SIN                      | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpreadsheet\Calculation\MathTrig\Trig\Sine::sin
-SINGLE                   | CATEGORY_UNCATEGORISED         | **Not yet Implemented**
+SINGLE                   | CATEGORY_MICROSOFT_INTERNAL    | \PhpOffice\PhpSpreadsheet\Calculation\Internal\ExcelArrayPseudoFunctions::single
 SINH                     | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpreadsheet\Calculation\MathTrig\Trig\Sine::sinh
 SKEW                     | CATEGORY_STATISTICAL           | \PhpOffice\PhpSpreadsheet\Calculation\Statistical\Deviations::skew
 SKEW.P                   | CATEGORY_STATISTICAL           | **Not yet Implemented**

docs/topics/calculation-engine.md

@@ -10,6 +10,8 @@ formula calculation capabilities. A cell can be of a value type
 which can be evaluated). For example, the formula `=SUM(A1:A10)`
 evaluates to the sum of values in A1, A2, ..., A10.
 
+Calling `getValue()` on a cell that contains a formula will return the formula itself.
+
 To calculate a formula, you can call the cell containing the formula’s
 method `getCalculatedValue()`, for example:
 
@@ -22,7 +24,18 @@ with PhpSpreadsheet, it evaluates to the value "64":
 
 ![09-command-line-calculation.png](./images/09-command-line-calculation.png)
 
-When writing a formula to a cell, formulae should always be set as they would appear in an English version of Microsoft Office Excel, and PhpSpreadsheet handles all formulae internally in this format. This means that the following rules hold:
+Calling `getCalculatedValue()` on a cell that doesn't contain a formula will simply return the value of that cell; but if the cell does contain a formula, then PhpSpreadsheet will evaluate that formula to calculate the result.
+
+There are a few useful mehods to help identify whether a cell contains a formula or a simple value; and if a formula, to provide further information about it:
+
+```php
+$spreadsheet->getActiveSheet()->getCell('E11')->isFormula();
+```
+will return a boolean true/false, telling you whether that cell contains a formula or not, so you can determine if a call to `getCalculatedVaue()` will need to perform an evaluation. 
+
+For more details on working with array formulas, see the [the recipes documentationn](./recipes.md/#array-formulas). 
+
+When writing a formula to a cell, formulas should always be set as they would appear in an English version of Microsoft Office Excel, and PhpSpreadsheet handles all formulas internally in this format. This means that the following rules hold:
 
  - Decimal separator is `.` (period)
  - Function argument separator is `,` (comma)
@@ -91,6 +104,11 @@ formula calculation is subject to PHP's language characteristics.
 Not all functions are supported, for a comprehensive list, read the
 [function list by name](../references/function-list-by-name.md).
 
+#### Array arguments for Function Calls in Formulas
+
+While most of the Excel function implementations now support array arguments, there are a few that should accept arrays as arguments but don't do so.
+In these cases, the result may be a single value rather than an array; or it may be a `#VALUE!` error.
+
 #### Operator precedence
 
 In Excel `+` wins over `&`, just like `*` wins over `+` in ordinary
@@ -161,7 +179,7 @@ number of seconds from the PHP/Unix base date. The PHP/Unix base date
 (0) is 00:00 UST on 1st January 1970. This value can be positive or
 negative: so a value of -3600 would be 23:00 hrs on 31st December 1969;
 while a value of +3600 would be 01:00 hrs on 1st January 1970. This
-gives PHP a date range of between 14th December 1901 and 19th January
+gives 32-bit PHP a date range of between 14th December 1901 and 19th January
 2038.
 
 #### PHP `DateTime` Objects

docs/topics/images/12-CalculationEngine-Array-Formula-2.png

@@ -168,6 +168,87 @@ Once you have created a reader object for the workbook that you want to
 load, you have the opportunity to set additional options before
 executing the `load()` method.
 
+All of these options can be set by calling the appropriate methods against the Reader (as described below), but some options (those with only two possible values) can also be set through flags, either by calling the Reader's `setFlags()` method, or passing the flags as an argument in the call to `load()`.
+Those options that can be set through flags are:
+
+Option             | Flag                                | Default
+-------------------|-------------------------------------|------------------------
+Empty Cells        | IReader::IGNORE_EMPTY_CELLS         | Load empty cells
+Rows with no Cells | IReader::IGNORE_ROWS_WITH_NO_CELLS  | Load rows with no cells
+Data Only          | IReader::READ_DATA_ONLY             | Read data, structure and style
+Charts             | IReader::LOAD_WITH_CHARTS           | Don't read charts
+
+Several flags can be combined in a single call:
+```php
+$inputFileType = 'Xlsx';
+$inputFileName = './sampleData/example1.xlsx';
+
+/**  Create a new Reader of the type defined in $inputFileType  **/
+$reader = \PhpOffice\PhpSpreadsheet\IOFactory::createReader($inputFileType);
+/** Set additional flags before the call to load() */
+$reader->setFlags(IReader::IGNORE_EMPTY_CELLS | IReader::LOAD_WITH_CHARTS);
+$reader->load($inputFileName);
+```
+or
+```php
+$inputFileType = 'Xlsx';
+$inputFileName = './sampleData/example1.xlsx';
+
+/**  Create a new Reader of the type defined in $inputFileType  **/
+$reader = \PhpOffice\PhpSpreadsheet\IOFactory::createReader($inputFileType);
+/** Set additional flags in the call to load() */
+$reader->load($inputFileName, IReader::IGNORE_EMPTY_CELLS | IReader::LOAD_WITH_CHARTS);
+```
+
+### Ignoring Empty Cells
+
+Many Excel files have empty rows or columns at the end of a worksheet, which can't easily be seen when looking at the file in Excel (Try using Ctrl-End to see the last cell in a worksheet).
+By default, PhpSpreadsheet will load these cells, because they are valid Excel values; but you may find that an apparently small spreadsheet requires a lot of memory for all those empty cells.
+If you are running into memory issues with seemingly small files, you can tell PhpSpreadsheet not to load those empty cells using the `setReadEmptyCells()` method. 
+
+```php
+$inputFileType = 'Xls';
+$inputFileName = './sampleData/example1.xls';
+
+/**  Create a new Reader of the type defined in $inputFileType  **/
+$reader = \PhpOffice\PhpSpreadsheet\IOFactory::createReader($inputFileType);
+/**  Advise the Reader that we only want to load cell's that contain actual content  **/
+$reader->setReadEmptyCells(false);
+/**  Load $inputFileName to a Spreadsheet Object  **/
+$spreadsheet = $reader->load($inputFileName);
+```
+
+Note that cells containing formulae will still be loaded, even if that formula evaluates to a NULL or an empty string.
+Similarly, Conditional Styling might also hide the value of a cell; but cells that contain Conditional Styling or Data Validation will always be loaded regardless of their value.
+
+This option is available for the following formats:
+
+Reader    | Y/N |Reader  | Y/N |Reader        | Y/N |
+----------|:---:|--------|:---:|--------------|:---:|
+Xlsx      | YES | Xls    | YES | Xml          | NO  |
+Ods       | NO  | SYLK   | NO  | Gnumeric     | NO  |
+CSV       | NO  | HTML   | NO
+
+This option is also available through flags.
+
+### Ignoring Rows With No Cells
+
+Similar to the previous item, you can choose to ignore rows which contain no cells.
+This can also help with memory issues.
+```php
+$inputFileType = 'Xlsx';
+$inputFileName = './sampleData/example1.xlsx';
+
+/**  Create a new Reader of the type defined in $inputFileType  **/
+$reader = \PhpOffice\PhpSpreadsheet\IOFactory::createReader($inputFileType);
+/**  Advise the Reader that we do not want rows with no cells  **/
+$reader->setIgnoreRowsWithNoCells(true);
+/**  Load $inputFileName to a Spreadsheet Object  **/
+$spreadsheet = $reader->load($inputFileName);
+```
+
+This option is available only for Xlsx. It is also available through flags.
+
 ### Reading Only Data from a Spreadsheet File
 
 If you're only interested in the cell values in a workbook, but don't
@@ -210,6 +291,8 @@ Xlsx      | YES | Xls    | YES | Xml          | YES |
 Ods       | YES | SYLK   | NO  | Gnumeric     | YES |
 CSV       | NO  | HTML   | NO
 
+This option is also available through flags.
+
 ### Reading Only Named WorkSheets from a File
 
 If your workbook contains a number of worksheets, but you are only
@@ -642,7 +725,7 @@ Xlsx      | NO  | Xls    | NO  | Xml          | NO  |
 Ods       | NO  | SYLK   | NO  | Gnumeric     | NO  |
 CSV       | YES | HTML   | NO
 
-### A Brief Word about the Advanced Value Binder
+## A Brief Word about the Advanced Value Binder
 
 When loading data from a file that contains no formatting information,
 such as a CSV file, then data is read either as strings or numbers
@@ -694,6 +777,9 @@ Xlsx      | NO  | Xls    | NO  | Xml          | NO
 Ods       | NO  | SYLK   | NO  | Gnumeric     | NO
 CSV       | YES | HTML   | YES
 
+Note that you can also use the Binder to determine how PhpSpreadsheet identified datatypes for values when you set a cell value without explicitly setting a datatype.
+Value Binders can also be used to set formatting for a cell appropriate to the value.
+
 ## Error Handling
 
 Of course, you should always apply some error handling to your scripts