Changelog

odoo/o-spreadsheet#7756

Created by BI, Vincent Schippefilt (vsc)

Merged at bb4353b04b518393903766324b5a3269890f1ae8

Statuses:

ci/runbot: (runtime 296s)
legal/cla: (runtime 15s)

label: odoo:master-squish-vsc
head: a34c02c927d3e8f2d8380595457964e613cad6d7
merged: 2 months ago by BI, Lucas Lefèvre (lul)

	odoo/o-spreadsheet
master	#7756

[IMP] export: reduce JSON size

Description:

Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass,
collecting all the formulae or changes that are the same and storing them under one single key in the JSON file.

Task: 5489478

API changes:

compiler.ts/compile(formula: string) --> CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)
fomulas/helpers.ts/getFunctionsFromTokens(tokens: Tokens[], functionNames: string[]) --> fomulas/helpers.ts/getFunctionsFromTokens(compiledFormula: CompiledFormula, functionNames: string[])
removed RangeCompiledFormula --> CompiledFormula
model.ts/export() --> export(shouldSquish?: boolean = false)
cell.ts/getTranslatedFormulaCell(sheetId: UID, offsetX: number, offsetY: number, tokens: Token[]) --> getTranslatedFormulaCell(sheetId: UID,offsetX: number,offsetY: number,compiledFormula: CompiledFormula | SerializedCompiledFormula )
removed plugins/core/cell.ts/ReferenceToken class --> it is implemented in CompiledFormulaformulas/helper.ts/isExportableToExcel&getFunctionsFromTokens--> CompiledFormula.areAllFunctionsExportableToExcel&getFunctionsFromTokens`
removed FormulaCellWithDependencies and ReferenceToken

Observed performance gains on large dashboard:

importing sheets:
~2300ms --> ~680 ms
1560 MB ram --> 580 MB

JSON Size:
40 MB -> 147 KB
4,4 MB -> 3KB
1,6 MB-> 43 KB

How does it work:

Shared formula

```cells: {
A1: "My Text",
A2: "=SUM(B2:B9)",
A3: "=SUM(B2:B9)",

As we can see, A2 and A3 share the same formula. They can be rewritten as
```cells: {
   A1: &#34;My Text&#34;,
   &#39;A2:A3&#39;: &#34;=SUM(B2:B9)&#34;,

removing duplication completely

Formula with small differences

Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite:
```cells: {
A1: "My Text",
A2: "=CONCAT(B2, $C$2, D2, "hello"),
A3: "=CONCAT(B3, $C$2, D3, "hello"),
A4: "=CONCAT(B4, $C$2, D4, "hello"),

to 
```cells: {
   A1: &#34;My Text&#34;,
   A2: &#34;=CONCAT(B2, $C$2, D2, &#34;hello&#34;),
   A3: { R : &#34;+R1|=|+R1&#34; },
   A4: { R : &#34;+R1|=|+R1&#34; },
   ...

now we see that A3 and A4 have the same transformation, so we can rewrite them to
```cells: {
A1: "My Text",
A2: "=CONCAT(B2, $C$2, D2, "hello"),
"A3:A4": { R : "+R1|=|+R1" },

We apply the same for number arguments and string arguments. 
A formula with all the arguments slightly changed might look like
```cells: {
   A1: &#34;=MY_FORMULA(1, B1, &#34;coucou&#34;),
   A2: { R : &#34;+R1&#34;, N: &#34;+1&#34;, S: [&#34;hello&#34;] },
   A3:A4: { R : &#34;+R1&#34;, N: &#34;+1&#34;}

can be read as
cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: "=MY_FORMULA(2, B2, "hello"), A3: "=MY_FORMULA(3, B3, "hello"), A4: "=MY_FORMULA(4, B4, "hello"),

When reading the compressed JSON, the order of the keys in the JSON itself do not matter, they are sorted on the first part of the key, so either the cell reference or the left part of the range.