Changelog

odoo/upgrade-util#284

Created by Victor Valet (vval)

Blocked

Merge method
Review
CI
- ci/runbot: Test upgrades between stable versions
- ci/upgrade_enterprise: Test upgrades for enterprise master

label: odoo-dev:master-batch-process-big-commands-vval
head: 508732d895fafe3db770b3bf02ec4550d7b2e424

	odoo/upgrade-util
master	#284 ^{missing r+}

[FIX] spreadsheet: batch process `spreadsheet_revision.commands`

[FIX] spreadsheet: batch process spreadsheet_revision.commands

Some dbs have spreadsheet_revision records with over 10 millions characters in commands. If the number of record is high, this leads to memory errors here. We distribute them in buckets of memory_cap maximum size, and use a named cursor to process them in buckets. Commands larger than memory_cap fit in one bucket.

select count(*),SUM(CHAR_LENGTH(commands)) as characters from spreadsheet_revision where commands like any (array[&#39;%&#34;DELETE_SHEET&#34;%&#39;, &#39;%&#34;MOVE_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;REMOVE_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;ADD_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;RENAME_SHEET&#34;%&#39;])
+-------+------------+
| count | characters |
|-------+------------|
| 2317  | 3419017646 |
+-------+------------+

select id,CHAR_LENGTH(commands) from spreadsheet_revision where commands like any (array[&#39;%&#34;DELETE_SHEET&#34;%&#39;, &#39;%&#34;MOVE_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;REMOVE_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;ADD_COLUMNS_ROWS&#34;%&#39;, &#39;%&#34;RENAME_SHEET&#34;%&#39;]) order by CHAR_LENGTH(commands) desc limit 10
+------+-------------+
| id   | char_length |
|------+-------------|
| 3871 | 13197157    |
| 4788 | 13197157    |
| 3290 | 13197157    |
| 3557 | 13197157    |
| 4179 | 13197157    |
| 4481 | 13197157    |
| 2757 | 13197157    |
| 3022 | 13197157    |
| 2492 | 13197157    |
| 5097 | 13197157    |
+------+-------------+

Fixes upg-2899961