odoo/upgrade#8987

Created by Upgrade, Carsten Wolff (cawo)

label

odoo-dev:master-imp_iter_browse_yield_chunks-cawo

head

70691f35f778592747e87a67dba8108b332175d3

merged

2 weeks ago by Upgrade, Alvaro Fuentes Suarez (afu)

	odoo/upgrade	odoo/upgrade-util
master	#8987	#362

[IMP] account_accountant/saas~18.3: performance

upg-3400299

This request times out afer 72h, because of this script:

2025-12-01 06:04:49,286 22562 INFO db_3400299 odoo.upgrade.util.orm: [38.04%] 62/163 account.reconcile.model processed in 1 day, 18:23:25.032493 (total estimated time: 4 days, 15:26:43.553167)

Because of a large number of records:

cawo_3400299=> SELECT am.company_id,
               count(DISTINCT absl.id) as st_line_ids,
               count(DISTINCT arm.id) as reco_models_ids
          FROM account_bank_statement_line absl
          JOIN account_move am ON absl.move_id = am.id
          JOIN account_reconcile_model arm ON arm.company_id = am.company_id
         WHERE absl.is_reconciled IS NOT TRUE
      GROUP BY am.company_id;
 company_id | st_line_ids | reco_models_ids
------------+-------------+-----------------
         15 |       67390 |             228
         16 |       16817 |             213
         17 |       18118 |             222
         18 |      734722 |             217
         19 |       19164 |             205
         20 |       23041 |             202
         21 |      117669 |             205
         22 |       34245 |             231
         23 |       19195 |             212
         24 |        9700 |             208
         25 |       28186 |             203
         26 |       23085 |             201
         27 |         255 |               2
         28 |        3262 |             138
         31 |           5 |             121
         72 |          17 |               2
        163 |         259 |               2

And the tight nested loops that calls into _apply_reconcile_models() millions of times.

Using the new yield_chunks option of iter_browse, this is reduced to a much smaller number of calls on recordsets, reducing the total runtime of the script to roughly 1h:

2025-12-01 15:23:02,240 44240 INFO cawo_3400299_19.0 odoo.modules.migration: module account_accountant: Running migration [$saas~18.3.1.1] end-migrate
2025-12-01 16:22:05,140 44240 INFO cawo_3400299_19.0 odoo.modules.migration: module hr_expense: Running migration [$saas~18.2.2.0] end-migrate

see odoo/upgrade-util#362

> Warning Please follow and check the relevant boxes for your PR, otherwise the review may take longer.

For a PR that adds a new upgrade script (for the Odoo's master branch or one of the RR versions):
- [ ] I've added the references to the Enterprise and/or Community PRs in the commit message, even if they are already merged.
- [ ] I've added an overall description of the changes made in the Community/Enterprise branches.

For a PR that fixes an issue on an already existing upgrade script:
- [x] I've added a clear description of the issue including information on how to reproduce it.
- [x] I've tested the fix and included references of upgrade requests where this fix will be applicable in the form of upg-<request_number>.
- [ ] I've included references to the OPW, if applicable, in the form of opw-<ticket_number>.
- [ ] I've tested this fix on other DBs, not just the ones with the issue.

For all PRs:
- [x] My commit's title follows the right format.
- [x] My commit's body includes information about the changes.
- [x] I've added odoo/upgrade as a reviewer.
- [x] I've read and understood this list.
- [x] I will ensure all the CI checks are green.

> Note More info can be found in the wiki.