odoo/enterprise#28648

Created by Accounting, Olivier Colson (oco)

label

odoo-dev:master-reportalypse-oco

head

ef6c8e8b7054ac7057fd45d0a552556816e5d62f

merged

3 years ago by Accounting, Laurent Smet (las)

	odoo/odoo	odoo/enterprise	odoo/upgrade	odoo/documentation
master	#94125	#28648	#3695	#2557

[IMP] account_reports, l10n_*: Introduce unified reporting engine

[IMP] account_reports, l10n_*: Introduce unified reporting engine

(aka Reportalypse)
(aka "v16 reports yeaaaaaaaaaaaah")

This commit introduces the new reporting engine, largely refactoring the account.report model, and replacing what used to be managed by account.financial.html.report, account.tax.report/account.generic.tax.report, and account.accounting.report.

Multiple improvements are brought by this commit:

1) "One engine to rule them all"

All reports (including custom ones) are now direct instances of the account.report model, which becomes a stored model whose filter options are handled by regular fields. account.financial.html.report, account.tax.report and account.accounting report objects don't exist anymore and have been converted to the new engine. Custom (i.e. python-defined) reports still exist, but have been modified to fit the new approach. Note that thanks to this change, a manual reconfiguration of the reports by the user is now possible for every report (to add or remove a filter, change the load more limit, allow comparisons, ...).

To allow the replacement of the tax reports, account.report's model declaration has been moved from enterprise to community. Indeed, those reports need to stay there, as they are referenced by the taxes requiring the account tags they generate in their repartition. All the logic applied on top of account.report's fields is still enterprise.

2) Introduce account.report.expression

Each account.report.line can define a set of account.report.expressions. Expressions are used for various advanced purposes (see below), but they can roughly be seen as a sub-variable needed by the line. The most basic case is for example what was done before with financial report lines using other invisible lines to do subcomputations: we don't need to use multiple not-obviously-connected lines to do that. Instead the line needing those subcomputations simply defines multiple expressions.

Each expression is associated to a formula engine, a formula, and an optional subformula. The formula engine defines how the formula and subformula will be interpreted, leading to much more modularity and expressivity.

The following formula engines are available natively. Other engines could be added in the future, possibly even by specific modules:

• domain:
  This engines corresponds to what used to be handled by account.financial.html.report. Its formulas are domains on account.move.line.

• tax_tags:
  Corresponds to account.tax.report. An expression using this engine will generate a + and - tag named after its formula, just as the tag_name used to work in the previous paradigm.

• account_codes:
  Especially useful for P&L or Balance Sheet reports. This engine uses a formula consisting of account code prefixes. For example "11 + 42" will be evaluated as the sum of the balances of all move lines made on accounts whose code starts with 11 or 42. This is the most basic case ; more advanced expressions are also allowed (for example, to handle what used to be done by sum_if_pos_groupby/sum_if_neg_groupby in old financial reports).

• aggregation:
  Used to aggregate expressions. These expressions can use the "code" field of report lines to reference their value, and express a computation on them, pretty much like the "formula" field of account.tax.report and account.financial.html.report did.

• external
  Use to reference values defined by account.report.external.value records (for carryover and manual values ; see below).

• custom
  For this engine, the formula simply consists of a function to be called. This function needs to respect the specification of a formula engine function, and can then benefit from all the advantages linked to non-custom formulas (batching, automatic handling of the groupby, ...). This is a way to define engines that are only needed by 1 or 2 reports without making them available everywhere because they are too specific.

Note that this architecture means expressions using different engines can be mixed within the same report in any way that is necessary.

3) Custom reports

Fully defining a report in Python is still possible, but differently than before. Now, you don't have to generate a new AbstractModel inheriting from account.report. Instead, you just need a regular _inherit on account.report, and defining a few functions with specific signatures. The report then needs to be declared as an XML data, and must reference those functions using dedicated fields.

4) Performance improvement

Formulas engines are batched. Depending on the implementation of the engine, this can improve the performance. account_codes engine works this way, leading to tremendous improvement on most P&L and Balance Sheets.

5) Allow multiple columns on non-custom reports

Before, only Python-defined reports could have more than 1 column. This is not the case anymore, with the introduction of the new account.report.column model. Columns fetch their values using labels assigned on account.report.expression objects to identify them. To define different formulas for different columns, just use different expressions with the right label.

6) Allow manual values

In the past, the only way for a user to input values into a report without creating move lines was to use a financial report with the from_context formula, and rely on a wizard when opening the report. This was too restrictive, as the input value was always the same whatever the period (so, comparisons did not work well), and the value didn't persist and needed to be re-entered every time the report was opened.

To solve this, we introduce a new model called account.report.external.value. It essentially represents a value which can't be computed from account.move.line records directly. Manual values are handled by using an expression with engine 'external' and subformula 'editable' as the value of a column. A little pen will then be displayed in the corresponding cell, allowing to modify its value.

7) Carryover refactoring

The introduction of account.report.external.value also allowed refactoring the way the carryover works. We remove account.tax.carryover.line and instead use account.report.external.value. Also, instead of computing the carryover as the cumulated balance of all existing account.tax.carryover.line, we directly create a external value with a balance equal to the carryover amount. When getting the carryover coming from previous periods, we only consider the external values that is in the previous tax period.

8) Reduce menuitems overhead in l10n modules

A lot of localizations define localized versions of generic reports, typically for the Balance Sheet, P&L and tax report. In v15, the tax report already was handled with a single menuitem and a selector on the report allowed choosing which country we wanted to view the report of. However, all other reports still created specific menuitems. This made the menus more complex, and overall impacted the user experience negatively. This was especially problematic in multicompany environment, since the menuitems of all companies' countries were always shown, whatever the active company.

We now generalize what was done by the tax report: each report can point towards another report via the root_report_id field, hence becoming a variant of this report. Variants are available through a selector on the report, and the default selected one is the one that matches best the company country.

All reports which used a wizard to input manual values (such as in Spain) now make use of editable expressions. Also, the tax report lines which did not create any tag have also been converted so that they use manual values.

9) Misc improvements

• Cleaning: merge l10n_es_reports_2021 into l10n_es_reports
• Cleaning: merge l10n_nl_report_intrastat into l10n_nl_intrastat, and make l10n_nl_intrastat depend on account_reports
• Remove old Indian reports an introduce new more standard ones instead (more correct business-wise)
• Remove support for Mexican 13th month closing: this should either be done by impacting the current year earnings account directly or by using a specific journal, and a group to exclude it.
• Remove a few unused reports whose information was redundant with the one in other, more complete reports

task-2524389