[ADD] developer/reference: add a reference page on performance
closes odoo/documentation#2793
X-original-commit: 4d8e3f9785
Signed-off-by: Antoine Vandevenne (anv) <anv@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Co-authored-by: Antoine Vandevenne (anv) <anv@odoo.com>
@ -59,6 +59,8 @@ debug mode, add `?debug=0` instead.
|
||||
:ref:`assets mode <frontend/framework/assets_debug_mode>`, and `?debug=tests` enables
|
||||
the :ref:`tests mode <frontend/framework/tests_debug_mode>`.
|
||||
|
||||
.. _developer-mode/mode-tools:
|
||||
|
||||
Locate the mode tools
|
||||
=====================
|
||||
|
||||
|
@ -589,32 +589,6 @@ Programming in Odoo
|
||||
- As in python, use ``filtered``, ``mapped``, ``sorted``, ... methods to
|
||||
ease code reading and performance.
|
||||
|
||||
|
||||
Make your method work in batch
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
When adding a function, make sure it can process multiple records by iterating
|
||||
on self to treat each record.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def my_method(self)
|
||||
for record in self:
|
||||
record.do_cool_stuff()
|
||||
|
||||
For performance issue, when developing a 'stat button' (for instance), do not
|
||||
perform a ``search`` or a ``search_count`` in a loop. It
|
||||
is recommended to use ``read_group`` method, to compute all value in only one request.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def _compute_equipment_count(self):
|
||||
""" Count the number of equipment per category """
|
||||
equipment_data = self.env['hr.equipment'].read_group([('category_id', 'in', self.ids)], ['category_id'], ['category_id'])
|
||||
mapped_data = dict([(m['category_id'][0], m['category_id_count']) for m in equipment_data])
|
||||
for category in self:
|
||||
category.equipment_count = mapped_data.get(category.id, 0)
|
||||
|
||||
|
||||
Propagate the context
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
The context is a ``frozendict`` that cannot be modified. To call a method with
|
||||
|
@ -709,8 +709,9 @@ Database Population
|
||||
.. program:: odoo-bin populate
|
||||
|
||||
Odoo CLI supports database population features. If the feature is
|
||||
:ref:`implemented on a given model <reference/testing/populate/methods>`, it allows automatic data
|
||||
generation of the model's records to test your modules in databases containing non-trivial amounts of records.
|
||||
:ref:`implemented on a given model <reference/performance/populate/methods>`, it allows automatic
|
||||
data generation of the model's records to test your modules in databases containing non-trivial
|
||||
amounts of records.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -727,8 +728,7 @@ generation of the model's records to test your modules in databases containing n
|
||||
of a given model (cf. the :file:`populate` folder of modules for further details).
|
||||
|
||||
.. seealso::
|
||||
|
||||
:ref:`reference/testing/populate`
|
||||
:ref:`reference/performance/populate`
|
||||
|
||||
.. _reference/cmdline/cloc:
|
||||
|
||||
|
@ -12,7 +12,6 @@ Tutorials
|
||||
howtos/website
|
||||
howtos/backend
|
||||
howtos/web_services
|
||||
howtos/profilecode
|
||||
howtos/company
|
||||
howtos/accounting_localization
|
||||
howtos/translations
|
||||
|
@ -1,131 +0,0 @@
|
||||
===================
|
||||
Profiling Odoo code
|
||||
===================
|
||||
|
||||
.. warning::
|
||||
|
||||
This tutorial requires :ref:`having installed Odoo <setup/install>`
|
||||
and :doc:`writing Odoo code <backend>`
|
||||
|
||||
Graph a method
|
||||
==============
|
||||
|
||||
Odoo embeds a profiler of code. This embedded profiler output can be used to
|
||||
generate a graph of calls triggered by the method, number of queries, percentage
|
||||
of time taken in the method itself as well as the time that the method took and
|
||||
its sub-called methods.
|
||||
|
||||
.. code:: python
|
||||
|
||||
from odoo.tools.misc import profile
|
||||
[...]
|
||||
@profile('/temp/prof.profile')
|
||||
def mymethod(...)
|
||||
|
||||
This produces a file called /temp/prof.profile
|
||||
|
||||
A tool called *gprof2dot* will produce a graph with this result:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
gprof2dot -f pstats -o /temp/prof.xdot /temp/prof.profile
|
||||
|
||||
A tool called *xdot* will display the resulting graph:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
xdot /temp/prof.xdot
|
||||
|
||||
Log a method
|
||||
============
|
||||
|
||||
Another profiler can be used to log statistics on a method:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from odoo.tools.profiler import profile
|
||||
[...]
|
||||
@profile
|
||||
@api.model
|
||||
def mymethod(...):
|
||||
|
||||
The statistics will be displayed into the logs once the method to be analysed is
|
||||
completely reviewed.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
2018-03-28 06:18:23,196 22878 INFO openerp odoo.tools.profiler:
|
||||
calls queries ms
|
||||
project.task ------------------------ /home/odoo/src/odoo/addons/project/models/project.py, 638
|
||||
|
||||
1 0 0.02 @profile
|
||||
@api.model
|
||||
def create(self, vals):
|
||||
# context: no_log, because subtype already handle this
|
||||
1 0 0.01 context = dict(self.env.context, mail_create_nolog=True)
|
||||
|
||||
# for default stage
|
||||
1 0 0.01 if vals.get('project_id') and not context.get('default_project_id'):
|
||||
context['default_project_id'] = vals.get('project_id')
|
||||
# user_id change: update date_assign
|
||||
1 0 0.01 if vals.get('user_id'):
|
||||
vals['date_assign'] = fields.Datetime.now()
|
||||
# Stage change: Update date_end if folded stage
|
||||
1 0 0.0 if vals.get('stage_id'):
|
||||
vals.update(self.update_date_end(vals['stage_id']))
|
||||
1 108 631.8 task = super(Task, self.with_context(context)).create(vals)
|
||||
1 0 0.01 return task
|
||||
|
||||
Total:
|
||||
1 108 631.85
|
||||
|
||||
Dump stack
|
||||
==========
|
||||
|
||||
Sending the SIGQUIT signal to an Odoo process (only available on POSIX) makes
|
||||
this process output the current stack trace to log, with info level. When an
|
||||
odoo process seems stuck, sending this signal to the process permit to know
|
||||
what the process is doing, and letting the process continue his job.
|
||||
|
||||
Tracing code execution
|
||||
======================
|
||||
|
||||
Instead of sending the SIGQUIT signal to an Odoo process often enough, to check
|
||||
where the processes are performing worse than expected, we can use the `py-spy`_ tool to
|
||||
do it for us.
|
||||
|
||||
Install py-spy
|
||||
--------------
|
||||
|
||||
.. code:: bash
|
||||
|
||||
python3 -m pip install py-spy
|
||||
|
||||
Record executed code
|
||||
--------------------
|
||||
|
||||
As py-spy is installed, we now record the executed code lines.
|
||||
This tool will record, multiple times a second, the stacktrace of the process.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# record to raw file
|
||||
py-spy record -o profile.json -f speedscope --pid <PID>
|
||||
|
||||
# OR record directly to svg
|
||||
py-spy record -o profile.svg --pid <PID>
|
||||
|
||||
where <PID> is the process ID of the odoo process you want to graph.
|
||||
|
||||
To open profile.json you can use online tool `speedscope.app`_.
|
||||
|
||||
To open profile.svg you should use browser, because other viewer may not
|
||||
support interactive part.
|
||||
|
||||
|
||||
.. image:: profilecode/flamegraph.svg
|
||||
|
||||
|
||||
.. _py-spy: https://github.com/benfred/py-spy
|
||||
|
||||
.. _speedscope.app: https://www.speedscope.app/
|
Before Width: | Height: | Size: 110 KiB |
@ -14,6 +14,7 @@ Python framework
|
||||
backend/module
|
||||
backend/reports
|
||||
backend/security
|
||||
backend/performance
|
||||
backend/testing
|
||||
backend/http
|
||||
backend/mixins
|
||||
|
540
content/developer/reference/backend/performance.rst
Normal file
@ -0,0 +1,540 @@
|
||||
:custom-css: performance.css
|
||||
|
||||
===========
|
||||
Performance
|
||||
===========
|
||||
|
||||
.. _performance/profiling:
|
||||
|
||||
Profiling
|
||||
=========
|
||||
|
||||
.. currentmodule:: odoo.tools.profiler
|
||||
|
||||
Profiling is about analysing the execution of a program and measure aggregated data. These data can
|
||||
be the elapsed time for each function, the executed SQL queries...
|
||||
|
||||
While profiling does not improve the performance of a program by itself, it can prove very helpful
|
||||
in finding performance issues and identifying which part of the program is responsible for them.
|
||||
|
||||
Odoo provides an integrated profiling tool that allows recording all executed queries and stack
|
||||
traces during execution. It can be used to profile either a set of requests of a user session, or a
|
||||
specific portion of code. Profiling results can be either inspected with the integrated `speedscope
|
||||
<https://github.com/jlfwong/speedscope>`_ :dfn:`open source app allowing to visualize a flamegraph`
|
||||
view or analyzed with custom tools by first saving them in a JSON file or in the database.
|
||||
|
||||
.. _performance/profiling/enable:
|
||||
|
||||
Enable the profiler
|
||||
-------------------
|
||||
|
||||
The profiler can either be enabled from the user interface, which is the easiest way to do so but
|
||||
allows profiling only web requests, or from Python code, which allows profiling any piece of code
|
||||
including tests.
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. tab:: Enable from the user interface
|
||||
|
||||
#. :ref:`Enable the developer mode <developer-mode>`.
|
||||
#. Before starting a profiling session, the profiler must be enabled globally on the database.
|
||||
This can be done in two ways:
|
||||
|
||||
- Open the :ref:`developer mode tools <developer-mode/mode-tools>`, then toggle the
|
||||
:guilabel:`Enable profiling` button. A wizard suggests a set of expiry times for the
|
||||
profiling. Click on :guilabel:`ENABLE PROFILING` to enable the profiler globally.
|
||||
|
||||
.. image:: performance/enable_profiling_wizard.png
|
||||
|
||||
- Go to :guilabel:`Settings --> General Settings --> Performance` and set the desired time to
|
||||
the field :guilabel:`Enable profiling until`.
|
||||
|
||||
#. After the profiler is enabled on the database, users can enable it on their session. To do
|
||||
so, toggle the :guilabel:`Enable profiling` button in the :ref:`developer mode tools
|
||||
<developer-mode/mode-tools>` again. By default, the recommended options :guilabel:`Record
|
||||
sql` and :guilabel:`Record traces` are enabled. To learn more about the different options,
|
||||
head over to :ref:`performance/profiling/collectors`.
|
||||
|
||||
.. image:: performance/profiling_debug_menu.png
|
||||
|
||||
When the profiler is enabled, all the requests made to the server are profiled and saved into
|
||||
an `ir.profile` record. Such records are grouped into the current profiling session which
|
||||
spans from when the profiler was enabled until it is disabled.
|
||||
|
||||
.. note::
|
||||
Odoo Online (SaaS) databases cannot be profiled.
|
||||
|
||||
.. tab:: Enable from Python code
|
||||
|
||||
Starting the profiler manually can be convenient to profile a specific method or a part of the
|
||||
code. This code can be a test, a compute method, the entire loading, etc.
|
||||
|
||||
To start the profiler from Python code, call it as a context manager. You may specify *what*
|
||||
you want to record through the parameters. A shortcut is available for profiling test classes:
|
||||
:code:`self.profile()`. See :ref:`performance/profiling/collectors` for more information on
|
||||
the `collectors` parameter.
|
||||
|
||||
.. example::
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with Profiler():
|
||||
do_stuff()
|
||||
|
||||
.. example::
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with Profiler(collectors=['sql', PeriodicCollector(interval=0.1)]):
|
||||
do_stuff()
|
||||
|
||||
.. example::
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with self.profile():
|
||||
with self.assertQueryCount(__system__=1211):
|
||||
do_stuff()
|
||||
|
||||
.. note::
|
||||
The profiler is called outside of the `assertQueryCount` in order to catch queries made
|
||||
when exiting the context manager (e.g., flush).
|
||||
|
||||
.. autoclass:: Profiler()
|
||||
:special-members: __init__
|
||||
|
||||
When the profiler is enabled, all executions of a test method are profiled and saved into an
|
||||
`ir.profile` record. Such records are grouped into a single profiling session. This is
|
||||
especially useful when using the :code:`@warmup` and :code:`@users` decorators.
|
||||
|
||||
.. tip::
|
||||
It can be complicated to analyze profiling results of a method that is called several times
|
||||
because all the calls are grouped together in the stack trace. Add an **execution context**
|
||||
as a context manager to break down the results into multiple frames.
|
||||
|
||||
.. example::
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
for index in range(max_index):
|
||||
with ExecutionContext(current_index=index): # Identify each call in speedscope results.
|
||||
do_stuff()
|
||||
|
||||
.. _performance/profiling/analyse:
|
||||
|
||||
Analyse the results
|
||||
-------------------
|
||||
|
||||
To browse the profiling results, make sure that the :ref:`profiler is enabled globally on the
|
||||
database <performance/profiling/enable>`, then open the :ref:`developer mode tools
|
||||
<developer-mode/mode-tools>` and click on the button in the top-right corner of the profiling
|
||||
section. A list view of the `ir.profile` records grouped by profiling session opens.
|
||||
|
||||
.. image:: performance/profiling_web.png
|
||||
:align: center
|
||||
|
||||
Each record has a clickable link that opens the speedscope results in a new tab.
|
||||
|
||||
.. image:: performance/flamegraph_example.png
|
||||
:align: center
|
||||
|
||||
Speedscope falls out of the scope of this documentation but there are a lot of tools to try: search,
|
||||
highlight of similar frames, zoom on frame, timeline, left heavy, sandwich view...
|
||||
|
||||
Depending on the profiling options that were activated, Odoo generates different view modes that you
|
||||
can access from the top menu.
|
||||
|
||||
.. image:: performance/speedscope_modes.png
|
||||
:align: center
|
||||
|
||||
- The :guilabel:`Combined` view shows all the SQL queries and traces merged togethers.
|
||||
- The :guilabel:`Combined no context` view shows the same result but ignores the saved execution
|
||||
context <performance/profiling/enable>`.
|
||||
- The :guilabel:`sql (no gap)` view shows all the SQL queries as if they were executed one after
|
||||
another, without any Python logic. This is useful for optimizing SQL only.
|
||||
- The :guilabel:`sql (density)` view shows only all the SQL queries, leaving gap between them. This
|
||||
can be useful to spot if eiter SQL or Python code is the problem, and to identify zones in where
|
||||
many small queries could be batched.
|
||||
- The :guilabel:`frames` view shows the results of only the :ref:`periodic collector
|
||||
<performance/profiling/collectors/periodic>`.
|
||||
|
||||
.. important::
|
||||
Even though the profiler has been designed to be as light as possible, it can still impact
|
||||
performance, especially when using the :ref:`Sync collector
|
||||
<performance/profiling/collectors/sync>`. Keep that in mind when analyzing speedscope results.
|
||||
|
||||
.. _performance/profiling/collectors:
|
||||
|
||||
Collectors
|
||||
----------
|
||||
|
||||
Whereas the profiler is about the *when* of profiling, the collectors take care of the *what*.
|
||||
|
||||
Each collector specializes in collecting profiling data in its own format and manner. They can be
|
||||
individually enabled from the user interface through their dedicated toggle button in the
|
||||
:ref:`developer mode tools <developer-mode/mode-tools>`, or from Python code through their key or
|
||||
class.
|
||||
|
||||
There are currently four collectors available in Odoo:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Name
|
||||
- Toggle button
|
||||
- Python key
|
||||
- Python class
|
||||
* - :ref:`SQL collector <performance/profiling/collectors/sql>`
|
||||
- :guilabel:`Record sql`
|
||||
- `sql`
|
||||
- `SqlCollector`
|
||||
* - :ref:`Periodic collector <performance/profiling/collectors/periodic>`
|
||||
- :guilabel:`Record traces`
|
||||
- `traces_async`
|
||||
- `PeriodicCollector`
|
||||
* - :ref:`QWeb collector <performance/profiling/collectors/qweb>`
|
||||
- :guilabel:`Record qweb`
|
||||
- `qweb`
|
||||
- `QwebCollector`
|
||||
* - :ref:`Sync collector <performance/profiling/collectors/sync>`
|
||||
- No
|
||||
- `traces_sync`
|
||||
- `SyncCollector`
|
||||
|
||||
By default, the profiler enables the SQL and the Periodic collectors. Both when it is enabled from
|
||||
the user interface or Python code.
|
||||
|
||||
.. _performance/profiling/collectors/sql:
|
||||
|
||||
SQL collector
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The SQL collector saves all the SQL queries made to the database in the current thread (for all
|
||||
cursors), as well as the stack trace. The overhead of the collector is added to the analysed thread
|
||||
for each query, which means that using it on a lot of small queries may impact execution time and
|
||||
other profilers.
|
||||
|
||||
It is especially useful to debug query counts, or to add information to the :ref:`Periodic collector
|
||||
<performance/profiling/collectors/periodic>` in the combined speedscope view.
|
||||
|
||||
.. autoclass:: SQLCollector
|
||||
|
||||
.. _performance/profiling/collectors/periodic:
|
||||
|
||||
Periodic collector
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This collector runs in a separate thread and saves the stack trace of the analysed thread at every
|
||||
interval. The interval (by default 10 ms) can be defined through the :guilabel:`Interval` option in
|
||||
the user interface, or the `interval` parameter in Python code.
|
||||
|
||||
.. warning::
|
||||
If the interval is set at a very low value, profiling long requests will generate memory issues.
|
||||
If the interval is set at a very high value, information on short function executions will be
|
||||
lost.
|
||||
|
||||
It is one of the best way to analyse performance as it should have a very low impact on the
|
||||
execution time thanks to its separate thread.
|
||||
|
||||
.. autoclass:: PeriodicCollector
|
||||
|
||||
.. _performance/profiling/collectors/qweb:
|
||||
|
||||
QWeb collector
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
This collector saves the Python execution time and queries of all directives. As for the :ref:`SQL
|
||||
collector <performance/profiling/collectors/sql>`, the overhead can be important when executing a
|
||||
lot of small directives. The results are different from other collectors in terms of collected data,
|
||||
and can be analysed from the `ir.profile` form view using a custom widget.
|
||||
|
||||
It is mainly useful for optimizing views.
|
||||
|
||||
.. autoclass:: QwebCollector
|
||||
|
||||
.. _performance/profiling/collectors/sync:
|
||||
|
||||
Sync collector
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
This collector saves the stack for every function's call and return and runs on the same thread,
|
||||
which greatly impacts performance.
|
||||
|
||||
It can be useful to debug and understand complex flows, and follow their execution in the code. It
|
||||
is however not recommended for performance analysis because the overhead is high.
|
||||
|
||||
.. autoclass:: SyncCollector
|
||||
|
||||
.. _performance/profiling/pitfalls:
|
||||
|
||||
Performance pitfalls
|
||||
--------------------
|
||||
|
||||
- Be careful with randomness. Multiple executions may lead to different results. E.g., a garbage
|
||||
collector being triggered during execution.
|
||||
- Be careful with blocking calls. In some cases, external `c_call` may take some time before
|
||||
releasing the GIL, thus leading to unexpected long frames with the :ref:`Periodic collector
|
||||
<performance/profiling/collectors/periodic>`. This should be detected by the profiler and give a
|
||||
warning. It is possible to trigger the profiler manually before such calls if needed.
|
||||
- Pay attention to the cache. Profiling before that the `view`/`assets`/... are in cache can lead to
|
||||
different results.
|
||||
- Be aware of the profiler's overhead. The :ref:`SQL collector
|
||||
<performance/profiling/collectors/sql>`'s overhead can be important when a lot of small queries
|
||||
are executed. Profiling is practical to spot a problem but you may want to disable the profiler in
|
||||
order to measure the real impact of a code change.
|
||||
- Profiling results can be memory intensive. In some cases (e.g., profiling an install or a long
|
||||
request), it is possible that you reach memory limit, especially when rendering the speedscope
|
||||
results, which can lead to an HTTP 500 error. In this case, you may need to start the server with
|
||||
a higher memory limit: `--limit-memory-hard $((8*1024**3))`.
|
||||
|
||||
.. _reference/performance/populate:
|
||||
|
||||
Database population
|
||||
===================
|
||||
|
||||
Odoo CLI offers a :ref:`database population <reference/cmdline/populate>` feature through the CLI
|
||||
command :command:`odoo-bin populate`.
|
||||
|
||||
Instead of the tedious manual, or programmatic, specification of test data, one can use this feature
|
||||
to fill a database on demand with the desired number of test data. This can be used to detect
|
||||
diverse bugs or performance issues in tested flows.
|
||||
|
||||
.. _reference/performance/populate/methods:
|
||||
|
||||
To populate a given model, the following methods and attributes can be defined.
|
||||
|
||||
.. currentmodule:: odoo.models
|
||||
|
||||
.. autoattribute:: Model._populate_sizes
|
||||
.. autoattribute:: Model._populate_dependencies
|
||||
.. automethod:: Model._populate
|
||||
.. automethod:: Model._populate_factories
|
||||
|
||||
.. note::
|
||||
You have to define at least :meth:`~odoo.models.Model._populate` or
|
||||
:meth:`~odoo.models.Model._populate_factories` on the model to enable database population.
|
||||
|
||||
.. example::
|
||||
.. code-block:: python
|
||||
|
||||
from odoo.tools import populate
|
||||
|
||||
class CustomModel(models.Model)
|
||||
_inherit = "custom.some_model"
|
||||
_populate_sizes = {"small": 100, "medium": 2000, "large": 10000}
|
||||
_populate_dependencies = ["custom.some_other_model"]
|
||||
|
||||
def _populate_factories(self):
|
||||
# Record ids of previously populated models are accessible in the registry
|
||||
some_other_ids = self.env.registry.populated_models["custom.some_other_model"]
|
||||
|
||||
def get_some_field(values=None, random=None, **kwargs):
|
||||
""" Choose a value for some_field depending on other fields values.
|
||||
|
||||
:param dict values:
|
||||
:param random: seeded :class:`random.Random` object
|
||||
"""
|
||||
field_1 = values['field_1']
|
||||
if field_1 in [value2, value3]:
|
||||
return random.choice(some_field_values)
|
||||
return False
|
||||
|
||||
return [
|
||||
("field_1", populate.randomize([value1, value2, value3])),
|
||||
("field_2", populate.randomize([value_a, value_b], [0.5, 0.5])),
|
||||
("some_other_id", populate.randomize(some_other_ids)),
|
||||
("some_field", populate.compute(get_some_field, seed="some_field")),
|
||||
('active', populate.cartesian([True, False])),
|
||||
]
|
||||
|
||||
def _populate(self, size):
|
||||
records = super()._populate(size)
|
||||
|
||||
# If you want to update the generated records
|
||||
# E.g setting the parent-child relationships
|
||||
records.do_something()
|
||||
|
||||
return records
|
||||
|
||||
Population tools
|
||||
----------------
|
||||
|
||||
Multiple population tools are available to easily create the needed data generators.
|
||||
|
||||
.. automodule:: odoo.tools.populate
|
||||
:members: cartesian, compute, constant, iterate, randint, randomize
|
||||
|
||||
.. _performance/good_practices:
|
||||
|
||||
Good practices
|
||||
==============
|
||||
|
||||
.. _performance/good_practices/batch:
|
||||
|
||||
Batch operations
|
||||
----------------
|
||||
|
||||
When working with recordsets, it is almost always better to batch operations.
|
||||
|
||||
.. example::
|
||||
Don't call a method that runs SQL queries while looping over a recordset because it will do so
|
||||
for each record of the set.
|
||||
|
||||
.. rst-class:: bad-example
|
||||
.. code-block:: python
|
||||
|
||||
def _compute_count(self):
|
||||
for record in self:
|
||||
domain = [('related_id', '=', record.id)]
|
||||
record.count = other_model.search_count(domain)
|
||||
|
||||
Instead, replace the `search_count` with a `read_group` to execute one SQL query for the entire
|
||||
batch of records.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
def _compute_count(self):
|
||||
if self.ids:
|
||||
domain = [('related_id', 'in', self.ids)]
|
||||
counts_data = other_model.read_group(domain, ['related_id'], ['related_id'])
|
||||
mapped_data = {
|
||||
count['related_id'][0]: count['related_id_count'] for count in counts_data
|
||||
}
|
||||
else:
|
||||
mapped_data = {}
|
||||
for record in self:
|
||||
record.count = mapped_data.get(record.id, 0)
|
||||
|
||||
.. note::
|
||||
This example is not optimal nor correct in all cases. It is only a substitute for a
|
||||
`search_count`. Another solution could be to prefetch and count the inverse `One2many` field.
|
||||
|
||||
.. example::
|
||||
Don't create records one after another.
|
||||
|
||||
.. rst-class:: bad-example
|
||||
.. code-block:: python
|
||||
|
||||
for name in ['foo', 'bar']:
|
||||
model.create({'name': name})
|
||||
|
||||
Instead, accumulate the create values and call the `create` method on the batch. Doing so has
|
||||
mostly no impact and helps the framework optimize fields computation.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
create_values = []
|
||||
for name in ['foo', 'bar']:
|
||||
create_values.append({'name': name})
|
||||
records = model.create(create_values)
|
||||
|
||||
.. example::
|
||||
Fail to prefetch the fields of a recordset while browsing a single record inside a loop.
|
||||
|
||||
.. rst-class:: bad-example
|
||||
.. code-block:: python
|
||||
|
||||
for record_id in record_ids:
|
||||
model.browse(record_id)
|
||||
record.foo # One query is executed per record.
|
||||
|
||||
Instead, browse the entire recordset first.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
records = model.browse(record_ids)
|
||||
for record in records:
|
||||
record.foo # One query is executed for the entire recordset.
|
||||
|
||||
We can verify that the records are prefetched in batch by reading the field `prefetch_ids` which
|
||||
includes each of the record ids.browsing all records together is unpractical,
|
||||
|
||||
If needed, the `with_prefetch` method can be used to disable batch prefetching:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
for values in values_list:
|
||||
message = self.browse(values['id']).with_prefetch(self.ids)
|
||||
|
||||
.. _performance/good_practices/algorithmic_complexity:
|
||||
|
||||
Reduce the algorithmic complexity
|
||||
---------------------------------
|
||||
|
||||
Algorithmic complexity is a measure of how long an algorithm would take to complete in regard to the
|
||||
size `n` of the input. When the complexity is high, the execution time can grow quickly as the input
|
||||
becomes larger. In some cases, the algorithmic complexity can be reduced by preparing the input's
|
||||
data correctly.
|
||||
|
||||
.. example::
|
||||
For a given problem, let's consider a naive algorithm crafted with two nested loops for which the
|
||||
complexity in in O(n²).
|
||||
|
||||
.. rst-class:: bad-example
|
||||
.. code-block:: python
|
||||
|
||||
for record in self:
|
||||
for result in results:
|
||||
if results['id'] == record.id:
|
||||
record.foo = results['foo']
|
||||
break
|
||||
|
||||
Assuming that all results have a different id, we can prepare the data to reduce the complexity.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
mapped_result = {result['id']: result['foo'] for result in results}
|
||||
for record in self:
|
||||
record.foo = mapped_result.get(record.id)
|
||||
|
||||
.. example::
|
||||
Choosing the bad data structure to hold the input can lead to quadratic complexity.
|
||||
|
||||
.. rst-class:: bad-example
|
||||
.. code-block:: python
|
||||
|
||||
invalid_ids = self.search(domain).ids
|
||||
for record in self:
|
||||
if record.id in invalid_ids:
|
||||
...
|
||||
|
||||
If `invalid_ids` is a list-like data structure, the complexity of the algorithm may be quadratic.
|
||||
|
||||
Instead, prefer using set operations like casting `invalid_ids` to a set.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
invalid_ids = set(invalid_ids)
|
||||
for record in self:
|
||||
if record.id in invalid_ids:
|
||||
...
|
||||
|
||||
Depending on the input, recordset operations can also be used.
|
||||
|
||||
.. rst-class:: good-example
|
||||
.. code-block:: python
|
||||
|
||||
invalid_ids = self.search(domain)
|
||||
for record in self - invalid_ids:
|
||||
...
|
||||
|
||||
.. _performance/good_practices/index:
|
||||
|
||||
Use indexes
|
||||
-----------
|
||||
|
||||
Database indexes can help fasten search operations, be it from a search in the or through the user
|
||||
interface.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
name = fields.Char(string="Name", index=True)
|
||||
|
||||
.. warning::
|
||||
Be careful not to index every field as indexes consume space and impact on performance when
|
||||
executing one of `INSERT`, `UPDATE`, and `DELETE`.
|
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 22 KiB |
After Width: | Height: | Size: 3.8 KiB |
After Width: | Height: | Size: 3.5 KiB |
After Width: | Height: | Size: 8.3 KiB |
After Width: | Height: | Size: 54 KiB |
After Width: | Height: | Size: 4.3 KiB |
@ -864,90 +864,6 @@ you can use the :meth:`~odoo.tests.common.BaseCase.assertQueryCount` method, int
|
||||
with self.assertQueryCount(11):
|
||||
do_something()
|
||||
|
||||
.. _reference/testing/populate:
|
||||
|
||||
Database population
|
||||
-------------------
|
||||
|
||||
Odoo CLI offers a :ref:`database population<reference/cmdline/populate>` feature.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
odoo-bin populate
|
||||
|
||||
Instead of the tedious manual, or programmatic, specification of test data,
|
||||
one can use this feature to fill a database on demand with the desired number of test data.
|
||||
This can be used to detect diverse bugs or performance issues in tested flows.
|
||||
|
||||
.. _reference/testing/populate/methods:
|
||||
|
||||
To specify this feature for a given model, the following methods and attributes can be defined.
|
||||
|
||||
.. currentmodule:: odoo.models
|
||||
|
||||
.. autoattribute:: Model._populate_sizes
|
||||
.. autoattribute:: Model._populate_dependencies
|
||||
.. automethod:: Model._populate
|
||||
.. automethod:: Model._populate_factories
|
||||
|
||||
.. note::
|
||||
|
||||
You have to define at least :meth:`~odoo.models.Model._populate` or :meth:`~odoo.models.Model._populate_factories`
|
||||
on the model to enable database population.
|
||||
|
||||
Example model
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from odoo.tools import populate
|
||||
|
||||
class CustomModel(models.Model)
|
||||
_inherit = "custom.some_model"
|
||||
_populate_sizes = {"small": 100, "medium": 2000, "large": 10000}
|
||||
_populate_dependencies = ["custom.some_other_model"]
|
||||
|
||||
def _populate_factories(self):
|
||||
# Record ids of previously populated models are accessible in the registry
|
||||
some_other_ids = self.env.registry.populated_models["custom.some_other_model"]
|
||||
|
||||
def get_some_field(values=None, random=None, **kwargs):
|
||||
""" Choose a value for some_field depending on other fields values.
|
||||
|
||||
:param dict values:
|
||||
:param random: seeded :class:`random.Random` object
|
||||
"""
|
||||
field_1 = values['field_1']
|
||||
if field_1 in [value2, value3]:
|
||||
return random.choice(some_field_values)
|
||||
return False
|
||||
|
||||
return [
|
||||
("field_1", populate.randomize([value1, value2, value3])),
|
||||
("field_2", populate.randomize([value_a, value_b], [0.5, 0.5])),
|
||||
("some_other_id", populate.randomize(some_other_ids)),
|
||||
("some_field", populate.compute(get_some_field, seed="some_field")),
|
||||
('active', populate.cartesian([True, False])),
|
||||
]
|
||||
|
||||
def _populate(self, size):
|
||||
records = super()._populate(size)
|
||||
|
||||
# If you want to update the generated records
|
||||
# E.g setting the parent-child relationships
|
||||
records.do_something()
|
||||
|
||||
return records
|
||||
|
||||
Population tools
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
Multiple population tools are available to easily create
|
||||
the needed data generators.
|
||||
|
||||
.. automodule:: odoo.tools.populate
|
||||
:members: cartesian, compute, constant, iterate, randint, randomize
|
||||
|
||||
.. _qunit: https://qunitjs.com/
|
||||
.. _qunit_config.js: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/qunit_config.js#L49
|
||||
.. _web.tests_assets: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/views/webclient_templates.xml#L594
|
||||
|
7
static/css/performance.css
Normal file
@ -0,0 +1,7 @@
|
||||
.bad-example {
|
||||
border-left: 3px solid red;
|
||||
}
|
||||
|
||||
.good-example {
|
||||
border-left: 3px solid green;
|
||||
}
|