diff --git a/content/applications/general/developer_mode.rst b/content/applications/general/developer_mode.rst index 47176f8ce..0a01bbefe 100644 --- a/content/applications/general/developer_mode.rst +++ b/content/applications/general/developer_mode.rst @@ -59,6 +59,8 @@ debug mode, add `?debug=0` instead. :ref:`assets mode `, and `?debug=tests` enables the :ref:`tests mode `. +.. _developer-mode/mode-tools: + Locate the mode tools ===================== diff --git a/content/contributing/development/coding_guidelines.rst b/content/contributing/development/coding_guidelines.rst index 802d3b776..9463be5d0 100644 --- a/content/contributing/development/coding_guidelines.rst +++ b/content/contributing/development/coding_guidelines.rst @@ -589,32 +589,6 @@ Programming in Odoo - As in python, use ``filtered``, ``mapped``, ``sorted``, ... methods to ease code reading and performance. - -Make your method work in batch -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When adding a function, make sure it can process multiple records by iterating -on self to treat each record. - -.. code-block:: python - - def my_method(self) - for record in self: - record.do_cool_stuff() - -For performance issue, when developing a 'stat button' (for instance), do not -perform a ``search`` or a ``search_count`` in a loop. It -is recommended to use ``read_group`` method, to compute all value in only one request. - -.. code-block:: python - - def _compute_equipment_count(self): - """ Count the number of equipment per category """ - equipment_data = self.env['hr.equipment'].read_group([('category_id', 'in', self.ids)], ['category_id'], ['category_id']) - mapped_data = dict([(m['category_id'][0], m['category_id_count']) for m in equipment_data]) - for category in self: - category.equipment_count = mapped_data.get(category.id, 0) - - Propagate the context ~~~~~~~~~~~~~~~~~~~~~ The context is a ``frozendict`` that cannot be modified. To call a method with diff --git a/content/developer/cli.rst b/content/developer/cli.rst index 9250935bd..e55858cb4 100644 --- a/content/developer/cli.rst +++ b/content/developer/cli.rst @@ -702,8 +702,9 @@ Database Population .. program:: odoo-bin populate Odoo CLI supports database population features. If the feature is -:ref:`implemented on a given model `, it allows automatic data -generation of the model's records to test your modules in databases containing non-trivial amounts of records. +:ref:`implemented on a given model `, it allows automatic +data generation of the model's records to test your modules in databases containing non-trivial +amounts of records. .. code-block:: console @@ -720,8 +721,7 @@ generation of the model's records to test your modules in databases containing n of a given model (cf. the :file:`populate` folder of modules for further details). .. seealso:: - - :ref:`reference/testing/populate` + :ref:`reference/performance/populate` .. _reference/cmdline/cloc: diff --git a/content/developer/howtos.rst b/content/developer/howtos.rst index 431b7b532..c7d96af7a 100644 --- a/content/developer/howtos.rst +++ b/content/developer/howtos.rst @@ -12,7 +12,6 @@ Tutorials howtos/website howtos/backend howtos/web_services - howtos/profilecode howtos/company howtos/accounting_localization howtos/translations diff --git a/content/developer/howtos/profilecode.rst b/content/developer/howtos/profilecode.rst deleted file mode 100644 index aa1c7a6c4..000000000 --- a/content/developer/howtos/profilecode.rst +++ /dev/null @@ -1,131 +0,0 @@ -=================== -Profiling Odoo code -=================== - -.. warning:: - - This tutorial requires :ref:`having installed Odoo ` - and :doc:`writing Odoo code ` - -Graph a method -============== - -Odoo embeds a profiler of code. This embedded profiler output can be used to -generate a graph of calls triggered by the method, number of queries, percentage -of time taken in the method itself as well as the time that the method took and -its sub-called methods. - -.. code:: python - - from odoo.tools.misc import profile - [...] - @profile('/temp/prof.profile') - def mymethod(...) - -This produces a file called /temp/prof.profile - -A tool called *gprof2dot* will produce a graph with this result: - -.. code:: bash - - gprof2dot -f pstats -o /temp/prof.xdot /temp/prof.profile - -A tool called *xdot* will display the resulting graph: - -.. code:: bash - - xdot /temp/prof.xdot - -Log a method -============ - -Another profiler can be used to log statistics on a method: - -.. code:: python - - from odoo.tools.profiler import profile - [...] - @profile - @api.model - def mymethod(...): - -The statistics will be displayed into the logs once the method to be analysed is -completely reviewed. - -.. code:: bash - - 2018-03-28 06:18:23,196 22878 INFO openerp odoo.tools.profiler: - calls queries ms - project.task ------------------------ /home/odoo/src/odoo/addons/project/models/project.py, 638 - - 1 0 0.02 @profile - @api.model - def create(self, vals): - # context: no_log, because subtype already handle this - 1 0 0.01 context = dict(self.env.context, mail_create_nolog=True) - - # for default stage - 1 0 0.01 if vals.get('project_id') and not context.get('default_project_id'): - context['default_project_id'] = vals.get('project_id') - # user_id change: update date_assign - 1 0 0.01 if vals.get('user_id'): - vals['date_assign'] = fields.Datetime.now() - # Stage change: Update date_end if folded stage - 1 0 0.0 if vals.get('stage_id'): - vals.update(self.update_date_end(vals['stage_id'])) - 1 108 631.8 task = super(Task, self.with_context(context)).create(vals) - 1 0 0.01 return task - - Total: - 1 108 631.85 - -Dump stack -========== - -Sending the SIGQUIT signal to an Odoo process (only available on POSIX) makes -this process output the current stack trace to log, with info level. When an -odoo process seems stuck, sending this signal to the process permit to know -what the process is doing, and letting the process continue his job. - -Tracing code execution -====================== - -Instead of sending the SIGQUIT signal to an Odoo process often enough, to check -where the processes are performing worse than expected, we can use the `py-spy`_ tool to -do it for us. - -Install py-spy --------------- - -.. code:: bash - - python3 -m pip install py-spy - -Record executed code --------------------- - -As py-spy is installed, we now record the executed code lines. -This tool will record, multiple times a second, the stacktrace of the process. - -.. code:: bash - - # record to raw file - py-spy record -o profile.json -f speedscope --pid - - # OR record directly to svg - py-spy record -o profile.svg --pid - -where is the process ID of the odoo process you want to graph. - -To open profile.json you can use online tool `speedscope.app`_. - -To open profile.svg you should use browser, because other viewer may not -support interactive part. - - -.. image:: profilecode/flamegraph.svg - - -.. _py-spy: https://github.com/benfred/py-spy - -.. _speedscope.app: https://www.speedscope.app/ diff --git a/content/developer/howtos/profilecode/flamegraph.svg b/content/developer/howtos/profilecode/flamegraph.svg deleted file mode 100644 index a20921ad0..000000000 --- a/content/developer/howtos/profilecode/flamegraph.svg +++ /dev/null @@ -1,315 +0,0 @@ -Flame GraphReset ZoomSearch/datas/progs/odoo/odoo/models.py:browse:4237 (11 samples, 8.33%)/datas/prog../datas/progs/odoo/odoo/fields.py:__get__:948 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:__get__:948 (38 samples, 28.79%)/datas/progs/odoo/odoo/fields.py:__get__:948/datas/progs/odoo/odoo/api.py:__new__:736 (5 samples, 3.79%)/dat..<string>:__new__:14 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_normalize_ids:5206 (1 samples, 0.76%)<frozen importlib._bootstrap>:_call_with_frames_removed:219 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:sudo:4310 (3 samples, 2.27%)/..<frozen importlib._bootstrap>:_call_with_frames_removed:219 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:736 (1 samples, 0.76%)<frozen importlib._bootstrap_external>:exec_module:678 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:convert_to_cache:1230 (6 samples, 4.55%)/data../datas/progs/odoo/odoo/api.py:__new__:736 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__hash__:777 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:736 (1 samples, 0.76%)/datas/progs/odoo/odoo/sql_db.py:execute:242 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_raw:1924 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/models.py:qualify:2612 (1 samples, 0.76%)/datas/progs/odoo/addons/auth_signup/models/ir_http.py:_dispatch:19 (128 samples, 96.97%)/datas/progs/odoo/addons/auth_signup/models/ir_http.py:_dispatch:19/datas/progs/odoo/odoo/api.py:__new__:742 (1 samples, 0.76%)<string>:change_digit:9 (6 samples, 4.55%)<stri../usr/local/lib/python3.6/encodings/utf_8.py:decode:16 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:convert_to_cache:1230 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/models.py:_read_group_resolve_many2one_fields:1944 (6 samples, 4.55%)/data../datas/progs/odoo/addons/stock_account/models/product.py:do_change_standard_price:126 (20 samples, 15.15%)/datas/progs/odoo/addon../datas/progs/odoo/odoo/models.py:read_group:1828 (13 samples, 9.85%)/datas/progs/o../datas/progs/odoo/odoo/api.py:__new__:736 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:convert_to_record:1989 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:recompute:4791 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:__get__:944 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:browse:4235 (2 samples, 1.52%)/datas/progs/odoo/odoo/api.py:add:939 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:<listcomp>:1204 (1 samples, 0.76%)<decorator-gen-112>:precision_get:2 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:get_unaccent_wrapper:455 (1 samples, 0.76%)/usr/local/lib/python3.6/site-packages/werkzeug/routing.py:add:1197 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_raw:1924 (3 samples, 2.27%)/../usr/local/lib/python3.6/site-packages/werkzeug/serving.py:run_wsgi:193 (132 samples, 100.00%)/usr/local/lib/python3.6/site-packages/werkzeug/serving.py:run_wsgi:193/datas/progs/odoo/odoo/api.py:get_todo:911 (1 samples, 0.76%)odoo-bin:<module>:8 (132 samples, 100.00%)odoo-bin:<module>:8/datas/progs/odoo/odoo/fields.py:__set__:966 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/modules/loading.py:load_modules:339 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_from_database:2618 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:<listcomp>:2618 (1 samples, 0.76%)all (132 samples, 100%)/usr/local/lib/python3.6/sre_compile.py:compile:566 (1 samples, 0.76%)/datas/progs/odoo/odoo/service/server.py:run:743 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:run:743/datas/progs/odoo/odoo/api.py:__call__:789 (9 samples, 6.82%)/datas/pr../datas/progs/odoo/odoo/cli/server.py:main:169 (132 samples, 100.00%)/datas/progs/odoo/odoo/cli/server.py:main:169/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities_dict:124 (13 samples, 9.85%)/datas/progs/o../datas/progs/odoo/odoo/models.py:__setitem__:4666 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/models.py:_read_group_raw:1889 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:__bool__:4521 (1 samples, 0.76%)/datas/progs/odoo/odoo/addons/base/res/ir_property.py:get_multi:160 (6 samples, 4.55%)/data../datas/progs/odoo/odoo/fields.py:__get__:948 (96 samples, 72.73%)/datas/progs/odoo/odoo/fields.py:__get__:948/datas/progs/odoo/odoo/fields.py:convert_to_cache:1227 (1 samples, 0.76%)/datas/progs/odoo/odoo/addons/base/res/ir_property.py:get_multi:150 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/models.py:_prefetch_field:2533 (27 samples, 20.45%)/datas/progs/odoo/odoo/models.py../usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_search:3664 (4 samples, 3.03%)/da../usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_where_calc:3469 (3 samples, 2.27%)/../datas/progs/odoo/odoo/addons/base/ir/ir_http.py:_find_handler:84 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:736 (2 samples, 1.52%)/datas/progs/odoo/odoo/api.py:__new__:103 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__call__:789 (6 samples, 4.55%)/data../datas/progs/odoo/addons/stock/models/product.py:<lambda>:200 (38 samples, 28.79%)/datas/progs/odoo/addons/stock/models/product../datas/progs/odoo/odoo/modules/registry.py:setup_models:276 (3 samples, 2.27%)/../datas/progs/odoo/odoo/api.py:__new__:735 (2 samples, 1.52%)/datas/progs/odoo/odoo/api.py:call_kw_multi:680 (1 samples, 0.76%)<frozen importlib._bootstrap>:_handle_fromlist:1017 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:dispatch:683 (127 samples, 96.21%)/datas/progs/odoo/odoo/http.py:dispatch:683/datas/progs/odoo/odoo/fields.py:determine_value:1049 (38 samples, 28.79%)/datas/progs/odoo/odoo/fields.py:determine_va../usr/local/lib/python3.6/site-packages/werkzeug/serving.py:handle_one_request:251 (132 samples, 100.00%)/usr/local/lib/python3.6/site-packages/werkzeug/serving.py:handle_one_request:251/datas/progs/odoo/odoo/sql_db.py:wrapper:155 (2 samples, 1.52%)/datas/progs/odoo/odoo/http.py:routing_map:978 (1 samples, 0.76%)<frozen importlib._bootstrap>:_handle_fromlist:1017 (1 samples, 0.76%)/datas/progs/odoo/addons/fetchmail/models/fetchmail.py:<module>:21 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:read:2509 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:_call_function:339 (127 samples, 96.21%)/datas/progs/odoo/odoo/http.py:_call_function:339/datas/progs/odoo/odoo/fields.py:compute_value:1015 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:59 (1 samples, 0.76%)/usr/local/lib/python3.6/sre_compile.py:_code:551 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:get_alias_from_query:380 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:determine_value:1049 (1 samples, 0.76%)/datas/progs/odoo/addons/stock_account/models/product.py:do_change_standard_price:131 (96 samples, 72.73%)/datas/progs/odoo/addons/stock_account/models/product.py:do_change_standard_price:131/datas/progs/odoo/odoo/fields.py:__get__:951 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:read:2509 (5 samples, 3.79%)/dat../datas/progs/odoo/addons/web/controllers/main.py:call_button:928 (126 samples, 95.45%)/datas/progs/odoo/addons/web/controllers/main.py:call_button:928/datas/progs/odoo/odoo/models.py:_read_group_raw:1854 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_where_calc:3462 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:determine_value:1059 (1 samples, 0.76%)/datas/progs/odoo/odoo/service/server.py:process_request:767 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:process_request:767/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities_dict:123 (9 samples, 6.82%)/datas/pr../datas/progs/odoo/odoo/tools/cache.py:lookup:81 (1 samples, 0.76%)/datas/progs/odoo/odoo/modules/module.py:load_module:82 (1 samples, 0.76%)/datas/progs/odoo/odoo/modules/loading.py:load_modules:346 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:__set__:966 (6 samples, 4.55%)/data../usr/local/lib/python3.6/sre_compile.py:_compile:81 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4218 (1 samples, 0.76%)<frozen importlib._bootstrap>:_call_with_frames_removed:219 (1 samples, 0.76%)<frozen importlib._bootstrap>:_load_unlocked:656 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:<listcomp>:2618 (1 samples, 0.76%)<frozen importlib._bootstrap>:_load_backward_compatible:626 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/query.py:get_sql:144 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (2 samples, 1.52%)/datas/progs/odoo/odoo/models.py:_read_from_database:2589 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4224 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/osv/expression.py:is_leaf:422 (1 samples, 0.76%)/datas/progs/odoo/odoo/service/server.py:process_work:776 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:process_work:776/datas/progs/odoo/odoo/fields.py:digits:1209 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:__get__:956 (3 samples, 2.27%)/../datas/progs/odoo/odoo/http.py:dispatch:1456 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/api.py:__call__:789 (4 samples, 3.03%)/da../usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_where_calc:3469 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (2 samples, 1.52%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:85 (8 samples, 6.06%)/datas/p../datas/progs/odoo/odoo/models.py:_where_calc:3471 (1 samples, 0.76%)/datas/progs/odoo/odoo/tools/lru.py:__getitem__:45 (1 samples, 0.76%)/usr/local/lib/python3.6/site-packages/werkzeug/serving.py:handle:216 (132 samples, 100.00%)/usr/local/lib/python3.6/site-packages/werkzeug/serving.py:handle:216/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:86 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/models.py:__getitem__:4657 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/service/server.py:process_spawn:539 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:process_spawn:539/usr/local/lib/python3.6/http/server.py:handle:418 (132 samples, 100.00%)/usr/local/lib/python3.6/http/server.py:handle:418/datas/progs/odoo/odoo/api.py:__new__:744 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:qualify:2612 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (1 samples, 0.76%)<frozen importlib._bootstrap>:_find_and_load_unlocked:955 (1 samples, 0.76%)/datas/progs/odoo/addons/product/models/product.py:name_get:372 (1 samples, 0.76%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:81 (72 samples, 54.55%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:81/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:84 (3 samples, 2.27%)/../datas/progs/odoo/odoo/addons/base/res/ir_property.py:_get_domain:134 (2 samples, 1.52%)/datas/progs/odoo/odoo/models.py:_read_from_database:2628 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__call__:789 (8 samples, 6.06%)/datas/p../datas/progs/odoo/odoo/models.py:_in_cache_without:4684 (27 samples, 20.45%)/datas/progs/odoo/odoo/models.py../datas/progs/odoo/odoo/osv/expression.py:__leaf_to_sql:1204 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_resolve_many2one_fields:1944 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:_compute_company_dependent:636 (11 samples, 8.33%)/datas/prog../datas/progs/odoo/odoo/fields.py:convert_to_cache:1230 (3 samples, 2.27%)/../datas/progs/odoo/odoo/api.py:__new__:735 (4 samples, 3.03%)/da..<frozen importlib._bootstrap>:_load_unlocked:665 (1 samples, 0.76%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities_dict:93 (38 samples, 28.79%)/datas/progs/odoo/addons/stock/models/product../datas/progs/odoo/addons/stock/models/product.py:_get_domain_locations:191 (38 samples, 28.79%)/datas/progs/odoo/addons/stock/models/product../usr/local/lib/python3.6/enum.py:__and__:804 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:call_kw_multi:680 (126 samples, 95.45%)/datas/progs/odoo/odoo/api.py:call_kw_multi:680/datas/progs/odoo/odoo/addons/base/res/res_company.py:_company_default_get:181 (3 samples, 2.27%)/../usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:with_context:4333 (9 samples, 6.82%)/datas/pr../datas/progs/odoo/odoo/models.py:_read_from_database:2618 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_add_field:288 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__getitem__:760 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:determine_value:1059 (96 samples, 72.73%)/datas/progs/odoo/odoo/fields.py:determine_value:1059<string>:change_digit:9 (6 samples, 4.55%)<stri../datas/progs/odoo/odoo/models.py:browse:4237 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:read:2520 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/osv/expression.py:parse:813 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_prefetch_field:2565 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/datas/progs/odoo/odoo/service/wsgi_server.py:application:166 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/wsgi_server.py:application:166/datas/progs/odoo/odoo/service/server.py:run:635 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:run:635/datas/progs/odoo/odoo/models.py:_inherits_join_calc:1985 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4219 (2 samples, 1.52%)/datas/progs/odoo/odoo/api.py:__new__:736 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/fields.py:convert_to_cache:1230 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/tools/lru.py:__setitem__:56 (1 samples, 0.76%)<frozen importlib._bootstrap_external>:exec_module:678 (1 samples, 0.76%)/datas/progs/odoo/odoo/__init__.py:registry:76 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/fields.py:convert_to_column:776 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (4 samples, 3.03%)/da..<frozen importlib._bootstrap>:_load_unlocked:665 (1 samples, 0.76%)<string>:change_digit:10 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:__set__:966 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:_compute_company_dependent:638 (7 samples, 5.30%)/datas../datas/progs/odoo/addons/stock/models/product.py:_get_domain_locations_new:200 (38 samples, 28.79%)/datas/progs/odoo/addons/stock/models/product../datas/progs/odoo/odoo/modules/loading.py:load_marked_modules:242 (1 samples, 0.76%)/datas/progs/odoo/odoo/sql_db.py:execute:232 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:call_kw:689 (1 samples, 0.76%)/datas/progs/odoo/odoo/modules/registry.py:new:85 (4 samples, 3.03%)/da..<decorator-gen-37>:check:2 (1 samples, 0.76%)<frozen importlib._bootstrap>:_find_and_load:971 (1 samples, 0.76%)/datas/progs/odoo/addons/web/controllers/main.py:_call_kw:916 (126 samples, 95.45%)/datas/progs/odoo/addons/web/controllers/main.py:_call_kw:916/datas/progs/odoo/odoo/fields.py:digits:1209 (7 samples, 5.30%)/datas../usr/local/lib/python3.6/socketserver.py:__init__:696 (132 samples, 100.00%)/usr/local/lib/python3.6/socketserver.py:__init__:696/datas/progs/odoo/odoo/service/model.py:wrapper:97 (127 samples, 96.21%)/datas/progs/odoo/odoo/service/model.py:wrapper:97/datas/progs/odoo/odoo/addons/base/ir/ir_http.py:_dispatch:208 (127 samples, 96.21%)/datas/progs/odoo/odoo/addons/base/ir/ir_http.py:_dispatch:208/datas/progs/odoo/odoo/http.py:__call__:1272 (132 samples, 100.00%)/datas/progs/odoo/odoo/http.py:__call__:1272/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (2 samples, 1.52%)<frozen importlib._bootstrap>:_find_and_load_unlocked:955 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_raw:1918 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:filtered:4439 (38 samples, 28.79%)/datas/progs/odoo/odoo/models.py:filtered:4439/datas/progs/odoo/addons/stock_account/models/product.py:do_change_standard_price:156 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/service/server.py:worker_spawn:457 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:worker_spawn:457/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities_dict:125 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_read_from_database:2597 (1 samples, 0.76%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities_dict:134 (9 samples, 6.82%)/datas/pr../datas/progs/odoo/odoo/osv/expression.py:__init__:668 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_where_calc:3471 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:__iter__:4531 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:cache_key:766 (2 samples, 1.52%)/datas/progs/odoo/odoo/fields.py:__set__:966 (8 samples, 6.06%)/datas/p../datas/progs/odoo/odoo/models.py:_read_group_raw:1851 (2 samples, 1.52%)/datas/progs/odoo/odoo/models.py:_prefetch_field:2565 (11 samples, 8.33%)/datas/prog../datas/progs/odoo/odoo/models.py:check_access_rights:2781 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_generate_translated_field:3553 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/usr/local/lib/python3.6/site-packages/werkzeug/wsgi.py:__call__:599 (132 samples, 100.00%)/usr/local/lib/python3.6/site-packages/werkzeug/wsgi.py:__call__:599/datas/progs/odoo/odoo/models.py:read_group:1830 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:response_wrap:512 (127 samples, 96.21%)/datas/progs/odoo/odoo/http.py:response_wrap:512/datas/progs/odoo/odoo/models.py:_browse:4224 (1 samples, 0.76%)<decorator-gen-112>:precision_get:2 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:<listcomp>:4439 (38 samples, 28.79%)/datas/progs/odoo/odoo/models.py:<listcomp>:4../datas/progs/odoo/odoo/sql_db.py:execute:239 (1 samples, 0.76%)/datas/progs/odoo/addons/fetchmail/models/__init__.py:<module>:4 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_resolve_many2one_fields:1943 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_raw:1924 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/api.py:__call__:789 (4 samples, 3.03%)/da../usr/local/lib/python3.6/_weakrefset.py:__enter__:24 (1 samples, 0.76%)/datas/progs/odoo/addons/web_editor/models/ir_http.py:_dispatch:21 (128 samples, 96.97%)/datas/progs/odoo/addons/web_editor/models/ir_http.py:_dispatch:21/datas/progs/odoo/odoo/sql_db.py:execute:232 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_in_cache_without:4684 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:_setup_attrs:446 (1 samples, 0.76%)/datas/progs/odoo/addons/http_routing/models/ir_http.py:_dispatch:325 (1 samples, 0.76%)/usr/local/lib/python3.6/enum.py:__call__:291 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/query.py:_get_alias_mapping:75 (1 samples, 0.76%)/datas/progs/odoo/odoo/addons/base/res/res_users.py:_get_company:217 (3 samples, 2.27%)/../datas/progs/odoo/odoo/service/wsgi_server.py:application_unproxied:154 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/wsgi_server.py:application_unproxied:154/datas/progs/odoo/odoo/api.py:__new__:736 (5 samples, 3.79%)/dat../usr/local/lib/python3.6/inspect.py:getmembers:333 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:dispatch:1473 (128 samples, 96.97%)/datas/progs/odoo/odoo/http.py:dispatch:1473/datas/progs/odoo/odoo/models.py:_add_field:284 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:distribute_not:302 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4224 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:747 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:to_sql:1271 (1 samples, 0.76%)<string>:change_digit:10 (1 samples, 0.76%)/datas/progs/odoo/addons/http_routing/models/ir_http.py:_dispatch:397 (127 samples, 96.21%)/datas/progs/odoo/addons/http_routing/models/ir_http.py:_dispatch:397/datas/progs/odoo/odoo/sql_db.py:wrapper:155 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_setup_base:2311 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:__get__:2523 (1 samples, 0.76%)/datas/progs/odoo/odoo/modules/registry.py:__new__:61 (4 samples, 3.03%)/da..<frozen importlib._bootstrap>:_find_and_load_unlocked:955 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:compute_value:1015 (96 samples, 72.73%)/datas/progs/odoo/odoo/fields.py:compute_value:1015/datas/progs/odoo/odoo/fields.py:cache_key:765 (2 samples, 1.52%)/datas/progs/odoo/odoo/fields.py:convert_to_cache:1230 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/sql_db.py:wrapper:155 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4222 (2 samples, 1.52%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_read_group_resolve_many2one_fields:1944 (5 samples, 3.79%)/dat../datas/progs/odoo/odoo/models.py:sudo:4310 (4 samples, 3.03%)/da..<frozen importlib._bootstrap>:_call_with_frames_removed:219 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:__leaf_to_sql:1197 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:to_sql:1271 (1 samples, 0.76%)/datas/progs/odoo/odoo/cli/command.py:main:60 (132 samples, 100.00%)/datas/progs/odoo/odoo/cli/command.py:main:60/datas/progs/odoo/odoo/models.py:read:2516 (1 samples, 0.76%)/usr/local/lib/python3.6/re.py:compile:233 (1 samples, 0.76%)/datas/progs/odoo/addons/stock/models/product.py:_compute_quantities:83 (6 samples, 4.55%)/data../datas/progs/odoo/addons/stock_account/models/product.py:do_change_standard_price:125 (9 samples, 6.82%)/datas/pr..<frozen importlib._bootstrap>:_find_and_load:971 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:create:3278 (1 samples, 0.76%)/datas/progs/odoo/odoo/sql_db.py:dbname:477 (1 samples, 0.76%)/datas/progs/odoo/addons/product/models/product.py:name_get:376 (6 samples, 4.55%)/data../datas/progs/odoo/odoo/api.py:__call__:789 (3 samples, 2.27%)/../datas/progs/odoo/odoo/http.py:__call__:927 (127 samples, 96.21%)/datas/progs/odoo/odoo/http.py:__call__:927/datas/progs/odoo/odoo/fields.py:__get__:948 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_add_magic_fields:330 (1 samples, 0.76%)/datas/progs/odoo/odoo/modules/registry.py:<dictcomp>:201 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:_compute_value:1006 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:read_group:1828 (8 samples, 6.06%)/datas/p../usr/local/lib/python3.6/site-packages/werkzeug/routing.py:bind:681 (1 samples, 0.76%)/datas/progs/odoo/odoo/tools/misc.py:__init__:990 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:determine_value:1057 (1 samples, 0.76%)/datas/progs/odoo/odoo/addons/base/ir/ir_http.py:routing_map:238 (1 samples, 0.76%)/usr/local/lib/python3.6/site-packages/werkzeug/routing.py:compile:742 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:checked_call:332 (127 samples, 96.21%)/datas/progs/odoo/odoo/http.py:checked_call:332/datas/progs/odoo/odoo/osv/expression.py:to_sql:1280 (1 samples, 0.76%)/datas/progs/odoo/addons/web/controllers/main.py:_call_kw:916 (1 samples, 0.76%)/datas/progs/odoo/addons/product/models/product.py:name_get:376 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/models.py:_inherits_join_calc:1973 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_setup_base:2308 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:__get__:948 (19 samples, 14.39%)/datas/progs/odoo/odo../datas/progs/odoo/odoo/modules/module.py:load_openerp_module:368 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:call_kw:689 (126 samples, 95.45%)/datas/progs/odoo/odoo/api.py:call_kw:689/datas/progs/odoo/odoo/fields.py:digits:1209 (6 samples, 4.55%)/data../datas/progs/odoo/odoo/models.py:_where_calc:3471 (1 samples, 0.76%)/datas/progs/odoo/odoo/http.py:__call__:1298 (132 samples, 100.00%)/datas/progs/odoo/odoo/http.py:__call__:1298/datas/progs/odoo/addons/account/models/account_move.py:create:126 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/osv/expression.py:__init__:665 (1 samples, 0.76%)/datas/progs/odoo/odoo/tools/cache.py:lookup:84 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:736 (2 samples, 1.52%)/usr/local/lib/python3.6/socketserver.py:finish_request:361 (132 samples, 100.00%)/usr/local/lib/python3.6/socketserver.py:finish_request:361/datas/progs/odoo/odoo/osv/query.py:add_join:136 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:set_all_attrs:360 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:__init__:660 (1 samples, 0.76%)/usr/local/lib/python3.6/re.py:_compile:301 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:_compute_value:1008 (18 samples, 13.64%)/datas/progs/odoo/od../datas/progs/odoo/odoo/fields.py:digits:1209 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/api.py:get:959 (1 samples, 0.76%)/datas/progs/odoo/odoo/cli/server.py:run:175 (132 samples, 100.00%)/datas/progs/odoo/odoo/cli/server.py:run:175/datas/progs/odoo/odoo/fields.py:determine_value:1059 (18 samples, 13.64%)/datas/progs/odoo/od../datas/progs/odoo/odoo/models.py:_read_from_database:2646 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:search:1407 (6 samples, 4.55%)/data../datas/progs/odoo/addons/stock_account/wizard/stock_change_standard_price.py:change_price:43 (126 samples, 95.45%)/datas/progs/odoo/addons/stock_account/wizard/stock_change_standard_price.py:change_price:43/datas/progs/odoo/odoo/models.py:compute_concurrency_field_with_access:356 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:60 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_search:3674 (2 samples, 1.52%)/datas/progs/odoo/odoo/models.py:read_group:1828 (3 samples, 2.27%)/../usr/local/lib/python3.6/_weakrefset.py:add:84 (1 samples, 0.76%)/datas/progs/odoo/odoo/osv/expression.py:get_unaccent_wrapper:455 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__new__:735 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:__set__:966 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/api.py:__new__:735 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:__call__:789 (3 samples, 2.27%)/../datas/progs/odoo/odoo/tools/func.py:wrapper:68 (1 samples, 0.76%)/datas/progs/odoo/odoo/service/server.py:start:991 (132 samples, 100.00%)/datas/progs/odoo/odoo/service/server.py:start:991/datas/progs/odoo/odoo/modules/registry.py:field_sequence:200 (1 samples, 0.76%)/datas/progs/odoo/odoo/tools/func.py:wrapper:68 (1 samples, 0.76%)<string>:change_digit:9 (3 samples, 2.27%)<../datas/progs/odoo/odoo/models.py:sudo:4310 (6 samples, 4.55%)/data../datas/progs/odoo/odoo/api.py:__new__:736 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_read_group_raw:1851 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/models.py:read:2520 (1 samples, 0.76%)/datas/progs/odoo/addons/product/models/product.py:name_get:376 (3 samples, 2.27%)/..<string>:<module>:4 (1 samples, 0.76%)/usr/local/lib/python3.6/socketserver.py:process_request:348 (132 samples, 100.00%)/usr/local/lib/python3.6/socketserver.py:process_request:348/datas/progs/odoo/odoo/api.py:get_records:1009 (27 samples, 20.45%)/datas/progs/odoo/odoo/api.py:ge..<frozen importlib._bootstrap>:_find_and_load:971 (1 samples, 0.76%)/datas/progs/odoo/odoo/tools/func.py:__get__:23 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:__getitem__:4657 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (2 samples, 1.52%)/datas/progs/odoo/odoo/api.py:__new__:736 (7 samples, 5.30%)/datas../datas/progs/odoo/odoo/osv/expression.py:__init__:660 (1 samples, 0.76%)/usr/local/lib/python3.6/_weakrefset.py:__iter__:61 (1 samples, 0.76%)/datas/progs/odoo/odoo/api.py:user:798 (3 samples, 2.27%)/../datas/progs/odoo/odoo/models.py:_create:3435 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:_compute_value:1006 (96 samples, 72.73%)/datas/progs/odoo/odoo/fields.py:_compute_value:1006/datas/progs/odoo/odoo/models.py:_normalize_ids:5197 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:setup_base:389 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:sudo:4310 (4 samples, 3.03%)/da../datas/progs/odoo/odoo/models.py:with_context:4333 (8 samples, 6.06%)/datas/p../datas/progs/odoo/odoo/modules/loading.py:load_module_graph:121 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_browse:4218 (1 samples, 0.76%)<string>:change_digit:9 (7 samples, 5.30%)<strin../datas/progs/odoo/odoo/addons/base/res/ir_property.py:_get_domain:138 (3 samples, 2.27%)/../datas/progs/odoo/odoo/fields.py:compute_value:1015 (18 samples, 13.64%)/datas/progs/odoo/od..<string>:change_digit:9 (6 samples, 4.55%)<stri../usr/local/lib/python3.6/site-packages/werkzeug/serving.py:execute:181 (132 samples, 100.00%)/usr/local/lib/python3.6/site-packages/werkzeug/serving.py:execute:181/datas/progs/odoo/odoo/api.py:<listcomp>:1010 (27 samples, 20.45%)/datas/progs/odoo/odoo/api.py:<l../datas/progs/odoo/addons/web/controllers/main.py:call_kw:924 (1 samples, 0.76%)/datas/progs/odoo/odoo/models.py:_setup_base:2312 (1 samples, 0.76%)/datas/progs/odoo/odoo/fields.py:digits:1209 (7 samples, 5.30%)/datas.. \ No newline at end of file diff --git a/content/developer/reference/backend.rst b/content/developer/reference/backend.rst index ead2caa7c..16938a300 100644 --- a/content/developer/reference/backend.rst +++ b/content/developer/reference/backend.rst @@ -14,6 +14,7 @@ Python framework backend/module backend/reports backend/security + backend/performance backend/testing backend/http backend/mixins diff --git a/content/developer/reference/backend/performance.rst b/content/developer/reference/backend/performance.rst new file mode 100644 index 000000000..1167d9e17 --- /dev/null +++ b/content/developer/reference/backend/performance.rst @@ -0,0 +1,540 @@ +:custom-css: performance.css + +=========== +Performance +=========== + +.. _performance/profiling: + +Profiling +========= + +.. currentmodule:: odoo.tools.profiler + +Profiling is about analysing the execution of a program and measure aggregated data. These data can +be the elapsed time for each function, the executed SQL queries... + +While profiling does not improve the performance of a program by itself, it can prove very helpful +in finding performance issues and identifying which part of the program is responsible for them. + +Odoo provides an integrated profiling tool that allows recording all executed queries and stack +traces during execution. It can be used to profile either a set of requests of a user session, or a +specific portion of code. Profiling results can be either inspected with the integrated `speedscope +`_ :dfn:`open source app allowing to visualize a flamegraph` +view or analyzed with custom tools by first saving them in a JSON file or in the database. + +.. _performance/profiling/enable: + +Enable the profiler +------------------- + +The profiler can either be enabled from the user interface, which is the easiest way to do so but +allows profiling only web requests, or from Python code, which allows profiling any piece of code +including tests. + +.. tabs:: + + .. tab:: Enable from the user interface + + #. :ref:`Enable the developer mode `. + #. Before starting a profiling session, the profiler must be enabled globally on the database. + This can be done in two ways: + + - Open the :ref:`developer mode tools `, then toggle the + :guilabel:`Enable profiling` button. A wizard suggests a set of expiry times for the + profiling. Click on :guilabel:`ENABLE PROFILING` to enable the profiler globally. + + .. image:: performance/enable_profiling_wizard.png + + - Go to :guilabel:`Settings --> General Settings --> Performance` and set the desired time to + the field :guilabel:`Enable profiling until`. + + #. After the profiler is enabled on the database, users can enable it on their session. To do + so, toggle the :guilabel:`Enable profiling` button in the :ref:`developer mode tools + ` again. By default, the recommended options :guilabel:`Record + sql` and :guilabel:`Record traces` are enabled. To learn more about the different options, + head over to :ref:`performance/profiling/collectors`. + + .. image:: performance/profiling_debug_menu.png + + When the profiler is enabled, all the requests made to the server are profiled and saved into + an `ir.profile` record. Such records are grouped into the current profiling session which + spans from when the profiler was enabled until it is disabled. + + .. note:: + Odoo Online (SaaS) databases cannot be profiled. + + .. tab:: Enable from Python code + + Starting the profiler manually can be convenient to profile a specific method or a part of the + code. This code can be a test, a compute method, the entire loading, etc. + + To start the profiler from Python code, call it as a context manager. You may specify *what* + you want to record through the parameters. A shortcut is available for profiling test classes: + :code:`self.profile()`. See :ref:`performance/profiling/collectors` for more information on + the `collectors` parameter. + + .. example:: + + .. code-block:: python + + with Profiler(): + do_stuff() + + .. example:: + + .. code-block:: python + + with Profiler(collectors=['sql', PeriodicCollector(interval=0.1)]): + do_stuff() + + .. example:: + + .. code-block:: python + + with self.profile(): + with self.assertQueryCount(__system__=1211): + do_stuff() + + .. note:: + The profiler is called outside of the `assertQueryCount` in order to catch queries made + when exiting the context manager (e.g., flush). + + .. autoclass:: Profiler() + :special-members: __init__ + + When the profiler is enabled, all executions of a test method are profiled and saved into an + `ir.profile` record. Such records are grouped into a single profiling session. This is + especially useful when using the :code:`@warmup` and :code:`@users` decorators. + + .. tip:: + It can be complicated to analyze profiling results of a method that is called several times + because all the calls are grouped together in the stack trace. Add an **execution context** + as a context manager to break down the results into multiple frames. + + .. example:: + + .. code-block:: python + + for index in range(max_index): + with ExecutionContext(current_index=index): # Identify each call in speedscope results. + do_stuff() + +.. _performance/profiling/analyse: + +Analyse the results +------------------- + +To browse the profiling results, make sure that the :ref:`profiler is enabled globally on the +database `, then open the :ref:`developer mode tools +` and click on the button in the top-right corner of the profiling +section. A list view of the `ir.profile` records grouped by profiling session opens. + +.. image:: performance/profiling_web.png + :align: center + +Each record has a clickable link that opens the speedscope results in a new tab. + +.. image:: performance/flamegraph_example.png + :align: center + +Speedscope falls out of the scope of this documentation but there are a lot of tools to try: search, +highlight of similar frames, zoom on frame, timeline, left heavy, sandwich view... + +Depending on the profiling options that were activated, Odoo generates different view modes that you +can access from the top menu. + +.. image:: performance/speedscope_modes.png + :align: center + +- The :guilabel:`Combined` view shows all the SQL queries and traces merged togethers. +- The :guilabel:`Combined no context` view shows the same result but ignores the saved execution + context `. +- The :guilabel:`sql (no gap)` view shows all the SQL queries as if they were executed one after + another, without any Python logic. This is useful for optimizing SQL only. +- The :guilabel:`sql (density)` view shows only all the SQL queries, leaving gap between them. This + can be useful to spot if eiter SQL or Python code is the problem, and to identify zones in where + many small queries could be batched. +- The :guilabel:`frames` view shows the results of only the :ref:`periodic collector + `. + +.. important:: + Even though the profiler has been designed to be as light as possible, it can still impact + performance, especially when using the :ref:`Sync collector + `. Keep that in mind when analyzing speedscope results. + +.. _performance/profiling/collectors: + +Collectors +---------- + +Whereas the profiler is about the *when* of profiling, the collectors take care of the *what*. + +Each collector specializes in collecting profiling data in its own format and manner. They can be +individually enabled from the user interface through their dedicated toggle button in the +:ref:`developer mode tools `, or from Python code through their key or +class. + +There are currently four collectors available in Odoo: + +.. list-table:: + :header-rows: 1 + + * - Name + - Toggle button + - Python key + - Python class + * - :ref:`SQL collector ` + - :guilabel:`Record sql` + - `sql` + - `SqlCollector` + * - :ref:`Periodic collector ` + - :guilabel:`Record traces` + - `traces_async` + - `PeriodicCollector` + * - :ref:`QWeb collector ` + - :guilabel:`Record qweb` + - `qweb` + - `QwebCollector` + * - :ref:`Sync collector ` + - No + - `traces_sync` + - `SyncCollector` + +By default, the profiler enables the SQL and the Periodic collectors. Both when it is enabled from +the user interface or Python code. + +.. _performance/profiling/collectors/sql: + +SQL collector +~~~~~~~~~~~~~ + +The SQL collector saves all the SQL queries made to the database in the current thread (for all +cursors), as well as the stack trace. The overhead of the collector is added to the analysed thread +for each query, which means that using it on a lot of small queries may impact execution time and +other profilers. + +It is especially useful to debug query counts, or to add information to the :ref:`Periodic collector +` in the combined speedscope view. + +.. autoclass:: SQLCollector + +.. _performance/profiling/collectors/periodic: + +Periodic collector +~~~~~~~~~~~~~~~~~~ + +This collector runs in a separate thread and saves the stack trace of the analysed thread at every +interval. The interval (by default 10 ms) can be defined through the :guilabel:`Interval` option in +the user interface, or the `interval` parameter in Python code. + +.. warning:: + If the interval is set at a very low value, profiling long requests will generate memory issues. + If the interval is set at a very high value, information on short function executions will be + lost. + +It is one of the best way to analyse performance as it should have a very low impact on the +execution time thanks to its separate thread. + +.. autoclass:: PeriodicCollector + +.. _performance/profiling/collectors/qweb: + +QWeb collector +~~~~~~~~~~~~~~ + +This collector saves the Python execution time and queries of all directives. As for the :ref:`SQL +collector `, the overhead can be important when executing a +lot of small directives. The results are different from other collectors in terms of collected data, +and can be analysed from the `ir.profile` form view using a custom widget. + +It is mainly useful for optimizing views. + +.. autoclass:: QwebCollector + +.. _performance/profiling/collectors/sync: + +Sync collector +~~~~~~~~~~~~~~ + +This collector saves the stack for every function's call and return and runs on the same thread, +which greatly impacts performance. + +It can be useful to debug and understand complex flows, and follow their execution in the code. It +is however not recommended for performance analysis because the overhead is high. + +.. autoclass:: SyncCollector + +.. _performance/profiling/pitfalls: + +Performance pitfalls +-------------------- + +- Be careful with randomness. Multiple executions may lead to different results. E.g., a garbage + collector being triggered during execution. +- Be careful with blocking calls. In some cases, external `c_call` may take some time before + releasing the GIL, thus leading to unexpected long frames with the :ref:`Periodic collector + `. This should be detected by the profiler and give a + warning. It is possible to trigger the profiler manually before such calls if needed. +- Pay attention to the cache. Profiling before that the `view`/`assets`/... are in cache can lead to + different results. +- Be aware of the profiler's overhead. The :ref:`SQL collector + `'s overhead can be important when a lot of small queries + are executed. Profiling is practical to spot a problem but you may want to disable the profiler in + order to measure the real impact of a code change. +- Profiling results can be memory intensive. In some cases (e.g., profiling an install or a long + request), it is possible that you reach memory limit, especially when rendering the speedscope + results, which can lead to an HTTP 500 error. In this case, you may need to start the server with + a higher memory limit: `--limit-memory-hard $((8*1024**3))`. + +.. _reference/performance/populate: + +Database population +=================== + +Odoo CLI offers a :ref:`database population ` feature through the CLI +command :command:`odoo-bin populate`. + +Instead of the tedious manual, or programmatic, specification of test data, one can use this feature +to fill a database on demand with the desired number of test data. This can be used to detect +diverse bugs or performance issues in tested flows. + +.. _reference/performance/populate/methods: + +To populate a given model, the following methods and attributes can be defined. + +.. currentmodule:: odoo.models + +.. autoattribute:: Model._populate_sizes +.. autoattribute:: Model._populate_dependencies +.. automethod:: Model._populate +.. automethod:: Model._populate_factories + +.. note:: + You have to define at least :meth:`~odoo.models.Model._populate` or + :meth:`~odoo.models.Model._populate_factories` on the model to enable database population. + +.. example:: + .. code-block:: python + + from odoo.tools import populate + + class CustomModel(models.Model) + _inherit = "custom.some_model" + _populate_sizes = {"small": 100, "medium": 2000, "large": 10000} + _populate_dependencies = ["custom.some_other_model"] + + def _populate_factories(self): + # Record ids of previously populated models are accessible in the registry + some_other_ids = self.env.registry.populated_models["custom.some_other_model"] + + def get_some_field(values=None, random=None, **kwargs): + """ Choose a value for some_field depending on other fields values. + + :param dict values: + :param random: seeded :class:`random.Random` object + """ + field_1 = values['field_1'] + if field_1 in [value2, value3]: + return random.choice(some_field_values) + return False + + return [ + ("field_1", populate.randomize([value1, value2, value3])), + ("field_2", populate.randomize([value_a, value_b], [0.5, 0.5])), + ("some_other_id", populate.randomize(some_other_ids)), + ("some_field", populate.compute(get_some_field, seed="some_field")), + ('active', populate.cartesian([True, False])), + ] + + def _populate(self, size): + records = super()._populate(size) + + # If you want to update the generated records + # E.g setting the parent-child relationships + records.do_something() + + return records + +Population tools +---------------- + +Multiple population tools are available to easily create the needed data generators. + +.. automodule:: odoo.tools.populate + :members: cartesian, compute, constant, iterate, randint, randomize + +.. _performance/good_practices: + +Good practices +============== + +.. _performance/good_practices/batch: + +Batch operations +---------------- + +When working with recordsets, it is almost always better to batch operations. + +.. example:: + Don't call a method that runs SQL queries while looping over a recordset because it will do so + for each record of the set. + + .. rst-class:: bad-example + .. code-block:: python + + def _compute_count(self): + for record in self: + domain = [('related_id', '=', record.id)] + record.count = other_model.search_count(domain) + + Instead, replace the `search_count` with a `read_group` to execute one SQL query for the entire + batch of records. + + .. rst-class:: good-example + .. code-block:: python + + def _compute_count(self): + if self.ids: + domain = [('related_id', 'in', self.ids)] + counts_data = other_model.read_group(domain, ['related_id'], ['related_id']) + mapped_data = { + count['related_id'][0]: count['related_id_count'] for count in counts_data + } + else: + mapped_data = {} + for record in self: + record.count = mapped_data.get(record.id, 0) + + .. note:: + This example is not optimal nor correct in all cases. It is only a substitute for a + `search_count`. Another solution could be to prefetch and count the inverse `One2many` field. + +.. example:: + Don't create records one after another. + + .. rst-class:: bad-example + .. code-block:: python + + for name in ['foo', 'bar']: + model.create({'name': name}) + + Instead, accumulate the create values and call the `create` method on the batch. Doing so has + mostly no impact and helps the framework optimize fields computation. + + .. rst-class:: good-example + .. code-block:: python + + create_values = [] + for name in ['foo', 'bar']: + create_values.append({'name': name}) + records = model.create(create_values) + +.. example:: + Fail to prefetch the fields of a recordset while browsing a single record inside a loop. + + .. rst-class:: bad-example + .. code-block:: python + + for record_id in record_ids: + model.browse(record_id) + record.foo # One query is executed per record. + + Instead, browse the entire recordset first. + + .. rst-class:: good-example + .. code-block:: python + + records = model.browse(record_ids) + for record in records: + record.foo # One query is executed for the entire recordset. + + We can verify that the records are prefetched in batch by reading the field `prefetch_ids` which + includes each of the record ids.browsing all records together is unpractical, + + If needed, the `with_prefetch` method can be used to disable batch prefetching: + + .. code-block:: python + + for values in values_list: + message = self.browse(values['id']).with_prefetch(self.ids) + +.. _performance/good_practices/algorithmic_complexity: + +Reduce the algorithmic complexity +--------------------------------- + +Algorithmic complexity is a measure of how long an algorithm would take to complete in regard to the +size `n` of the input. When the complexity is high, the execution time can grow quickly as the input +becomes larger. In some cases, the algorithmic complexity can be reduced by preparing the input's +data correctly. + +.. example:: + For a given problem, let's consider a naive algorithm crafted with two nested loops for which the + complexity in in O(n²). + + .. rst-class:: bad-example + .. code-block:: python + + for record in self: + for result in results: + if results['id'] == record.id: + record.foo = results['foo'] + break + + Assuming that all results have a different id, we can prepare the data to reduce the complexity. + + .. rst-class:: good-example + .. code-block:: python + + mapped_result = {result['id']: result['foo'] for result in results} + for record in self: + record.foo = mapped_result.get(record.id) + +.. example:: + Choosing the bad data structure to hold the input can lead to quadratic complexity. + + .. rst-class:: bad-example + .. code-block:: python + + invalid_ids = self.search(domain).ids + for record in self: + if record.id in invalid_ids: + ... + + If `invalid_ids` is a list-like data structure, the complexity of the algorithm may be quadratic. + + Instead, prefer using set operations like casting `invalid_ids` to a set. + + .. rst-class:: good-example + .. code-block:: python + + invalid_ids = set(invalid_ids) + for record in self: + if record.id in invalid_ids: + ... + + Depending on the input, recordset operations can also be used. + + .. rst-class:: good-example + .. code-block:: python + + invalid_ids = self.search(domain) + for record in self - invalid_ids: + ... + +.. _performance/good_practices/index: + +Use indexes +----------- + +Database indexes can help fasten search operations, be it from a search in the or through the user +interface. + +.. code-block:: python + + name = fields.Char(string="Name", index=True) + +.. warning:: + Be careful not to index every field as indexes consume space and impact on performance when + executing one of `INSERT`, `UPDATE`, and `DELETE`. diff --git a/content/developer/reference/backend/performance/enable_profiling_wizard.png b/content/developer/reference/backend/performance/enable_profiling_wizard.png new file mode 100644 index 000000000..7fa2870d3 Binary files /dev/null and b/content/developer/reference/backend/performance/enable_profiling_wizard.png differ diff --git a/content/developer/reference/backend/performance/flamegraph_example.png b/content/developer/reference/backend/performance/flamegraph_example.png new file mode 100644 index 000000000..6fe54aaa8 Binary files /dev/null and b/content/developer/reference/backend/performance/flamegraph_example.png differ diff --git a/content/developer/reference/backend/performance/profiling_debug_menu.png b/content/developer/reference/backend/performance/profiling_debug_menu.png new file mode 100644 index 000000000..ac5af3f6f Binary files /dev/null and b/content/developer/reference/backend/performance/profiling_debug_menu.png differ diff --git a/content/developer/reference/backend/performance/profiling_example_after.png b/content/developer/reference/backend/performance/profiling_example_after.png new file mode 100644 index 000000000..b832aa4ca Binary files /dev/null and b/content/developer/reference/backend/performance/profiling_example_after.png differ diff --git a/content/developer/reference/backend/performance/profiling_example_before.png b/content/developer/reference/backend/performance/profiling_example_before.png new file mode 100644 index 000000000..a61f8eb1a Binary files /dev/null and b/content/developer/reference/backend/performance/profiling_example_before.png differ diff --git a/content/developer/reference/backend/performance/profiling_web.png b/content/developer/reference/backend/performance/profiling_web.png new file mode 100644 index 000000000..a636e02ac Binary files /dev/null and b/content/developer/reference/backend/performance/profiling_web.png differ diff --git a/content/developer/reference/backend/performance/speedscope_modes.png b/content/developer/reference/backend/performance/speedscope_modes.png new file mode 100644 index 000000000..e08a70620 Binary files /dev/null and b/content/developer/reference/backend/performance/speedscope_modes.png differ diff --git a/content/developer/reference/backend/testing.rst b/content/developer/reference/backend/testing.rst index f89425ee9..b10bd8807 100644 --- a/content/developer/reference/backend/testing.rst +++ b/content/developer/reference/backend/testing.rst @@ -732,90 +732,6 @@ you can use the :meth:`~odoo.tests.common.BaseCase.assertQueryCount` method, int with self.assertQueryCount(11): do_something() -.. _reference/testing/populate: - -Database population -------------------- - -Odoo CLI offers a :ref:`database population` feature. - -.. code-block:: console - - odoo-bin populate - -Instead of the tedious manual, or programmatic, specification of test data, -one can use this feature to fill a database on demand with the desired number of test data. -This can be used to detect diverse bugs or performance issues in tested flows. - -.. _reference/testing/populate/methods: - -To specify this feature for a given model, the following methods and attributes can be defined. - -.. currentmodule:: odoo.models - -.. autoattribute:: Model._populate_sizes -.. autoattribute:: Model._populate_dependencies -.. automethod:: Model._populate -.. automethod:: Model._populate_factories - -.. note:: - - You have to define at least :meth:`~odoo.models.Model._populate` or :meth:`~odoo.models.Model._populate_factories` - on the model to enable database population. - -Example model -~~~~~~~~~~~~~ - -.. code-block:: python - - from odoo.tools import populate - - class CustomModel(models.Model) - _inherit = "custom.some_model" - _populate_sizes = {"small": 100, "medium": 2000, "large": 10000} - _populate_dependencies = ["custom.some_other_model"] - - def _populate_factories(self): - # Record ids of previously populated models are accessible in the registry - some_other_ids = self.env.registry.populated_models["custom.some_other_model"] - - def get_some_field(values=None, random=None, **kwargs): - """ Choose a value for some_field depending on other fields values. - - :param dict values: - :param random: seeded :class:`random.Random` object - """ - field_1 = values['field_1'] - if field_1 in [value2, value3]: - return random.choice(some_field_values) - return False - - return [ - ("field_1", populate.randomize([value1, value2, value3])), - ("field_2", populate.randomize([value_a, value_b], [0.5, 0.5])), - ("some_other_id", populate.randomize(some_other_ids)), - ("some_field", populate.compute(get_some_field, seed="some_field")), - ('active', populate.cartesian([True, False])), - ] - - def _populate(self, size): - records = super()._populate(size) - - # If you want to update the generated records - # E.g setting the parent-child relationships - records.do_something() - - return records - -Population tools -~~~~~~~~~~~~~~~~ - -Multiple population tools are available to easily create -the needed data generators. - -.. automodule:: odoo.tools.populate - :members: cartesian, compute, constant, iterate, randint, randomize - .. _qunit: https://qunitjs.com/ .. _qunit_config.js: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/qunit_config.js#L49 .. _web.tests_assets: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/views/webclient_templates.xml#L594 diff --git a/static/css/performance.css b/static/css/performance.css new file mode 100644 index 000000000..25b6af75d --- /dev/null +++ b/static/css/performance.css @@ -0,0 +1,7 @@ +.bad-example { + border-left: 3px solid red; +} + +.good-example { + border-left: 3px solid green; +}