Commit Graph

1424 Commits

Author SHA1 Message Date
Xavier-Do
e0856b2245 [IMP] runbot: improve build_error management
The build error view was unstructured and contains to much information.

This commit organize fields in groups and also validate some
modification on records in order to avoid build error manager to
disable test-tags by mistake.

An error cannot be deactivated if it appeared less than 24 hours ago to
avoid disabling a non forxardported pr that will fail the next nightly
generating another build error.

Test tags can only be added or removed by administrators.

Also adds a menu for easier User managerment

Also fixed the dname search and display.
2022-11-08 14:43:43 +01:00
Christophe Monniez
ac010405dc [FIX] runbot: use gethostname instead of getfqdn
At boot time, it appears that when the runbot tries to get its hostname
it may get a wrong fqdn.
2022-11-07 10:12:15 +01:00
Christophe Monniez
964a88cb36 [FIX] runbot: take number of builds from config data
Oversight from previous improvement in b79c4a5a52.
2022-11-07 10:12:15 +01:00
Christophe Monniez
04459cdda7 [FIX] runbot: proper line feed in default odoorc 2022-11-07 10:12:15 +01:00
Xavier Morel
1449937e00 [ADD] mergebot: support for coverage during tests
Runs the test instances of Odoo using `coverage` in parallel mode.

- useful for finding out under-tested parts of the code
- because it only instruments mergeport/forwardport, and the tests do
  so much IO, the wallclock performance impacts are minimal (~2%
  increase with branch coverage analysis, for an increase in CPU
  of ~20%, for the entire testsuite)
- for reporting, the scattered coverage reports need to be aggregated
  using `coverage combine`, followed by rendering with `coverage
  html`, these work out of the box, no parameterization is necessary
- coverage does not run on the test suite, only the modules under test
2022-10-27 11:25:25 +02:00
Xavier Morel
13f239826e [FIX] forwardport: avoid logging git error if there's no stream data
If no stream data was captured (no stderr and no stdout), would just
log

    git call error:

as error, with no further information.

Don't do that if we have neither stderr nor stdout data, since we're
re-raising the exception anyway, it's just confusing.
2022-10-26 14:47:00 +02:00
Xavier Morel
22c3406659 [FIX] forwardport: error reporting when git command fails
- if stderr was empty or had been redirected to stdout, no useful
  information would be show, making debugging more complicated
- the fallback is the error itself, but since it's reraised odds are
  pretty high the caller will eventually log the error itself, so
  it's redundant

=> fallback to stdout if stderr is empty, and only log if either is
non-empty
2022-10-26 14:47:00 +02:00
Xavier Morel
6281c86d5e [FIX] conftest: local webhook support
Not sure how I missed this but apparently pytest fixtures can't return
or yield, it's one or the other.

Which makes a lot of sense but means the tunnel fixture was broken
when using local webhooks (= no tunnel) as it returned the local url
rather than yield it.
2022-10-26 14:47:00 +02:00
Xavier ALT
fffc27d2fa [FIX] runbot: fix creation of new runbot.version from the backend
Traceback (most recent call last):
  File "/home/odoo/src/odoo/15.0/odoo/addons/base/models/ir_http.py", line 237, in _dispatch
    result = request.dispatch()
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 687, in dispatch
    result = self._call_function(**self.params)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 359, in _call_function
    return checked_call(self.db, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/service/model.py", line 94, in wrapper
    return f(dbname, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 348, in checked_call
    result = self.endpoint(*a, **kw)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 916, in __call__
    return self.method(*args, **kw)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 535, in response_wrap
    response = f(*args, **kw)
  File "/home/odoo/src/odoo/15.0/addons/web/controllers/main.py", line 1347, in call_kw
    return self._call_kw(model, method, args, kwargs)
  File "/home/odoo/src/odoo/15.0/addons/web/controllers/main.py", line 1339, in _call_kw
    return call_kw(request.env[model], method, args, kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/api.py", line 464, in call_kw
    result = _call_kw_multi(method, model, args, kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/api.py", line 451, in _call_kw_multi
    result = method(recs, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6489, in onchange
    snapshot1 = Snapshot(record, nametree)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6271, in __init__
    self.fetch(name)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6281, in fetch
    self[name] = record[name]
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 5888, in __getitem__
    return self._fields[key].__get__(self, type(self))
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1054, in __get__
    self.recompute(record)
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1243, in recompute
    self.compute_value(recs)
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1265, in compute_value
    records._compute_field_value(self)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 4255, in _compute_field_value
    getattr(self, field.compute)()
  File "/home/odoo/runbot/extra/runbot/models/version.py", line 36, in _compute_version_number
    version.number = '.'.join([elem.zfill(2) for elem in re.sub(r'[^0-9\.]', '', version.name).split('.')])
  File "/usr/lib/python3.8/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 643, in _handle_exception
    return super(JsonRequest, self)._handle_exception(exception)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 301, in _handle_exception
    raise exception.with_traceback(None) from new_cause
TypeError: expected string or bytes-like object
2022-10-21 11:59:55 +02:00
Xavier ALT
1095b25270 [FIX] runbot: fix help msg, 'additionnal_env' is split on semi-column 2022-10-21 11:59:55 +02:00
Christophe Monniez
309aeaa32e [IMP] runbot: speedup build garbage collecting
When the builds directory is filled with a lot of build directories
(around 100000) the garbage collection process may take up to 2 minutes.
The root cause is that each build directory is scanned to clean it up
even if it was already cleaned.

With this commit, a stamp file is used to mark directories that were
already garbage collected.
2022-10-21 11:32:46 +02:00
Christophe Monniez
7642bffda3 [FIX] runbot: download phantomjs from nightly server
In order to avoid bitbucket rate limiting, we prefer to download this
old pal from our server.
2022-10-21 11:17:05 +02:00
Xavier-Do
903ee7d983 [FIX] runbot: manage falsy value for frontend_url 2022-10-21 10:39:20 +02:00
Xavier-Do
e27a1b8f71 [IMP] runbot: cleaup settings view 2022-10-21 10:32:40 +02:00
Xavier-Do
0287dcaab7 [IMP] runbot: update install documentation 2022-10-21 10:32:40 +02:00
Xavier-Do
303638e507 [FIX] runbot: don't hardcode user odoo 2022-10-21 10:32:40 +02:00
Xavier-Do
e4af8d7b6b [FIX] runbot: remove fqdn calls 2022-10-21 10:32:40 +02:00
Xavier-Do
785e2dc3bc [IMP] runbot: don't use local odoorc 2022-10-21 10:32:40 +02:00
Xavier-Do
bfc75cd7fe [IMP] runbot: don't active cron by default 2022-10-21 10:32:40 +02:00
Xavier-Do
50a7120949 [FIX] runbot: don't crash when there is no accessible project 2022-10-21 10:32:40 +02:00
Xavier-Do
16e71d064a [FIX] runbot: remove useless/invalid check 2022-10-21 10:32:40 +02:00
Xavier-Do
b79c4a5a52 [IMP] runbot: various build_config improvements 2022-10-21 10:32:40 +02:00
Xavier-Do
7bb2f06501 [IMP] runbot: make build_error button visible for users 2022-10-21 10:32:40 +02:00
Xavier-Do
b606a2199b [IMP] runbot: improve build error display in logs 2022-10-21 10:32:40 +02:00
Xavier-Do
831f5f7197 [IMP] runbot: make link to backend have the runbot menu 2022-10-21 10:32:40 +02:00
Xavier-Do
eaf8964701 [IMP] runbot: change trigger start logic
and add sequence on project
2022-10-21 10:32:40 +02:00
Xavier-Do
02eb8014a3 [IMP] runbot: make build_error triggers searchable 2022-10-21 10:32:40 +02:00
Xavier-Do
a294348cd0 [IMP] runbot: improve views 2022-10-21 10:32:40 +02:00
xmo-odoo
d72b17862e
[MERGE] from 15.0
Forward-port support for no tunnel.
2022-08-24 11:20:45 +02:00
xmo-odoo
d5095dac63
[MERGE] from 14.0
Forward-port support for no tunnel.
2022-08-24 11:19:41 +02:00
xmo-odoo
da14496e13
[IMP] conftest: support local webhooking
With dummy central now kinda sorta working, it makes sense to avoid going
through a tunnel for webhooks, especially as ngrok has lowered the number of
tunnels to 2 per agent for free accounts (or possibly fixed the bug which made
it not be enforced?)
2022-08-24 11:17:01 +02:00
Xavier Morel
389ea03544 [MERGE] from 14.0
Get mergebot updates from since the runbot was upgraded.

Also convert assets to new system for compatibility.
2022-08-24 08:40:18 +02:00
Xavier-Do
70e8b15690 [IMP] runbot: add websocket support 2022-08-23 16:30:24 +02:00
Xavier-Do
e91611b0f8 [FIX] runbot: reduce load when builder are idle
The sleep 1 was ok with a few builder bur regarding the number of
request on the database when no build are running, this become
problematic.

An ideal solution would be to detect if
-> me managed some testing build
-> There is load (pendings)

In both case, we don't want to sleep to much.
In other cases, we may want to wait a little longer.

A simple quick fix will just wait longer in all cases.
2022-08-23 15:54:50 +02:00
Xavier Morel
65c2ffc997 [MERGE] from 13.0
Get mergebot updates from since the runbot was upgraded.

NOTE: updates forwardport.models.forwardport.Queue with slots for
compatibility with commit
odoo/odoo@ea3e39506a "use slots for
BaseModel", otherwise we get

    TypeError: __bases__ assignment: 'ForwardPortTasks' object layout differs from 'BaseModel'
2022-08-23 14:41:35 +02:00
Xavier-Do
b44ed5f7a6 [IMP] runbot: active trigger and filter manual
- dont display manual trigger having a team if not member of that team
- add an active field on triggers
2022-08-17 16:50:23 +02:00
Xavier-Do
326ac36b5e [FIX] runbot: display stats of failed and scheduled builds 2022-08-17 12:18:40 +02:00
Xavier-Do
35e3e7cee0 [FIX] runbot: make Upgrade exception message no update and step id clickable 2022-08-17 11:46:47 +02:00
Xavier-Do
d4f1f01b46 [IMP] runbot: add buttons to compare with next line on build_search page 2022-08-17 11:46:47 +02:00
Xavier-Do
05c0c29e0f [IMP] runbot: add bundle name in link title 2022-08-17 11:43:53 +02:00
Romain Derie
8af81d72bb [IMP] runbot: add sort and filter capability on the build error page
Before this commit, the build errors page was neither sortable neither
filterable.

This commit adds a way to filter by:
- all
- unassigned
- seen more than once

It also allows to sort by:
- last seen
- nb seen

Typical need is to sort by nb seen and filter out the only seen once to
be able to figure which one is supposed to be checked in priority.
2022-08-17 11:42:36 +02:00
Xavier-Do
a382977d39 [IMP] runbot: add teams on trigger.
This will be mainy usefull in trigger categories custom view and for
dashboards.
2022-08-17 11:40:56 +02:00
Xavier-Do
5f2fe9a451 [FIX] runbot: mitigate docker status race condition
In some rare cases, a docker container has a status of exited, removed
or in removal and the `end-`file has been written right after the code
checked the existence of the file.

To mitigate the issue, this commit checks the `end-` file existence
after the container status have been checked. That way, if the file
exists we can be pretty confident that the build really ended.
2022-08-17 11:40:12 +02:00
Xavier Morel
ebbe77b849 [IMP] runbot_merge: reorg test
Test seems to fail from time to time with one of the PRs getting
lost. Tried to move code around trying to investigate, can't repro
anymore. Possibly a race condition because the `to_pr` call was
performed too early, before the webhook had run (and thus before the
PR object had been created on the odoo side).

By moving the `to_pr` calls to after the cron run, we really ensure
the webhooks will have run.

Also update `to_pr` to ensure exactly one PR was retrieved, as
currently nothing is checked so we might have gotten none (yet), which
should be noticed early and clearly. In theory this also guards
against multiple PRs, but PRs should be unique on (repo, number).
2022-08-05 15:35:51 +02:00
Xavier Morel
c85735870a [FIX] runbot_merge: bugs in log of branch stagings (in frontend)
- Some batches in a few stagings are apparently empty (e.g. batch
  71771 from staging 32511), the listing was not resilient to such
  issues.

  Update the code to suppress the display of empty batches.
- The possibility of accessing `/runbot_merge/<id>` with a
  non-existent branch had not been considered and triggered a
  rendering error in the template.

Fixes #630, fixes #631
2022-08-05 15:35:51 +02:00
Xavier Morel
c3b90454a2 [IMP] runbot_merge: a few backend screens
- override the staging's name_get to provide a slightly more useful
  display_name (though still not great as the staging object remains
  quite technical and inimical to human interaction)
- show individual PRs in a batch (as m2m tags) for readability
- update PR views to show the author and reviewer, except in the list
  of delegations of users where it's a lot less useful

/cc #632
2022-08-05 15:35:51 +02:00
Xavier Morel
0c882fc0df [IMP] runbot_merge: automation around branch deactivation
Currently deactivating a branch kinda leaves users in the dark, with
little way to know what has happened aside from inferring it from the
branch having disappeared from the main dashboard.

- surface the state of the branch in the PR dashboard (also surface
  the target branch at all so users can see if their PR is targeted
  as they expect as far as the mergebot is concerned)
- close & notify every PR to a branch being deactivated
- cancel any current staging to the branch (as a consequence of the
  above)

Closes #632
2022-08-05 15:35:51 +02:00
Xavier Morel
e9278c021d [IMP] forwardport: commenting of test_author_can_close_via_fwbot 2022-08-05 15:35:51 +02:00
Xavier Morel
21899af758 [FIX] runbot_merge: brainfart in dashboard warning 2022-08-05 15:35:51 +02:00
Xavier Morel
5176ff094c [FIX] runbot_merge: home page styling
Previously if a branch name could not be line-broken (because it was
full of underscores) it would break the layout by making "its"
staging's column much wider than the expected 1/2 / 1/4 /
1/6 (depending on window width), compressing the width of its
sibling's columns.

By disabling content-based width (only taking in account flex-width)
and setting overflow to hidden, the overlong branch names get cut off
instead.
2022-08-05 15:35:51 +02:00