Commit Graph

953 Commits

Author SHA1 Message Date
David James
1532bcab0f [FIX] fix typo in selection field value 2023-02-28 11:58:34 +01:00
David James
fa3808bf1e [FIX] runbot: fix dictionary changed size during iteration 2023-02-28 11:16:46 +01:00
Xavier-Do
f5caa87da1 [FIX] runbot: fix time related test 2023-02-28 10:59:47 +01:00
Xavier-Do
d3e3228921 [FIX] runbot: get_module remove prefix
Since removeprefix was not available in ubuntu 20.04, a easier alternative
using rebase was used.

The initial assumption was that the prefix would look like `odoo/addons/`
and won't be in the filename.

When the repo is enterprise, the prefix is `enterprise/` meaning that
module name ending with `enterprise` will be truncated

`repo._get_module('enterprise/mail_enterprise/static/src/widgets/form_renderer/form_renderer.js')`

will output

`mail_static`
2023-02-28 10:59:47 +01:00
Xavier-Do
f7a12c6359 [FIX] runbot: fix fetch tests 2023-02-06 10:16:07 +01:00
Xavier-Do
763b1af1a5 [FIX] runbot: disable host_default 2023-01-17 15:24:49 +01:00
Xavier-Do
27ac733df6 [IMP] runbot: automatic base master bundle
When adding a new project, if no branch matches a base name,
the created bundles won't have a version and it will fail.

A simple fix will be to add a master bundle for all projects.
2023-01-17 13:41:01 +01:00
Xavier-Do
9e0f0e6a75 [IMP] runbot: display errors for critical/warn 2023-01-13 15:57:25 +01:00
Christophe Monniez
1a108669ce [IMP] runbot: do not install wkhtml when value is empty 2023-01-13 15:33:05 +01:00
Xavier-Do
12c0a0edf6 [IMP] runbot: automatically transfer responsible
When an error is linked to another one, we don't expect it  to appear on
team and user dashboard. When adding a parent, this will transfer the
responsible from the child to the parent when applicable.
2023-01-13 15:32:17 +01:00
avs
6fffa0264a [IMP] runbot: make runbot.bundle::write() callable through xml-rpc
The XML-RPC implementation does not allow for receiving or sending
`None` values (both as query parameters and response).

Since the `write` method of `runbot.bundle` was overriden without
returning a value, an exception is raised when the method is called
through the external API.

This makes the `write` method return the value from its call
to `super()` which should be equal to `True` if all went well.
2023-01-13 15:32:01 +01:00
Xavier-Do
d011bb7f8c [IMP] runbot: make disable host configurable
The auto disable host is mainly usefull when there are a lot of host for
well configured repositories.

If for any reason a repo is corrupted on one host, this host will be
disabled until a manual intervention cleans the repo.

For other cases, where thjere are many repositories with not so many
host, it is most likely that a fetch will fail because of an invalid
repository configuration. Disabling the host in this case is not a good
idea.

With this commit, a settings allows to enable or disable this feature.
2023-01-13 14:46:02 +01:00
Xavier-Do
ecd105218c [IMP] runbot: parse warning and critical 2023-01-13 14:27:17 +01:00
Xavier-Do
2b8242cf42 [IMP] runbot: default empty value for result 2023-01-13 14:27:17 +01:00
Xavier-Do
6892546d94 [IMP] runbot: fail before contacting github if user cannot write on team 2023-01-13 14:27:17 +01:00
Xavier-Do
cd4c6b5e43 [IMP] runbot: better configure upgrade descriptions 2023-01-13 14:27:17 +01:00
Xavier-Do
37df18d54d [FIX] runbot: use params db_name when provided for upgrades 2023-01-12 11:54:53 +01:00
Xavier-Do
c9c64bccc7 [FIX] runbot: fetch member should be able to access token for users 2023-01-11 13:35:02 +01:00
Xavier-Do
06f9f62b1f [FIX] runbot: fix was alive 2023-01-11 12:41:39 +01:00
Xavier-Do
16fa0e1ad8 [IMP] runbot: add freeze list feature 2023-01-11 11:20:29 +01:00
Xavier-Do
788114776b [IMP] runbot: add python_code filter 2023-01-10 10:48:56 +01:00
Xavier-Do
fdb144a3af [IMP] runbot: allow to ignore forward-ports for a team 2023-01-10 10:47:59 +01:00
Christophe Monniez
1c49aaacf7 [IMP] runbot: add a test for _get_refs 2023-01-10 10:37:06 +01:00
Xavier-Do
56c08be631 [FIX] runbot: filter numerical branch names
Right now, a branch with a numerical name will be added to the database,
but it can conflict with pr since the name of a pr is a number.
This means that a unique (name, repo_id) constraints can be broken.
We could use the 'is_pr' in the unicity constraints to avoid this issue
but searching on branch name will give confusing result if some of them
can be numerical.

Moreover, a runbot branch name should start with the version name
meaning that a numerical branch name was a bad idea from the begining.
2022-12-27 10:37:30 +01:00
Xavier-Do
6d3663e9f6 [IMP] runbot: disable auto-tags from custom trigger 2022-12-26 12:31:09 +01:00
Xavier-Do
910843f06d [FIX]: fix codeowner
A corner case was discovered when a pr is open but the pr is not in the
codeowner commits.
2022-12-24 12:39:55 +01:00
Xavier-Do
ad573d64cf [FIX] runbot: manage falsy path_glob 2022-12-22 14:40:13 +01:00
Xavier-Do
5540f84667 [IMP] runbot: add priority queue for debuging
The main idea is to allow some build to use an extra slot from all host
if a bundle is in priority mode.

This is mainly for quick step debuging, mainly when modifying python
steps when the runbot is fully loaded.

This will also be usefull to concurrency test, by starting a build on
each host at the same time, even when some host are already fully loaded
2022-12-22 14:26:10 +01:00
Christophe Monniez
76cc74fcd9 [IMP] runbot: improve team form and build frontend page
* Hide useless pages on backend team form
* Add a team exceptions page on team form
* Show fixed errors as fixed on front build page
2022-12-22 14:21:59 +01:00
Xavier-Do
356843c680 [IMP] runbot: reduce concurrency
Right now, multiple build are read when managing build to schedule.

This is not usefull since the transaction is commited between each
of them. Moreover, the read build can be written from another host
adding another possibility to have a conccurent update.

Removing the prefetch_ids may help a little.
2022-12-22 11:25:04 +01:00
Xavier-Do
205d299c7d [IMP] runbot: add trigger list to team 2022-12-22 10:47:24 +01:00
Christophe Monniez
9ccf5a2c0e [IMP] runbot: add a reassign team wizard 2022-12-21 15:20:42 +01:00
Xavier-Do
c83f89fc23 [IMP] runbot: documentation and views imps
Add a small documentation for users, mainly about teams and codeowners.
Improves some views and hide some menu_items to keep interface easier
to navigate.
2022-12-21 13:00:42 +01:00
Xavier-Do
ada69222c9 [IMP] runbot: add an action to reassign team automaticaly 2022-12-21 10:58:30 +01:00
Xavier-Do
af7a0751fc [FIX] runbot: avoid recursion in create steps
Since the custom create was introduced, if a config is added in the
config data of a create step, the config can be dynamic. If the given
config contains a create step, this become recursive.

This is fixed in this commit by:
- Checking the parent_path depth in add_child. This will also work for
  python config.
- Consuming the params when adding the child
- Also cleanup the base custom multi config to use a specific step
2022-12-20 09:30:08 +01:00
Xavier-Do
92ed23c237 [FIX] runbot: manage team without github logins 2022-12-19 11:14:07 +01:00
Xavier-Do
25557e6116 [FIX] runbot: manage team without github logins 2022-12-19 11:09:15 +01:00
Xavier-Do
a67e51e600 [FIX] runbot: revert log_access and write in cache
Removing log_access has as side effect to add the foreign key to the
create_uid and write_uid fields. This is quite slow and will slow the
insert

Removing the fields is also not an good idea on such a large table

Puthing the value in cache and flushing should do the trick.
2022-12-16 16:59:59 +01:00
Xavier-Do
0d61f6c93b [IMP] runbot: add github login
This idea was postpone for a while since this was most a mergebot
responsability but having the github login of the user will help
for some team feature requests.

The main one is to only ping a team if the pr was not opened by a member
of the team. We want to let the team manager manage that as much as
possible so the team manager group will be able to write the user
github login (as well as the user himself) and add a list of non user
github_login to consider if not all user have a account on runbot.

This commit also improves the views for team managers.
2022-12-16 15:32:39 +01:00
Xavier-Do
3f2bf0112f [FIX] runbot: custom log_access fields shouldn't be automatic 2022-12-15 14:20:11 +01:00
Xavier-Do
02ed33c498 [FIX] runbot: keep log create date 2022-12-15 13:38:55 +01:00
Xavier-Do
0d6ec3e038 [FIX] runbot: fix invalid db_name log local
If the db_name does not stat with a build ind (or at least an int)
the query will fail because of 'local_pg_cursor'

Since a database can be create with an invalid name from the db manager
but still log in the runbot_logs, we need to manage all format.
Also add a limit to catchup if the db is full of logs, to avoid a Memory
error.
2022-12-15 12:54:50 +01:00
Xavier-Do
a6ce3dceb0 [IMP] runbot: add step description 2022-12-12 10:35:23 +01:00
Xavier-Do
2a82f3c1f7 [FIX] runbot: fw-bot codeowner fix 2022-12-01 11:29:25 +01:00
Xavier-Do
ee9b3b7570 [FIX] runbot: avoid excessive log_counter updates 2022-11-30 14:07:55 +01:00
Xavier-Do
7cdf77ce18 [IMP] runbot: hide some buttons
Force new batch buttons can be sometimes confusing for user.
Creating a group to show this button for advanced user only will help
avoiding useless new batch when it's not needed.

New batch is only needed:
- to create a new slot when a new trigger is added/modified through a
custom trigger
- take last databases into account for upgrades, mainly when backporting
a new field or strange usually forbiden operations
- avoiding to need to push again to rebase when a r+ was added on one
pr but one of them needs to be rebased or adapted.

Thos case are unusuall but the button is used most of the time thinking
this is a kind or rebuild or maybe it will rebase and push the branch
on the pr. Only user with basic knowledge of when it is needed should
have access to these buttons.
2022-11-29 16:07:34 +01:00
Xavier-Do
3664eabd90 [FIX] runbot: manage empty dbname 2022-11-28 14:38:05 +01:00
Christophe Monniez
2cad0542f4 [IMP] runbot: queue build logs in a local database
Before the commit the build ir_logging was sent from the build instance
to the main runbot ir.logging table. As the number of runbot hosts
increases, it introduce a lot of concurrency.
e.g.: 80 hosts with 8 builds means 640 instances trying to insert
records in the ir.logging table.

With this commit, a special database is used on the builder host in
order to receive ir.logging's from the build instances.

Regulary, the table is emptied by the builder and the logs are inserted
in the runbot leader ir.logging table.
2022-11-28 06:46:49 +01:00
Xavier-Do
66e37b9323 [FIX] runbot: renable blacklist 2022-11-25 10:41:52 +01:00
Xavier-Do
891d2d71e8 [IMP] runbot: add draft detection form titl 2022-11-24 16:24:42 +01:00
Christophe Monniez
3a9832d747 [IMP] runbot: add a close error wizard
When marking multiple build error as fixed, it's sometimes necessary to
explain why it was decided to close the error. When working with a few
errors, this can be done manually ... But most of the time we want to
close a lot of false negatives in batch.

With this commit, a simple wizard is made available that will post a
reason in the chatter of the build_errors.
2022-11-24 15:18:14 +01:00
Xavier-Do
1e8e059734 [FIX] fix codeowner 2022-11-24 15:17:53 +01:00
Xavier-Do
ff41311cb5 [FIX] runbot: add coverage access 2022-11-22 13:19:38 +01:00
Xavier-Do
c5e42b5529 [IMP] runbot: add missing fields 2022-11-22 13:19:38 +01:00
Xavier-Do
2b53455a9c [IMP] runbot: make ownership multi_edit 2022-11-22 10:34:24 +01:00
Xavier-Do
cd1360d716 [IMP] runbot: add search view 2022-11-22 10:12:48 +01:00
Xavier-Do
0ca706c56c [IMP] runbot: add team manager group 2022-11-22 09:34:29 +01:00
Xavier-Do
22abf95bca [FIX] runbot: ownership improvements
Tweaking the view to make it easier to use
2022-11-22 09:34:29 +01:00
Xavier-Do
f2d71a0b79 [FIX] runbot: fix pull info 2022-11-21 16:48:54 +01:00
Xavier-Do
2e77a55ddb [IMP] runbot: add codeowner management 2022-11-21 16:32:25 +01:00
Xavier-Do
410a01d13b [REF] runbot: move teams stuff 2022-11-21 16:32:25 +01:00
Xavier-Do
2ca7a3de6e [FIX] runbot: trigger with no config fallback 2022-11-09 12:29:00 +01:00
Xavier-Do
f72c4a8baa [FIX] runbot: fix cleanup 2022-11-08 15:19:33 +01:00
Xavier-Do
e0856b2245 [IMP] runbot: improve build_error management
The build error view was unstructured and contains to much information.

This commit organize fields in groups and also validate some
modification on records in order to avoid build error manager to
disable test-tags by mistake.

An error cannot be deactivated if it appeared less than 24 hours ago to
avoid disabling a non forxardported pr that will fail the next nightly
generating another build error.

Test tags can only be added or removed by administrators.

Also adds a menu for easier User managerment

Also fixed the dname search and display.
2022-11-08 14:43:43 +01:00
Christophe Monniez
ac010405dc [FIX] runbot: use gethostname instead of getfqdn
At boot time, it appears that when the runbot tries to get its hostname
it may get a wrong fqdn.
2022-11-07 10:12:15 +01:00
Christophe Monniez
964a88cb36 [FIX] runbot: take number of builds from config data
Oversight from previous improvement in b79c4a5a52.
2022-11-07 10:12:15 +01:00
Christophe Monniez
04459cdda7 [FIX] runbot: proper line feed in default odoorc 2022-11-07 10:12:15 +01:00
Xavier ALT
fffc27d2fa [FIX] runbot: fix creation of new runbot.version from the backend
Traceback (most recent call last):
  File "/home/odoo/src/odoo/15.0/odoo/addons/base/models/ir_http.py", line 237, in _dispatch
    result = request.dispatch()
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 687, in dispatch
    result = self._call_function(**self.params)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 359, in _call_function
    return checked_call(self.db, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/service/model.py", line 94, in wrapper
    return f(dbname, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 348, in checked_call
    result = self.endpoint(*a, **kw)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 916, in __call__
    return self.method(*args, **kw)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 535, in response_wrap
    response = f(*args, **kw)
  File "/home/odoo/src/odoo/15.0/addons/web/controllers/main.py", line 1347, in call_kw
    return self._call_kw(model, method, args, kwargs)
  File "/home/odoo/src/odoo/15.0/addons/web/controllers/main.py", line 1339, in _call_kw
    return call_kw(request.env[model], method, args, kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/api.py", line 464, in call_kw
    result = _call_kw_multi(method, model, args, kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/api.py", line 451, in _call_kw_multi
    result = method(recs, *args, **kwargs)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6489, in onchange
    snapshot1 = Snapshot(record, nametree)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6271, in __init__
    self.fetch(name)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 6281, in fetch
    self[name] = record[name]
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 5888, in __getitem__
    return self._fields[key].__get__(self, type(self))
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1054, in __get__
    self.recompute(record)
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1243, in recompute
    self.compute_value(recs)
  File "/home/odoo/src/odoo/15.0/odoo/fields.py", line 1265, in compute_value
    records._compute_field_value(self)
  File "/home/odoo/src/odoo/15.0/odoo/models.py", line 4255, in _compute_field_value
    getattr(self, field.compute)()
  File "/home/odoo/runbot/extra/runbot/models/version.py", line 36, in _compute_version_number
    version.number = '.'.join([elem.zfill(2) for elem in re.sub(r'[^0-9\.]', '', version.name).split('.')])
  File "/usr/lib/python3.8/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 643, in _handle_exception
    return super(JsonRequest, self)._handle_exception(exception)
  File "/home/odoo/src/odoo/15.0/odoo/http.py", line 301, in _handle_exception
    raise exception.with_traceback(None) from new_cause
TypeError: expected string or bytes-like object
2022-10-21 11:59:55 +02:00
Xavier ALT
1095b25270 [FIX] runbot: fix help msg, 'additionnal_env' is split on semi-column 2022-10-21 11:59:55 +02:00
Christophe Monniez
309aeaa32e [IMP] runbot: speedup build garbage collecting
When the builds directory is filled with a lot of build directories
(around 100000) the garbage collection process may take up to 2 minutes.
The root cause is that each build directory is scanned to clean it up
even if it was already cleaned.

With this commit, a stamp file is used to mark directories that were
already garbage collected.
2022-10-21 11:32:46 +02:00
Christophe Monniez
7642bffda3 [FIX] runbot: download phantomjs from nightly server
In order to avoid bitbucket rate limiting, we prefer to download this
old pal from our server.
2022-10-21 11:17:05 +02:00
Xavier-Do
903ee7d983 [FIX] runbot: manage falsy value for frontend_url 2022-10-21 10:39:20 +02:00
Xavier-Do
e27a1b8f71 [IMP] runbot: cleaup settings view 2022-10-21 10:32:40 +02:00
Xavier-Do
0287dcaab7 [IMP] runbot: update install documentation 2022-10-21 10:32:40 +02:00
Xavier-Do
303638e507 [FIX] runbot: don't hardcode user odoo 2022-10-21 10:32:40 +02:00
Xavier-Do
e4af8d7b6b [FIX] runbot: remove fqdn calls 2022-10-21 10:32:40 +02:00
Xavier-Do
785e2dc3bc [IMP] runbot: don't use local odoorc 2022-10-21 10:32:40 +02:00
Xavier-Do
bfc75cd7fe [IMP] runbot: don't active cron by default 2022-10-21 10:32:40 +02:00
Xavier-Do
50a7120949 [FIX] runbot: don't crash when there is no accessible project 2022-10-21 10:32:40 +02:00
Xavier-Do
16e71d064a [FIX] runbot: remove useless/invalid check 2022-10-21 10:32:40 +02:00
Xavier-Do
b79c4a5a52 [IMP] runbot: various build_config improvements 2022-10-21 10:32:40 +02:00
Xavier-Do
7bb2f06501 [IMP] runbot: make build_error button visible for users 2022-10-21 10:32:40 +02:00
Xavier-Do
b606a2199b [IMP] runbot: improve build error display in logs 2022-10-21 10:32:40 +02:00
Xavier-Do
831f5f7197 [IMP] runbot: make link to backend have the runbot menu 2022-10-21 10:32:40 +02:00
Xavier-Do
eaf8964701 [IMP] runbot: change trigger start logic
and add sequence on project
2022-10-21 10:32:40 +02:00
Xavier-Do
02eb8014a3 [IMP] runbot: make build_error triggers searchable 2022-10-21 10:32:40 +02:00
Xavier-Do
a294348cd0 [IMP] runbot: improve views 2022-10-21 10:32:40 +02:00
Xavier-Do
70e8b15690 [IMP] runbot: add websocket support 2022-08-23 16:30:24 +02:00
Xavier-Do
e91611b0f8 [FIX] runbot: reduce load when builder are idle
The sleep 1 was ok with a few builder bur regarding the number of
request on the database when no build are running, this become
problematic.

An ideal solution would be to detect if
-> me managed some testing build
-> There is load (pendings)

In both case, we don't want to sleep to much.
In other cases, we may want to wait a little longer.

A simple quick fix will just wait longer in all cases.
2022-08-23 15:54:50 +02:00
Xavier-Do
b44ed5f7a6 [IMP] runbot: active trigger and filter manual
- dont display manual trigger having a team if not member of that team
- add an active field on triggers
2022-08-17 16:50:23 +02:00
Xavier-Do
326ac36b5e [FIX] runbot: display stats of failed and scheduled builds 2022-08-17 12:18:40 +02:00
Xavier-Do
35e3e7cee0 [FIX] runbot: make Upgrade exception message no update and step id clickable 2022-08-17 11:46:47 +02:00
Xavier-Do
d4f1f01b46 [IMP] runbot: add buttons to compare with next line on build_search page 2022-08-17 11:46:47 +02:00
Xavier-Do
05c0c29e0f [IMP] runbot: add bundle name in link title 2022-08-17 11:43:53 +02:00
Romain Derie
8af81d72bb [IMP] runbot: add sort and filter capability on the build error page
Before this commit, the build errors page was neither sortable neither
filterable.

This commit adds a way to filter by:
- all
- unassigned
- seen more than once

It also allows to sort by:
- last seen
- nb seen

Typical need is to sort by nb seen and filter out the only seen once to
be able to figure which one is supposed to be checked in priority.
2022-08-17 11:42:36 +02:00
Xavier-Do
a382977d39 [IMP] runbot: add teams on trigger.
This will be mainy usefull in trigger categories custom view and for
dashboards.
2022-08-17 11:40:56 +02:00
Xavier-Do
5f2fe9a451 [FIX] runbot: mitigate docker status race condition
In some rare cases, a docker container has a status of exited, removed
or in removal and the `end-`file has been written right after the code
checked the existence of the file.

To mitigate the issue, this commit checks the `end-` file existence
after the container status have been checked. That way, if the file
exists we can be pretty confident that the build really ended.
2022-08-17 11:40:12 +02:00
Christophe Monniez
2dce6d048e [IMP] runbot: freeze files modification time at export
When exporting a commit, it will be useful to freeze the modification
time of the exported files. The idea behind that is to pre-generate
bundles at the install time of the Odoo instances, that way when running
the post install tests, the bundles does not have to be generated for
each test.
2022-07-26 13:57:18 +02:00
xmo-odoo
6d96f8c2cd [FIX] runbot: redirect to current page after login
Because the login link redirected to `/`, when logging in the user would have
to re-navigate to their previous page unless they'd remembered this issue and
kept the original page around.

Fix this because it's annoying and dumb: we know our URL when rendering
templates, so we can redirect back there after login.

Consideration: this could also be done on logout, however it seems likely that
in that case the original page is "privileged" and when coming back we'd just
get an access error. So don't do it for now.
2022-07-25 12:12:02 +02:00
Xavier-Do
9fb7e94de9 [IMP] add load info page 2022-07-25 12:10:48 +02:00