Commit Graph

465 Commits

Author SHA1 Message Date
Xavier-Do
b078275c94 [IMP] runbot: add commit_export_ids 2023-03-23 13:32:50 +01:00
Xavier-Do
2488228b7b [IMP] runbot: make params work in create multi
The main motivation is to allow to create params from data
the "new" method was called with a value list instead of a dict.

Also, makes it possible to update params when the registry is not laoded
2023-03-23 13:32:50 +01:00
Xavier-Do
d164d9a745 [IMP] runbot: host improvement 2023-03-23 13:32:50 +01:00
Xavier-Do
d2f9330043 [REF] runbot: various refactoring
The initial motivation is to remove the flush when a log_counter is
written. This flush was initially usefull when the limit was in a
psql trigger, but finally add a side effect to flush everything before
starting the docker. This was limiting concurrent update after starting
the docker, but we still have no garantee that the transaction is
commited after starting the docker. The use case where the docker is
started but the transaction is not commited was not handled well and was
leading to an infinite loop of trying to start a docker (while the
docker was already started)

This refactoring returns the docker to the scheduler so that the
schedulter can commit before starting the docker.

To achieve this, it is ideal to have only one method that could return
a callable in the _scheduler loop. This is done by removing the run_job
from the init_pending method. All satellite method like make result
are also modified and adapted to make direct write: the old way was
technical debt, useless optimization from pre-v13.

Other piece of code are moved arround to prepare for future changes,
mainly to make the last commit easier to revert if needed.

[FIX] runbot: adapt tests to previous refactoring
2023-03-23 13:32:50 +01:00
Xavier-Do
688900edb1 [FIX] runbot: remove hardcoded runbot_logs 2023-03-23 13:32:50 +01:00
Xavier-Do
0e14e6d922 [IMP] runbot: serealisation imps
Trying to log when the transaction is in error is useless and create
noise in the logs.

Flushing is also useless there now that we have the local logs,
and it makes the error confusing since the error does not come from the
log_counter update but from the update of the global state on the
parents global_results.
2023-03-23 13:32:50 +01:00
Xavier-Do
937747caec [IMP] runbot: remove dead code 2023-03-23 13:32:50 +01:00
Xavier-Do
c965c2c35a [IMP] runbot: add message queue
Message queue squeletton for future changes
2023-03-23 10:09:12 +01:00
Xavier-Do
2579a2d3fe [FIX] runbot: fix run step for no install config 2023-03-22 16:17:30 +01:00
Christophe Monniez
6fdd35ed50 [FIX] runbot: avoid empty negative test tags
When parenting a build error, if a test_tag is set on it, the tag is
transferred to the parent and cleared to an empty string.
In that case, a single `-` appears in the disabling tags and leads to an
apocalyptic situation ... the runbot builds don't not run any tests.

With this commit, the test_tags is set to `False`.
2023-02-28 12:16:07 +01:00
David James
1532bcab0f [FIX] fix typo in selection field value 2023-02-28 11:58:34 +01:00
David James
fa3808bf1e [FIX] runbot: fix dictionary changed size during iteration 2023-02-28 11:16:46 +01:00
Xavier-Do
d3e3228921 [FIX] runbot: get_module remove prefix
Since removeprefix was not available in ubuntu 20.04, a easier alternative
using rebase was used.

The initial assumption was that the prefix would look like `odoo/addons/`
and won't be in the filename.

When the repo is enterprise, the prefix is `enterprise/` meaning that
module name ending with `enterprise` will be truncated

`repo._get_module('enterprise/mail_enterprise/static/src/widgets/form_renderer/form_renderer.js')`

will output

`mail_static`
2023-02-28 10:59:47 +01:00
Xavier-Do
763b1af1a5 [FIX] runbot: disable host_default 2023-01-17 15:24:49 +01:00
Xavier-Do
27ac733df6 [IMP] runbot: automatic base master bundle
When adding a new project, if no branch matches a base name,
the created bundles won't have a version and it will fail.

A simple fix will be to add a master bundle for all projects.
2023-01-17 13:41:01 +01:00
Xavier-Do
9e0f0e6a75 [IMP] runbot: display errors for critical/warn 2023-01-13 15:57:25 +01:00
Xavier-Do
12c0a0edf6 [IMP] runbot: automatically transfer responsible
When an error is linked to another one, we don't expect it  to appear on
team and user dashboard. When adding a parent, this will transfer the
responsible from the child to the parent when applicable.
2023-01-13 15:32:17 +01:00
avs
6fffa0264a [IMP] runbot: make runbot.bundle::write() callable through xml-rpc
The XML-RPC implementation does not allow for receiving or sending
`None` values (both as query parameters and response).

Since the `write` method of `runbot.bundle` was overriden without
returning a value, an exception is raised when the method is called
through the external API.

This makes the `write` method return the value from its call
to `super()` which should be equal to `True` if all went well.
2023-01-13 15:32:01 +01:00
Xavier-Do
d011bb7f8c [IMP] runbot: make disable host configurable
The auto disable host is mainly usefull when there are a lot of host for
well configured repositories.

If for any reason a repo is corrupted on one host, this host will be
disabled until a manual intervention cleans the repo.

For other cases, where thjere are many repositories with not so many
host, it is most likely that a fetch will fail because of an invalid
repository configuration. Disabling the host in this case is not a good
idea.

With this commit, a settings allows to enable or disable this feature.
2023-01-13 14:46:02 +01:00
Xavier-Do
ecd105218c [IMP] runbot: parse warning and critical 2023-01-13 14:27:17 +01:00
Xavier-Do
2b8242cf42 [IMP] runbot: default empty value for result 2023-01-13 14:27:17 +01:00
Xavier-Do
6892546d94 [IMP] runbot: fail before contacting github if user cannot write on team 2023-01-13 14:27:17 +01:00
Xavier-Do
cd4c6b5e43 [IMP] runbot: better configure upgrade descriptions 2023-01-13 14:27:17 +01:00
Xavier-Do
37df18d54d [FIX] runbot: use params db_name when provided for upgrades 2023-01-12 11:54:53 +01:00
Xavier-Do
c9c64bccc7 [FIX] runbot: fetch member should be able to access token for users 2023-01-11 13:35:02 +01:00
Xavier-Do
06f9f62b1f [FIX] runbot: fix was alive 2023-01-11 12:41:39 +01:00
Xavier-Do
16fa0e1ad8 [IMP] runbot: add freeze list feature 2023-01-11 11:20:29 +01:00
Xavier-Do
fdb144a3af [IMP] runbot: allow to ignore forward-ports for a team 2023-01-10 10:47:59 +01:00
Xavier-Do
56c08be631 [FIX] runbot: filter numerical branch names
Right now, a branch with a numerical name will be added to the database,
but it can conflict with pr since the name of a pr is a number.
This means that a unique (name, repo_id) constraints can be broken.
We could use the 'is_pr' in the unicity constraints to avoid this issue
but searching on branch name will give confusing result if some of them
can be numerical.

Moreover, a runbot branch name should start with the version name
meaning that a numerical branch name was a bad idea from the begining.
2022-12-27 10:37:30 +01:00
Xavier-Do
6d3663e9f6 [IMP] runbot: disable auto-tags from custom trigger 2022-12-26 12:31:09 +01:00
Xavier-Do
910843f06d [FIX]: fix codeowner
A corner case was discovered when a pr is open but the pr is not in the
codeowner commits.
2022-12-24 12:39:55 +01:00
Xavier-Do
ad573d64cf [FIX] runbot: manage falsy path_glob 2022-12-22 14:40:13 +01:00
Xavier-Do
5540f84667 [IMP] runbot: add priority queue for debuging
The main idea is to allow some build to use an extra slot from all host
if a bundle is in priority mode.

This is mainly for quick step debuging, mainly when modifying python
steps when the runbot is fully loaded.

This will also be usefull to concurrency test, by starting a build on
each host at the same time, even when some host are already fully loaded
2022-12-22 14:26:10 +01:00
Xavier-Do
356843c680 [IMP] runbot: reduce concurrency
Right now, multiple build are read when managing build to schedule.

This is not usefull since the transaction is commited between each
of them. Moreover, the read build can be written from another host
adding another possibility to have a conccurent update.

Removing the prefetch_ids may help a little.
2022-12-22 11:25:04 +01:00
Xavier-Do
205d299c7d [IMP] runbot: add trigger list to team 2022-12-22 10:47:24 +01:00
Christophe Monniez
9ccf5a2c0e [IMP] runbot: add a reassign team wizard 2022-12-21 15:20:42 +01:00
Xavier-Do
c83f89fc23 [IMP] runbot: documentation and views imps
Add a small documentation for users, mainly about teams and codeowners.
Improves some views and hide some menu_items to keep interface easier
to navigate.
2022-12-21 13:00:42 +01:00
Xavier-Do
ada69222c9 [IMP] runbot: add an action to reassign team automaticaly 2022-12-21 10:58:30 +01:00
Xavier-Do
af7a0751fc [FIX] runbot: avoid recursion in create steps
Since the custom create was introduced, if a config is added in the
config data of a create step, the config can be dynamic. If the given
config contains a create step, this become recursive.

This is fixed in this commit by:
- Checking the parent_path depth in add_child. This will also work for
  python config.
- Consuming the params when adding the child
- Also cleanup the base custom multi config to use a specific step
2022-12-20 09:30:08 +01:00
Xavier-Do
92ed23c237 [FIX] runbot: manage team without github logins 2022-12-19 11:14:07 +01:00
Xavier-Do
25557e6116 [FIX] runbot: manage team without github logins 2022-12-19 11:09:15 +01:00
Xavier-Do
a67e51e600 [FIX] runbot: revert log_access and write in cache
Removing log_access has as side effect to add the foreign key to the
create_uid and write_uid fields. This is quite slow and will slow the
insert

Removing the fields is also not an good idea on such a large table

Puthing the value in cache and flushing should do the trick.
2022-12-16 16:59:59 +01:00
Xavier-Do
0d61f6c93b [IMP] runbot: add github login
This idea was postpone for a while since this was most a mergebot
responsability but having the github login of the user will help
for some team feature requests.

The main one is to only ping a team if the pr was not opened by a member
of the team. We want to let the team manager manage that as much as
possible so the team manager group will be able to write the user
github login (as well as the user himself) and add a list of non user
github_login to consider if not all user have a account on runbot.

This commit also improves the views for team managers.
2022-12-16 15:32:39 +01:00
Xavier-Do
3f2bf0112f [FIX] runbot: custom log_access fields shouldn't be automatic 2022-12-15 14:20:11 +01:00
Xavier-Do
02ed33c498 [FIX] runbot: keep log create date 2022-12-15 13:38:55 +01:00
Xavier-Do
0d6ec3e038 [FIX] runbot: fix invalid db_name log local
If the db_name does not stat with a build ind (or at least an int)
the query will fail because of 'local_pg_cursor'

Since a database can be create with an invalid name from the db manager
but still log in the runbot_logs, we need to manage all format.
Also add a limit to catchup if the db is full of logs, to avoid a Memory
error.
2022-12-15 12:54:50 +01:00
Xavier-Do
a6ce3dceb0 [IMP] runbot: add step description 2022-12-12 10:35:23 +01:00
Xavier-Do
2a82f3c1f7 [FIX] runbot: fw-bot codeowner fix 2022-12-01 11:29:25 +01:00
Xavier-Do
ee9b3b7570 [FIX] runbot: avoid excessive log_counter updates 2022-11-30 14:07:55 +01:00
Xavier-Do
3664eabd90 [FIX] runbot: manage empty dbname 2022-11-28 14:38:05 +01:00