Commit Graph

502 Commits

Author SHA1 Message Date
Xavier-Do
92157ad71a [IMP] runbot: make access to build customizable 2023-07-03 15:52:53 +02:00
Xavier-Do
d09b9961cd [IMP] runbot: log user beside ip 2023-07-03 15:52:53 +02:00
Christophe Monniez
3c8f821695 [IMP] runbot: unify build error wizards
There are two wizards for the runbot build errors:
- One to close an error with a reason
- One to update the team/user or PR

With this commit, the two wizards are merged into one wizard that helps
to update errors in bulk.

Also, a button is added in the list view that allow to save a mouse
click.

The `NEW` button is removed from the tree view as it should not be of
any use.
2023-06-30 10:57:01 +02:00
Christophe Monniez
1cc73a606d [IMP] runbot: allow only advanced users to wake up a child
As it happens that some users are waking up child builds, let's only
allow advanced users to do that.
2023-06-30 10:57:01 +02:00
Xavier-Do
506ff03e07 [IMP] runbot: pause, and profile.
Pausing a host can be usefull in some case, mainly when testing new code
The loop will have no effect avoiding to break some build wainting for
testing.

Profile will help to identify potential performance flows during the
loop.
2023-06-30 10:54:50 +02:00
Christophe Monniez
2b95e0dc01 [FIX] runbot: remove search_count hack
Since odoo/odoo@a01e8b5232 this hack should not be necessary anymore.
2023-06-30 10:37:28 +02:00
Xavier-Do
cd8cc1f77f [FIX] runbot: fix onchange 2023-06-27 14:42:00 +02:00
Xavier-Do
2a003f00de [FIX] runbot: fix dump_db 2023-06-27 14:28:08 +02:00
xdo
24d35988a4
[IMP] runbot: add auto restore for custom trigger (#784)
One of the most common custom trigger is to restore a build before
starting some test, either to create a multibuild or make the execution
and debug of some test faster.

It is somethimes tedious to use because we need to give an url of a
build to restore. This build must correspond to the right commits,
must still exixt, ... this means that the dump url must be adapted
everytime a branch is rebased.

The way the dump_url is defined is by going on the last batch, following
the link to the `base_reference_batch_id`, finding a slot corresponding
to the right repo set, (ex: Custom enterprise -> enterprise), and
copying the dump_url in this build.

The base_reference_batch_id is eay to automated but we have to find the
right trigger, this is now a parameter of the custom trigger wizard.
There are actually 2 strategy now to define how to download the dump:
- `url`, using `restore_ dump_url`
- `auto`,  using `restore_trigger_id` and `restore_database_suffix`

To ease the setup, a `restore_trigger_id` is added on a trigger, so that
when selecting a trigger, lets say `Custom enterprise`, the defined
`trigger.restore_trigger_id` is automatically chosen for the
`custom_trigger.restore_trigger_id` and the `restore_mode` is setted to
auto.

Two actions are also added to the header of a bundle, a shorcut to
setup a multi build (restore in children) or a restore and test build
(restore in parent).
2023-06-27 14:13:12 +02:00
Xavier-Do
727d71d103 [IMP] runbot: add an action to check token 2023-06-22 16:43:05 +02:00
Xavier-Do
ebb531f4a4 [FIX] runbot: adapt for 16.0 2023-06-22 14:27:32 +02:00
Xavier-Do
614040720b [IMP] runbot: add useful indexes remaining from upgrade 2023-06-21 10:08:14 +02:00
Xavier-Do
45104b635f [REL] adapt for 16.0 2023-06-20 12:34:32 +02:00
Xavier-Do
d3f998f88c [IMP] runbot: keep log create_date 2023-06-20 09:18:48 +02:00
Xavier-Do
a80dc25699 [FIX] runbot: only top parent is marked killable 2023-06-14 10:30:02 +02:00
Xavier-Do
03667b703c [IMP] runbot: fallback on python step for upgrade step 2023-06-02 14:44:48 +02:00
Xavier-Do
ad4131789c [FIX] runbot: fix false positive message 2023-06-02 11:10:51 +02:00
Xavier-Do
7845a718b7 [IMP] runbot: add use_ssl to settings 2023-06-02 10:49:05 +02:00
Martin Trigaux
8153fdc4b2 [FIX] runbot: download over https
Firefox blocks downloads from http link if you are on an https page
Allow to deactivate via an ICP in case the runbot is configured over
HTTP (you shouldn't really)
2023-06-02 10:49:05 +02:00
Christophe Monniez
236554b588 [FIX] runbot: catch exceptions during db drop
When the runbot tries to drop a local database, if the that raises an
exception, it goes in a loop failure. It mays happen for example if
someone forgot to close a psql during an investigation :-)

With this commit, the exceptions are catched and at least the database
name is logged.
2023-06-01 16:16:06 +02:00
Xavier-Do
7523dc8000 [FIX] runbot: fix dockerfile choice order
Since all versions will have a defined dockerfile, the project one
will alway be ignored. The idea here is that for a project, we may
definea default dockerfile_id so that we don't have to set it on all
bundle to make it work.
2023-06-01 16:13:40 +02:00
Xavier-Do
db38794f9a [FIX] runbot: avoid sending sattus on running kill
The _kill method was called in multiple case, usually when something
wrong happen:
- exception initiating pending
- kill requested manually
- testing time exceeded
- exception running a job
- ...

But it will also be called when killing a running build.

It was usually not an issue since the status remains the same, but it is
not true if the same commit is used in two build, the new one is green,
the old one is red (enterprise commit remaining the same but community
commit changed as an example)

In this situation, the enterprise commit may receive the red ci from the
old build while the last one is green.

Since with the last version, the github status responsibility is left to
write method, this github status is not useful anymore, updating the
state and result is enough.

This commit also removes the commit since it is not always a god idea.
Most of the time the transaction will be comited quite fast after that
with the new scheduler.

Note that checking in github status if no status has a more recent build
may be a good idea. Only the most recent build using a commit could
sending a status? This would not alway be helpful Imagine a commit used
in 2 branches by mistake, the last build is not always the one we want
(usually fixed by rebuilding a subbuild of the good build)
2023-06-01 15:40:16 +02:00
Xavier-Do
64d3c59ed9 [FIX] runbot: limit max log size
In some case, a build can add a lot of info in a log, there
is already a limit to the number of entry but not to the size of an
entry. This will limit the database usage in case of mistake/abuse.
2023-06-01 15:33:47 +02:00
Xavier-Do
5a5e7693d4 [IMP] runbot: add an option on step to disable logdb 2023-06-01 15:13:01 +02:00
Christophe Monniez
45fb4f8319 [IMP] runbot: add activities on runbot build errors 2023-06-01 15:03:13 +02:00
Christophe Monniez
bdd98b07ec [IMP] runbot: rename active to active
Error is not fixed was too disturbing.
2023-06-01 15:03:13 +02:00
Christophe Monniez
2e002c2dd7 [IMP] runbot: improve the build errors wizard
With this commit, the wizard now allows to set a fixing PR and/or a
commit text to multiple errors.
2023-06-01 15:03:13 +02:00
Christophe Monniez
388eeb377a [IMP] runbot: improve build errors views
* add a link to the fixing PR on github
* add a warning ribbon on test-tagged errors
* show different colors in tree view to spot fixed PR's
* add some search filters
2023-06-01 15:03:13 +02:00
Xavier-Do
e3d87b5b5d [IMP] runbot: improve test-tags support
The current post_install build mecanism is using extra params to give
test-tags. Unfortunately this disables the support for auto tags
and this have to be done manually. This means that auto tags are in the
build extra-params and not dynamic at rebuild of a post_install.

Also, using extraparams in the post install creation was removing
extra_params comming from custom trigger.

With this commit, the test-tags can be given inside config_data
and will be combined with config step test-tags and auto-tags.

This was an opportunity to simplify the logic.

This commit also fixes the test_install_tags that was broken.
2023-04-06 11:33:01 +02:00
Xavier-Do
e88f679c87 [FIX] runbot: fix python step wakeup 2023-03-30 13:05:35 +02:00
Xavier-Do
8c2e7a5781 [FIX] runbot: manage case when there is no start
SInce the previous version the build end is written when going in any
done state. This means that when a build is skipped, it has a end
but no start.

Adapat the build dime to manage this use case.
2023-03-29 14:58:14 +02:00
Xavier-Do
f7d29f87a4 [FIX] runbot: fix build end
Previous fix was not enough because based on global_state, meaning
that the build if the build goes waiting, it need to update its end
anyway.
2023-03-24 13:09:19 +01:00
Xavier-Do
30c74e2434 [FIX] runbot: update build end 2023-03-24 10:26:20 +01:00
Xavier-Do
ee58a93e9a [IMP] runbot: avoid link to killed build
When a build is created, it will first check for another
build having the same params. It is usually a good idea to avoid
to much load. In some case, a build can be found, but a killed one.

This is not what we want:
The first scenario is to consecutive force push,
commit1 -> commit2 -> commit1

The build of commit1 may be killed because of commit2, then when
forcepushing commit1 again, it will be linked to a killed build.

A even more problematic problem was discovered because of a delay In
odoo/odoo repo hook. An odoo-dev/odoo 16.0-... branch was discovered
first using this commit, and a build was created.
Then, the branch was forcedpushed and the build was killed.
Finally, the 16.0 commit was discovered, and was linked to the killed
build. This was mainly an issue because the build was a template.

With this changes, the 16.0 would have created a new build, not linking
to a killed one.

Note that linking to a red build is not an error. Only a killed one.
2023-03-24 10:26:20 +01:00
Xavier-Do
3e5d5e88a1 [FIX] runbot: global_state not written in compute 2023-03-23 16:56:34 +01:00
Xavier-Do
74f0c8e1ad [FIX] runbot: fix wakeup 2023-03-23 14:44:08 +01:00
Xavier-Do
2acca91a97 [FIX] runbot: fix missing return
This was missing to make python (and other non docker) steps really
blazing fast
2023-03-23 14:17:02 +01:00
Xavier-Do
b078275c94 [IMP] runbot: add commit_export_ids 2023-03-23 13:32:50 +01:00
Xavier-Do
2488228b7b [IMP] runbot: make params work in create multi
The main motivation is to allow to create params from data
the "new" method was called with a value list instead of a dict.

Also, makes it possible to update params when the registry is not laoded
2023-03-23 13:32:50 +01:00
Xavier-Do
d164d9a745 [IMP] runbot: host improvement 2023-03-23 13:32:50 +01:00
Xavier-Do
d2f9330043 [REF] runbot: various refactoring
The initial motivation is to remove the flush when a log_counter is
written. This flush was initially usefull when the limit was in a
psql trigger, but finally add a side effect to flush everything before
starting the docker. This was limiting concurrent update after starting
the docker, but we still have no garantee that the transaction is
commited after starting the docker. The use case where the docker is
started but the transaction is not commited was not handled well and was
leading to an infinite loop of trying to start a docker (while the
docker was already started)

This refactoring returns the docker to the scheduler so that the
schedulter can commit before starting the docker.

To achieve this, it is ideal to have only one method that could return
a callable in the _scheduler loop. This is done by removing the run_job
from the init_pending method. All satellite method like make result
are also modified and adapted to make direct write: the old way was
technical debt, useless optimization from pre-v13.

Other piece of code are moved arround to prepare for future changes,
mainly to make the last commit easier to revert if needed.

[FIX] runbot: adapt tests to previous refactoring
2023-03-23 13:32:50 +01:00
Xavier-Do
688900edb1 [FIX] runbot: remove hardcoded runbot_logs 2023-03-23 13:32:50 +01:00
Xavier-Do
0e14e6d922 [IMP] runbot: serealisation imps
Trying to log when the transaction is in error is useless and create
noise in the logs.

Flushing is also useless there now that we have the local logs,
and it makes the error confusing since the error does not come from the
log_counter update but from the update of the global state on the
parents global_results.
2023-03-23 13:32:50 +01:00
Xavier-Do
937747caec [IMP] runbot: remove dead code 2023-03-23 13:32:50 +01:00
Xavier-Do
c965c2c35a [IMP] runbot: add message queue
Message queue squeletton for future changes
2023-03-23 10:09:12 +01:00
Xavier-Do
2579a2d3fe [FIX] runbot: fix run step for no install config 2023-03-22 16:17:30 +01:00
Christophe Monniez
6fdd35ed50 [FIX] runbot: avoid empty negative test tags
When parenting a build error, if a test_tag is set on it, the tag is
transferred to the parent and cleared to an empty string.
In that case, a single `-` appears in the disabling tags and leads to an
apocalyptic situation ... the runbot builds don't not run any tests.

With this commit, the test_tags is set to `False`.
2023-02-28 12:16:07 +01:00
David James
1532bcab0f [FIX] fix typo in selection field value 2023-02-28 11:58:34 +01:00
David James
fa3808bf1e [FIX] runbot: fix dictionary changed size during iteration 2023-02-28 11:16:46 +01:00
Xavier-Do
d3e3228921 [FIX] runbot: get_module remove prefix
Since removeprefix was not available in ubuntu 20.04, a easier alternative
using rebase was used.

The initial assumption was that the prefix would look like `odoo/addons/`
and won't be in the filename.

When the repo is enterprise, the prefix is `enterprise/` meaning that
module name ending with `enterprise` will be truncated

`repo._get_module('enterprise/mail_enterprise/static/src/widgets/form_renderer/form_renderer.js')`

will output

`mail_static`
2023-02-28 10:59:47 +01:00