When updating a bundle and changing the is_base field to True, it
creates a new version based on the bundle name. This can potentially
breaks builds and moreover, it can breaks the whole runbot when a
duplicate version name is created.
With this commit:
- a constraint on the version name uniqueness is added
- the is_base field is now readonky
- a version cannot be created by the compute version if the version name
does not match the above mentioned regular expression
Using dev branch from foreign project to fill missing commits looks like
a bad idea, mainly because the lifecycle of the branches is not the same
In some case, it can be useful to allow that to test a branch with a
future change in the base project that will be needed to make the branch
work. As an example introducing a small api change in odoo to make an
override easier, or introducing a module that may be needed to use
the feature.
This commit changes that by allowing to configure on the project or
bundle if we allow to use foreign bundle as reference *before* checking
the base bundle.
- clean thread username
- allow to write on params for debug (was mainly usefull to forbid it
at the beginning)
- imrpove some guidelines about method and actions naming/ ordering
- move some code for a cleaner organisation.
- remove some useless request.env.user (not useful anymore)
One of the most common custom trigger is to restore a build before
starting some test, either to create a multibuild or make the execution
and debug of some test faster.
It is somethimes tedious to use because we need to give an url of a
build to restore. This build must correspond to the right commits,
must still exixt, ... this means that the dump url must be adapted
everytime a branch is rebased.
The way the dump_url is defined is by going on the last batch, following
the link to the `base_reference_batch_id`, finding a slot corresponding
to the right repo set, (ex: Custom enterprise -> enterprise), and
copying the dump_url in this build.
The base_reference_batch_id is eay to automated but we have to find the
right trigger, this is now a parameter of the custom trigger wizard.
There are actually 2 strategy now to define how to download the dump:
- `url`, using `restore_ dump_url`
- `auto`, using `restore_trigger_id` and `restore_database_suffix`
To ease the setup, a `restore_trigger_id` is added on a trigger, so that
when selecting a trigger, lets say `Custom enterprise`, the defined
`trigger.restore_trigger_id` is automatically chosen for the
`custom_trigger.restore_trigger_id` and the `restore_mode` is setted to
auto.
Two actions are also added to the header of a bundle, a shorcut to
setup a multi build (restore in children) or a restore and test build
(restore in parent).
The initial motivation is to remove the flush when a log_counter is
written. This flush was initially usefull when the limit was in a
psql trigger, but finally add a side effect to flush everything before
starting the docker. This was limiting concurrent update after starting
the docker, but we still have no garantee that the transaction is
commited after starting the docker. The use case where the docker is
started but the transaction is not commited was not handled well and was
leading to an infinite loop of trying to start a docker (while the
docker was already started)
This refactoring returns the docker to the scheduler so that the
schedulter can commit before starting the docker.
To achieve this, it is ideal to have only one method that could return
a callable in the _scheduler loop. This is done by removing the run_job
from the init_pending method. All satellite method like make result
are also modified and adapted to make direct write: the old way was
technical debt, useless optimization from pre-v13.
Other piece of code are moved arround to prepare for future changes,
mainly to make the last commit easier to revert if needed.
[FIX] runbot: adapt tests to previous refactoring
The XML-RPC implementation does not allow for receiving or sending
`None` values (both as query parameters and response).
Since the `write` method of `runbot.bundle` was overriden without
returning a value, an exception is raised when the method is called
through the external API.
This makes the `write` method return the value from its call
to `super()` which should be equal to `True` if all went well.
The main idea is to allow some build to use an extra slot from all host
if a bundle is in priority mode.
This is mainly for quick step debuging, mainly when modifying python
steps when the runbot is fully loaded.
This will also be usefull to concurrency test, by starting a build on
each host at the same time, even when some host are already fully loaded
This idea was postpone for a while since this was most a mergebot
responsability but having the github login of the user will help
for some team feature requests.
The main one is to only ping a team if the pr was not opened by a member
of the team. We want to let the team manager manage that as much as
possible so the team manager group will be able to write the user
github login (as well as the user himself) and add a list of non user
github_login to consider if not all user have a account on runbot.
This commit also improves the views for team managers.
The actual way to add a custom multibuild on a dev bundle will be:
- use the multibuild wizard to create 2 configs and 2 steps
- add this config on the bundle
- customize this config to make it fatser by installing/restoring...
The usefull parameters are the number of build, the test tags and
modules to install.
Another usefull operation is to restore a dump instead of installing,
especially for post install test only breaking on full databases.
This commit proposes to replace the multi build wizard with a custom
trigger wizard. The main idea is to have generic config, parametrized by
config_data. The wizard will only help to generate the corresponding
config_data.
Runbot layout modifies the website/portal base layout to remove navbar,
footer, overides some custom styles. A lot of assets are loaded but not
used. The only real usefull elements are base assets (bootstrap, ...)
and the login button.
Migrating to the next version of odoo is usually painfull because some
xpath may break, extra element added, or some style change may break the
page, needing to add more and more xpath, css rules, ... for very little
benefits.
This cleanup creates a custom base layout for runbot independant from
base odoo templates.
Also add a breadcrumb, navigation arrow, and improve batch links
- searching on number will search for both pr and branche name
- hooks are now using payload to define repo when not given in url
- fixes .git cleaning in repo
(remove rstrip since it can fail for repo starting with g, i, t)
- recompute base on prepare if base was not found
- remove local_result form write values if there is a single record
(instead of raising, makes python step easier to write).
- avoid stucked build/loop after removing a step from a config.
- avoid to send ci for linked base_commit
- add a fallback mechanism for base if no master branch is found
- add option on project to avoid to keep sticky running, usefull
when using a lots of projects
WARNING: this is a change of default behaviour, need to update
existing projects.
- always discover new commits for branch matching base paterns.
This is especially usefull to discover old versions on project with
low merge frequency.
- always create a batch, event if there is now trigger. This helps to
notice that commits are discovered
- add line-through on death branches/pr
- manual trigger are now displayed on main page
- Add draft pr management to avoid to trigger code owner on draft pr.
- Add check on falsy config on trigger id (avoid crash, usefull to disable trigger)
- Add extra params on custom trigger to avoid to write specific config every time.
- Trigger a new batch automaticaly when updating target/draft
If you try to manually create bundle, Odoo will crash, because it will
try to use `name` value that is not set yet. For that we start computing
host_id once `name` is entered.
For now, the sticky flag is used to define bundle to use as target to upgrade.
The plan for future is to continue to test upgrade, even if the bundle is not sticky anymore.
This new flag will allow to remove sticky for old branche.
This will also allow to disable upgrda tests for still sticky branches when needed.
This commit also filter source bundle based on this flag
(before that all base bundle where used as source, even if not sticky)
Currently, runbot is using a single Dockerfile maintained in a data file
in the source code. This situation is not convenient for testing Odoo in
different environments.
With this commit, a Dockerfile Odoo model is used to allow usage of
multiple Docker containers.
This model comes with a pre-defined Dockerfile that can be used to build
the current Odoo supported versions (12.0 up to 14.0).
Runbot initial architechture was working for a single odoo repo, and was
adapted to build enterprise. Addition of upgrade repo and test began
to make result less intuitive revealing more weakness of the system.
Adding to the oddities of duplicate detection and branch matching,
there was some room for improvement in the runbot models.
This (small) commit introduce the runbot v5.0, designed for a closer
match of odoo's development flows, and hopefully improving devs
experience and making runbot configuration more flexible.
**Remotes:** remote intoduction helps to detect duplicate between odoo and
odoo-dev repos: a commit is now on a repo, a repo having multiple remote.
If a hash is in odoo-dev, we consider that it is the same in odoo.
Note: github seems to manage commit kind of the same way. It is possible
to send a status on a commit on odoo when the commit only exists in
odoo-dev.
This change also allows to remove some repo duplicate configuration
between a repo and his dev corresponding repo.
(modules, server files, manifests, ...)
**Trigger:** before v5.0, only one build per repo was created, making it
difficult to tweak what test to execute in what case. The example use
case was for upgrade. We want to test upgrade to master when pushing on
odoo. But we also want to test upgrade the same way when pushing on
upgrade. We introduce a build that should be ran on pushing on either
repo when each repo already have specific tests.
The trigger allows to specify a build to create with a specific config.
The trigger is executed when any repo of the trigger repo is pushed.
The trigger can define depedencies: only build enterprise when pushing
enterprise, but enterprise needs odoo. Test upgrade to master when pushing
either odoo or upgrade.
Trigger will also allows to extract some build like cla that where
executed on both enterprise and odoo, and hidden in a subbuild.
**Bundle:** Cross repo branches/pr branches matching was hidden in build
creation and can be confusing. A build can be detected as a duplicate
of a pr, but not always if naming is wrong or traget is invalid/changes.
This was mainly because of how a community ref will be found. This was
making ci on pr undeterministic if duplicate matching fails. This was
also creating two build, with one pointing to the other when duplicate
detection was working, but the visual result can be confusing.
Associtaions of remotes and bundles fix this by adding all pr and
related branches from all repo in a bundle. First of all this helps to
visualise what the runbot consider has branch matching and that should
be considered as part of the same task, giving a place where to warn
devs of some possible inconsistencies. Associate whith repo/remote, we
can consider branches in the same repo in a bundle as expected to have
the same head. Only one build is created since trigger considers repo,
not remotes.
**Batch:** A batch is a group of build, a batch on a bundle can be
compared to a build on a branch in previous version. When a branch
is pushed, the corresponding bundle creates a new batch, and wait for
new commit. Once no new update are detected in the batch for 60 seconds,
All the trigger are executed if elligible. The created build are added
to the batch in a batch_slot. It is also possible that an corresponding
build exists (duplicate) and is added to the slot instead of creating a
new build.
Co-authored-by d-fence <moc@odoo.com>
