The previous code of runbot and runbot_cla was made for Odoo API version
8.0. This commit makes it work with Odoo API 11.0 and Python 3.
Also, the present refactoring splits the code into multiple files to
make it easier to read (I hope).
The main change due to Python 3 is the job locking mechanism:
Since PEP-446 file descriptors are non-inheritable by default.
A new method (os.set_inheritable) was introduced to explicitely make
fd inheritable. Also, the close_fds parameter of the subprocess.Popen
method is now True by default.
Finally, PEP-3151 changed the exception raised by fcntl.flock from IOError to OSError
(and IOError became an alias of OSError).
As a consequence of all that, the runbot locking mechanism to check if a
job is finished was not working in python3.
The linked build is extracted from dbname via a trigger on new row
creation.
This improve rendering of build page.
A migration script is added to link existing logging entries to
corresponding build and remove invalid ones.
Note that this migration script can take a long time.
Allow logged users to kill builds that are pending or testing.
The kill button solves the following problem:
When you have a community and enterprise branch with pull request.
The enterprise PR gets fetched before the branch by runbot, so it
builds with the community master branch instead of the corresponding
community branch. As the branch shares the same HEAD as the PR,
so no other build is done for the branch.
Previously you had to wait the build to finish building, then rebuild it
from the branch, taking twice the time necessary. Now you'll be able to
kill & rebuild it just after pushing the branch.
Also having a kill button is globally a nice to have feature.
In a repository with dependencies
e.g. `odoo-dev/enterprise` depending on `odoo-dev/odoo`
consider a build as a duplicate of another if
- The HEAD is identical (as before)
- AND the branches used for the depending repositories are the same
e.g.
- pushing `master-staging-dle` to `odoo-dev/enterprise`,
- with as HEAD the same HEAD than `master`,
- but for which a branch `master-staging-dle` exists in `odoo-dev/odoo`
- a duplicate would be a build having as HEAD the HEAD of master
AND the `odoo-dev` branch `master-staging-dle`
Before, it would have used the latest build of the `master` branch,
as the HEAD was the same, despite the fact it didn't use the branch
`master-staging-dle` of `odoo-dev/odoo`
The `git gc --auto` command often fails and requires an explicit
execution of `git gc --prune=all`. When it happens, it crashes the
update, failing the cron execution.
Executing `git gc prune` every time would be very slow and dispruptive
during the cron's critical sections.
As the initial goal was to avoid triggering the automatic GC during the
next update/pull command, we prefer to adopt the global strategy of
disabling the auto GC globally with:
git config --global gc.auto 0
And to configure a system cron job to periodically GC/prune all
repositories, to get the performance benefit.
Due to locking issues, that cronjob should be scheduled outside peak
build hours.
Example bash script for the system cron:
```
#!/bin/bash
set -e
base_repo_dir='/path/to/runbot/static/repo'
for repo in $base_repo_dir/*
do
if [[ -d $repo ]];then
git --git-dir="$repo" gc --prune=all
fi
done
```
Catch exceptions occurring within job methods, to avoid
stalling the runbot by repeatedly trying to call it -
and crashing every time.
Mark the build as failed immediately and skip it.
Most of the non-CRUD methods of runbot models should be private, as
there is no need to access them via RPC.
WARNING: This change is backwards compatible with existing
installations and existing runbot databases, but will most
likely BREAK all third-party extensions, due to the renamed
methods.
Fortunately the runbot module is not ruled by Odoo's "stable policy" ;-)
Reported by @nilshamerlinck - thanks!
When doing a `git archive`, all the files in the tar archive are set to
the commit date. Ignore this date during extraction.
Having the files with dates in future will make some tests about assets
to fail. These tests force the asset regeneration by `touch`ing one
js/css file. If other bundle files are still older that the one we touch,
the asset is not regenerated and the test fail.
This commit() is the root cause of the famous false-positive error
`_check_module_names invalid module names, ignored: False`.
The first step (pull code) is a long one. if a push-hook request
happen during this step, the current transaction wont be able to be
commited (the list of module to test is written on the build). Due to
this early commit(), the build is left in an intermediate state with
the module list unpopulated (=False).
To find out the list of manually created dbs,
rev. 4d94f45 used the cursor that is connected to the
master `runbot` database, which may reside on a different
cluster/host.
- Add a helper to run commands on the local PG cluster
instead ("postgres" database).
- Modify other commands for local cluster (create/drop db)
to use the same cursor. Rename those methods to _local_*
to better indicate their local effect.
- pos_blackbox_be: because it makes the POS unuseable without a blackbox
- pos_cache: because it's too confusing for runbot users, everytime they
update a product they have to also update the cache, but
most people don't realise that.
As of Odoo 9 the first l10n module installed will automatically
install its chart of account on the main company, so installing
the first one (l10n_ae) is surprising/strange.
Branches that want to build l10n_* modules should list
them explicitly in the "modules to install" field (modules).
In addition, having demo data to set the country of the
main company will automatically installe the relevant
l10n module if it exists. This is the default in Odoo 9
with l10n_us.
This allows configuring an explicit list of modules to install
on a repository without having orange builds (with warnings)
whenever a branch does not have one of these modules for some
reason.
This lets us use them in README files
on GitHub, which uses an image proxy
that does not have access to private
repositories.
This is an acceptable disclosure of
information about private repos.
The web addon is supposed to be duplicate, and needs to be installed
(in the enterprise version) over the community edition.
A better fix would be to keep the folder and use the --addons-path key
to start the server with the correct addon, but this is an urgent
change.
closes#69
pushing new commits on your branch should not make you loose rank in schuler
build order is based on sequence which is the id of the build, take lowest
When a build is skipped, `build.host` is `False`.
The proposed link is then just `http://rubot/static/build/...` which is invalid.
As there is no logs anyway, hide the buttons.
The base field is used to compose URL (e.g. "https://%s/pulls" % repo.base).
The URL given by github can contains trailing '.git' so this should be removed.
