Commit Graph

1441 Commits

Author SHA1 Message Date
Xavier-Do
7519406e3f [FIX] runbot: to_upgrade for complement should use base_id 2020-11-26 13:35:09 +01:00
David James
190f2ed356 [IMP] runbot: serve badges for projects which are not accessible to the public 2020-11-24 15:18:20 +01:00
Xavier-Do
4ab669e371 [IMP] runbot: add to_upgrade option
For now, the sticky flag is used to define bundle to use as target to upgrade.
The plan for future is to continue to test upgrade, even if the bundle is not sticky anymore.
This new flag will allow to remove sticky for old branche.
This will also allow to disable upgrda tests for still sticky branches when needed.

This commit also filter source bundle based on this flag
(before that all base bundle where used as source, even if not sticky)
2020-11-24 15:14:23 +01:00
Christophe Monniez
6aab868883 [IMP] runbot: add a commit form and search 2020-11-24 15:13:59 +01:00
Xavier-Do
d76a41537c [IMP] runbot: add batch_dependent triggers
Params mechanism are a way to avoid to rebuild with the exact same params.
Commit is the main identifier to know if two builds are the same, ususally
commit_link are not usefull to uniquify params, exept when the mergebase is
in use as in security check (based on diff).

We would also like to avoid to have red diff-based build in sticky/base branches when the
ci is ignored on a pr, but we want to keep the diff display on new batches
(lines and files changed).
We need a way to know if it is a is_base bundle. This was an initially forbidden behaviour
because of duplicate detection.

This commit add a batch_dependent flag. This will enable the use of the batch in the
fingerprint, making it unique per batch and disabling duplicate detection/ in this
case we can use the "is_base" information from the bundle + avoid false duplicate
detection if trigger is based on mergebase-commit diff. (pr based on another pr now merged)
2020-11-24 15:13:00 +01:00
Christophe Monniez
2556145843 [FIX] runbot: do not count linked errors
On the build errors frontend page, the linked errors are displayed too.
With this commit, they are removed from the search query.
2020-11-23 12:15:21 +01:00
Xavier Morel
5c19342bf6 [CHG] runbot_merge, forwardport: remove labelling
Because github materialises every labels change in the
timeline (interspersed with comments), the increasing labels churn
contributes to PRs being difficult to read and review.

This change removes the update of labels on PRs, instead the mergebot
will automatically send a comment to created PRs serving as a
notification that the PR was noticed & providing a link to the
mergebot's dashboard for that PR where users should be able to see the
PR state in detail in case they wonder what's what.

Lots of tests had to be edited to:

- remove any check on the labels of the PR
- add checks on the PR dashboard (to ensure that they're at least on
  the correct "view")
- add a helper to handle the comment now added to every PR by the 'bot
- since that helper is needed by both mergebot and forwardbot, the
  utils modules were unified and moved out of the odoo modules

Probably relevant note: no test was added for the dashboard
ACL, though since I had to explicitly unset the group on the repo used
for tests for things to work it looks to me like it at least excludes
people just fine.

Fixes #419
2020-11-20 07:41:54 +01:00
Xavier Morel
c80d8048f7 [ADD] runbot_merge: PR dashboard
Provides a view of the "ongoing" state of a PR in order to more easily
know what might be blocking / problematic.
2020-11-20 07:41:54 +01:00
Xavier Morel
47e8b5b014 [IMP] runbot_merge: overridable CI
Convert overridable CI to an m2m from partners, it's significantly
more convenient to manipulate as multiple users can (and likely will)
have access to the same overrides, add a name_search so the override
is easy to find from a partner, and provide a view for the
overrides (with partners as tags).

Also make the repository optional on CI overrides.

Fixes #420
2020-11-20 07:41:54 +01:00
Xavier Morel
8abf1be278 [FIX] runbot_merge: cancel approval (r+) when fetching PRs
When retrieving unknown PRs, the process would apply all comments,
thereby applying eventual r+ without taking in account their
relationship to a force push. This means it was possible for a
mergebot-unknown PR to be r+'d, updated, retargeted, and the mergetbot
would consider it good to go.

The possible damage would be somewhat limited but still, not great.

Sadly Github simply doesn't provide access to the entire event stream
of the PR, so there is no way to even know whether the PR was updated,
let alone when in relation to comments. Therefore just resync the PR
after having applied comments: we still want to apply the merge method
& al, we just want to reset back to un-approved.

An other minor fix (for something we never actually hit but could):
reviews are treated more or less as comments, but separate at github's
level. The job would apply all comments then all reviews, so the
relative order of comments and reviews would be wrong.

Combine and order comments and reviews so they are applied
in (hopefully) the correct order of their creation / submission.

Closes #416
2020-11-20 07:41:54 +01:00
Xavier Morel
963ba9e911 [FIX] runbot_merge: name_get using the wrong field on override rights 2020-11-20 07:41:54 +01:00
Xavier Morel
be7788be0a [FIX] forwardport: git conflict markers
2.29 changed the formatting of conflict labels (the stuff added at the
end of the closing conflict marker):
6cceea19eb

Used to be

    >>>>>>> <commit>... <text>

now is

    >>>>>>> <commit> (<text>)

Just ignore everything after the commit hash.
2020-11-20 07:41:45 +01:00
Christophe Monniez
733ab2dd43 [IMP] runbot: try to improve dashboard bundle line title 2020-11-18 16:49:45 +01:00
Christophe Monniez
9637976fef [IMP] runbot: bring back copy bundle name button 2020-11-18 16:49:45 +01:00
Christophe Monniez
9c7cee657d [IMP] runbot: improve build error frontend page
On the build error web page, a regular assigned error is not shown to
all users.

With this commit, a regular build error (not only random) will be shown
if the error is assigned.
2020-11-18 16:49:45 +01:00
Christophe Monniez
c6a1d202a1 [IMP] runbot: parse log redirects to the created error
When builds logs are parsed by using the contextual button, the client
stays on the same page even if a build error is created.

With this commit, the client is now redirected to the created/found
build error(s).
2020-11-18 16:49:45 +01:00
Christophe Monniez
e51a27bbbb [FIX] runbot: fix build error creation form 2020-11-18 16:49:45 +01:00
Xavier-Do
87512ae78c [IMP] runbot: improve coverage
- execute coverage results in case of failure
- fix coverage links
- use coverage directly
2020-11-18 13:36:41 +01:00
Xavier-Do
559e10754e [FIX] runbot: fix wakeup 2020-11-12 14:02:00 +01:00
Xavier-Do
3657a65b20 [IMP] runbot: call docker_run outside steps
When calling a step from a python step, it is impossible to alter some parameter, the only solution is to copy all step code.
With this change, python step are now able to override docker_run parameter of another step by modifying the dict returned by run_* steps
and assigning the result to "docker_params".

This will mainly be used to set the correct docker_image for migration pre and post test. This can also be used to alter a command,
like removing the pip install or adding extra pre/post operation on the command.
2020-11-11 10:47:26 +01:00
Christophe Monniez
4dec2c5f5c [FIX] runbot: fix warning during dockerfile test 2020-11-10 17:33:18 +01:00
Christophe Monniez
79f7fc9ed2 [FIX] runbot: fix default image_tag
Also fix Docker build process return code.
2020-11-10 11:03:30 +01:00
Christophe Monniez
981cb7e1b6 [IMP] runbot: use a Dockerfile model
Currently, runbot is using a single Dockerfile maintained in a data file
in the source code. This situation is not convenient for testing Odoo in
different environments.

With this commit, a Dockerfile Odoo model is used to allow usage of
multiple Docker containers.

This model comes with a pre-defined Dockerfile that can be used to build
the current Odoo supported versions (12.0 up to 14.0).
2020-11-10 09:52:48 +01:00
Xavier-Do
92ac1d4737 [FIX] runbot: fix field label 2020-10-20 15:26:41 +02:00
David James
237e931e57 [IMP] runbot: add extra parameters and environment variables to run stages 2020-10-19 11:29:39 +02:00
Xavier-Do
7e2d7c65a1 [FIX] runbot: fix message formating 2020-10-19 01:30:39 +02:00
Xavier-Do
43075a59e6 [IMP] runbot: allow to define errors policy for _git decode 2020-10-15 16:54:47 +02:00
Xavier-Do
1985917702 [FIX] runbot: add limit on log length 2020-10-15 15:55:33 +02:00
Xavier-Do
45034d362d [FIX] runbot: inverted condition 2020-10-15 12:09:40 +02:00
Xavier-Do
bd69c0dfa0 [FIX] runbot: manual is on trigger, not slot 2020-10-15 11:28:20 +02:00
Xavier-Do
f9272cb207 [IMP]: add linitng tools (+bunch of small fixes) 2020-10-15 10:38:36 +02:00
Xavier-Do
c3cc9013df [IMP] runbot: use last pylint version 2020-10-14 18:08:47 +02:00
David James
58eefd3b52 [FIX] runbot: serve badges for non-default projects 2020-10-12 16:39:57 +02:00
Xavier Morel
dd0905f8d3 [IMP] runbot_merge: handling of PRs on disabled branches
Historically PRs to disabled branches were treated like PRs to
un-managed branches: ignored.

However because they cay *already exist* when the branch is disabled,
the effects can be subtly different, and problematically so
e.g. ignoring all PR events on PRs targeting disabled branches means
we can't close them anymore, which is less than great.

So don't ignore events on PRs to disabled branches (creation, sync,
closing, and reopening) but also send feedback on PRs to disabled or
un-managed branches to indicate that they're not merge-able.

Fixes #410
2020-10-06 14:45:49 +02:00
Xavier Morel
c61c1c74a0 [FIX] runbot_merge: remove widget on branch_filter
Apparently:

* this requires a `model` option (makes sense)
* this blows up in a list view (makes less sense)
2020-10-05 08:02:14 +02:00
Xavier Morel
3b28d7801d [IMP] runbot_merge: be more optimistic about PR mergeability
If we can't stage a PR, rather than immediately put them in error wait
until they were the first we tried staging, otherwise they might have
been conflicting with the previous batch which ultimately won't be
merged for other reason and they would have worked as part of the next
batch.

Note: this commit will lead to false negatives because it's
batch-based rather than repo-based, so if the first batch only affects
repo A and the second batch only affects repo B, if the second batch
triggers a merge error it should be rejected immediately (as it's
applied on a "pristine" repo B) but this change will just bump it to
the next staging.

fixes #209
2020-10-02 15:28:36 +02:00
Xavier Morel
c78ffb9e3f [CHG] runbot_merge: branch_ids -> branch_filter
On per-repo status configurations, convert the "branch_ids" filter to
a domain on branches. Since the selection is generally
binary (statuses either apply to the master branch or apply to
non-master branch) this avoids error-prone missed updates where we
forget to enable statuses pretty much every time we fork off a new
branch.

Fixes #404
2020-10-02 15:28:36 +02:00
Xavier Morel
2fafd5419a [FIX] runbot_merge: attempt to stage PRs on disabled branches
Normally opening a PR against a disabled branch is like opening a PR
against a branch which is not configured at all: the PR id ignored
entirely.

However if the PR already exists then the state of the branch isn't
normally checked when interacting with the branch, and it is possible
to trigger its staging, at which point the staging itself will crash:
on a project the branches are `active_test=False` so they're all
visible in the form, but when repos go search()-ing for the branch
they won't find it and will blow up.

Solution: only try staging on branches which are active. Fixes
odoo/runbot#408. Also do the same for checking stagings.

And while at it, fix #409 by wrapping each checking or staging into a
try/except and a savepoint. This way if a staging blows up it should
move on to the next branch instead of getting stuck.
2020-10-02 15:28:36 +02:00
Xavier Morel
c93ce7c2c2 [FIX] runbot_merge: blocked flag false positive
The "blocked" computation would not take branch targets in account, so
PRs with the same label targeting *different branches* (possible if
somewhat rare due to our naming conventions) could block one another,
despite really being unrelated.

Also fix up some messages:

* if a PR is blocked due to having no merge method, it should say
  that, not "has no merge" (no merge what?)
* format un-managed branches as `$repo:$branch` in logging messages,
  `$repo#$thing` is for issues / PRs and `$branch` alone can be very
  unhelpful

Closes #407
2020-10-02 15:28:36 +02:00
Christophe Monniez
91de4f9314 runbot: bump pdfminer.six to version 20200720
Odoo.sh team discrovered that pip also install pycryptodome alongside
with pdfminer 20181108. This installation causes clashes with pycrypto
that may be used elsewehere in Odoo code. This version was used in order
to stick with the Debian version but it appears that Debian fixed that
issue in the deb package [1] and [2]. As the runbot uses the pypi
version instead of the patched Debian package, the runbot could face the
same problems.

So as the issue is fixed in 20200720, this commit will install it in the
Dockerfile.

So we can stay sync with Odoo.sh. Thanks guys.

1.https://metadata.ftp-master.debian.org/changelogs/main/p/pdfminer/pdfminer_20181108+dfsg-3_changelog
2.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919008
3.https://github.com/pdfminer/pdfminer.six/releases/tag/20200720
2020-10-01 20:15:45 +02:00
Xavier-Do
67f6eb12c5 [IMP] runbot: improve build order
The old order based on status is not really needed anymore:

- Scheduled builds have a special condition and already have a low priority.
- Indirect builds don't exist anymore
- It is actually questionnable to postpone rebuild. Sometimes they are needed but stuck.

This new proposition will keep subbuild scheduling close to parent build. Some build may take 2 hours
because they are parallelized and children are stuck. Priority should be defined by top parent.
2020-09-25 15:20:26 +02:00
Christophe Monniez
6c3f14ed61 [IMP] runbot: add bundle_ids to ErrorLog SQL view 2020-09-25 11:18:46 +02:00
Xavier-Do
0e2f75c5d6 [FIX] runbot: typo 2020-09-23 14:09:23 +02:00
Xavier-Do
03b82e712a [FIX] runbot: allow to disable screencast globaly 2020-09-23 13:49:35 +02:00
Xavier-Do
b991eb5b70 [IMP] runbot: enable screencast, add access control, small fixes 2020-09-23 13:27:59 +02:00
Christophe Monniez
17f0458d06 [FIX] runbot: host must be null instead of emptry string 2020-09-23 00:29:44 +02:00
Christophe Monniez
6cf0b3e128 [IMP] runbot: add ffmpeg tool to Dockerfile
This will allow screenscasts on runbot.
2020-09-22 19:57:49 +02:00
Christophe Monniez
73426d9348 [IMP] runbot: add pdf417 library
The pdf417gen python library is needed for the l10n_cl_edi. As this module
is not mandatory, the library was not added in the requirements.txt
file.

In order to test the feature on the runbot, this commit adds the library
in the Docker container.

Related work:
    odoo/odoo#54995
    odoo/enterprise#12080
    odoo/enterprise@8290bfaf
2020-09-15 13:58:56 +02:00
Shyju Kanaprath
85c2ef4ee8 Update README.md 2020-09-15 13:06:37 +02:00
Shyju Kanaprath
9aa62ac6f1 Update README.md 2020-09-15 13:06:37 +02:00