Commit Graph

1179 Commits

Author SHA1 Message Date
Xavier Morel
52699d901a [IMP] forwardport: ping on CI failure
It's especially important as users / assignees don't get
pinged *during* the forwardport process.

closes #203
2019-09-18 08:32:38 +02:00
Xavier Morel
ee8f81be2a [CHG] forwardport: automatically delegate original PR author on FP PRs
This way the original author can r+ the forward ports if they succeed
(and probably requires no attention).

Closes #195
2019-09-17 14:43:21 +02:00
Xavier Morel
73f27873a3 [IMP] forwardport: less spammy reminder cron
* don't warn on every PR on the dot every week, instead wait for the
  PRs to be "sufficiently old" (at least 3 days)
* after discussion with bugfix, the reminder ping should be sent every
  day following the PR being "old enough"
* run the cron every day instead of every week
* add an override context key for test purposes

Closes #198
2019-09-16 15:28:03 +02:00
Xavier Morel
bf1d6c510d [CHG] forwardport: let PR author set the FP limit
Closes #200
2019-09-16 15:07:40 +02:00
Xavier Morel
a1a7d65ebe [IMP] forwardport: move working copy to the cache dir
Working copies were created in tempdir under the assumption that
they're, well, temporary.

However after thinking about it more there are two issues with this:

* tempdirs might not be in the same FS as the cache dir, meaning
  meaning `git clone` can't hardlink the repo objects and has to
  copy them
* tempdirs are often on RAM-backed tmpfs, which is not great when we're
  filling them with multiple GB worth of git repository...
2019-09-16 15:07:40 +02:00
Xavier Morel
f8da17994a [FIX] forwardport: not being properly notified on last FP of a seq
If the default limit of a forward-port sequence is not a valid
target (either disabled or not actually forward-ported to), the last
effective forward port in a sequence will be commented on as any
intermediate PR rather than get a proper ping and r+ instructions.

Also remove a bunch of leftover prints in the tests.

Fixes #192
2019-09-16 15:07:40 +02:00
Xavier Morel
8976f9e310 [FIX] forwardport: attempt no renamelimit & filter out messages
* a high renamelimit increases the cherrypick runtime and can
  apparently fail for reasons other than actual conflicts (cf
  odoo/odoo#36893) so first attempt to cherrypick with the default
  renamelimit, and fallback to renamelimit=0 if that fails
* in that case, filter out the spam of "progress" messages about
  inexact rename detection as it's not really useful. There seems to
  be no built-in way to suppress these.

Closes #196
2019-09-16 15:05:42 +02:00
Xavier Morel
0a64699070 [FIX] forwardport: cached repo setup
Turns out bare repositories (unlike normal ones) don't have any
refspecs by default. So adding the PR refspec would... replace the
default behaviour (apparently?) and the base branches would never get
fetched at all.

--mirror looks to be an other option as it has a default refspec but
it's a bit of an odd duck: it has pull requests in refs/pull but not
all of them? And open / closed doesn't seem to be a factor.
2019-09-12 15:03:45 +02:00
Xavier Morel
60f4db8687 [FIX] forwardport: forgot token when getting PR commits
So would break any time it needed to fetch the commits of a PR in a
private repo.
2019-09-12 14:35:56 +02:00
Xavier Morel
426bc69ea1 [FIX] forwardport: ask github what the PR's commits are
Attempts to use rev-list seem to work locally but they fail *a lot*
when live. The previous attempt to fix it in
0f4c22490c made things worse rather than
better once deployed.

Give up on that (at least for now?), and just ask github what the PR's
commits are then cherrypick exactly that.
2019-09-12 11:58:10 +02:00
Martin Trigaux
48c8e53df2 [FIX] forwardport: clarify forward port message
The ping message was not clear
- don't know if anything is needed from the author
- don't know if the last step in the chain

Ping the authors only when something needs to be done (error or last
step).

Do not ping non-reviewers as they will not have the rights to r+ or
modify the PR anyway

Closes #192
2019-09-12 09:05:49 +02:00
Xavier Morel
0f4c22490c [IMP] forwardport: refs
There's an issue of too many commits being selected for
cherry-picking. It still isn't quite clear, but the forward ports to
11.0 systematically seem to get everything from
5045b5f4c346e60c9b127403ef1fde453d49396a to the PR head, and that
commit was one of the first to land after the last merge-based forward
port.

So the commits selection seems to behave as if the commits range was
`..origin/pull/36692` rather than the `origin/10.0..origin/pull/36692`
which is passed to rev-list. This might be an issue of concurrent
access / update of the refs (though I can't reproduce it locally,
missing refs generate an error).

This change attempts to "pin" the local branch in the working copy
rather than go and get it from the repo. It's unclear that this will
change / fix anything, but it might just work.

Also stop creating shared clones, that seems like an unnecessary
risk (and might be the actual source of the issue).
2019-09-12 09:05:49 +02:00
Xavier Morel
a69ed7996a [FIX] forwardport: change method for conflicting working copy
The original method was to `git diff | git apply` in order to get a
complete overview of conflicts generated by the forward port (if
any).

However this turns out to have a huge issue in the presence of renamed
or removed files: in that case `git apply` will simply not do
anything, and fail with a completely clean working copy. Which is very
much undesirable.

-> alternative method, squash the PR to a single commit then
cherry-pick that single commit, this should provide us with proper
conflicts & their markers.

Also add tests for conflicts due to deleted files...
2019-09-12 09:05:49 +02:00
Xavier Morel
0e3d873f0f [IMP] forwardport: logging & rename during cherrypick
* add slightly better logging to the cherrypick process (init)
* cherrypick with an infinite renamelimit, seems fine according to
  linus[0] and significantly reduces "unsuccessful" forward ports

[0] http://git.661346.n2.nabble.com/Merging-limitations-after-directory-renames-interesting-test-repo-td6041103.html#a6041833
2019-09-12 09:05:49 +02:00
Xavier Morel
4ce4a3bda7 [FIX] forwardport: disable creation of draft PRs
That was only indicative and it doesn't work on private repos.
2019-09-12 09:05:49 +02:00
Christophe Monniez
5f8e80ea77 [IMP] runbot: add dbfread to the Docker image
The new feature in odoo/enterprise#4892 needs the dbfread module.
As this lib is not required for other Odoo modules, it cannot be added
to the requirements.txt file.

In order to run the tests and use this new feature on the runbot, this
commit installs the dbfread lib in the Docker image.
2019-09-10 10:07:54 +02:00
Xavier Morel
4fcebed563 [FIX] forwardport: PR message
* avoid losing original PR message
* add PR source to description
2019-09-10 09:54:25 +02:00
Xavier Morel
79777662df [FIX] forwardport: clone op and add logging 2019-09-10 09:28:30 +02:00
Xavier ALT
f25ab94517 [FIX] runbot: correctly order repo by sequence
Even if present and editable the user, changing the sequence
of a repository has no effect

Fix by adding missing _order on runbot.repo model
2019-09-06 11:01:15 +02:00
Xavier Morel
f671dcc828 [ADD] forwardbot
* Cherrypicking is handrolled because there seems to be no easy way to
  programmatically edit commit messages during the cherrypicking
  sequence: `-n` basically squashes all commits and `-e` invokes a
  subprocess. `-e` with `VISUAL=false` kinda sorta works (in that it
  interrupts the process before each commit), however there doesn't
  seem to be clean status codes so it's difficult to know if the
  cherrypick failed or if it's just waiting for a commit of this step.

  Instead, cherrypick commits individually then edit / rewrite their
  commit messages:

  * add a reference to the original commit
  * convert signed-off-by to something else as the original commit was
    signed off but not necessarily this one

* Can't assign users when creating PRs: only repository collaborators
  or people who commented on the issue / PR (which we're in the
  process of creating) can be assigned.

  PR authors are as likely to be collaborators as not, and we can have
  non-collaborator reviewers. So pinging via a regular comment seems
  less fraught as a way to notify users.
2019-09-05 10:00:07 +02:00
Xavier Morel
1b1aa637fe [IMP] runbot_merge: commit message edition abstraction
Prepares for more complex edition operations on the forwardbot side

* split out the pseudo-headers from the message body
* don't separate the co-authored-by headers from the others, seems
  unnecessary, we just need to ensure they're at the end so github
  doesn't miss them (/it)
2019-09-05 10:00:07 +02:00
Christophe Monniez
c5582ce154 [FIX] runbot: remove skip old builds logic
When finding new commits, if there is more pending builds on a repo than
the running_max parameter, the exceeding builds are skipped.

As a result, when nightly builds are created on the runbot, it happens
that some of them are skipped.

Also, since e51412d , only refs newer than max_age are builded; thus the
logic is not needed to prevent rebuild of olds refs in case of a fresh
runbot install.
2019-09-04 11:05:43 +02:00
Christophe Monniez
cc388715bc [FIX] runbot: properly compute children_build_ids
The Many2many related on a Many2many does not map the ids as expected.
With this commit, the records are mapped in a compute.

It also fixes an uppercase letter was used in the children_build_ids field name.
2019-09-03 15:33:39 +02:00
Xavier-Do
e7d59d4f88 [IMP] runbot: setting multi to 20 builds 2019-09-03 11:10:38 +02:00
Xavier-Do
9a9de6ad85 [FIX] runbot: fix build_end and port
When killed a build could have his build end changed (problematic when
killing a running since build_time must represent the testing time)
-> if a build already has a build end, don't overwrite it.

Port also needs to be reset on wake_up since another build would have
recycle the current one since port unicity is limited to build not in
done state. This was working most of the time before since port unicity
was determined cross runbots, thus we only had one chance over 17
to have a conflict on wake up. (this changed with prevous commit)
2019-09-03 11:10:38 +02:00
Xavier-Do
2e44c51678 [FIX] runbot: limit port search to local host.
With the increasing number of runbot servers (17), the total number of docker
instances can reach more than 3570 only for running build. Starting at 2000,
this covers the posrt 5432 used by postgress and make the build run step fail.

This commit simply limit the port unicity constraint by host.
2019-09-03 11:10:38 +02:00
Christophe Monniez
5297ac0f5f [IMP] runbot: show no_build on branch form 2019-09-03 11:10:38 +02:00
Christophe Monniez
4c75a8e107 [IMP] runbot: add an action to parse logs from build error views 2019-09-03 11:10:38 +02:00
Christophe Monniez
c49e422a25 [IMP] runbot: add an error log model based on a SQL view
With this commit, a new model is introduced to facilitate the tracking
of the build errors.

Its based on an SQL view (Thanks @Xavier-Do), that way, there is no new
table in DB and this view is also useful from the PSQL CLI.

In the UI, the search for errors easier than manipulate the ir_logging
view because the builds informations can be used in search and filters.
2019-09-03 11:10:38 +02:00
Xavier-Do
7382aa6b79 [FIX] runbot: various small fixes and imps 2019-09-03 11:10:38 +02:00
Christophe Monniez
f94c8d7390 [FIX] runbot: use odoo repo for google-chrome
The since google chrome 74, a random bug makes it crash at startup,
making the odoo tests crash.

With this commit, an odoo custom deb repo is used on nightly with a
known working chrome version.
2019-09-03 11:10:38 +02:00
Xavier-Do
a91b08897f [IMP] runbot: improve archive failure logging 2019-09-03 11:10:38 +02:00
Christophe Monniez
57a32ee937 [IMP] runbot: log error instead of info when wake up fails 2019-09-03 11:10:38 +02:00
Christophe Monniez
8d199b4c16 [FIX] runbot: catch exceptions on wake-up
When a build is wake-up and something goes wrong during the
_run_odoo_run method, the "fetch and build" cron is broken and the
concerned runbot host stops working.

With this commit, the exception is catched and the build goes back to
the "done" state whith a log.
2019-09-03 11:10:38 +02:00
Christophe Monniez
0da30a9f60 [IMP] runbot: add build_error model
With this commit, a new RunbotBuilError model is added in order to
classify and manage errors that appears during runbot builds.  This is
an helper to find undeterministic bugs in Odoo builds.  Build logs can
be parsed on demand, during the parsing, the logs are cleaned with some
regexes stored on the RunbotErrorRegex model.  A hash is computed on the
cleaned log, if a build error already exists with the same fingerprint,
the build is appended on the build error.

Errors can also be manually linked together with a parent/children
relation in case of a related error log. e.g. the error message is
different in two different branches but the bug is the same.

Also, a new build_url field is added to the runbot_build in order to
access the build web page from the backend.
2019-09-03 11:10:38 +02:00
Xavier-Do
02d2cc4528 [WIP] runbot: monitoring tools
Add a new model runbot.host to keep info and configuration about
hosts (worker servers), like number of worker, reserved or not,
ping times (last start loop, successful iteration, end loop, ...)
and also last errors, number of testing per host, psql connection
count, ...

A new monitoring frontend page is created, similar to glances
but with additionnal information like hosts states and
last_monitored builds (for nightly)

Later this model will be used for runbot_build host instead of char.

Host are automaticaly created when running _scheduler.
2019-09-03 11:10:38 +02:00
Xavier Morel
ef24adad88 [FIX] runbot_merge: cancel button on staging
* split action_cancel (UI button) from cancel (internal): since the
  xhr mapping is weird, if there are available args the mapper thinks
  it should pass the call context as reason which is unexpected
* make cancel a no-op when called on already inactive stagings
* make cancel work when called on multiple statgings
* make computing the active staging work properly in an
  active_test=False context (e.g. when it's interacted with from the
  form view because that comes from the list view which is
  active_test=False, probably so we can see not just the stagings but
  recursively see deactivated batches in deactivated stagings)
* don't show the cancel button on inactive stagings
2019-08-27 12:28:53 +02:00
Xavier Morel
b1b959d472 [FIX] runbot_merge: properly handle retarget from deactivated branch
A deactivated branch is generally treated as unmanaged which is mostly
correct except for the case of retargeting an existing PR.

When a branch is deactivated the corresponding PRs are not removed, so
it's possible to have live PRs associated with ~unmanamaged
branches. When retargeting those PRs to active branches, the mergebot
would assume there was no existing PR and would create a duplicate,
then either get completely lost (before
a84595ea04) or blow up (after the same).

Properly search amongst deactivated branches for retargeting sources
so we update the relevant PR instead of trying to create duplicates.

Fixes #169
2019-08-27 11:27:02 +02:00
Xavier Morel
e40e814b90 [IMP] runbot_merge: show heads on stagings
Stagings have a "statuses" field which was shown but useless (as it's
a binary), they also have a "heads" field which only provides a
mapping of repository names to commits.

This change provides the staging heads as a commits m2m.

Fixes #178
2019-08-26 17:22:21 +02:00
Xavier Morel
1981bd68e6 [IMP] alter ngrok startup to allow concurrent runs
Running multiple ngrok concurrently is only allowed from pro and up
(OOTB and without shenanigans) is only allowed from Pro and up. However
multiple tunnels through a single ngrok is allowed

-> when tunneling through ngrok, start the process without any tunnel,
use the API to create then remove the local tunnel, and shut down ngrok
IIF there's no tunnel left.

There's plenty of race conditions but given how slow tests are when they
involve github that's probably not an issue.
2019-08-26 13:53:37 +02:00
Xavier Morel
28bcc6b5d7 [IMP] runbot_merge: refactor some bits
* extract method to create a PR object from a github result (from the
  PR endpoint)
* move some of the remote's fixtures to a global conftest (so they can
  be reused in the forwardbot)
2019-08-26 13:53:37 +02:00
xmo-odoo
02d85ad523
[FIX] runbot_merge: less restrictive commands matching
Fixes #167 ignores casing when matching bot name
Fixes #168 ignores leading whitespace when matching commands lines
2019-08-26 13:41:33 +02:00
xmo-odoo
8b74e79da9
[FIX] runbot_merge: FF error logging message
In case of error while fast-forwarding a staging to its source, we'd
log the target to which we couldn't FF. Sadly this relied on a
`repo_name` variable which (likely since the introduction of the
"safety dance" fast forwarding) can not actually be set in case of
failure.

So stash the relevant bit (the repo name) inside the FF error exception
and use that to compose our logging message instead of a variable which
can only be None.
2019-08-26 13:41:11 +02:00
Xavier Morel
222f591deb [IMP] runbot_merge: ACL & PR name
* add missing ACL for PR feedback object
* configure name_get for PRs (which don't have a name), fixes some
  layout issues & stuff
2019-08-26 13:22:29 +02:00
Xavier Morel
0bfb018e49 [IMP] runbot_merge: table for staging history
That way all staging labels (timestamps) have the same width, and PRs
/ batches being wrapped don't look like weird-named stagings.
2019-08-26 11:56:42 +02:00
xmo-odoo
c4b7604999
[ADD] runbot_merge: staging history per branch
Closes #175
2019-08-21 14:15:10 +02:00
xmo-odoo
a84595ea04
[FIX] runbot_merge: indexing of PR objects
Github constrains a single issue (/PR) number per repository, having
different targets does not allow two PRs to share a number.

Doesn't fix but should mitigate #169 slightly.
2019-08-21 11:21:06 +02:00
xmo-odoo
429257d013
[FIX] runbot_merge: resync tags on stage change
Before this change mergebot assumes github's tags are in sync with its
"previous" state, but because tags update was highly non-atomic (one
call per removal plus one for additions) and state can further change
between a failure and an update retry (especially as the labels endpoint
fails *a lot*), it's possible for set tags (in github) to be completely
desync'd from the mergebot state, leading to very misleading on-pr
indications.

This first fetches the current tagstate from github (to not lose non-
mergebot tags) then (hopefully atomically) resets all tags tags based on
the current mergebot state. This should avoid desyncs, and eventually
resync PRs (if they change state).

Fixes #170
2019-08-21 11:17:04 +02:00
Xavier-Do
2a18ef4195 [FIX] runbot: missing indirect
In some case _force can return an empty recordset,
if the corresponding branch is in no_build mode in other
repo may be an explanation here.

This commit avoid to stuck the fetch and build loop in this case.
2019-08-19 12:13:55 +02:00
mreficent
9b174ab706 [FIX] runbot: typo (closet -> closest) 2019-08-14 11:43:03 +02:00