If a PR is *merged*, enqueue it for deletion (with a 2 weeks delay).
Mainly to avoid FW branches staying around long after they've been
merged (possibly eventually closed?), will also clean up regular
merged branches, including historical merges forgotten by their
author.
Fixes#230
In the case where we have a co-dependent forward port (co-dependent
PRs got forward-ported, which they should be together) where *one* of
the PRs got explicitly updated, the batch would "fall into a hole"
being handled as neither "this is part of a forward-port sequence" nor
"this is a new merge to forward-port" (the latter being the proper
one).
Modify & remove guards which checked that either no or all PRs in a
batch have parents: should be either all or not all.
Fixes#231
Turns out tagging PRs requires a pretty significant level of ACLs
which we may not want to give to the forwardbot?
Anyway use the mergebot ACLs (which already include tagging) for this.
Due to the title formatting of FP PRs, we'd get incorrectly formatted
commit messages if the PR was *merged* (rather than squashed /
fast-forwarded) due to either "merge" or "rebase-merge" integration
mode: in that case the PR message would be used as message for the
merge commit and that'd be along the lines of "Forward Port of #xxx to
<somebranch> (failed)", followed by the old PR message (e.g. see this
commit message).
* re-extract and reuse original PR title, just prefix with "[FW]"
* finally add support for tagging, and use that to tag the PRs,
especially for the failed / conflict marker which is quite important
Closes#229
When posting a reminder that there are open / waiting forward ports on
a source PR, also post *which* PRs those are.
While at it, move the cron code in a proper python file (so we can use
stuff from odoo.tools), and fix display_name so we can straight use
display_name as a github ref' ({owner}/{repo}#{number}). This impacts
log-grepping but it seems like an improvement nonetheless.
Closesodoo/runbot#228
* shorten the postfix, forwardbot is now a bigram!
* shorten the uniquifier: go from 5 to 3 bytes, and use urlsafe base64
that way we only have a 4-char uniquifier instead of 8
* while at it, fix deprecated calls to logging.warn (should be
logging.warning)
Fixes#226
The queue would get items to process one at a time, process, commit,
and go to the next. However this is an issue if one of the item fails
systematically for some reason (aka it's not just a transient
failure): the cron fails, then restarts at the exact same point, and
fails again with the same issue, leading to following items never
getting processed.
Fix by getting all the queue contents at once, processing them one by
one and "skipping" any item which fails (leaving it in place so it can
get re-processed later).
That way, even if an item causes issues, the rest of the queue gets
processed normally. The interruption was an issue following
odoo/enterprise#5670 not getting properly updated in the
backend (backend didn't get notified of the last two updates /
force-push to the PR, so it was trying to forward-port a commit which
didn't exist - and failing).
Attempt to avoid some of the comment spam by dedup-ing input (only
signaling when the status actually changes and ignoring identity
transformations) and in case of failing CI keeping the last failed
status and not signaling on the next update if it's the same failure.
Closes#225
When running tests on some machines, it's apparently possible for the
PR-creation webhook to come back before the PR creation request has,
leading to the creation of the PR from the API call duplicating that of
the webhook and blowing up.
To fix, immediately commit the transaction then check if we already have
the PR we just created in the system, and only create it explicitly if
not.
User should probably be warned when they try to set the limit ("up
to") in a context where it's going to be ignored:
* on a forward port PR (should be set on the source)
* on a merged source (should be set before the PR is merged)
Closes#213
In selecting the parent commits to list on the last PR, we would miss
the *first* forward-port of the sequence. Not sure why we added a
detrimental check on source_id there.
Also add a missing space between "chain" and "containing" in the case
where there's at least one forward-port PR other than the final one.
Fixes#212
Test is probably more complex than necessary (thinking about it, the
failed staging is probably unnecessary) but that triggers the issue
and matches the original scenario.
The problem was really with new CI events being received on the last
forward-port PR of a sequence: previous PRs would have a child PR so
the check would abort, but for the last PR it would go through, fail
to find an active batch, then blow up as it tries to create a
forwardport.batch without an actual batch.
Change this to use the existence of an inactive batch not linked to a
staging as a flag that the PR has been processed and forward-ported.
Closes#218
If a PR is explicitly updated, it gets converted to a normal
PR[0]. Before this, users had no indication that this had happened and
might be wondering what they're supposed to do (or try to r+ via the
forwardbot, which doesn't work on a root PR).
[0] to an extent: the PR still has a source and might have children,
in which case the followups will be created from the source &
existing followups should be updated to match
Closes#206
When creating the conflicting commit of a single commit PR, reuse the
original commit's meta-information so the developer / fixer can more
easily update it in-place.
Closes#204
In the case where an FP sequence is interrupted (e.g. there was a
conflict during one of the intermediate steps), followups get linked
to the original source but don't get linked to the "interruption" PR
which is a bit confusing.
Link FP PRs to both source and root if they're different.
* always allow specifying the PR's own branch as a forward-port limit
/ target (even if deactivated or disabled)
* add an "ignore" alias to "up to <pr target>" for clarity
* add dedicated feedback when deactivating forward-port of a PR
Fixes#191
Having all the feedback be sent by the mergebot user (github_token) is
confusing. Add a way to specify which field of project should be used to
source the token used when sending feedback.
Fixes#190
Working copies were created in tempdir under the assumption that
they're, well, temporary.
However after thinking about it more there are two issues with this:
* tempdirs might not be in the same FS as the cache dir, meaning
meaning `git clone` can't hardlink the repo objects and has to
copy them
* tempdirs are often on RAM-backed tmpfs, which is not great when we're
filling them with multiple GB worth of git repository...
If the default limit of a forward-port sequence is not a valid
target (either disabled or not actually forward-ported to), the last
effective forward port in a sequence will be commented on as any
intermediate PR rather than get a proper ping and r+ instructions.
Also remove a bunch of leftover prints in the tests.
Fixes#192
* a high renamelimit increases the cherrypick runtime and can
apparently fail for reasons other than actual conflicts (cf
odoo/odoo#36893) so first attempt to cherrypick with the default
renamelimit, and fallback to renamelimit=0 if that fails
* in that case, filter out the spam of "progress" messages about
inexact rename detection as it's not really useful. There seems to
be no built-in way to suppress these.
Closes#196
Turns out bare repositories (unlike normal ones) don't have any
refspecs by default. So adding the PR refspec would... replace the
default behaviour (apparently?) and the base branches would never get
fetched at all.
--mirror looks to be an other option as it has a default refspec but
it's a bit of an odd duck: it has pull requests in refs/pull but not
all of them? And open / closed doesn't seem to be a factor.
Attempts to use rev-list seem to work locally but they fail *a lot*
when live. The previous attempt to fix it in
0f4c22490c made things worse rather than
better once deployed.
Give up on that (at least for now?), and just ask github what the PR's
commits are then cherrypick exactly that.
The ping message was not clear
- don't know if anything is needed from the author
- don't know if the last step in the chain
Ping the authors only when something needs to be done (error or last
step).
Do not ping non-reviewers as they will not have the rights to r+ or
modify the PR anyway
Closes#192
There's an issue of too many commits being selected for
cherry-picking. It still isn't quite clear, but the forward ports to
11.0 systematically seem to get everything from
5045b5f4c346e60c9b127403ef1fde453d49396a to the PR head, and that
commit was one of the first to land after the last merge-based forward
port.
So the commits selection seems to behave as if the commits range was
`..origin/pull/36692` rather than the `origin/10.0..origin/pull/36692`
which is passed to rev-list. This might be an issue of concurrent
access / update of the refs (though I can't reproduce it locally,
missing refs generate an error).
This change attempts to "pin" the local branch in the working copy
rather than go and get it from the repo. It's unclear that this will
change / fix anything, but it might just work.
Also stop creating shared clones, that seems like an unnecessary
risk (and might be the actual source of the issue).
The original method was to `git diff | git apply` in order to get a
complete overview of conflicts generated by the forward port (if
any).
However this turns out to have a huge issue in the presence of renamed
or removed files: in that case `git apply` will simply not do
anything, and fail with a completely clean working copy. Which is very
much undesirable.
-> alternative method, squash the PR to a single commit then
cherry-pick that single commit, this should provide us with proper
conflicts & their markers.
Also add tests for conflicts due to deleted files...
* Cherrypicking is handrolled because there seems to be no easy way to
programmatically edit commit messages during the cherrypicking
sequence: `-n` basically squashes all commits and `-e` invokes a
subprocess. `-e` with `VISUAL=false` kinda sorta works (in that it
interrupts the process before each commit), however there doesn't
seem to be clean status codes so it's difficult to know if the
cherrypick failed or if it's just waiting for a commit of this step.
Instead, cherrypick commits individually then edit / rewrite their
commit messages:
* add a reference to the original commit
* convert signed-off-by to something else as the original commit was
signed off but not necessarily this one
* Can't assign users when creating PRs: only repository collaborators
or people who commented on the issue / PR (which we're in the
process of creating) can be assigned.
PR authors are as likely to be collaborators as not, and we can have
non-collaborator reviewers. So pinging via a regular comment seems
less fraught as a way to notify users.
