Commit Graph

2062 Commits

Author SHA1 Message Date
Xavier-Do
5406d29f8f [FIX] runbot: fix prepare 2024-01-25 08:40:42 +01:00
Christophe Monniez
c6f9d1f0c5 [IMP] runbot: use real log date on build errors
In the build error view, a list of build is displayed with a confusing
create date. The create date in the list is the creation date of the
build, leading to a confusion with the creation of the build log
creation.

With this commit, the real log creation is used in this view.

To achieve that, the many2many relation is extended with a
log_date which is filled when a build log entry is parsed.
2024-01-24 13:02:22 -01:00
Christophe Monniez
d2bb42264e [IMP] runbot: give more information about known errors 2024-01-24 13:02:22 -01:00
Christophe Monniez
7c0c03753d [IMP] runbot: add warning ribbon when an error is a link
It happens that a user edits or annotates a build error form without
noticing that the error is linked to another one. In that case, the
modification or the note is probably useless. So a warning ribbon should
grab his attention.

While at it, we change the warning ribbon when an error has a test-tag
to not be confused with the link ribbon and also because removing a
test-tag can lead to a real danger (for the mergebot stagings).
2024-01-24 13:02:22 -01:00
Christophe Monniez
ef045c8540 [IMP] runbot: move builds on host page into a tab 2024-01-24 13:02:22 -01:00
Christophe Monniez
05d072d6eb [IMP] runbot: add a frontend button to view batch in backend 2024-01-24 13:02:22 -01:00
Christophe Monniez
c322458c5b [IMP] runbot: add a frontend btn to parse a single log 2024-01-24 13:02:22 -01:00
Christophe Monniez
68892398cd [FIX] runbot: fix error_log parse_log method 2024-01-24 13:02:22 -01:00
Christophe Monniez
c837e7330c [IMP] runbot: remove useless prefix on build errors 2024-01-24 08:54:39 -01:00
Christophe Monniez
6030988560 [IMP] runbot: allow to filter team errors by trigger 2024-01-24 08:54:39 -01:00
Christophe Monniez
bef9dc0e49 [FIX] runbot: properly compute error versions and triggers 2024-01-24 08:54:39 -01:00
Christophe Monniez
f47beb31ba [FIX] runbot: fix the favorites filter bug 2024-01-24 08:54:39 -01:00
Christophe Monniez
2999f92cdb [IMP] runbot: allow to search errors in a single version
When filtering the build error tree view based on the versions equality,
the results may not be what you expect.

e.g.: searching for `versions is equal to 16.0` gives the errors that
appeared in `16.0` (hopefully) but also those which appeared in other
versions too.

With this commit, this search will give the errors that appeared in the
specified version only. When the user wants to list errors that appeared
in `16.0` and other versions too, he has to use the `contains 16.0`
criteria.
2024-01-24 08:54:39 -01:00
Christophe Monniez
08c614a86e [IMP] runbot: add optional fingerprint in error list view 2024-01-24 07:43:33 -01:00
Christophe Monniez
bddac0a645 [IMP] runbot: allow to choose the replacement string
When defining cleaning regex, the replacement character is always the
percent sign as it's hard coded in various methods.

With this commit, a replacement string can be defined by cleaning regex
and fallback to the percent sign by default.
2024-01-24 07:43:33 -01:00
Christophe Monniez
7ce79b434e [IMP] runbot: improve build error cleaning action
With this commit, when build errors are re-cleaned, they are also merged
if the fingerprints when fingerprints are matching.

Also, this fixes the ir_logging compute that associate a build error
so that the active build error is preferred over an inactive one.
2024-01-24 07:43:33 -01:00
Xavier-Do
c513edc6ad [IMP] runbot: allow to use dev foreigh branches
Using dev branch from foreign project to fill missing commits looks like
a bad idea, mainly because the lifecycle of the branches is not the same

In some case, it can be useful to allow that to test a branch with a
future change in the base project that will be needed to make the branch
work. As an example introducing a small api change in odoo to make an
override easier, or introducing a module that may be needed to use
the feature.

This commit changes that by allowing to configure on the project or
bundle if we allow to use foreign bundle as reference *before* checking
the base bundle.
2024-01-24 09:30:10 +01:00
Xavier Morel
95393afde8 [FIX] runbot_merge: extraction of authorship info during rebase
Turns out I've always been mistaken about the handling of quotes
*inside* shell parameters, apparently they are always consumed by the
shell unless nested so

    --foo="bar"

reaches the underlying program as

    --foo=bar

This means when using subprocess (without shell=True), adding the
quotes leads to mishandling of the parameters (as the subprocess now
has quotes it's not equipped to deal with).

This exact error is made in the `--pretty` parameter of git show,
locally this results in the author name and the committer email being
terminated by double quotes although somehow other layers seem to
exclude those from the end result (I assume `commit-tree` strips the
quotes from the envvars under the assumption that users can mistakenly
quote them or something?

Anyway while it does not seem harmful (so far), better safe than
sorry.
2024-01-22 15:36:37 +01:00
Xavier Morel
64d80c276b [FIX] *: tests not working with github actual
Add intermediate forks to a pair of tests, because github now (?)
requires being able to write on a branch to create a PR from it, so
the non-collaborator reviewers were not able to create a PR from a
branch created by user.
2024-01-16 15:07:25 +01:00
Xavier Morel
4b9fb513eb [IMP] *: make to_pr more resilient to webhook delays
Github delivery delays keep getting worse. Depending on what comes
before `to_pr`, this leads it to fail more often as it runs before the
PR it's looking for was signaled to the mergebot.

In order to mitigate this issue, add a wait loop in `to_pr`, waiting
up to 4 seconds for the PR it's looking for before aborting.

Also replace manual lookups by `to_pr` in every method of
`TestPRUpdate` while at it since it hit a few of the issues. And
remove the xfail test case since it seems unlikely github will change
tack (maybe? could be worth testing to be sure).
2024-01-16 15:03:45 +01:00
Xavier Morel
7054c865d7 [FIX] *: ngrok tunnel for 3.x
ngrok 3 scrambled some of the tunnel configuration keys. Most notably,
it replaced the ill-named `bind_tls` by an explicit list of http
`schemes`. Although it *removed* `bind_tls` so the fix is necessary
for ngrok to work again (especially as ngrok2 is reaching EOL).

While at it, improve the tunnel setup somewhat:

- remove fixme which we're probably not going to fix after all
- if we spawn ngrok ourselves, keep the handle around so we can
- kill the process we spawned directly instead of looking it up
  somewhat awkwardly
2024-01-16 14:59:01 +01:00
Xavier Morel
cea1b62ac2 [FIX] runbot_merge: commit messages should be trimmed indeed
Reverts commit 85a7890023 which
untrimmed the commits: while it's *probably* true that git and
github's APIs differ in their treatment of whitespace (in that git
pretty much always terminates the commit message with a newline while
github does not, as far as I understand, though I didn't really
validate it) the issue was that github also trims on *output* when
fetching over the API, something the fake did not do.

So rather than update the test data I should have fixed the fake, but
I failed to realise that at the time. I only realised when I decided
to re-run against github actual (something I rarely do anymore as it's
painfully slow) and it went on to choke on every message I'd updated.
2024-01-16 10:51:37 +01:00
Xavier Morel
45f0c8cc81 [FIX] runbot_merge: rebase logging
The logging line was copied over from the github-api version, but it
was not correctly fixed up to match, leading to a lot of spam on
stderr when debug is enabled (aka spams journalctl on the production
server).

Splat the logging call out of `rebase` and into the various callers,
so they have access to the pr object to log it.
2024-01-16 09:53:57 +01:00
Xavier Morel
1cb31cf2c2 [FIX] runbot_merge: 1.9 version & migration
Forgot to bump the version when creating the migration. Also convert
the migration to a single sql query, although the migration will never
run because I ran the query manually to fix things up after finding
out the data was "dirty" since the new code (assuming only modern
statuses) was merged without running the migration.

Thankfully it looks like the impact was not too severe (because the
legacy statuses should only be present on very old commits / PRs), I
don't remember when I deployed the update but apparently just a pair
of PRs got affected, because their `previous_failure` was the old
style and thus broke the "new failure" check.
2024-01-16 09:44:13 +01:00
Xavier Morel
994cea467c [FIX] runbot_merge: typo in freeze wizard
Forgot to deref the id of the staging we're trying to lock, so the
specific case where we start a freeze with a bump PR and an
outstanding staging in master would instantly blow up.
2024-01-16 07:54:43 +01:00
Xavier-Do
3f260ba08f [FIX] runbot: wakeup child without parent db
A check was add to avoid to wakeup a child if there is a parent database
Most of the time, it was a mistake.

In some case it can be legit, if the parent only creates subbuid without
installing any database.
This commit fixes that by allowing to wake up child if the parent has no
database.
2023-12-15 10:41:21 +01:00
Xavier-Do
fe87569044 [FIX] runbot: allow rebuild when killed
The feature was there but broken, lets fix it.
2023-12-05 10:06:18 +01:00
Xavier-Do
bf2528dbd0 [FIX] runbot: avoid non local redirect
The goal of this feature is to redirect to the build
2023-12-05 10:06:18 +01:00
Xavier Morel
b21fbaf9cc [IMP] runbot_merge: prevent merging empty commits
The low-level APIs used by the staging process don't do any merge
check, so because of the way Git works it's possible for them to merge
commits with content as empty commits, e.g. if something was merged
then backported and the backport was merged on top. This should
trigger a merge failure as we don't really want to merge newly
empty. This is a feature which some high level commands of git
support, kind-of, e.g. by default `git rebase --interactive` will ask
about newly empty commits.

Take care to allow merging already-empty commits, as these do have a
use for signaling, freezes, ....

Fixes #809
2023-11-30 12:45:39 +01:00
Xavier Morel
2cd3fb8999 [IMP] runbot_merge: make uniquifier commit optional
Prepares the possibility of either more direct communication with the
CI platform(s) or just assuming CI has gotten reliable enough and
colleagues intelligent enough that this is not an issue anymore
because they've stopped pushing empty branches (which we know is not
the case).

Fixes #806
2023-11-30 12:45:39 +01:00
Xavier Morel
a15086a8a9 [FIX] runbot_merge: "not something we can merge" freeze error
During the 17.0 freezeathon, the freeze wizard blew up with

    MergeError: merge-tree: {oid} - not something we can merge

Turns out when freezes were moved to local
(4d2c0f86e1) I forgot to fetch the heads
of the release and bump PRs into the local repo, so rebasing them atop
their branch would fail because the local repository would just not
find the object being rebased.

I had missed that case in testing as well, but in fairness even if I
had tried testing it I'd likely have missed it: implementation
limitations (shortcuts) of dummy central mean it currently ignores
what objects the client requests and bundles everything it can find
associated with the repository (meaning it sends the entire network).

This is not usually an issue because the test repos are pretty small,
but it means the client can have objects they should not because they
never requested them and might not even be supposed to be aware of
their existence.

Anyway solve by doing the obvious: fetch the heads of the release and
bump PRs at the same time we update the branch being forked off. Also
update the freeze tests to trigger the issue (by creating the release
/ bump PRs in different repos) and running the tests against github
actual to make sure we can actually see them fail (correctly, the
merge error we expect) not via errors in the test), and we do fix
them.

Fixes #821
2023-11-30 12:45:39 +01:00
Xavier Morel
134ce03053 [FIX] forwardport: fix sourcing on reparenting
When reparenting a commit (mostly when inserting a new forwardport in
an existing chain after a freeze / branch insertion), the new source
should be the source of the new parent (which is likely a not-change
of the source).

This was miscomputed to the root of the new parent, which often
matches but breaks if there was a conflict or a mid-port update,
leading to inconsistent presentation. Nothing critical, just somewhat
annoying.
2023-11-30 12:45:39 +01:00
Xavier Morel
f44b0c018e [IMP] forwardport: allow updating the fw limit after merging the source
Currently, once a source PR has been merged it's not possible to set
or update a limit, which can be inconvenient (e.g. people might have
forgotten to set it, or realise after the fact that setting one is not
useful, or might realise after the fact that they should *unset* it).

This PR relaxes that constraint (which is not just a relaxation as it
requires a bunch of additional work and validation), it should now be
possible to:

- update the limit to any target, as long as that target is at or
  above the current forwardport tip's
  - with the exception of a tip being forward ported, as that can't be
    cancelled
- resume a forward port stopped by a previously set limit (just
  increase it to whatever)
- set a limit from any forward-port PR
- set a limit from a source, post-merge

The process should also provide pretty chatty feedback:

- feedback on the source and closest root
- feedback about adjustments if any (e.g. setting the limit to B but
  there's already a forward port to C, the 'bot should set the limit
  to C and tell you that it happened and why)

Fixes #506
2023-10-06 13:19:01 +02:00
Christophe Monniez
383352d88e [FIX] runbot: fix _get_commit_infos 2023-10-04 14:35:23 +02:00
Christophe Monniez
c2ef253ec9 [IMP] runbot: retrieve commit infos when needed
In some circumstances, a commit is created from scratch with only a hash
for sole information. This is not very convenient and can lead to issues
that may be difficult to investigate.

With this commit, when creating this kind of commit object, we try to
get commit informations from git repo.
2023-10-04 09:00:35 +02:00
Christophe Monniez
b5b90557fd [IMP] runbot: add a backend commit links tree view
It's sometimes usefull to find a commit in the commit links.
2023-10-04 09:00:35 +02:00
Christophe Monniez
929bc33569 [IMP] runbot: allow to call the commit route with a hash
With this commit, it's not possible to go directly to the commit
frontend page with a commit hash with its id or directly with the hash
value.
2023-10-04 09:00:35 +02:00
Christophe Monniez
fef9ae9801 [FIX] runbot: ensure that commit date is always set
When exporting a commit, the commit date is used in the `tar` command to
set the date of the exported folder. On the other hand it happens that a
commit is not found in the database and should be quickly created on the
fly. e.g.: with the `_get` method. In this case, if the commit needs to
be exported later, the method fails and may break a runbot build.

It happened with a custom python step.
2023-10-04 09:00:35 +02:00
Xavier-Do
ce47661bbc [IMP] runbot: allow to define a per host nb run 2023-10-03 10:38:55 +02:00
Xavier-Do
06e088f82e [IMP] runbot: check dir exists upgrade path 2023-09-28 22:55:55 +02:00
Xavier-Do
269e59538e [FIX] runbot: fix wake_up 2023-09-25 11:15:29 +02:00
Xavier-Do
435ac449f5 [FIX] runbot: various fixes and ref
- clean thread username
- allow to write on params for debug (was mainly usefull to forbid it
at the beginning)
- imrpove some guidelines about method and actions naming/ ordering
- move some code for a cleaner organisation.
- remove some useless request.env.user (not useful anymore)
2023-09-25 10:52:16 +02:00
Xavier-Do
8d619d11f8 [FIX] runbot: use upgrade path
Manage multiple upgrade_paths + small fix
2023-09-20 11:40:07 +02:00
Xavier-Do
f627ebaf0e [IMP] runbot: allow pr creation on closed pr
Even if we prefer to avoid this use case, some devs like to use the same
branch for multiple pr. This change allows to reopen a pr once one is
closed.
2023-09-15 13:23:06 +02:00
Christophe Monniez
b018e5a1a4 [IMP] runbot: resize the child extra parameters
When configuring a custom trigger on a bundle by using the wizard, the
child extra params field is often too small to display all the
parameters.
e.g., specify two long test-tags as it's often the case for
multi-builds.

With this commit, the field span over 4 columns.
2023-09-15 11:03:17 +02:00
Christophe Monniez
3c7f59a817 [REF] runbot: update settings view
The runbot settings view is a bit messy and the 16.0 upgrade added mess
on the existing one.

This commit is an attempt to make it a bit clearer and cleaner.
2023-09-15 11:03:17 +02:00
Xavier-Do
9923b18429 [IMP] runbot: use upgrade path
This commit will replace the symlink used for upgrade by the
upgrade-path.

The symlink was used before because old version does not support upgrade
path, but the decision was taken to now limit the testing to version
suporting upgrade paths in order to be able to support utils in another
repository latter.
2023-09-15 10:59:22 +02:00
Christophe Monniez
f50c099d93 [IMP] runbot: avoid modal in build error view
When trying to open a linked error or an error from the history, the
object is opened in a useless modal. With this commit, the object is
opened in a regular form view.
2023-09-06 10:58:21 +02:00
Christophe Monniez
c00579cdf2 [FIX] runbot: fix invalid condition in build template 2023-09-06 10:58:21 +02:00
Xavier Morel
ea2857ec31 [IMP] forwardport: parametrize main fw limits
Probably a bit more expensive, but makes the code shorter and more
straightforward.
2023-08-31 12:31:59 +02:00