Commit Graph

1508 Commits

Author SHA1 Message Date
Xavier Morel
ef52785d82 [IMP] forwardport: store and display detachment reason
Currently, if a PR forward-port PR gets detached the reason for it is
not always obvious, and may have to be hunted in the logs or in
"sibling" PRs.

By writing a forward port reason (hopefully) ever time we detach a PR,
and displaying that reason in the form and dashboard, the
justification should be a lot more obvious.

Fixes #679
2023-01-19 14:23:41 +01:00
Xavier Morel
2742fe18fd [FIX] forwardport: correct batching of intermediate PRs
When inserting a new branch, if there are extant forward ports
sequences spanning over the insertion, new forward ports must be
created for the new branch.

This was the case, *however* one of the cases was handled incorrectly:
PR batches (PRs to different repositories staged together) must be
forward-ported as batches. However when creating the "insert" batches,
these PRs would be split apart, leading to either inconsistent states
or an inability to merge the new forward ports (if they are
co-dependent which is a common case). This is because the handler
would look for all the relevant PRs to forward ports, but it would
fail to group them by label thereafter.

Fix that, create the `insert` batches correctly, and augment the
relevant test with an additional repository to test this case.

No gh issue was created, but this problem was reported on
odoo/odoo#110029 and odoo/enterprise#35838 (they were re-linked by
hand in the runbot and mergebot, but on github the labels remain
visibly different, one having the slug ZCwE while the other
hasO7Pr).

It's possible that the problem had occurred previously and not been
noticed (or reported), though it's also possible this issue had never
occurred before: for the latest freeze (16.1) there were 18 forward
ports spanning the insertion, but of them only 2 were the same batch.
2023-01-19 10:57:09 +01:00
Xavier Morel
652b1ff9ae [IMP] runbot_merge: more logging on synchronize
Was missing a logging message in the case where the current and sync'd
head are identical, which seems to occur from time to time but can
only be inferred (by seeing a sync event then nothing happening).

Add a logging warning (because it's a strange situation) in order to
explicitely note the issue.

Also make the sync logging messages more regular for clarity.

And add the delivery information (delivery id and user-agent) to event
log, so it's more possible to report issues to github.
2023-01-19 10:56:11 +01:00
Xavier-Do
763b1af1a5 [FIX] runbot: disable host_default 2023-01-17 15:24:49 +01:00
Xavier-Do
27ac733df6 [IMP] runbot: automatic base master bundle
When adding a new project, if no branch matches a base name,
the created bundles won't have a version and it will fail.

A simple fix will be to add a master bundle for all projects.
2023-01-17 13:41:01 +01:00
Xavier-Do
9e0f0e6a75 [IMP] runbot: display errors for critical/warn 2023-01-13 15:57:25 +01:00
Christophe Monniez
1a108669ce [IMP] runbot: do not install wkhtml when value is empty 2023-01-13 15:33:05 +01:00
Xavier-Do
12c0a0edf6 [IMP] runbot: automatically transfer responsible
When an error is linked to another one, we don't expect it  to appear on
team and user dashboard. When adding a parent, this will transfer the
responsible from the child to the parent when applicable.
2023-01-13 15:32:17 +01:00
avs
6fffa0264a [IMP] runbot: make runbot.bundle::write() callable through xml-rpc
The XML-RPC implementation does not allow for receiving or sending
`None` values (both as query parameters and response).

Since the `write` method of `runbot.bundle` was overriden without
returning a value, an exception is raised when the method is called
through the external API.

This makes the `write` method return the value from its call
to `super()` which should be equal to `True` if all went well.
2023-01-13 15:32:01 +01:00
Xavier-Do
d011bb7f8c [IMP] runbot: make disable host configurable
The auto disable host is mainly usefull when there are a lot of host for
well configured repositories.

If for any reason a repo is corrupted on one host, this host will be
disabled until a manual intervention cleans the repo.

For other cases, where thjere are many repositories with not so many
host, it is most likely that a fetch will fail because of an invalid
repository configuration. Disabling the host in this case is not a good
idea.

With this commit, a settings allows to enable or disable this feature.
2023-01-13 14:46:02 +01:00
Xavier-Do
ecd105218c [IMP] runbot: parse warning and critical 2023-01-13 14:27:17 +01:00
Xavier-Do
2b8242cf42 [IMP] runbot: default empty value for result 2023-01-13 14:27:17 +01:00
Xavier-Do
6892546d94 [IMP] runbot: fail before contacting github if user cannot write on team 2023-01-13 14:27:17 +01:00
Xavier-Do
cd4c6b5e43 [IMP] runbot: better configure upgrade descriptions 2023-01-13 14:27:17 +01:00
Xavier-Do
37df18d54d [FIX] runbot: use params db_name when provided for upgrades 2023-01-12 11:54:53 +01:00
Xavier-Do
c9c64bccc7 [FIX] runbot: fetch member should be able to access token for users 2023-01-11 13:35:02 +01:00
Xavier-Do
06f9f62b1f [FIX] runbot: fix was alive 2023-01-11 12:41:39 +01:00
Christophe Monniez
794a6bf7e2 [IMP] runbot: add a monitoring tool to the builder
At this moment it's difficult to monitor a build cpu and memory usage.
This must be done from outside the container to avoid to kill the cat by
opening the box.

This commit adds a daemonic thread launched by the builder. This thread
will read the memory and cpu stats of the running dockers and report
their stats in a log file into the logs directory of the builds.

The log file format is made to be easily mixed with the build logs and
spot memory leaks or cpu overload during builds.
2023-01-11 11:33:19 +01:00
Xavier-Do
16fa0e1ad8 [IMP] runbot: add freeze list feature 2023-01-11 11:20:29 +01:00
Xavier-Do
788114776b [IMP] runbot: add python_code filter 2023-01-10 10:48:56 +01:00
Xavier-Do
fdb144a3af [IMP] runbot: allow to ignore forward-ports for a team 2023-01-10 10:47:59 +01:00
Christophe Monniez
1c49aaacf7 [IMP] runbot: add a test for _get_refs 2023-01-10 10:37:06 +01:00
Xavier-Do
56c08be631 [FIX] runbot: filter numerical branch names
Right now, a branch with a numerical name will be added to the database,
but it can conflict with pr since the name of a pr is a number.
This means that a unique (name, repo_id) constraints can be broken.
We could use the 'is_pr' in the unicity constraints to avoid this issue
but searching on branch name will give confusing result if some of them
can be numerical.

Moreover, a runbot branch name should start with the version name
meaning that a numerical branch name was a bad idea from the begining.
2022-12-27 10:37:30 +01:00
Xavier-Do
6d3663e9f6 [IMP] runbot: disable auto-tags from custom trigger 2022-12-26 12:31:09 +01:00
Xavier-Do
910843f06d [FIX]: fix codeowner
A corner case was discovered when a pr is open but the pr is not in the
codeowner commits.
2022-12-24 12:39:55 +01:00
Xavier-Do
ad573d64cf [FIX] runbot: manage falsy path_glob 2022-12-22 14:40:13 +01:00
Xavier-Do
5540f84667 [IMP] runbot: add priority queue for debuging
The main idea is to allow some build to use an extra slot from all host
if a bundle is in priority mode.

This is mainly for quick step debuging, mainly when modifying python
steps when the runbot is fully loaded.

This will also be usefull to concurrency test, by starting a build on
each host at the same time, even when some host are already fully loaded
2022-12-22 14:26:10 +01:00
Christophe Monniez
76cc74fcd9 [IMP] runbot: improve team form and build frontend page
* Hide useless pages on backend team form
* Add a team exceptions page on team form
* Show fixed errors as fixed on front build page
2022-12-22 14:21:59 +01:00
Xavier-Do
356843c680 [IMP] runbot: reduce concurrency
Right now, multiple build are read when managing build to schedule.

This is not usefull since the transaction is commited between each
of them. Moreover, the read build can be written from another host
adding another possibility to have a conccurent update.

Removing the prefetch_ids may help a little.
2022-12-22 11:25:04 +01:00
Xavier-Do
205d299c7d [IMP] runbot: add trigger list to team 2022-12-22 10:47:24 +01:00
Christophe Monniez
9ccf5a2c0e [IMP] runbot: add a reassign team wizard 2022-12-21 15:20:42 +01:00
Xavier-Do
c83f89fc23 [IMP] runbot: documentation and views imps
Add a small documentation for users, mainly about teams and codeowners.
Improves some views and hide some menu_items to keep interface easier
to navigate.
2022-12-21 13:00:42 +01:00
Xavier-Do
ada69222c9 [IMP] runbot: add an action to reassign team automaticaly 2022-12-21 10:58:30 +01:00
Xavier-Do
af7a0751fc [FIX] runbot: avoid recursion in create steps
Since the custom create was introduced, if a config is added in the
config data of a create step, the config can be dynamic. If the given
config contains a create step, this become recursive.

This is fixed in this commit by:
- Checking the parent_path depth in add_child. This will also work for
  python config.
- Consuming the params when adding the child
- Also cleanup the base custom multi config to use a specific step
2022-12-20 09:30:08 +01:00
Xavier-Do
92ed23c237 [FIX] runbot: manage team without github logins 2022-12-19 11:14:07 +01:00
Xavier-Do
25557e6116 [FIX] runbot: manage team without github logins 2022-12-19 11:09:15 +01:00
Xavier-Do
a67e51e600 [FIX] runbot: revert log_access and write in cache
Removing log_access has as side effect to add the foreign key to the
create_uid and write_uid fields. This is quite slow and will slow the
insert

Removing the fields is also not an good idea on such a large table

Puthing the value in cache and flushing should do the trick.
2022-12-16 16:59:59 +01:00
Xavier-Do
0d61f6c93b [IMP] runbot: add github login
This idea was postpone for a while since this was most a mergebot
responsability but having the github login of the user will help
for some team feature requests.

The main one is to only ping a team if the pr was not opened by a member
of the team. We want to let the team manager manage that as much as
possible so the team manager group will be able to write the user
github login (as well as the user himself) and add a list of non user
github_login to consider if not all user have a account on runbot.

This commit also improves the views for team managers.
2022-12-16 15:32:39 +01:00
Xavier-Do
3f2bf0112f [FIX] runbot: custom log_access fields shouldn't be automatic 2022-12-15 14:20:11 +01:00
Xavier-Do
02ed33c498 [FIX] runbot: keep log create date 2022-12-15 13:38:55 +01:00
Xavier-Do
0d6ec3e038 [FIX] runbot: fix invalid db_name log local
If the db_name does not stat with a build ind (or at least an int)
the query will fail because of 'local_pg_cursor'

Since a database can be create with an invalid name from the db manager
but still log in the runbot_logs, we need to manage all format.
Also add a limit to catchup if the db is full of logs, to avoid a Memory
error.
2022-12-15 12:54:50 +01:00
Xavier-Do
a6ce3dceb0 [IMP] runbot: add step description 2022-12-12 10:35:23 +01:00
Xavier Morel
3361e87988 [FIX] forwardport: relax limits on all git operations
Apparently it's not just GC which causes trouble, on the new
configuration anyway.
2022-12-12 07:40:39 +01:00
Xavier Morel
a563fcf907 [REM] forwardport: fp_sequence field
It's almost certainly not useful, save as a minor convenience for
tests: decorrelating the branch sequence and the fp sequence seems
like it would only be extremely confusing, and on the mergebot all the
fp_sequence values are set to the default while the sequence values
are set to something useful and sensible (kinda).

Fixes #584
2022-12-08 10:46:22 +01:00
Xavier Morel
629e1aea4a [IMP] *: remove default group operator on objects
After review, there doesn't seem to be a single integer field created
by the mergebot or fortwardbot modules for which a `group_operator`
makes sense, let alone the default of `sum`.

So just disable them all.

Fixes #674
2022-12-08 10:46:22 +01:00
Xavier Morel
2631b17ec4 [IMP] forwardport: ACLs on maintenance
Not actually useful since it's a table with no records (just exists to
run a cron), but the log complains.
2022-12-08 10:46:22 +01:00
Xavier Morel
03c9a7e492 [ADD] runbot_merge: stagings canceling wizard
- allows providing a canceling reason
- allows nuking the splits at the same time (hopefully)

Fixes #661
2022-12-08 10:46:22 +01:00
Xavier Morel
2746a13163 [FIX] runbot_merge: multi-commit squashing
If commits have different authors (/ committers), the mergebot would
ask github to create a commit with an author (/ committer) of `None` /
`null`.

Apparently github really does not like that, and complains that

> nil is not an object

So remove the key entirely. Also fix the collision between `author`
and the `Co-Authored-By` list, which could lead to trying to set an
`author` of `[name, email]` instead of an object, which is also not
accepted by github.
2022-12-08 10:46:22 +01:00
Xavier Morel
eb6dbf5d9b [IMP] forwardport: enable negative refspecs
Requires Git 2.29, new mergebot has access to Git 2.34
2022-12-08 10:46:22 +01:00
Xavier Morel
0f3647b7c7 [FIX] *: freeze wizard take 3
Fixes to the new bits which didn't really work:

- Fix borked view layout
- Add some help to the label fields
- Improve the resolution of label -> pr, and fix
- Also make the feature actually work for bump PRs
- Also make pr -> label work more reliably, now allows setting one PR
  and getting the other PRs of the same batch (with the same label)
  even without setting the label by hand

An autocomplete for the label has been considered but there is no
autocomplete field for char/selection fields, and it seems way too
much work for the utility:

- either create a brand new widget for 15.0 which will have to be
  entirely rewritten in 16
- or create a transient model composed entirely of fake records to
  provide an m2o to records which don't actually exist as label
  bearers, which is also a lot of unnecessary work

NOTE: we want to support partial freezing (aka not freeze all the
      branches because some of them have different release models
      than others), so some project repos *not* having a release
      PR is fine and normal, such a validation should not be added.

Fixes #664
2022-12-08 10:46:22 +01:00