Since Odoo 14.0, the recommended Ubuntu LTS release is Focal, the
default Docker should be updated accordingly.
A custom template with bionic have to be manually created on runbot
instances that still build Odoo < 14.0.
* A small change is made in the templates logic that builds the
Dockerfile: a `runbot_pip` dict entry now exists in order to install the
python libs required by the runbot. On the other hand, `additional_pip`
should only be used to install optional python libs for Odoo.
* Upgrade Chrome version to 90.0.4430.93-1 as this one is currently in
use on our current runbot instance. Just keep in mind that the Odoo
screencast feature does not work anymore since Chrome 88.
* gsfont is added because of a bug [0] that affects python-reportlab in
Focal.
* pyCrypto package is removed. It was used in an Odoo addon that
disappeared in odoo/odoo@2738341c21
* dbfread, websocket-client are now installed as a deb package as they exists in Bionic and
Focal
* pdfminer.six is now removed because a deb package exists in Focal but
not in Bionic. It means that it has to be added in deb_packages_python
in Dockerfiles for odoo > 13.0 and in additional_pip for odoo <= 13.0
[0]: https://bugs.launchpad.net/ubuntu/+source/python-reportlab/+bug/1918107
The test in the original code will never fire because the value searched
for is not in the keys of the dictionary, but in one of the lists which
are in the values. Work around this by maintaining a reverse dictionary
module name -> commit and use this for the test.
When a dockerfile to_build field is False for any reason and a new
runbot is setup, it's easy to miss the point and builds that involves
this particular Dockerfile will fail on the new runbot.
With this commit, the Dockerfiles tree view is improved to easily spot
those kind of problems:
- to_build field is visible and the Falsy lines are in yellow (warning)
- the empty Dockerfile's are in red (danger)
- versions are now visible in the tree view too
When a build slot is hidden in the batch tile but is responsible of the
batch failure, the failure reason may not be obvious for the user.
With this commit, an hidden slot appears if the slot build is in
failure.
Sometimes a pr pull info can fail.
- Most of the time it is only temporary and it will be successfull on next try.
- In some rare case the pr will always fail (github inconsistency) The pr exist in git but not on github api.
For this rare case, we store the pr in memory in order to unstuck other pr/branches update.
We consider that this error should not remain, in this case github needs to fix the inconsistency.
This is why the runbot model don't handle such a case for now.
Another solution would be to create the pr with fake pull info. This idea is not the best one
since we want to avoid to have many pr with fake pull_info in case of temporary failure of giothub services.
With this solution, the pr will be retried once every cron loop.
We dont except to have pr with this kind of persistent failure more than every few mounths/years.
When code blocks were containing markdown like text, the inside of the code
block was also formated.
This commit removes the code blocks before applying other formating and
place them back at the end.
closes#481
- Adds a complete legend enabelling to display a custom subset of modules.
This is mainly to enable a vertical scroll on list since chart-js default
legend will be displayed on multiple column.
- Adds a "Noisy" order mode to find non-deterministic modules.
- Changes the build selection mode to a center one to easylly center
build of interrest and add a forward button.
- Small ui tweaks/fix to match new selection logic.
Since 360e31ade4, it's possible to add statistics values to build
results but there was no practical way to analyze them.
With this commit, there is a new button on the bundle page that leads to
a chart page that displays those values.
The default reference build is last known good build of the bundle.
Values are filtered by key and only the most significant values are
displayed. The user can then refine the chart by changing the reference
build or the key and a few other options.
Co-author: Xavier-Do <xdo@odoo.com>
Before this commit, the `requirements.txt` from a specified odoo branch
(master by default) was always installed in the Dockerfile's.
We now allow to disable this feature to test Odoo with vanilla
distributions.
When choosing to install Chrome in a Dockerfile, the chrome version is
downloaded from Odoo nightly server. This make it difficult to test
with different versions of Chrome.
With this commit, we allow to install from Google in Docker files.
By default, the install remains from Odoo Nightly server but if the key
`custom_values['chrome_source']` is set to 'google' in a Dockerfile,
the specified version will be downloaded from Google servers when the
Docker image is built.
If you try to manually create bundle, Odoo will crash, because it will
try to use `name` value that is not set yet. For that we start computing
host_id once `name` is entered.
If a PR got merged to master (or whatever the current development
branch is), there's no easy way to know what maintenance branch it
ended up landing in, except by asking git which branches contain the
commit (which can be rather slow).
Add a special case on merge which labels the PR with a pseudo-branch
patterned after the second-to-last branch of the project:
* if the branch ends with a number, increment the number by one
e.g. 2.0 -> 2.1, 5 -> 5.1
* otherwise, just prefix with `post-` e.g. "maint" ->
"post-maint" (that one doesn't sound very helpful, but I guess it's
nice for the weirdoes who call their branches "natty narwhal" and
shit)
Fixes#450
Fix#457 hopefully: I didn't manage to repro / create a test for.
It looks like in some cases during the update process the PR ref lags
behind the branch itself. This means `forwardport.updates` creates a
new commit, pushes it, then on the next iteration updates the local
cache, tries to find the commit we just pushed... and that fails.
I can only assume this is because when there's enough load on the
github side the update to the `info/refs` pseudo-file can fall
behind (it's now 4MB and holding nearly 65k refs).
So cheat: take the commit we just pushed to the dev remote
and... immediately push it to the local cache under a dummy branch,
which we delete. Since we only gc "1 day ago" this should not vacuum.
On edition of an intermediate PR in a chain, merging the PR would lead
to *it* being forward-ported, duplicating the PRs already created
from *its* source.
Add a check for PRs in the target branch with the same source,
suppresses the forward-porting of the newly merged PR.
Fixes#451 (hopefully)
append `git status` data to stderr, should be somewhat more
informative especially when a conflict is a DU (where the file has
been deleted on one side, so there is no conflict marker anywhere).
Fixes#461
- When updating the local repo cache, always capture both stdout and
stderr and log them out rather than having them in journalctl hard
to relate to the main log.
- In the git layer, capture stderr by default and log it automatically
on command failure.
The process did properly update the state, but not the squash state.
It's somewhat unclear whether the state should be fully reset and
require reapproval though. Maybe only the validation should be reset?
The CI will eventually run and either succeed (re-validating) or
fail (devalidating, hopefully) but I'm not entirely sure this is
correct.
Apparently I wrote this when I was a dumbshit and did not think that
Python sets are implemented completely separately from dicts
and *remain unordered*.
As a result, the simplistic set union would not properly conserve
record order, which it should.
Apparently the "reason phrase" has been removed from HTTP/2 and github
has enabled HTTP/2, meaning responses may or may not contain a reason
phrase depending whether they're HTTP/1.1 or HTTP/2. Now I don't
understand how this triggers because requests isn't supposed to
support HTTP/2, and in a shell I do get a reason, but w/e.
Anyway the reason is used by a few tests to check more specifically
for the response they're getting, as that's used as the assertion
message by `get_ref`.
I guess I could replace the AssertionError by an HTTPError
but... w/e. Seems simpler to just fallback to the generic reason based
on the status code. It fixes both failing
tests (test_straightforward_flow and test_delete_normal) with very
little change.
aka: make it clean
This build database_ids field was generated a few months, waiting for the database to update with
this new sceme before using it.
It is still a little early to use it in cleanup methods, but this can be used to
generate the connect links dynamicaly.
To follow al specs introduced in previous commit, main fa-sign-in button should link to the -all.
It will almost always be the first one in database_ids in alphabetic order.
Then, in the dropdown, all other database are listed.
This will fix the previously broken design_theme connect link (no base nor all).
For this purpose nginx regex needs to be adapted to accept database name with underscore.
Using database name as subdomain will set the web.base.url correctly when connection for the first time using this link
This will unfortunatelly break connect all and connect base links if dns and nginx are not setup.
This won't fix the web.base.url when connecting with the Connect (database selector) button either.
When a status cannot be sent to github, the status is created in
database but the `sent_date` field is not set.
Because of that, the status cannot be sent manually from the frontend.
With this commit, that kind of status can by tried again.
5cf3617eef intended to create merge
messages with only the content of PR descriptions before the first
thematic break. However this processing was incorrectly applied
to all messages being processed (meaning rebased / squashed commit
messages as well).
Properly apply thematic break processing to only commit messages
created from PR descriptions.
Before this, we would "roughly" select stagings by looking at stagings
whose heads matched a specific sha then validating them all. This
could perform extra validations on stagings once in a while but this
was assumed not to be much an issue, at least originally.
However two changes later on have contributed to this likely being the
cause of #429 (stagings never timing out):
* heads of the staging branches are uniquifier commits stored in the
heads map, but the actual heads of the stagings are also stored
there, some of which are no-ops (hence the uniquifiers) so assuming
repos A and B, if a staging contains PRs touching A then the head of
B actual will also be a head of B
* when a staging is validated, if it *contains* any pending result the
timeout limit gets bumped back
The issue here is that if a success / failure status is lost (which
would be the most common reason for timeouts) *and* someone has forked
and is regularly rebuilding a branch-head used as-is by a staging,
each of those rebuilds will trigger a validation of the staging, which
will find that one of the statuses is still pending (because we missed
the success / failure), which will bump up the timeout limit,
continuing until the branch stops getting rebuilt.
This is probably one of the reasons why some stagings last for *way*
more than 2h, though it is far from explaining all of them: 90% of the
stagings lasting more than *3*h end up succeeding. Tho it's always
possible that this is because someone notices and resends a success
for the missing status it seems somewhat doubtful. Oh well.
Also fix the incorrect log call on `update_timeout_limit` triggering.
* fix small error which generated an extra commit in case of conflict
probably (would create a `temp` commit, then put the conflict
information on an empty commit on top of it). Avoids committing the
cherrypick though a probable alternative would be to commit the
message with `amend`.
* improve the cherrypick header: instead of one log line per commit,
put all commits within the sole header log line (with a newline),
should make things less noisy
* put *all* log records below the correct logger (two of them were on
the toplevel logger)
* log a purported inflateInit error as warning, should be sent to
Sentry instead of having to wait for people to wonder why the thing
is completely broken
I'd forgotten that in order to better handle cases where the CI is
highly backed up (and / or slow for some reason), we actually update
the CI timeout to really take the last "pending" status as the "true
start" of the CI. This might explain why lots of stagings needed extra
time: as of right now, out of 28835 stagings
- 20086 had their timeout bumped by more than 15mn
- 6967 had their timeout bumped by more than 30mn
- 264 had their timeout bumped by more than 1h
- 30 had their timeout bumped by more than 2h
Add some logging every time the CI is bumped this way, so we have
better visibility into that event.
Closes#429
The mergebot has a feature to ping users when an approved PR or
forward-port suffers from a CI failure, as those PRs might be somewhat
unattended (so the author needs to be warned explicitly).
Because the runbot can send the same failure information multiple
times, the mergebot also has a *deduplication* feature, however this
deduplication feature was too weak to handle the case where the PR has
2+ failures e.g. ci and linting as it only stores the last-seen
failure, and there would be two different failures here.
Worse, because the validation step looks at all required statuses, in
that case it would send a failure ping message for each failed
status *on each inbound status*: first it'd notify about the ci
failure and store that, then it'd see the linting failure, check
against the previous (ci), consider it a new failure, notify, and
store that. Rinse and repeat every time runbot sends a ci *or* lint
failure, leading to a lot of dumb and useless spam.
Fix by storing the entire current failure state (a map of context:
status) instead of just the last-seen status data.
Note: includes a backwards-compatibility shim where we just convert a
stored status into a full `{context: status}` map. This uses the
"current context" because we don't have the original, but if it was a
different context it's not going to match anyway (the target_url
should be different) and if it was the same context then there's a
chance we skip sending a redundant notification.
Fixes#435
Before this change, a CI override would have to be replicated on most
/ all forward-ports of the base PR. This was intentional to see how it
would shake out, the answer being that it's rather annoying.
Also add a `statuses_full` computed field on PRs for the aggregate
status: the existing `statuses` field is just a copy of the commit
statuses which I didn't remember I kept free of the overrides so the
commit statuses could be displayed "as-is" in the backend (the
overrides are displayed separately). And while at it fix the PR
dashboard to use that new field: that was basically the intention but
then I went on to use the "wrong" field hence #433.
Mebbe the UI part should be displayed using a computed M2M (?)
as a table or as tags instead? This m2m could indicate whether the
status is an override or an "intrinsic" status.
Also removed some dead code:
* leftover from the removed tagging feature (removed the tag
manipulation but forgot some of the setup / computations)
* unused local variables
* an empty skipped test case
Fixes#439.
Fixes#433.
When a new branch is created between other branches, the process to
try and look for forward-port PRs to create to "fill in" currently has
no logging so it's very difficult to figure out why it decides not to
do something.
Add some logging to the process to try and better understand what
happens.
Fixes#441
