Commit Graph

87 Commits

Author SHA1 Message Date
Xavier-Do
b55e38ae2d [FIX] runbot: wait for chart.js to be loaded
Since the frontend_assets are loaded with `defer="defer"`,
the page sometimes fail with the message:
```
stats.js:212 Uncaught ReferenceError: Chart is not defined
    at updateChart (stats.js:212)
    at stats.js:158
    at XMLHttpRequest.xhttp.onreadystatechange (stats.js:53)
```

This commit checks that Chart is available before tring to render the graph.

Thanks to @kebeclibre for the help.
2021-08-19 15:49:41 +02:00
Xavier-Do
726c7ba423 [IMP] runbot: add static_run button to slots buttons. 2021-07-20 13:43:02 +02:00
Xavier-Do
d9a09806e2 [IMP] runbot: add a static run link for documentation builds 2021-07-20 13:22:57 +02:00
Christophe Monniez
67d020a8b3 [IMP] runbot: update docker default for focal
Since Odoo 14.0, the recommended Ubuntu LTS release is Focal, the
default Docker should be updated accordingly.

A custom template with bionic have to be manually created on runbot
instances that still build Odoo < 14.0.

* A small change is made in the templates logic that builds the
  Dockerfile: a `runbot_pip` dict entry now exists in order to install the
  python libs required by the runbot. On the other hand, `additional_pip`
  should only be used to install optional python libs for Odoo.

* Upgrade Chrome version to 90.0.4430.93-1 as this one is currently in
  use on our current runbot instance. Just keep in mind that the Odoo
  screencast feature does not work anymore since Chrome 88.

* gsfont is added because of a bug [0] that affects python-reportlab in
  Focal.

* pyCrypto package is removed. It was used in an Odoo addon that
  disappeared in odoo/odoo@2738341c21

* dbfread, websocket-client are now installed as a deb package as they exists in Bionic and
  Focal

* pdfminer.six is now removed because a deb package exists in Focal but
  not in Bionic. It means that it has to be added in deb_packages_python
  in Dockerfiles for odoo > 13.0 and in additional_pip for odoo <= 13.0

[0]: https://bugs.launchpad.net/ubuntu/+source/python-reportlab/+bug/1918107
2021-06-29 09:22:41 +02:00
Xavier-Do
e65ebb570d [IMP] runbot: change ahead/behind visibility 2021-05-19 11:45:06 +02:00
Christophe Monniez
642844fdb5 [IMP] runbot: show red slots in batch tile
When a build slot is hidden in the batch tile but is responsible of the
batch failure, the failure reason may not be obvious for the user.

With this commit, an hidden slot appears if the slot build is in
failure.
2021-05-17 14:11:03 +02:00
Xavier-Do
6279dfa442 [IMP] runbot: imp stat display:
- Adds a complete legend enabelling to display a custom subset of modules.
This is mainly to enable a vertical scroll on list since chart-js default
legend will be displayed on multiple column.

- Adds a "Noisy" order mode to find non-deterministic modules.

- Changes the build selection mode to a center one to easylly center
build of interrest and add a forward button.

- Small ui tweaks/fix to match new selection logic.
2021-04-13 10:35:30 +02:00
Xavier-Do
84d9425bdc [FIX] runbot: various view fixes
- fix missing batch references on builds
- fix group on action buttons
2021-04-09 14:58:17 +02:00
Christophe Monniez
57bd00672d [IMP] runbot: add a chart page for build stats
Since 360e31ade4, it's possible to add statistics values to build
results but there was no practical way to analyze them.

With this commit, there is a new button on the bundle page that leads to
a chart page that displays those values.

The default reference build is last known good build of the bundle.
Values are filtered by key and only the most significant values are
displayed. The user can then refine the chart by changing the reference
build or the key and a few other options.

Co-author: Xavier-Do <xdo@odoo.com>
2021-04-09 14:10:37 +02:00
Christophe Monniez
785a7796fb [IMP] runbot: allow conditional pip requirements
Before this commit, the `requirements.txt` from a specified odoo branch
(master by default) was always installed in the Dockerfile's.

We now allow to disable this feature to test Odoo with vanilla
distributions.
2021-04-07 11:35:33 +02:00
Christophe Monniez
6bd74c07c4 [IMP] runbot: allow to install Chrome from google
When choosing to install Chrome in a Dockerfile, the chrome version is
downloaded from Odoo nightly server. This make it difficult to test
with different versions of Chrome.

With this commit, we allow to install from Google in Docker files.

By default, the install remains from Odoo Nightly server but if the key
`custom_values['chrome_source']` is set to 'google' in a Dockerfile,
the specified version will be downloaded from Google servers when the
Docker image is built.
2021-04-07 11:35:33 +02:00
Christophe Monniez
50f803ec31 [FIX] runbot: allow larger upload on runbot odoo instances 2021-04-07 11:01:35 +02:00
Xavier-Do
46269ada70 [FIX] runbot: fix logged message 2021-03-15 13:12:48 +01:00
Xavier-Do
c538bd17f8 [IMP] runbot: use build database_ids to generate connect links
aka: make it clean

This build database_ids field was generated a few months, waiting for the database to update with
this new sceme before using it.

It is still a little early to use it in cleanup methods, but this can be used to
generate the connect links dynamicaly.

To follow al specs introduced in previous commit, main fa-sign-in button should link to the -all.
It will almost always be the first one in database_ids in alphabetic order.
Then, in the dropdown, all other database are listed.

This will fix the previously broken design_theme connect link (no base nor all).

For this purpose nginx regex needs to be adapted to accept database name with underscore.
2021-03-01 15:49:23 +01:00
Xavier-Do
26ad74051f [IMP] runbot: apply al connect button spec 2021-02-26 12:13:39 +01:00
Xavier-Do
38b52411fa [FIX] runbot: use database name subdomain
Using database name as subdomain will set the web.base.url correctly when connection for the first time using this link
This will unfortunatelly break connect all and connect base links if dns and nginx are not setup.
This won't fix the web.base.url when connecting with the Connect (database selector) button either.
2021-02-22 16:03:51 +01:00
Christophe Monniez
f5034e5718 [IMP] runbot: add a button to create a new rebased batch 2020-12-09 13:32:52 +01:00
Christophe Monniez
733ab2dd43 [IMP] runbot: try to improve dashboard bundle line title 2020-11-18 16:49:45 +01:00
Christophe Monniez
9637976fef [IMP] runbot: bring back copy bundle name button 2020-11-18 16:49:45 +01:00
Christophe Monniez
9c7cee657d [IMP] runbot: improve build error frontend page
On the build error web page, a regular assigned error is not shown to
all users.

With this commit, a regular build error (not only random) will be shown
if the error is assigned.
2020-11-18 16:49:45 +01:00
Xavier-Do
87512ae78c [IMP] runbot: improve coverage
- execute coverage results in case of failure
- fix coverage links
- use coverage directly
2020-11-18 13:36:41 +01:00
Christophe Monniez
981cb7e1b6 [IMP] runbot: use a Dockerfile model
Currently, runbot is using a single Dockerfile maintained in a data file
in the source code. This situation is not convenient for testing Odoo in
different environments.

With this commit, a Dockerfile Odoo model is used to allow usage of
multiple Docker containers.

This model comes with a pre-defined Dockerfile that can be used to build
the current Odoo supported versions (12.0 up to 14.0).
2020-11-10 09:52:48 +01:00
Xavier-Do
45034d362d [FIX] runbot: inverted condition 2020-10-15 12:09:40 +02:00
Xavier-Do
bd69c0dfa0 [FIX] runbot: manual is on trigger, not slot 2020-10-15 11:28:20 +02:00
Xavier-Do
f9272cb207 [IMP]: add linitng tools (+bunch of small fixes) 2020-10-15 10:38:36 +02:00
Xavier-Do
b991eb5b70 [IMP] runbot: enable screencast, add access control, small fixes 2020-09-23 13:27:59 +02:00
Xavier-Do
45721cdf6c [IMP] runbot: runbot 5.0
Runbot initial architechture was working for a single odoo repo, and was
adapted to build enterprise. Addition of upgrade repo and test began
to make result less intuitive revealing more weakness of the system.

Adding to the oddities of duplicate detection and branch matching,
there was some room for improvement in the runbot models.

This (small) commit introduce the runbot v5.0, designed for a closer
match of odoo's development flows, and hopefully improving devs
experience and making runbot configuration more flexible.

**Remotes:** remote intoduction helps to detect duplicate between odoo and
odoo-dev repos: a commit is now on a repo, a repo having multiple remote.
If a hash is in odoo-dev, we consider that it is the same in odoo.
Note: github seems to manage commit kind of the same way. It is possible
to send a status on a commit on odoo when the commit only exists in
odoo-dev.
This change also allows to remove some repo duplicate configuration
between a repo and his dev corresponding repo.
(modules, server files, manifests, ...)

**Trigger:** before v5.0, only one build per repo was created, making it
difficult to tweak what test to execute in what case. The example use
case was for upgrade. We want to test upgrade to master when pushing on
odoo. But we also want to test upgrade the same way when pushing on
upgrade. We introduce a build that should be ran on pushing on either
repo when each repo already have specific tests.
The trigger allows to specify a build to create with a specific config.
The trigger is executed when any repo of the trigger repo is pushed.
The trigger can define depedencies: only build enterprise when pushing
enterprise, but enterprise needs odoo. Test upgrade to master when pushing
either odoo or upgrade.
Trigger will also allows to extract some build like cla that where
executed on both enterprise and odoo, and hidden in a subbuild.

**Bundle:** Cross repo branches/pr branches matching was hidden in build
creation and can be confusing. A build can be detected as a duplicate
of a pr, but not always if naming is wrong or traget is invalid/changes.
This was mainly because of how a community ref will be found. This was
making ci on pr undeterministic if duplicate matching fails. This was
also creating two build, with one pointing to the other when duplicate
detection was working, but the visual result can be confusing.
Associtaions of remotes and bundles fix this by adding all pr and
related branches from all repo in a bundle. First of all this helps to
visualise what the runbot consider has branch matching and that should
be considered as part of the same task, giving a place where to warn
devs of some possible inconsistencies. Associate whith repo/remote, we
can consider branches in the same repo in a bundle as expected to have
the same head. Only one build is created since trigger considers repo,
not remotes.

**Batch:** A batch is a group of build, a batch on a bundle can be
compared to a build on a branch in previous version. When a branch
is pushed, the corresponding bundle creates a new batch, and wait for
new commit. Once no new update are detected in the batch for 60 seconds,
All the trigger are executed if elligible. The created build are added
to the batch in a batch_slot. It is also possible that an corresponding
build exists (duplicate) and is added to the slot instead of creating a
new build.

Co-authored-by d-fence <moc@odoo.com>
2020-09-10 13:44:38 +02:00
Christophe Monniez
59d4eeaf24 [FIX] runbot: use proxy-mode for running instances
When deployed behind nginx reverse proxy and https, the Odoo proxy-mode
must be enabled.
2020-03-30 14:35:15 +02:00
Christophe Monniez
e30d2fbeb1 [FIX] runbot: show connect links when build is a duplicate
When a build is a duplicate, the connections links are not visible
because the local_state is "duplicate".
2020-03-18 11:41:42 +01:00
Martin Trigaux
60a4a9de85 [FIX] runbot: displayed collapsed error
By default, the error was displayed with arrow pointing UP, not down
2020-03-16 15:32:13 +01:00
Christophe Monniez
3b00d2576c [IMP] runbot: allow pseudo markdown in log messages
With this commit, a kind of markdown is allowed in log messages and
build.description.
Following patterns are supported:
  **strong**
  ~~striketrough~~
  __underline__
`code`
 [link](target)
 @icon-font-awesome-class
2020-03-09 13:10:49 +01:00
Xavier-Do
7e49b4cd2b [IMP] runbot: add a public route for monitoring 2020-02-24 14:45:23 +01:00
Christophe Monniez
d9ff43fa6b [REM] runbot: remove reverse dependency build feature
When a new commit is found, a rebuild is forced on sticky branches
builds in repositories that depends on the new commit repository.

If one of the repo is protected by groups, the main page gives access
errors for public users and finally leads to a CPU overload.

With this commit, the feature is removed as it will be re-implemented in
custom config steps.
2020-02-20 10:58:20 +01:00
Xavier-Do
c816ba2161 [IMP] runbot: add latest builds of same branch 2020-02-11 17:15:20 +01:00
Xavier-Do
a7ecf6a631 [FIX] runbot: fix build creation from view form
Before this commit, creating branch/build from view form may lead to errors
because of missing fields/ inexisting id.
2020-02-05 15:59:57 +01:00
Christophe Monniez
61101eaf46 [FIX] runbot: fix separtor class in build template 2020-01-22 13:32:27 +01:00
Xavier Morel
5d1772a9a5 [FIX] runbot: redirect to the relevant build on actions
Currently killing, waking or rebuilding a build then redirects to the
repository root which is worse than useless: you've lost the build you
come from, and for rebuilds it's no help getting to the new build.

Since it's very easy to go from a build to its repository, redirect to
the "most useful" build instead:

* if rebuilding the current build (from its page), open the page of
  the new build which was just created
* otherwise reload the current page whatever it is
2020-01-21 09:21:02 +01:00
Christophe Monniez
a7f1b94c41 [IMP] runbot: add a description field on build 2020-01-15 13:32:38 +01:00
Xavier-Do
163fa6ca59 [FIX] runbot: fix views 2020-01-14 13:24:58 +01:00
Xavier-Do
acf31c2820 [FIX runbot: remove _update_nb_children] 2020-01-14 13:24:58 +01:00
Xavier-Do
93b901dcdd [IMP] runbot: simplify views and adapt to odoo13
mainly fixes datetime and bootstrap layouts
2020-01-14 13:24:58 +01:00
Xavier-Do
cede4b936e [FIX] runbot: fix docker state and rebuild button
When docker is stopped, end command cannot execute.
Thus the state is considered as running.
2019-12-18 16:30:01 +01:00
Christophe Monniez
6c64bbb49b [IMP] runbot: improve build log view
Co-authored-by: Xavier-Do <xdo@odoo.com>
2019-12-17 16:08:31 +01:00
Xavier-Do
af4037261f [IMP] runbot: display childrens on duplicate 2019-10-28 10:54:46 +01:00
Christophe Monniez
d9b20d6961 [IMP] runbot: dump database at the end of step
Dump a db at the end of a build, using a new 'finals' cmd part
added in order to execute dump even if build fails.

Add a link in last step log to download dump.
2019-10-21 16:23:05 +02:00
Christophe Monniez
54f0488b26 [IMP] runbot: add a copy to clipboard button for branch
When using the runbot frontend, it's sometimes very frustrating when
trying to copy branch name, some mouse gym is necessary.

With this commit, a copy to clipboard button is added near the branch
name on the frontend.
2019-09-18 13:10:45 +02:00
Christophe Monniez
0da30a9f60 [IMP] runbot: add build_error model
With this commit, a new RunbotBuilError model is added in order to
classify and manage errors that appears during runbot builds.  This is
an helper to find undeterministic bugs in Odoo builds.  Build logs can
be parsed on demand, during the parsing, the logs are cleaned with some
regexes stored on the RunbotErrorRegex model.  A hash is computed on the
cleaned log, if a build error already exists with the same fingerprint,
the build is appended on the build error.

Errors can also be manually linked together with a parent/children
relation in case of a related error log. e.g. the error message is
different in two different branches but the bug is the same.

Also, a new build_url field is added to the runbot_build in order to
access the build web page from the backend.
2019-09-03 11:10:38 +02:00
Xavier-Do
02d2cc4528 [WIP] runbot: monitoring tools
Add a new model runbot.host to keep info and configuration about
hosts (worker servers), like number of worker, reserved or not,
ping times (last start loop, successful iteration, end loop, ...)
and also last errors, number of testing per host, psql connection
count, ...

A new monitoring frontend page is created, similar to glances
but with additionnal information like hosts states and
last_monitored builds (for nightly)

Later this model will be used for runbot_build host instead of char.

Host are automaticaly created when running _scheduler.
2019-09-03 11:10:38 +02:00
Xavier-Do
2aed7e1a48 [IMP] runbot: imp build result display
If children are killed, they will all look the same in the parent view
making difficult to find the failed one in staging branches.

This commit displays result rather than status in priority if build
is in failure.
2019-08-13 14:12:31 +02:00
Xavier-Do
22fe792e9f [IMP] runbot: add wake_up button where connect button usually stands
This commit add a wake up button in place of connect button when build is
not running and may be wake up.

Connect button will also be visible immediatly when local_state is 'running'
since we don't need to wait sub build to finish.
2019-08-13 10:33:37 +02:00