Commit Graph

835 Commits

Author SHA1 Message Date
Xavier-Do
2f160b27e7 [FIX] dont allow to wake up a build if db where drop 2022-10-21 11:44:35 +02:00
Christophe Monniez
309aeaa32e [IMP] runbot: speedup build garbage collecting
When the builds directory is filled with a lot of build directories
(around 100000) the garbage collection process may take up to 2 minutes.
The root cause is that each build directory is scanned to clean it up
even if it was already cleaned.

With this commit, a stamp file is used to mark directories that were
already garbage collected.
2022-10-21 11:32:46 +02:00
Christophe Monniez
7642bffda3 [FIX] runbot: download phantomjs from nightly server
In order to avoid bitbucket rate limiting, we prefer to download this
old pal from our server.
2022-10-21 11:17:05 +02:00
Xavier-Do
903ee7d983 [FIX] runbot: manage falsy value for frontend_url 2022-10-21 10:39:20 +02:00
Xavier-Do
e27a1b8f71 [IMP] runbot: cleaup settings view 2022-10-21 10:32:40 +02:00
Xavier-Do
0287dcaab7 [IMP] runbot: update install documentation 2022-10-21 10:32:40 +02:00
Xavier-Do
303638e507 [FIX] runbot: don't hardcode user odoo 2022-10-21 10:32:40 +02:00
Xavier-Do
e4af8d7b6b [FIX] runbot: remove fqdn calls 2022-10-21 10:32:40 +02:00
Xavier-Do
785e2dc3bc [IMP] runbot: don't use local odoorc 2022-10-21 10:32:40 +02:00
Xavier-Do
bfc75cd7fe [IMP] runbot: don't active cron by default 2022-10-21 10:32:40 +02:00
Xavier-Do
50a7120949 [FIX] runbot: don't crash when there is no accessible project 2022-10-21 10:32:40 +02:00
Xavier-Do
16e71d064a [FIX] runbot: remove useless/invalid check 2022-10-21 10:32:40 +02:00
Xavier-Do
b79c4a5a52 [IMP] runbot: various build_config improvements 2022-10-21 10:32:40 +02:00
Xavier-Do
7bb2f06501 [IMP] runbot: make build_error button visible for users 2022-10-21 10:32:40 +02:00
Xavier-Do
b606a2199b [IMP] runbot: improve build error display in logs 2022-10-21 10:32:40 +02:00
Xavier-Do
831f5f7197 [IMP] runbot: make link to backend have the runbot menu 2022-10-21 10:32:40 +02:00
Xavier-Do
eaf8964701 [IMP] runbot: change trigger start logic
and add sequence on project
2022-10-21 10:32:40 +02:00
Xavier-Do
02eb8014a3 [IMP] runbot: make build_error triggers searchable 2022-10-21 10:32:40 +02:00
Xavier-Do
a294348cd0 [IMP] runbot: improve views 2022-10-21 10:32:40 +02:00
Xavier-Do
70e8b15690 [IMP] runbot: add websocket support 2022-08-23 16:30:24 +02:00
Xavier-Do
e91611b0f8 [FIX] runbot: reduce load when builder are idle
The sleep 1 was ok with a few builder bur regarding the number of
request on the database when no build are running, this become
problematic.

An ideal solution would be to detect if
-> me managed some testing build
-> There is load (pendings)

In both case, we don't want to sleep to much.
In other cases, we may want to wait a little longer.

A simple quick fix will just wait longer in all cases.
2022-08-23 15:54:50 +02:00
Xavier-Do
b44ed5f7a6 [IMP] runbot: active trigger and filter manual
- dont display manual trigger having a team if not member of that team
- add an active field on triggers
2022-08-17 16:50:23 +02:00
Xavier-Do
326ac36b5e [FIX] runbot: display stats of failed and scheduled builds 2022-08-17 12:18:40 +02:00
Xavier-Do
35e3e7cee0 [FIX] runbot: make Upgrade exception message no update and step id clickable 2022-08-17 11:46:47 +02:00
Xavier-Do
d4f1f01b46 [IMP] runbot: add buttons to compare with next line on build_search page 2022-08-17 11:46:47 +02:00
Xavier-Do
05c0c29e0f [IMP] runbot: add bundle name in link title 2022-08-17 11:43:53 +02:00
Romain Derie
8af81d72bb [IMP] runbot: add sort and filter capability on the build error page
Before this commit, the build errors page was neither sortable neither
filterable.

This commit adds a way to filter by:
- all
- unassigned
- seen more than once

It also allows to sort by:
- last seen
- nb seen

Typical need is to sort by nb seen and filter out the only seen once to
be able to figure which one is supposed to be checked in priority.
2022-08-17 11:42:36 +02:00
Xavier-Do
a382977d39 [IMP] runbot: add teams on trigger.
This will be mainy usefull in trigger categories custom view and for
dashboards.
2022-08-17 11:40:56 +02:00
Xavier-Do
5f2fe9a451 [FIX] runbot: mitigate docker status race condition
In some rare cases, a docker container has a status of exited, removed
or in removal and the `end-`file has been written right after the code
checked the existence of the file.

To mitigate the issue, this commit checks the `end-` file existence
after the container status have been checked. That way, if the file
exists we can be pretty confident that the build really ended.
2022-08-17 11:40:12 +02:00
Christophe Monniez
2dce6d048e [IMP] runbot: freeze files modification time at export
When exporting a commit, it will be useful to freeze the modification
time of the exported files. The idea behind that is to pre-generate
bundles at the install time of the Odoo instances, that way when running
the post install tests, the bundles does not have to be generated for
each test.
2022-07-26 13:57:18 +02:00
xmo-odoo
6d96f8c2cd [FIX] runbot: redirect to current page after login
Because the login link redirected to `/`, when logging in the user would have
to re-navigate to their previous page unless they'd remembered this issue and
kept the original page around.

Fix this because it's annoying and dumb: we know our URL when rendering
templates, so we can redirect back there after login.

Consideration: this could also be done on logout, however it seems likely that
in that case the original page is "privileged" and when coming back we'd just
get an access error. So don't do it for now.
2022-07-25 12:12:02 +02:00
Xavier-Do
9fb7e94de9 [IMP] add load info page 2022-07-25 12:10:48 +02:00
Xavier-Do
a52c14957e [IMP] runbot: log manual batch and redirect
1. Log who started a manull build
2. redirect if the batch given in a build url is not containing this
build (manual url)
2022-07-12 13:39:44 +02:00
Xavier-Do
af8a5b8e8e [FIX] runbot: avoid error when there is no build 2022-07-08 15:53:43 +02:00
Christophe Monniez
e1161a03e6 [FIX] runbot: avoid sudo when using pip
In bd4cf76b7 a new argument `--progress-bar off` was added to the pip
command line. Unfortunately, it appears that this argument is not
available in some pip versions, typically the one from Ubuntu Bionic
(9.0.1) [0]. The argument appeared in version 10.0.0 [1].

Also, in the DockerFile, we allow the user to use `sudo` for
`/usr/bin/pip3` and the main reason for that was to avoid having `pip`
to re-install the packages that are pre-installed by the distribution
package manager.

That said, it appears that since pip 10.0.0 when installing packages
locally for the user, pip now properly detects distribution installed
packages.

So, as the solution to fix the progress bar issue is to upgrade pip to a
newer version, we can take advantage of it to get rid of the `sudo`.

Besides the fact that `--user` is the default on Debian based
distributions [2] , we enforce it in case another distribution is used
in the Dockerfile.

[0] https://packages.ubuntu.com/bionic-updates/python3-pip
[1] https://pip.pypa.io/en/stable/news/#id744
[2] https://wiki.debian.org/Python#Deviations_from_upstream
2022-07-06 09:48:20 +02:00
Xavier-Do
bd4cf76b76 [FIX] runbot: hide progressbar for pip 2022-07-01 15:47:13 +02:00
Xavier-Do
03732638f3 [IMP] runbot: add an automatic generated message for upgrade exceptions 2022-07-01 15:47:13 +02:00
Christophe Monniez
b25aa52fb9 [IMP] runbot: avoid core dumps in containers
In some conditions, the google-chrome crashes and a core dump is written
in the bind mounted directory.

With this commit, the core dumps are disabled in the containers.
2022-06-30 10:58:38 +02:00
Christophe Monniez
78697d2acb [IMP] runbot: use docker python lib
While using the docker cli as a subprocess was KISS and convenient,
the python Docker SDK is mature, easy to use, available as a Debian
package and much more powerful.

It will permit to monitor the containers memory consumption and will
help to spot memory leaks.
2022-06-30 10:58:38 +02:00
Christophe Monniez
9fb48f48cf [IMP] runbot: improve git gc warning 2022-06-27 09:11:02 +02:00
Christophe Monniez
3f042c44a8 [FIX] runbot: fix gc warning 2022-06-24 08:41:32 +02:00
Christophe Monniez
c006bf953d [FIX] runbot: apply small fixes 2022-06-20 14:37:02 +02:00
Christophe Monniez
d534f5a5e2 [FIX] runbot: count only really active hosts 2022-06-20 10:32:10 +02:00
Christophe Monniez
250d48e266 [IMP] runbot: schedule git gc on repositories
On the actual runbot deployments, the `git gc` command is handled by a
unix cron. From time to time, some repositories get corrupted and we
suspect that some concurrent action may be involved as stated in
documentation [0].

For those reasons, with this commit, the `git gc` will be run by the
runbot clients themselves in order to avoid concurrent operations.

By default, the first gc will occur a few minutes after the start of the
client and the next gc are scheduled a two hours and a few minutes later.

Also, this commit ensures that the git config is written regularly in
case of change.

[0] https://git-scm.com/docs/git-gc
2022-06-20 10:32:10 +02:00
Xavier-Do
cdaae9b3ed [IMP] runbot: speedup get_refs
Regarding the number of refs in odoo repo (arround 18 million at this
time), the parsing of the date was significant when filtering old refs.

Using unix time allows a direct comparaison without parsing the date,
and improved performance, going from ~7 seconds to ~1.3 seconds.
2022-06-20 10:20:37 +02:00
Xavier-Do
cbfc8401a8 [FIX] runbot: various FIX 2022-06-17 13:52:37 +02:00
Xavier-Do
26a3ad20f1 [IMP] runbot: status management
The status are currently sent by leader when build are created and by
workers on post_commit.

If the leader fails during the preparing of a batch (while creating
builds) the transaction is rollbacked and the status are send again.

The number of status to send makes it quite slow, making the transaction
longuer, and the retry even more expensive. This leads to preparing time
being quite important, sometimes ten minutes after many retries.

This commit proposes to send status in another dedicated transaction.
Since status are sent in batch, we can also try to use a unique session,
and uniquify commit+context status.

This allows to remove the postcommit logic

A further improvement would be to wait before sending status in order to
skip the pending status if the build is verry fast.

An option is also added on the remote to skip the status: if the remote
is a fork, sending the satus on the main remote should be enough.
2022-06-17 12:32:14 +02:00
Christophe Monniez
b997119588 [FIX] runbot: avoid children in team errors
When an error is assigned to a team and the error is a child of another
one, the two errors are displayed on the team dashboard and adds
confusion.
2022-06-17 10:40:57 +02:00
Christophe Monniez
69cd228b6b [IMP] runbot: use the lovely boolean_toggle widget 2022-06-16 13:39:46 +02:00
Xavier-Do
47c12fe00b [FIX] runbot: add MULTILINE to ReProxy 2022-06-15 12:30:41 +02:00