Commit Graph

2062 Commits

Author SHA1 Message Date
Olivier Dony
8dfee8af40 [FIX] runbot: stop forcing git gc --auto during updates
The `git gc --auto` command often fails and requires an explicit
execution of `git gc --prune=all`. When it happens, it crashes the
update, failing the cron execution.

Executing `git gc prune` every time would be very slow and dispruptive
during the cron's critical sections.

As the initial goal was to avoid triggering the automatic GC during the
next update/pull command, we prefer to adopt the global strategy of
disabling the auto GC globally with:

  git config --global gc.auto 0

And to configure a system cron job to periodically GC/prune all
repositories, to get the performance benefit.
Due to locking issues, that cronjob should be scheduled outside peak
build hours.

Example bash script for the system cron:

```
   #!/bin/bash
   set -e
   base_repo_dir='/path/to/runbot/static/repo'

   for repo in $base_repo_dir/*
   do
       if [[ -d $repo ]];then
           git --git-dir="$repo" gc --prune=all
       fi
   done
```
2017-05-16 11:00:01 +02:00
Christophe Simonis
a5b88ff0ab [FIX] runbot: failing to update a repo should not stop others 2017-05-05 11:38:08 +02:00
Olivier Dony
99c200afcb [FIX] runbot: recover from failed job method call
Catch exceptions occurring within job methods, to avoid
stalling the runbot by repeatedly trying to call it -
and crashing every time.

Mark the build as failed immediately and skip it.
2017-05-02 14:32:13 +02:00
Olivier Dony
1d46bf9ff5 [FIX] runbot: stop wasting a cycle for coverage check
Build jobs that return a -2 PID are considered immediately done and the
next job is started without waiting for a cron cycle.
2017-05-02 13:25:42 +02:00
Olivier Dony
63b997a4ed [FIX] runbot: make checkout safer wrt addons layout
Avoid errors/crash when the server/addons directory in the build branch
contains symlinks.

Exhibited by e.g. odoo/odoo#16720
2017-05-02 13:07:02 +02:00
Christophe Simonis
f889f2095c [IMP] runbot: let nginx serve static files 2017-04-27 16:43:48 +02:00
Olivier Dony
d66047230b [FIX] runbot*: make most methods private
Most of the non-CRUD methods of runbot models should be private, as
there is no need to access them via RPC.

WARNING: This change is backwards compatible with existing
installations and existing runbot databases, but will most
likely BREAK all third-party extensions, due to the renamed
methods.

Fortunately the runbot module is not ruled by Odoo's "stable policy" ;-)

Reported by @nilshamerlinck - thanks!
2017-04-27 14:54:39 +02:00
Olivier Dony
e1ecb49f89 [FIX] runbot: use proper manager visibility
The token is not meant to be used by anyone except runbot managers.

Reported by @nilshamerlinck - thanks!
2017-04-27 14:54:39 +02:00
Christophe Simonis
3b1145b4ab [IMP] runbot: ignore pad modules 2017-03-10 11:56:23 +01:00
Christophe Simonis
f242db17b3 [FIX] runbot: only kill builds on same host 2017-03-10 11:51:27 +01:00
Nicolas Seinlet
12691bdbcd Private method for quick connect URL, to allow an easy override 2017-02-21 15:18:20 +01:00
Christophe Simonis
33584cea10 [FIX] runbot: when running via coverage, skip sys.executable in command line 2017-02-09 17:03:39 +01:00
Christophe Simonis
9282b10034 [FIX] runbot: respect repo sequence in dashboard 2017-02-08 13:53:53 +01:00
Christophe Simonis
efd049da1d [FIX] runbot: index repo sequence 2017-02-08 13:30:18 +01:00
Christophe Simonis
0fa8bbe620 [ADD] runbot: add support for running coverage when testing. 2017-01-24 14:14:20 +01:00
Christophe Simonis
e29c25c7dc [IMP] runbot: add sequence field on repo model 2017-01-24 14:06:34 +01:00
Nicolas Seinlet
65e86b564e [IMP] Clean old, unused and not deleted runbot builds databases 2017-01-10 16:42:24 +01:00
Christophe Simonis
b04488f22f [IMP] runbot: handle new binary for 10.0 2016-09-29 14:33:15 +02:00
Olivier Dony
48e652436e [IMP] runbot: support v10-style manifest naming convention 2016-09-02 19:23:02 +02:00
Julien Legros
10c3e87579 [IMP] runbot: optimize new branch detection 2016-09-02 13:05:22 +02:00
Christophe Simonis
a835d1e800 [IMP] runbot: add link to original build for duplicate builds 2016-09-02 13:02:56 +02:00
Christophe Simonis
287c466637 [FIX] runbot: hide bad link to github for messages from runbot itself 2016-09-02 13:02:11 +02:00
Christophe Simonis
b46980ab91 [FIX] runbot: export git tree at current time
When doing a `git archive`, all the files in the tar archive are set to
the commit date. Ignore this date during extraction.

Having the files with dates in future will make some tests about assets
to fail. These tests force the asset regeneration by `touch`ing one
js/css file. If other bundle files are still older that the one we touch,
the asset is not regenerated and the test fail.
2016-09-01 12:36:36 +02:00
Alexandre Fayolle
920c3edcee [FIX] runbot: avoid 404 errors in fast_launch url for odoo >= 9
Closes https://github.com/odoo/odoo-extra/pull/104
2016-07-27 17:49:29 +02:00
Christophe Simonis
97001fbf72 [FIX] runbot: remove useless commit()
This commit() is the root cause of the famous false-positive error
`_check_module_names invalid module names, ignored: False`.

The first step (pull code) is a long one. if a push-hook request
happen during this step, the current transaction wont be able to be
commited (the list of module to test is written on the build). Due to
this early commit(), the build is left in an intermediate state with
the module list unpopulated (=False).
2016-07-26 18:28:24 +02:00
Christophe Simonis
40e502ecd5 [FIX] runbot: track and log prefix match of closest branch 2016-04-27 16:09:04 +02:00
Christophe Simonis
fd8e7f269b [FIX] runbot: check if branch exists when matching 2016-04-04 19:24:02 +02:00
Christophe Simonis
f7b2a40f43 [FIX] runbot: favor sticky branches during matching 2016-02-23 11:48:18 +01:00
Olivier Dony
d9c9642675 [FIX] runbot: workaround for nginx bug leaving orphan worker, blocking port 2016-02-12 15:24:32 +01:00
Christophe Simonis
72bfc504cc [IMP] runbot: hide stopped hosts from dashboard page 2016-02-09 14:48:38 +01:00
Christophe Simonis
f77c52119e [IMP] runbot: log matching closest branches 2016-02-09 14:38:38 +01:00
Christophe Simonis
7e9ddee6ea [IMP] runbot: cleanup builds after 7 days.
By cleaning job only after 7 days, we ensure we keep logs of killed
jobs (or jobs that doesn't even start) for easier debuging.
2016-02-09 14:11:49 +01:00
Olivier Dony
0535f99616 [FIX] runbot_cla: avoid unicode errors when reading CLA files 2016-02-03 15:50:07 +01:00
Jeremy Kersten
5ef0b2ee6d [FIX] runbot: avoid traceback when build_ids was empty (eg: filter by search) or not declared.
equivalent to max(build_ids, default=0) in python 3
2016-01-27 17:29:09 +01:00
Jeremy Kersten
fccd90f755 [IMP] runbot: keep search when we switch of repo from shorcut at top right 2016-01-26 16:59:45 +01:00
Olivier Dony
93dd0f5ee0 [FIX] runbot: do not use remote cursor for dropping local dbs
To find out the list of manually created dbs,
rev. 4d94f45 used the cursor that is connected to the
master `runbot` database, which may reside on a different
cluster/host.

- Add a helper to run commands on the local PG cluster
instead ("postgres" database).
- Modify other commands for local cluster (create/drop db)
to use the same cursor. Rename those methods to _local_*
to better indicate their local effect.
2016-01-18 18:47:17 +01:00
Olivier Dony
be34ac72c9 [FIX] runbot: reduce serialization failures (TransactionRollbackError)
Both the cron and the tests themselves need to write to the builds,
which triggers the update of the `repo_id` field, as it is a stored
related.
2016-01-18 18:09:05 +01:00
Olivier Dony
431a3f6d5b [IMP] runbot: do not display hosts with no recent builds (considered stopped) 2016-01-18 18:09:05 +01:00
Joren Van Onder
fa68c54477 [IMP] runbot: blacklist a couple of POS related modules
- pos_blackbox_be: because it makes the POS unuseable without a blackbox
- pos_cache: because it's too confusing for runbot users, everytime they
             update a product they have to also update the cache, but
             most people don't realise that.
2015-12-10 09:26:59 +01:00
Moises Lopez
7a842e5e45 [FIX] runbot: Fix csrf error in rebuild method
Runbot is intended to be run on a 8.0 instance but if runs on a 9.0, CSRF needs
to be disabled. Will not break on 8.0 code.

Closes #84
2015-10-22 12:20:37 +02:00
moylop260
3b5669bd01 [FIX] runbot: Fix no such file FETCH_HEAD
Closes #80
2015-10-09 14:47:43 +02:00
Olivier Dony
d1cff975be [IMP] runbot: exclude l10n_ modules unless explicit
As of Odoo 9 the first l10n module installed will automatically
install its chart of account on the main company, so installing
the first one (l10n_ae) is surprising/strange.

Branches that want to build l10n_* modules should list
them explicitly in the "modules to install" field (modules).

In addition, having demo data to set the country of the
main company will automatically installe the relevant
l10n module if it exists. This is the default in Odoo 9
with l10n_us.
2015-10-09 14:41:46 +02:00
Olivier Dony
19f6c36156 [IMP] runbot: detect and warn for builds where server branch cannot be properly determined
More useful recently for enterprise builds where an
appropriate server branch is always required
2015-09-23 16:06:35 +02:00
Olivier Dony
42ae78f647 [FIX] runbot: typo, branch is a dict from search_read() 2015-09-11 12:24:21 +02:00
Odoo Online
d40460d9f1 [IMP] runbot: ir.cron frequency from 0.016 Hz to 0.1 Hz
Monkey patch ir.cron to increase frequency to 0.1 Hz.

In a setup with 6 workers every worker should have the lock every minute.
2015-09-09 01:16:37 +02:00
Odoo Online
44771039ce [FIX] runbot: repo hook 2015-09-09 00:35:45 +02:00
Antony Lesuisse
b9c2a1f9e1 [IMP] runbot add github hook support 2015-09-08 17:04:30 +02:00
Olivier Dony
544bbe5a10 [FIX] runbot: typo s/local/repo 2015-08-24 10:40:13 +02:00
Christophe Simonis
abce301ed1 [IMP] runbot: search of closest branch
When searching the closest branch, search through all duplicated
repositories.
2015-08-21 14:48:21 +02:00
Christophe Simonis
bab6221882 [IMP] runbot: drop all db with build prefix on cleanup 2015-08-19 19:36:49 +02:00