Commit Graph

569 Commits

Author SHA1 Message Date
Xavier-Do
9215a23cc1 [FIX] change version number to avoid second execution when migrating to 13.0 2020-01-14 13:24:57 +01:00
Xavier-Do
3ef33a5320 [FIX] runbot: clear environment, do not reset
Resetting env will make this env unusable with the new orm,
breaking field recomputes logic
2020-01-14 13:24:57 +01:00
Xavier-Do
20f97798d9 [FIX] runbot: use create_single decorator everywhere
All create in odoo are now create_multi. This is a quick and dirty
fix to make it work with runbot.
2020-01-14 13:24:57 +01:00
Xavier-Do
8dcaa6669b [FIX] runbot: allow portal user to see frontend 2020-01-14 13:24:57 +01:00
Xavier-Do
4e694ffefa [IMP] runbot: addapt tests to 13.0 2020-01-14 13:24:57 +01:00
Xavier-Do
1dcea28f45 [FIX] runbot: make field name unique 2020-01-14 13:24:57 +01:00
Xavier-Do
1069598616 [REF] runbot: use selection_add 2020-01-14 13:24:57 +01:00
Xavier-Do
df157f7742 [REF] runbot: add descriptions 2020-01-14 13:24:57 +01:00
Xavier-Do
2b53c7cfd0 [REF] runbot: adapt cron path 2020-01-14 13:24:57 +01:00
Xavier-Do
e706bb70c2 [REF] runbot: remove deprecated model_cr 2020-01-14 13:24:57 +01:00
Xavier-Do
24189fd0ee [REF] runbot: runbot deprecated api.multi 2020-01-14 13:24:57 +01:00
Christophe Monniez
926ab4a9d9 [FIX] runbot: fix build_error test tags validation
When trying to remove test_tags on a build_error, the validation fails
because it tries to iterate on False. Also, the ValidationError
exception was not properly imported.

With this commit, the validation is fixed and a test is added.
2020-01-06 11:52:13 +01:00
Christophe Monniez
501142c561 [IMP] runbot: allow to rebuild a branch
When a repo is set to "no_build", there is no way to force a branch to
build from the backend.

With this commit, a button is added on a branch form to ask rebuild of the
branch, even when the repo is set to "no_build".
2019-12-19 15:41:11 +01:00
Xavier-Do
eaee010b25 [IMP] runbot: add relation between branches
Migration tests comming on runbot, it will be usefull to have quick
way to obtain branches related to current build.

This commit adds a field for the colsest sticky branch, previous version
and intermediates stickies.

Example when last sticky is saas-13.2:
branch_name: master-test-tri
closest_sticky: master
previous_version: 13.0
intermediate_stickies: saas-13.1, saas-13.2
2019-12-19 15:38:53 +01:00
Xavier-Do
328ba2d9c2 [IMP] runbot: adding some log 2019-12-18 17:37:06 +01:00
Xavier-Do
9ae816a23d [IMP] runbot: add some log 2019-12-18 17:27:33 +01:00
Christophe Monniez
7606c03c35 [IMP] runbot: allow to change prefix in multi wizard
Also fix the modules that were not set on the config step.
2019-12-18 16:36:14 +01:00
Xavier-Do
cede4b936e [FIX] runbot: fix docker state and rebuild button
When docker is stopped, end command cannot execute.
Thus the state is considered as running.
2019-12-18 16:30:01 +01:00
Xavier-Do
426b7af2cb [REF] runbot: refactor sheduler 2019-12-18 16:30:01 +01:00
Xavier-Do
630e1d55f7 [IMP] runbot: custom python result
This commit add the possibility to add custom checks to python steps,
as well as ignoring triggered result if log of level error/warning
is not considered as a problem.
2019-12-18 14:31:49 +01:00
Christophe Monniez
6c64bbb49b [IMP] runbot: improve build log view
Co-authored-by: Xavier-Do <xdo@odoo.com>
2019-12-17 16:08:31 +01:00
Christophe Monniez
5552c16102 [IMP] runbot: create a broken symlink in all server sources
As the --upgrade-paths options does not work as expected in Odoo, a
symbolic link has to be created in odoo/addons/base/maintenance pointing
to the migration scripts.

The runbot uses Docker read-only volumes to access the sources that are
shared between builds, preventing the creation of such a link.

With this commit, a symbolic link is created right after the export of a
commit only when the repo is a "server" repo.

This link is broken outside of the Docker volumes but uses the mount
points of the sources inside the container.

Two ir.config_parameter's are used to enable and configure this feature:

* runbot_migration_ln: the relative path where the link should be placed
  (typically odoo/addons/base/maintenance)

* runbot_migration_repo_id: the id of the migration scripts repo, used
to determine the name of the mount point inside the Docker container

A change is also made in the "reverse dependcy build" to avoid the
creation of a build in the migration repo for each push in its
dependency. Simply set the no_build field on this migration repo.
2019-12-16 13:10:55 +01:00
Xavier-Do
8d20c2ce79 [IMP] runbot: add Commit to run_python steps context 2019-12-13 13:44:01 +01:00
Xavier-Do
2ed0b9c4d4 [FIX] runbot: ref_time rounding.
`getmtime` will return a 6 digit float when postgress will only store 5.
Depending on rounding, _get_refs have 1/2 chance to make an update
when it shouldn't. Rounding below psql precision before comparing and
storing should fix this.
2019-12-09 16:32:27 +01:00
Xavier-Do
e5a2f98c59 [IMP] runbot: store times in other tables.
Sometimes, sheduler may have a hard time to create build.
The transaction can be verry long when there are many repo and
a lot of new commits. Writing get_ref_time on repo will fail
due to concurrent update rollbacking the whole transaction.

This is supposedly because of hook occuring during the transaction.
With this new model, hook will only perform an insert, and shouldn't
interfer with ref_times.
2019-12-09 15:40:02 +01:00
Christophe Monniez
3f806416d6 [FIX] runbot: adapt test to new docker_state method 2019-12-06 11:40:38 +01:00
Xavier-Do
a3abdf8505 [IMP] runbot: add docker state
docker_is_running is ambiguous since we dont know if it was started once.
This new feature tries to add tools to know if a docker was started or not.

The main reason of this is that sometimes docker_run may take more than 15 seconds
creating unpredictable errors on build when the second step is launched and
the previous one is still running. Hopefully this fix will help to solve this
issue and detect late docker run.
2019-12-06 10:20:11 +01:00
Christophe Monniez
83651c408e [IMP] runbot: keep txt files when cleaning builds dirs 2019-12-04 16:20:17 +01:00
Xavier-Do
b69d40af59 [FIX] runbot: ignore rebuilds in _find_new_commits
When rebuilding a build/subbuild, the last commit of the branch
won't have the same sha of the branch last commit. find_new_commit
will kill the running builds in the branch and create a new one,
that may be a duplicate of one of the killed commits.

This fix only take normal builds into account when checking for
existing builds.
2019-12-04 15:55:08 +01:00
Xavier-Do
f2db93fc63 [FIX] runbot: create nginx.conf if not exist
When starting a new runbot, nginx.conf does not exist. This fix will
allow _reload_nginx to initialize it the first time.
2019-12-04 13:58:03 +01:00
Xavier-Do
7f8547f4ec [IMP] runbot: add log on get_param failure 2019-12-04 12:21:23 +01:00
Xavier-Do
12febd1920 [FIX] runbot: reintroduce try exept in get_params
This commit was initially there for tests, when no repo exist, but
get_param will also crash if commit does not exist, wich may be
a problem on user rebuild.
2019-12-04 12:13:18 +01:00
Xavier-Do
6681e40231 [FIX] runbot: better source cleaing
A build can sometimes fail and be stuck in a running state
without corresponding sources. In this case, source are not gc anymore
This commit fixes that by always applying gc even when an inconsistency
is detected.
2019-12-04 11:26:54 +01:00
Xavier-Do
9e5ee700f3 [FIX] runbot: sha instead of commit 2019-12-03 16:58:30 +01:00
Christophe Monniez
8ab89d2e2f [FIX] runbot: fix percent in format string and typos 2019-12-03 16:07:44 +01:00
Xavier-Do
9cc46d1d3a [IMP] runbot: add force option to cleanup specific build now 2019-12-03 10:20:12 +01:00
Christophe Monniez
c9732ed68d [FIX] runbot: remove dash options in generated config file
When creating an .odoorc file to store configuration that are proper for
the runbot, the command line options were used as key.
The problem is that the Odoo config file use undescore instead of dash
for options keys.

Also, the Command object was not recreated with the config_tuples
parameter. Because of that, when adding a command in the list, the
config_tuples were losts.

As a consequence, on the Odoo runbot instance, the data dir was created
in the default dir and thus, not included in the zip file of the dump,
causing some runbot steps to fail.
2019-12-02 16:45:13 +01:00
Christophe Monniez
73f720a55c [IMP] runbot: refactor tests
A lot of things have to be mocked during runbot tests, as a consequence,
a lot of patch decorators accumulate in a big stack uppon some tests
methods.

Also, a lot of mocks are used multiple times among tests.

With this commit, a new RunbotClass is added that comes with patches
ready to be started. A start_patcher helper method is available to start
a patch and add the appropriate stop in a cleanup.

Also, when a build is created in the tests, the _get_params method is
always called, resulting in an annoying git warning.

With this commit, a create_build method is added on the test class, that
way the _get_params is always mocked when a build is created.
2019-12-02 16:36:56 +01:00
Xavier-Do
7d1283492b [IMP] runbot: dynamic requirements 2019-12-02 16:36:56 +01:00
Xavier-Do
4ee0d93abf [IMP] runbot: general runbot log improvement 2019-11-27 15:37:58 +01:00
Xavier-Do
54ecee8c4e [IMP] runbot: create build when old HEAD is force-pushed
_find_new_commits will check if a build exists with current branch HEAD
before creating build. This is crutial to avoid to create a new build
at each loop turn. The problem is that in some rare cases, when
force-pushing an old head on a branch, the build won't appear and the only
way to update the branch is to find the corresponding build that may be
hidden in hitory. This may be confusing for the user that will rebuild the
created build with a commit that doesn't represent the head of the branch.

This commit only search for the last build of each branch, in order to
only skip build creation if the last build as the same hash. The new
created build should be marked as the duplicate of the first one.
2019-11-27 15:37:58 +01:00
Christophe Monniez
064546441f [IMP] runbot: use a config file to simplify args
When starting an odoo instance with Docker, a very long command line is
computed and appears in the logs.

With this commit, an .odoorc configuration file is written ind the build
dir and mounted in the Docker container.

Previously, the runbot .odoorc/.openerprc file was mounted to share some
parameters. Now, if that file exsists, its content is merged with build
.odoorc.
2019-11-27 15:37:43 +01:00
Christophe Monniez
e8eb4eea10 [IMP] runbot: add unzip tool in Docker image 2019-11-22 15:42:48 +01:00
Christophe Monniez
f8614c4abd [FIX] runbot: grep for enhanced test-tag for auto_tags
When using auto_tags, most of the time, the enhanced version are used.
For example, using "-:TestPoSStock" to disable the test class.
If the tested Odoo version does not support this kind of tag, they are
considered as simple tags, thus disabling all tests.

It 's the case for Odoo saas-11.3.

With this commit, the auto_tags are only used on Odoo versions that
support the new test tags.
2019-11-19 14:54:16 +01:00
Xavier-Do
1e102b7d76 [FIX] runbot: check support before adding auto-tags 2019-11-08 15:40:36 +01:00
Xavier-Do
0c4d78f497 [FIX] runbot: various small fixes 2019-11-08 15:33:30 +01:00
Xavier-Do
a7992e9f50 [IMP] runbot: add no_build on repo
When runbot is installed to test customs addons, we don't
want to build all odoo commit, but we need to update branches
in order to make _get_closest_branch work.

This commit will allow a user to set odoo in poll mode
with no_build set to True, to create branches only.

(And a small fix for additionnal_env)
2019-11-08 12:08:02 +01:00
Christophe Monniez
631b29fcf8 [FIX] runbot: fix container cli tests
Since 81fefee, the container.py CLI does not work as expected.

With this commit, the CLI is working, a new arg was added to test
flamegraphs and the dump is adapted to mimic the runbot.

Also, a small issue is fixed in the zip file creation. Before the zip
creation, the directory is changed, if the directory change fails, the
zip is created from the current directory which is removed by zip at the
end of the process. That could lead to the deletion of the build dir.
2019-11-08 10:29:51 +01:00
Xavier-Do
685cc282ba [IMP] runbot: add env parameter support for docker in config steps 2019-11-08 10:29:51 +01:00
Xavier-Do
58c683030d [IMP] runbot: add auto test-tags management.
A typical use case when an error is detected is to disable
this test by adding a negated test-tags on config
step 'all' and 'split_all'. This commit will help
to do that by adding a test_tags management on build error.

The user define a test_tag that will only execute failling test.
if a config step has the flag enable_auto_tags, the test tag will
be negated and added to config test-tags.

This commit also add some information for monitoring.
2019-11-07 09:51:05 +01:00