runbot/runbot_merge/models/crons/git_maintenance.py
Xavier Morel b1d3278de1 [CHG] forwardport: perform forward porting without working copies
The goal is to reduce maintenance and odd disk interactions &
concurrency issues, by not creating concurrent clones, not having to
push forks back in the repository, etc... it also removes the need to
cleanup "scratch" working copies though that looks not to have been an
issue in a while.

The work is done on isolated objects without using or mutating refs,
so even concurrent work should not be a problem.

This turns out to not be any more verbose (less so if anything) than
using `cherry-pick`, as that is not really designed for scripted /
non-interactive use, or for squashing commits thereafter. Working
directly with trees and commits is quite a bit cleaner even without a
ton of helpers.

Much of the credit goes to Julia Evans for [their investigation of
3-way merges as the underpinnings of cherry-picking][3-way merge],
this would have been a lot more difficult if I'd had to rediscover the
merge-base trick independently.

A few things have been changed by this:

- The old trace/stderr from cherrypick has disappeared as it's
  generated by cherrypick, but for a non-interactive use it's kinda
  useless anyway so I probably should have looked into removing it
  earlier (I think the main use was investigation of the inflateinit
  issue).
- Error on emptied commits has to be hand-rolled as `merge-tree`
  couldn't care less, this is not hard but is a bit annoying.
- `merge-tree`'s conflict information only references raw commits,
  which makes sense, but requires updating a bunch of tests. Then
  again so does the fact that it *usually* doesn't send anything to
  stderr, so that's usually disappearing.

Conveniently `merge-tree` merges the conflict marker directly in the
files / tree so we don't have to mess about moving them back out of
the repository and into the working copy as I assume cherry-pick does,
which means we don't have to try and commit them back in ether. That
is a huge part of the gain over faffing about with the working copy.

Fixes #847

[3-way merge]: https://jvns.ca/blog/2023/11/10/how-cherry-pick-and-revert-work/
2024-07-08 14:37:14 +02:00

38 lines
1.3 KiB
Python

import logging
import subprocess
from odoo import models
from ...git import get_local
_gc = logging.getLogger(__name__)
class GC(models.TransientModel):
_name = 'runbot_merge.maintenance'
_description = "Weekly maintenance of... cache repos?"
def _run(self):
# lock out crons which use the local repo cache to avoid concurrency
# issues while we're GC-ing it
Stagings = self.env['runbot_merge.stagings']
crons = self.env.ref('runbot_merge.staging_cron', Stagings) | self.env.ref('forwardport.port_forward', Stagings)
if crons:
self.env.cr.execute("""
SELECT 1 FROM ir_cron
WHERE id = any(%s)
FOR UPDATE
""", [crons.ids])
# run on all repos with a forwardport target (~ forwardport enabled)
for repo in self.env['runbot_merge.repository'].search([]):
repo_git = get_local(repo, clone=False)
if not repo_git:
continue
_gc.info('Running maintenance on %s', repo.name)
r = repo_git\
.stdout(True)\
.with_config(stderr=subprocess.STDOUT, text=True, check=False)\
.gc('--prune=now', aggressive=True)
if r.returncode:
_gc.warning("Maintenance failure (status=%d):\n%s", r.returncode, r.stdout)