Compare commits

..
Author SHA1 Message Date
Dr.Lt.Data ef991cf059 fix(git_compat): follow-ups for pygit2 fallback hardening
- Route list_remotes() fetch through _fetch_remote so the proxy and
  SSH-to-HTTPS rewrite apply to every fetch entry point, and provide
  pull on its proxies (parity with get_remote and the GitPython proxy)
  via a shared _pull_remote helper
- Rework _to_https_url: handle ssh://git@host:port/... URLs (drop the
  custom SSH port instead of mangling it into the path) and collapse
  leading slashes so git@host:/abs/path no longer yields a double-slash
  URL. Behavior narrowing: colon-less git@host/path (not a valid
  scp-form URL), ssh:// URLs with a port but no path, and IPv6 ssh://
  URLs are now returned unchanged instead of being wrongly converted
- clone_repo: omit the proxy= kwarg when no proxy is configured, so
  proxy-less installs keep working on pygit2 older than 1.18
- Simplify redundant except (KeyError, Exception) clause
- Require pygit2>=1.18 (clone_repository proxy= parameter)
- Add unit tests for _to_https_url (incl. negative cases pinning the
  narrowing) and a regression test that list_remotes proxies route
  fetch/pull through their own remote (late-binding guard)
2026-06-10 18:15:19 +09:00
Jedrzej KosinskiandAmp 622a7077a5 fix(git_compat): harden pygit2 fallback path; prefer system git when available (#2972)
* fix(git_compat): ignore global git config in pygit2 backend

Under Desktop 2.0 the launcher sets CM_USE_PYGIT2=1, so the pygit2 backend ran clone_repository/remote.fetch with no credentials callback and honored the user's global git config. An insteadOf rewrite (https->ssh) or credential helper then forced authentication, failing with 'authentication required but no callback set'.

Blank the system/global/XDG config search path at import time so libgit2 operations are hermetic, and normalize SSH-form GitHub URLs to anonymous HTTPS on clone and when opening a repo.

Amp-Thread-ID: https://ampcode.com/threads/T-019eafa0-16a1-726e-91a4-dac1a40d4481
Co-authored-by: Amp <amp@ampcode.com>

* fix(git_compat): preserve corporate http.proxy in pygit2 backend

Snapshot http.proxy from the global git config before blanking the config search path, then pass it explicitly (proxy=) to clone_repository and every remote.fetch() in the pygit2 backend, so corporate-lockdown proxy setups keep working after the insteadOf/SSH hardening.

Amp-Thread-ID: https://ampcode.com/threads/T-019eafa0-16a1-726e-91a4-dac1a40d4481
Co-authored-by: Amp <amp@ampcode.com>

* fix(git_compat): stop rewriting repo remotes on disk under pygit2 backend

Removing _normalize_remote_urls(): persistently rewriting a repo's SSH origin
to HTTPS mutates on-disk repo state, which is risky if interrupted. The pygit2
backend already neutralizes auth-forcing global config (insteadOf, credential
helpers) by blanking libgit2's config search path, so anonymous HTTPS fetch
works without touching the stored remote.

Manager already prefers the GitPython/system-git backend when a system git is
present (which honors the user's full git config including insteadOf https->ssh
and proxies), and only uses the bundled pygit2 when system git is absent or
CM_USE_PYGIT2=1 is set.

Amp-Thread-ID: https://ampcode.com/threads/T-019eafa0-16a1-726e-91a4-dac1a40d4481
Co-authored-by: Amp <amp@ampcode.com>

* fix(git_compat): fetch SSH-origin repos via in-memory anonymous HTTPS

Consolidate the five pygit2 fetch sites into a single _fetch_remote helper.
When a repo's stored origin is SSH-form (git@host:owner/repo), the bundled
pygit2 (no SSH transport) would fail with an auth error; fetch through an
in-memory anonymous remote over HTTPS instead, leaving .git/config untouched.
Also closes a proxy hole where get_remote() exposed remote.fetch without the
preserved http.proxy.

Validated against both backends (pygit2 1.19.2 + GitPython): all 47
git_compat tests pass; verified create_anonymous fetch updates
refs/remotes/origin/* without persisting any remote or rewriting origin.

Amp-Thread-ID: https://ampcode.com/threads/T-019eafa0-16a1-726e-91a4-dac1a40d4481
Co-authored-by: Amp <amp@ampcode.com>

---------

Co-authored-by: Amp <amp@ampcode.com>
2026-06-10 16:33:18 +09:00
4 changed files with 223 additions and 23 deletions
+117 -21
View File
@@ -14,11 +14,40 @@ Exports:
"""
import os
import re
import sys
from abc import ABC, abstractmethod
from collections import deque
from datetime import datetime, timezone, timedelta
# Snapshot of the user's global `http.proxy` (captured before the config
# search path is blanked under the pygit2 backend) so corporate proxy
# settings survive and can be passed explicitly to fetch/clone. None means
# "no proxy".
_HTTP_PROXY = None
def _to_https_url(url):
"""Rewrite an SSH-form git URL to its anonymous HTTPS equivalent.
Handles `git@host:owner/repo(.git)` and `ssh://git@host[:port]/owner/repo(.git)`.
A custom SSH port is dropped — the HTTPS endpoint of a hosting service
lives on the standard port regardless of its SSH port. Leading slashes in
the path part are collapsed so `git@host:/abs/path` does not yield a
double-slash URL. Returns the URL unchanged if it is not SSH-form, so
pygit2 clone/fetch of public repos never requires SSH credentials even
when a repo's own config stores an SSH origin.
"""
if not url:
return url
m = re.match(r"^ssh://git@([^:/]+)(?::\d+)?/(.+)$", url)
if m is None:
m = re.match(r"^git@([^:/]+):(.+)$", url)
if m:
return "https://%s/%s" % (m.group(1), m.group(2).lstrip("/"))
return url
# ---------------------------------------------------------------------------
# Backend selection
# ---------------------------------------------------------------------------
@@ -49,6 +78,41 @@ if USE_PYGIT2:
# See CVE-2022-24765 for context on this validation.
_pygit2.option(_pygit2.GIT_OPT_SET_OWNER_VALIDATION, 0)
# Snapshot the global http.proxy BEFORE blanking the config search
# path, so a corporate proxy survives and can be passed explicitly
# to clone/fetch below.
try:
_global_cfg = _pygit2.Config.get_global_config()
try:
_HTTP_PROXY = _global_cfg["http.proxy"] or None
except Exception:
_HTTP_PROXY = None
except Exception:
_HTTP_PROXY = None
# Ignore system/global/XDG git config for libgit2 operations. A
# user's global config can carry `insteadOf` rewrites (e.g.
# https->ssh) or credential helpers that force authentication,
# which libgit2 cannot satisfy without a credentials callback
# ("authentication required but no callback set"). The bundled
# pygit2 has no SSH transport, so an SSH rewrite can never succeed;
# blanking the config search path keeps clone/fetch on anonymous
# HTTPS.
try:
from pygit2.enums import ConfigLevel as _ConfigLevel
_cfg_levels = [_ConfigLevel.SYSTEM, _ConfigLevel.XDG, _ConfigLevel.GLOBAL]
except (ImportError, AttributeError):
_cfg_levels = [
_pygit2.GIT_CONFIG_LEVEL_SYSTEM,
_pygit2.GIT_CONFIG_LEVEL_XDG,
_pygit2.GIT_CONFIG_LEVEL_GLOBAL,
]
for _lvl in _cfg_levels:
try:
_pygit2.settings.search_path[_lvl] = ""
except Exception:
pass
if not USE_PYGIT2:
import git as _git
@@ -388,6 +452,28 @@ class _Pygit2Repo(GitRepo):
self._repo = _pygit2.Repository(git_dir)
self._working_dir = repo_path
def _fetch_remote(self, remote, refspecs=None):
"""Fetch *remote* over the preserved proxy, transparently rewriting an
SSH-form origin to anonymous HTTPS in memory.
The bundled pygit2 has no SSH transport, so a stored `git@host:...`
origin would fail with an auth error. When the URL is SSH-form we fetch
through an in-memory anonymous remote over HTTPS, leaving `.git/config`
untouched (no on-disk rewrite).
"""
https_url = _to_https_url(remote.url)
if https_url != remote.url:
anon = self._repo.remotes.create_anonymous(https_url)
anon.fetch(
list(refspecs) if refspecs is not None
else list(remote.fetch_refspecs),
proxy=_HTTP_PROXY,
)
elif refspecs is not None:
remote.fetch(list(refspecs), proxy=_HTTP_PROXY)
else:
remote.fetch(proxy=_HTTP_PROXY)
@property
def working_dir(self):
return self._working_dir
@@ -440,24 +526,26 @@ class _Pygit2Repo(GitRepo):
name = name[len('refs/remotes/'):]
return name.split('/')[0]
def _pull_remote(self, remote):
"""Fetch *remote* and fast-forward the active branch onto its upstream."""
self._fetch_remote(remote)
branch_name = self.active_branch_name
branch = self._repo.branches.get(branch_name)
if branch and branch.upstream:
remote_commit = branch.upstream.peel(_pygit2.Commit)
analysis, _ = self._repo.merge_analysis(remote_commit.id)
if analysis & _pygit2.GIT_MERGE_ANALYSIS_FASTFORWARD:
self._repo.checkout_tree(self._repo.get(remote_commit.id))
branch_ref = self._repo.references.get(f'refs/heads/{branch_name}')
if branch_ref is not None:
branch_ref.set_target(remote_commit.id)
self._repo.head.set_target(remote_commit.id)
def get_remote(self, name):
remote = self._repo.remotes[name]
def _pull():
remote.fetch()
branch_name = self.active_branch_name
branch = self._repo.branches.get(branch_name)
if branch and branch.upstream:
remote_commit = branch.upstream.peel(_pygit2.Commit)
analysis, _ = self._repo.merge_analysis(remote_commit.id)
if analysis & _pygit2.GIT_MERGE_ANALYSIS_FASTFORWARD:
self._repo.checkout_tree(self._repo.get(remote_commit.id))
branch_ref = self._repo.references.get(f'refs/heads/{branch_name}')
if branch_ref is not None:
branch_ref.set_target(remote_commit.id)
self._repo.head.set_target(remote_commit.id)
return _RemoteProxy(remote.name, remote.url, remote.fetch, _pull)
return _RemoteProxy(remote.name, remote.url,
lambda: self._fetch_remote(remote),
lambda: self._pull_remote(remote))
def has_ref(self, ref_name):
for prefix in [f'refs/remotes/{ref_name}', f'refs/heads/{ref_name}',
@@ -490,7 +578,9 @@ class _Pygit2Repo(GitRepo):
def list_remotes(self):
result = []
for r in self._repo.remotes:
result.append(_RemoteProxy(r.name, r.url, r.fetch))
result.append(_RemoteProxy(r.name, r.url,
lambda r=r: self._fetch_remote(r),
lambda r=r: self._pull_remote(r)))
return result
def get_remote_url(self, index_or_name):
@@ -654,7 +744,7 @@ class _Pygit2Repo(GitRepo):
raise GitCommandError(f"No upstream for branch '{branch_name}'")
remote_name = upstream.remote_name
self._repo.remotes[remote_name].fetch()
self._fetch_remote(self._repo.remotes[remote_name])
upstream = self._repo.branches.get(branch_name).upstream
if upstream is None:
@@ -779,12 +869,12 @@ class _Pygit2Repo(GitRepo):
def fetch_remote_by_index(self, index):
remotes = list(self._repo.remotes)
remotes[index].fetch()
self._fetch_remote(remotes[index])
def pull_remote_by_index(self, index):
remotes = list(self._repo.remotes)
remote = remotes[index]
remote.fetch()
self._fetch_remote(remote)
# After fetch, try to ff-merge tracking branch
try:
branch_name = self.active_branch_name
@@ -824,7 +914,13 @@ def clone_repo(url, dest, progress=None):
(checkout, clear_cache, close, etc.).
"""
if USE_PYGIT2:
_pygit2.clone_repository(url, dest)
# The proxy= kwarg requires pygit2>=1.18; omit it when no proxy is
# configured so proxy-less installs keep working on older pygit2.
# (A configured proxy still needs >=1.18 — see requirements.txt.)
if _HTTP_PROXY is not None:
_pygit2.clone_repository(_to_https_url(url), dest, proxy=_HTTP_PROXY)
else:
_pygit2.clone_repository(_to_https_url(url), dest)
repo = _Pygit2Repo(dest)
repo.submodule_update()
return repo
+1 -1
View File
@@ -39,7 +39,7 @@ dependencies = [
]
[project.optional-dependencies]
dev = ["pre-commit", "pytest", "ruff", "pytest-cov", "pygit2"]
dev = ["pre-commit", "pytest", "ruff", "pytest-cov", "pygit2>=1.18"]
[project.urls]
Repository = "https://github.com/ltdrdata/ComfyUI-Manager"
+1 -1
View File
@@ -1,5 +1,5 @@
GitPython
pygit2
pygit2>=1.18 # clone_repository(proxy=...) requires 1.18.0+
PyGithub
# matrix-nio
transformers
+104
View File
@@ -824,6 +824,110 @@ finally:
self.assertEqual(p2['sha'], self.first_sha)
self.assertTrue(p2['detached'])
# === list_remotes fetch routing (pygit2) ===
def test_list_remotes_fetch_routing_pygit2(self):
"""Each list_remotes() proxy must route fetch/pull through
_fetch_remote/_pull_remote bound to its OWN remote (pins the
``lambda r=r:`` late-binding fix and the pull_fn parity)."""
snippet = """
import tempfile, shutil
dest = tempfile.mkdtemp()
try:
repo = clone_repo(REPO_PATH, os.path.join(dest, 'cloned'))
repo._repo.remotes.create('alt', REPO_PATH)
calls = []
repo._fetch_remote = lambda remote, refspecs=None: calls.append(remote.name)
proxies = repo.list_remotes()
for p in proxies:
p.fetch()
pulled = []
for p in proxies:
p.pull()
pulled.append(p.name)
repo.close()
print(json.dumps({"names": [p.name for p in proxies],
"fetch_calls": calls[:len(proxies)],
"pulled": pulled}))
finally:
shutil.rmtree(dest, ignore_errors=True)
"""
p2 = _run_snippet(snippet, self.repo_path, use_pygit2=True)
self.assertEqual(len(p2['names']), 2)
self.assertEqual(p2['fetch_calls'], p2['names'])
self.assertEqual(p2['pulled'], p2['names'])
class TestToHttpsUrl(unittest.TestCase):
"""Unit tests for the pure SSH→HTTPS URL rewrite helper."""
@classmethod
def setUpClass(cls):
sys.path.insert(0, COMPAT_DIR)
from git_compat import _to_https_url
cls.convert = staticmethod(_to_https_url)
@classmethod
def tearDownClass(cls):
try:
sys.path.remove(COMPAT_DIR)
except ValueError:
pass
def test_scp_form(self):
self.assertEqual(
self.convert('git@github.com:owner/repo.git'),
'https://github.com/owner/repo.git')
def test_scp_form_without_suffix(self):
self.assertEqual(
self.convert('git@github.com:owner/repo'),
'https://github.com/owner/repo')
def test_ssh_scheme(self):
self.assertEqual(
self.convert('ssh://git@github.com/owner/repo.git'),
'https://github.com/owner/repo.git')
def test_ssh_scheme_with_port(self):
self.assertEqual(
self.convert('ssh://git@git.example.com:2222/owner/repo.git'),
'https://git.example.com/owner/repo.git')
def test_https_unchanged(self):
url = 'https://github.com/owner/repo.git'
self.assertEqual(self.convert(url), url)
def test_non_git_user_unchanged(self):
url = 'ssh://alice@git.example.com/owner/repo.git'
self.assertEqual(self.convert(url), url)
def test_local_path_unchanged(self):
url = '/home/user/repos/local-repo'
self.assertEqual(self.convert(url), url)
def test_empty_and_none(self):
self.assertEqual(self.convert(''), '')
self.assertIsNone(self.convert(None))
def test_scp_absolute_path_no_double_slash(self):
self.assertEqual(
self.convert('git@git.example.com:/srv/git/repo.git'),
'https://git.example.com/srv/git/repo.git')
def test_slash_form_without_colon_unchanged(self):
# Not a valid scp-form URL (git treats it as a local path).
url = 'git@github.com/owner/repo'
self.assertEqual(self.convert(url), url)
def test_ipv6_unchanged(self):
url = 'ssh://git@[2001:db8::1]/owner/repo.git'
self.assertEqual(self.convert(url), url)
def test_port_without_path_unchanged(self):
url = 'ssh://git@git.example.com:2222'
self.assertEqual(self.convert(url), url)
if __name__ == '__main__':
unittest.main()