Vito
ef97d01dbd
* ci: run frontend coverage in four shards Coverage shards reached their 18-minute step limit three times on 2026-10-31 with every test passing. Split the run into four shards with the limits unchanged; the merge job now expects four raw coverage artifacts, and the vendor checks move to shard 4. Closes #563 * test(tags): pin the 390px window check to the first page's last row Reaching the last row loads the next page (D183), and last() then followed row 200, far outside the window, so the WebKit check failed in 4-5 of 20 runs on 2 CPUs. Pinned to row 100, it passed 90 of 90 across WebKit, Chromium and Firefox. Closes #566
269 lines
14 KiB
YAML
269 lines
14 KiB
YAML
name: Viewer.js vendor and compatibility
|
|
|
|
on:
|
|
merge_group:
|
|
pull_request:
|
|
branches: [main]
|
|
push:
|
|
branches: [main]
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
# Superseded PR pushes cancel their older run. Every main push gets its own
|
|
# group (run_id), so no main commit's run is ever cancelled or dropped as pending.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.run_id }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
|
|
# Timeouts are the measured job time plus at least 50%, and never under 20 minutes for a browser job,
|
|
# because runner speed varies. Before sharding (main 2cecfcf, one worker): frontend coverage 32m10s,
|
|
# page workflows firefox 18m59s, webkit 19m26s. First sharded run (PR #274 run 39712909047, 3 workers,
|
|
# no npm cache, where npm ci stalled for 5m in six of eight jobs): coverage shards 11m19s/10m47s/10m49s
|
|
# (tests 5m20s/4m42s/4m46s), merge 5m52s (report and check 4s), firefox shards 7m53s/9m27s (tests
|
|
# 4m41s/3m29s), webkit shards 12m25s+ (cancelled at the old 12-minute limit after its 267 tests passed in
|
|
# 6m29s) and 4m53s (tests 3m08s). With the npm cache hit (run 39725176320 at cbc8a61, npm ci 0-2s):
|
|
# coverage shards 7m34s/6m04s/5m22s (tests 6m38s/4m53s/4m04s), merge 54s, firefox 7m57s/5m21s (tests
|
|
# 7m08s/4m34s), webkit 8m16s/5m03s (tests 7m26s/3m58s); a cache miss adds up to 5m to npm ci, which
|
|
# the 20-minute limits absorb.
|
|
# Hangs (#425): a job killed at its limit keeps no log and skips its upload steps, so each browser
|
|
# test step has its own timeout-minutes (18) below the job's 20. At the step limit, SIGTERM makes
|
|
# tools/vendor/running-tests-reporter.mjs print the tests still running and hand over to Playwright's
|
|
# interrupt, which writes the blob report; the always() uploads then run.
|
|
# Measured on main (runs 40725289094, 40719375442, 40700439399): setup before the test step <= 52s,
|
|
# uploads <= 35s; test steps up to 14m42s (webkit shard 1/2) and 13m05s for coverage (shard 1/3:
|
|
# 12m53s, node units and the viewer suite before 11.8m of workflows), so coverage is the tight one
|
|
# in its 18 minutes. This holds only when the npm cache hits: a cold cache adds up to 5m to npm ci,
|
|
# and then the job limit can still end the job before the step limit does.
|
|
# #473: firefox/webkit shard 1/2 reached its 18-minute step limit (queue run 40866252975: webkit 622 tests
|
|
# passed in 17m56s, 48.5 worker-minutes vs 28.0 in shard 2/2; recent shard 1/2 jobs 13.7m-18.9m, shard 2/2
|
|
# 7.3m-12.2m), because --shard splits by test count and the slow concept specs sort into shard 1. The page
|
|
# workflows now run in three shards per browser with the limits unchanged; the per-shard times are
|
|
# re-measured after three green main runs.
|
|
# #563: coverage shards reached their 18-minute step limit three times on 2026-10-31 with no failing
|
|
# test (PR #560 shard 2/3: 814 passed in 17.9m; queue run 41206495451 shard 1/3: 827 passed; PR #564
|
|
# shard 2/3 twice), against recent green shards of 11m-17m. Coverage now runs in four shards with the
|
|
# limits unchanged; the per-shard times are re-measured after three green main runs.
|
|
jobs:
|
|
frontend-coverage:
|
|
name: frontend coverage (shard ${{ matrix.shard }}/4)
|
|
runs-on: ubuntu-latest
|
|
# Layout comparisons depend on the fonts each browser falls back to, so browser jobs run in the
|
|
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
|
|
# tests/vendor/browser-image.test.mjs checks it). Browsers and fonts come from the image.
|
|
container:
|
|
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
|
options: --user 1001
|
|
timeout-minutes: 20
|
|
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
|
|
# server, leave one CPU for the servers and the runner.
|
|
env:
|
|
PM_TEST_WORKERS: 3
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3, 4]
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
|
|
# builds, whose entry points name host paths that do not exist in the container. The image has no
|
|
# ensurepip and the job has no root, so a checksum-pinned pip wheel bootstraps a user-space venv.
|
|
- name: Python environment
|
|
env:
|
|
PIP_WHEEL: https://files.pythonhosted.org/packages/f3/6e/1736e5b4ae2b778ef2f81c47d797de9f891d4d8acb047a24ca37a60294dd/pip-26.2.1-py3-none-any.whl
|
|
PIP_SHA256: 71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e
|
|
run: |
|
|
venv="$RUNNER_TEMP/venv" wheel="$RUNNER_TEMP/pip-26.2.1-py3-none-any.whl"
|
|
python3 -m venv --without-pip "$venv"
|
|
curl -fsSL --retry 3 -o "$wheel" "$PIP_WHEEL"
|
|
echo "$PIP_SHA256 $wheel" | sha256sum -c -
|
|
"$venv/bin/python" "$wheel/pip" install --no-index --no-cache-dir "$wheel"
|
|
"$venv/bin/python" -m pip install --no-cache-dir -r requirements-dev.txt
|
|
echo "$venv/bin" >> "$GITHUB_PATH"
|
|
- run: python --version && node --version && node --version | grep -q '^v24\.'
|
|
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
|
|
# which npm also uses. node_modules itself is still built fresh by npm ci.
|
|
- name: Cache npm downloads
|
|
uses: actions/cache@v6
|
|
with:
|
|
path: ~/.npm
|
|
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
|
|
- run: npm ci --prefix tools/vendor --ignore-scripts
|
|
# Shard 1 also runs the node units and the viewer suite, so the vendor pins and the tooling tests
|
|
# run on the lightest shard instead.
|
|
- if: matrix.shard == 4
|
|
run: npm run --prefix tools/vendor check
|
|
- if: matrix.shard == 4
|
|
run: npm run --prefix tools/vendor test
|
|
- name: Page workflows with coverage, shard ${{ matrix.shard }}/4
|
|
timeout-minutes: 18
|
|
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/4
|
|
- name: Upload raw coverage
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: coverage-raw-${{ matrix.shard }}
|
|
path: coverage/js/raw/
|
|
if-no-files-found: error
|
|
retention-days: 3
|
|
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
|
|
- name: Upload Chromium and touch traces
|
|
if: always()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: frontend-traces-shard-${{ matrix.shard }}
|
|
path: tools/vendor/test-results/
|
|
retention-days: 14
|
|
|
|
frontend-coverage-merge:
|
|
name: frontend coverage (merge)
|
|
needs: frontend-coverage
|
|
runs-on: ubuntu-latest
|
|
# Same image as the shards: the raw files name sources by the container's workspace path.
|
|
container:
|
|
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
|
options: --user 1001
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- run: node --version && node --version | grep -q '^v24\.'
|
|
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
|
|
# which npm also uses. node_modules itself is still built fresh by npm ci.
|
|
- name: Cache npm downloads
|
|
uses: actions/cache@v6
|
|
with:
|
|
path: ~/.npm
|
|
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
|
|
- run: npm ci --prefix tools/vendor --ignore-scripts
|
|
# Each artifact lands in its own coverage-raw-N directory, so the next step can count them.
|
|
- uses: actions/download-artifact@v8
|
|
with:
|
|
pattern: coverage-raw-*
|
|
path: ${{ runner.temp }}/coverage-raw
|
|
# coverage:merge only checks that raw files exist; merging three of four shards would report
|
|
# partial coverage, so a missing or empty shard artifact fails here.
|
|
- name: Verify every shard's raw coverage arrived
|
|
shell: bash
|
|
env:
|
|
STAGED_RAW: ${{ runner.temp }}/coverage-raw
|
|
EXPECTED_SHARDS: 4
|
|
run: |
|
|
found=0
|
|
for shard in $(seq 1 "$EXPECTED_SHARDS"); do
|
|
directory="$STAGED_RAW/coverage-raw-$shard"
|
|
count=0
|
|
if [ -d "$directory" ]; then count=$(find "$directory" -maxdepth 1 -type f -name '*.json' | wc -l); fi
|
|
echo "coverage-raw-$shard: $count raw files"
|
|
if [ "$count" -gt 0 ]; then found=$((found + 1)); fi
|
|
done
|
|
echo "shard artifacts with raw coverage: $found of $EXPECTED_SHARDS"
|
|
if [ "$found" -ne "$EXPECTED_SHARDS" ]; then
|
|
echo "::error::Expected raw coverage from $EXPECTED_SHARDS shards, got $found of $EXPECTED_SHARDS"
|
|
exit 1
|
|
fi
|
|
staged=$(find "$STAGED_RAW" -mindepth 2 -maxdepth 2 -type f -name '*.json' | wc -l)
|
|
mkdir -p coverage/js/raw
|
|
# cp -n never overwrites; a clash shows up as fewer merged files than staged ones.
|
|
find "$STAGED_RAW" -mindepth 2 -maxdepth 2 -type f -name '*.json' -exec cp -n -t coverage/js/raw {} + || true
|
|
merged=$(find coverage/js/raw -maxdepth 1 -type f -name '*.json' | wc -l)
|
|
if [ "$merged" -ne "$staged" ]; then
|
|
echo "::error::Shards wrote clashing raw file names: $staged staged, $merged merged"
|
|
exit 1
|
|
fi
|
|
echo "merged $merged raw files"
|
|
- name: Frontend coverage report and baseline check
|
|
run: npm run --prefix tools/vendor coverage:merge
|
|
- name: Upload frontend coverage report
|
|
if: always()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: frontend-test-reports
|
|
path: |
|
|
coverage/js/
|
|
!coverage/js/raw/**
|
|
retention-days: 14
|
|
|
|
browser-workflows:
|
|
name: page workflows (${{ matrix.browser }}, shard ${{ matrix.shard }}/3)
|
|
# Merge queue, main and manual runs only (#420); pull requests run the Chromium/touch coverage shards.
|
|
if: github.event_name != 'pull_request'
|
|
runs-on: ubuntu-latest
|
|
# Layout comparisons depend on the fonts each browser falls back to, so browser jobs run in the
|
|
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
|
|
# tests/vendor/browser-image.test.mjs checks it). Browsers and fonts come from the image.
|
|
container:
|
|
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
|
options: --user 1001
|
|
timeout-minutes: 20
|
|
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
|
|
# server, leave one CPU for the servers and the runner.
|
|
env:
|
|
PM_TEST_WORKERS: 3
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
browser: [firefox, webkit]
|
|
shard: [1, 2, 3]
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
|
|
# builds, whose entry points name host paths that do not exist in the container. The image has no
|
|
# ensurepip and the job has no root, so a checksum-pinned pip wheel bootstraps a user-space venv.
|
|
- name: Python environment
|
|
env:
|
|
PIP_WHEEL: https://files.pythonhosted.org/packages/f3/6e/1736e5b4ae2b778ef2f81c47d797de9f891d4d8acb047a24ca37a60294dd/pip-26.2.1-py3-none-any.whl
|
|
PIP_SHA256: 71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e
|
|
run: |
|
|
venv="$RUNNER_TEMP/venv" wheel="$RUNNER_TEMP/pip-26.2.1-py3-none-any.whl"
|
|
python3 -m venv --without-pip "$venv"
|
|
curl -fsSL --retry 3 -o "$wheel" "$PIP_WHEEL"
|
|
echo "$PIP_SHA256 $wheel" | sha256sum -c -
|
|
"$venv/bin/python" "$wheel/pip" install --no-index --no-cache-dir "$wheel"
|
|
"$venv/bin/python" -m pip install --no-cache-dir -r requirements-dev.txt
|
|
echo "$venv/bin" >> "$GITHUB_PATH"
|
|
- run: python --version && node --version && node --version | grep -q '^v24\.'
|
|
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
|
|
# which npm also uses. node_modules itself is still built fresh by npm ci.
|
|
- name: Cache npm downloads
|
|
uses: actions/cache@v6
|
|
with:
|
|
path: ~/.npm
|
|
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
|
|
- run: npm ci --prefix tools/vendor --ignore-scripts
|
|
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/3
|
|
timeout-minutes: 18
|
|
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/3
|
|
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
|
|
- name: Upload browser traces
|
|
if: always()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: browser-traces-${{ matrix.browser }}-shard-${{ matrix.shard }}
|
|
path: tools/vendor/test-results/
|
|
retention-days: 14
|
|
|
|
# Branch protection requires the exact context "viewerjs". This job keeps that
|
|
# name and fails unless every coverage shard and the coverage merge succeeded and
|
|
# the browser shards succeeded, or, on a pull request only, were skipped (#420).
|
|
# Any failure, cancellation or other skip fails it.
|
|
viewerjs:
|
|
needs: [frontend-coverage, frontend-coverage-merge, browser-workflows]
|
|
if: always()
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
steps:
|
|
- name: Require every frontend job to succeed
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
COVERAGE_RESULT: ${{ needs.frontend-coverage.result }}
|
|
MERGE_RESULT: ${{ needs.frontend-coverage-merge.result }}
|
|
WORKFLOWS_RESULT: ${{ needs.browser-workflows.result }}
|
|
run: |
|
|
queue_only=success
|
|
if [ "$EVENT_NAME" = pull_request ]; then queue_only=skipped; fi
|
|
echo "event: $EVENT_NAME (browser-workflows must be $queue_only)"
|
|
echo "frontend-coverage: $COVERAGE_RESULT"
|
|
echo "frontend-coverage-merge: $MERGE_RESULT"
|
|
echo "browser-workflows: $WORKFLOWS_RESULT"
|
|
test "$COVERAGE_RESULT" = success && test "$MERGE_RESULT" = success && test "$WORKFLOWS_RESULT" = "$queue_only"
|