Make ecosystem all check more generic (#4629)

* Don't assume unique repo names in ecosystem checks

This fixes a bug where previously repositories with the same name would have been overwritten.

I tested with `scripts/check_ecosystem.py -v --checkouts target/checkouts_main .venv/bin/ruff target/release/ruff` and ruff 0.0.267 that changes are shown. I confirmed with `scripts/ecosystem_all_check.sh check --select RUF008` (next PR) that the checkouts are now complete.

* Make ecosystem all check more generic

This allows passing arguments to the ecosystem all check script, e.g. you can now do `scripts/ecosystem_all_check.sh check --select RUF008`.

Tested with
```
$ cat target/ecosystem_all_results/*.stdout.txt | head
src/fi_parliament_tools/parsing/data_structures.py:33:17: RUF008 Do not use mutable default values for dataclass attributes
src/fi_parliament_tools/parsing/data_structures.py:76:17: RUF008 Do not use mutable default values for dataclass attributes
src/fi_parliament_tools/parsing/data_structures.py:178:17: RUF008 Do not use mutable default values for dataclass attributes
Found 3 errors.
braid_triggers/tasks.py:46:17: RUF008 Do not use mutable default values for dataclass attributes
Found 1 error.
src/boards/RaspberryPi3.py:15:22: RUF008 Do not use mutable default values for dataclass attributes
src/boards/board.py:21:26: RUF008 Do not use mutable default values for dataclass attributes
src/boards/board.py:22:32: RUF008 Do not use mutable default values for dataclass attributes
src/boards/board.py:23:37: RUF008 Do not use mutable default values for dataclass attributes
$ cat target/ecosystem_all_results/*.stdout.txt | wc -l
115
```

scripts/ecosystem_all_check.py

@@ -6,6 +6,7 @@ It's a less elaborate, more hacky version of check_ecosystem.py
 
 import json
 import subprocess
+import sys
 from pathlib import Path
 from subprocess import CalledProcessError
 from typing import NamedTuple, Optional
@@ -22,8 +23,9 @@ class Repository(NamedTuple):
 
 
 def main() -> None:
+    ruff_args = sys.argv[1:]
     checkouts = Path("checkouts")
-    out_dir = Path("ecosystem_fix_all_results")
+    out_dir = Path("ecosystem_all_results")
     github_search_json = Path("github_search.jsonl")
     # Somehow it doesn't like plain ruff
     ruff = Path.cwd().joinpath("ruff")
@@ -54,14 +56,6 @@ def main() -> None:
             continue
 
         try:
-            ruff_args = [
-                "check",
-                "--no-cache",
-                "--exit-zero",
-                "--select",
-                "ALL",
-                "--fix",
-            ]
             output = subprocess.run(
                 [ruff, *ruff_args, "."],
                 cwd=project_dir,

scripts/ecosystem_all_check.sh

@@ -1,9 +1,14 @@
 #!/bin/bash
-# This is @konstin's setup for checking an entire checkout of ~2.1k packages for
+# This is @konstin's setup for checking an entire checkout of ~3k packages for
 # panics, autofix errors and similar problems.
 #
 # We put this in a docker container because processing random scraped code from GitHub is
 # [kinda dangerous](https://moyix.blogspot.com/2022/09/someones-been-messing-with-my-subnormals.html)
+#
+# Usage:
+# ```
+# scripts/ecosystem_all_check.sh check --select RUF200
+# ```
 
 # https://stackoverflow.com/a/246128/3549270
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
@@ -11,16 +16,16 @@ SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 time docker run --rm -it \
   -w /app \
   -v "${SCRIPT_DIR}/../target/checkouts:/app/checkouts" \
-  -v "${SCRIPT_DIR}/../target/ecosystem_fix_all_results:/app/ecosystem_fix_all_results" \
+  -v "${SCRIPT_DIR}/../target/ecosystem_all_results:/app/ecosystem_all_results" \
-  -v "${SCRIPT_DIR}/../target/release/ruff:/app/ruff" \
+  -v "${SCRIPT_DIR}/../target/x86_64-unknown-linux-musl/release/ruff:/app/ruff" \
   -v "${SCRIPT_DIR}/../ecosystem_all.py:/app/ecosystem_all.py" \
   -v "${SCRIPT_DIR}/../github_search.jsonl:/app/github_search.jsonl" \
   -v "${SCRIPT_DIR}/../.venv-3.11:/app/.venv" \
-  -v "${SCRIPT_DIR}/ecosystem_fix_all_check_entrypoint.sh:/app/ecosystem_fix_all_check_entrypoint.sh" \
+  -v "${SCRIPT_DIR}/ecosystem_all_check_entrypoint.sh:/app/ecosystem_all_check_entrypoint.sh" \
-  -v "${SCRIPT_DIR}/ecosystem_fix_all_check.py:/app/ecosystem_fix_all_check.py" \
+  -v "${SCRIPT_DIR}/ecosystem_all_check.py:/app/ecosystem_all_check.py" \
-  python:3.11 ./ecosystem_fix_all_check_entrypoint.sh
+  python:3.11 ./ecosystem_all_check_entrypoint.sh "$@"
 
 # grep the autofix errors
-grep -R "the rule codes" "${SCRIPT_DIR}/../target/ecosystem_fix_all_results" | sort > "${SCRIPT_DIR}/../target/autofix-errors.txt"
+grep -R "the rule codes" "${SCRIPT_DIR}/../target/ecosystem_all_results" | sort > "${SCRIPT_DIR}/../target/autofix-errors.txt"
 # Make sure we didn't have an early exit
 echo "Done"

scripts/ecosystem_all_check_entrypoint.sh

@@ -1,9 +1,9 @@
 #!/bin/bash
-# Wrapper for ecosystem_fix_all_check.py
+# Wrapper for ecosystem_all_check.py
 
 if [ ! -d ".venv/bin" ]; then
   python -m venv .venv
   .venv/bin/pip install tqdm
 fi
 
-.venv/bin/python ecosystem_fix_all_check.py
+.venv/bin/python ecosystem_all_check.py "$@"