Skip to content

Commit d502ab3

Browse files
leliaflowstate
andauthored
Add legal artifact presets, FOSSA-compatible outputs (#199)
* add legal presets, file-based compliance artifacts Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * add unit test coverage for new config defaults and outputs Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * document legal preset flag and default artifacts Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * update lockfile Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * bump version to pass checks Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * update lockfile to match new version Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * handle missing SBOM and package data in legal artifacts Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * update tests to check for missing data issues Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * add opt-in FOSSA-compatible legal artifact mode Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * cover FOSSA-compatible legal outputs/sample scenarios Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * update README docs to include FOSSA compatibility mode Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * bump version for release Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * update uv lockfile version Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * fix compatibility unit tests Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * bump version for release Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * fix version check sync logic to check pypi properly Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> * test: add FOSSA reference fixtures and parity harness * fix: emit fossa project.id as <locator>\$<revision> Real FOSSA artifacts use \$ as the revision separator in project.id, not \-. Update _build_project_metadata and add two tests that pin the correct separator and fallback behaviour. * test: lock fossa analyze top-level keyset invariants * fix: source vulnerability version ranges from Socket field names * fix: emit consistent defaults for FOSSA fields with no Socket source Adds customRiskScore: None to vulnerability entries (FOSSA samples include this field, sometimes null). Documents all gap fields and their defaults in the module docstring. Locks the new key in EXPECTED_VULNERABILITY_KEYS. * feat: reshape fossa-sbom.json to 5-key attribution top-level Replaces the 2-key {project, dependencies} shape with the real FOSSA attribution shape: copyrightsByLicense, deepDependencies, directDependencies, licenses, project. The SBOM project field is now the 2-key {name, revision} subset rather than the 6-key analyze project shape. _partition_dependencies is a stub returning ([], []) until Tasks 7-9 fill in per-dependency entries. * feat: build FOSSA-shape Dependency entries with attribution text Add _build_dependency_entry and _build_dependency_licenses to produce the 14-key per-dependency dict that matches real FOSSA attribution output. License entries prefer licenseAttrib (full attribText + spdxExpr), fall back to declared license string, or emit [] when unlicensed. Also removes the stale test_fossa_attribution_payload_shape_is_stable test, which asserted the pre-Task-6 two-key shape and was already failing. * feat: partition SBOM dependencies into direct and deep * feat: compute SBOM dependencyPaths from topLevelAncestors Replaces the stub that always returned [package.name] with real logic: direct deps emit just their name; transitive deps emit one "<ancestor> > <package>" chain per top-level ancestor, falling back to name-only when ancestors are absent or not in the lookup. * fix: write fossa-sbom.json with indent=2 for consistency * test: assert structural parity against real FOSSA fixtures * test: update legacy FOSSA-shape assertions for wrapper parity Pin project.id to dollar separator, replace 2-key SBOM with 5-key shape, and update per-dependency assertions to the 14-key _build_dependency_entry contract. * docs: document partialFix/completeFix collapse and otherLicenses gap * style: apply ruff --fix to touched files * fix: always include unchanged alerts in FOSSA output FOSSA's /api/v2/issues endpoint returns a point-in-time snapshot of all issues at the scan revision, not only diff-new ones. The previous implementation only included unchanged alerts when --strict-blocking was set, causing FOSSA-mode output to under-represent project-wide findings compared to the typical FOSSA pipeline. * test: sanitize customer references in FOSSA reference fixtures Replace customer org ID and project name with generic placeholders (1234/example-validation-project) across all four fixtures and the README. Structural shape, key sets, value types, and per-field cardinality are unchanged. Parity tests assert keysets only, so the substitution is transparent to test behavior. * docs: correct README description of FOSSA SBOM shape The SBOM artifact now matches FOSSA's `report --json attribution` shape with five top-level keys, not the previously documented `project` / `dependencies` two-key payload. --------- Signed-off-by: lelia <2418071+lelia@users.noreply.github.com> Co-authored-by: Eric Hibbs <eric@socket.dev>
1 parent 0445421 commit d502ab3

20 files changed

Lines changed: 3140 additions & 115 deletions

.github/workflows/version-check.yml

Lines changed: 36 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,16 +35,43 @@ jobs:
3535
MAIN_VERSION=$(git show origin/main:socketsecurity/__init__.py | grep -o "__version__.*" | awk '{print $3}' | tr -d "'")
3636
echo "MAIN_VERSION=$MAIN_VERSION" >> $GITHUB_ENV
3737
38-
# Compare versions using Python
39-
python3 -c "
38+
export PR_VERSION
39+
export MAIN_VERSION
40+
41+
# Compare against both main and latest published PyPI release.
42+
python3 <<'PY'
43+
import json
44+
import os
45+
import urllib.request
4046
from packaging import version
41-
pr_ver = version.parse('${PR_VERSION}')
42-
main_ver = version.parse('${MAIN_VERSION}')
43-
if pr_ver <= main_ver:
44-
print(f'❌ Version must be incremented! Main: {main_ver}, PR: {pr_ver}')
45-
exit(1)
46-
print(f'✅ Version properly incremented from {main_ver} to {pr_ver}')
47-
"
47+
48+
pr_ver = version.parse(os.environ["PR_VERSION"])
49+
main_ver = version.parse(os.environ["MAIN_VERSION"])
50+
51+
with urllib.request.urlopen("https://pypi.org/pypi/socketsecurity/json") as response:
52+
pypi_data = json.load(response)
53+
54+
published_versions = []
55+
for raw in pypi_data.get("releases", {}).keys():
56+
parsed = version.parse(raw)
57+
if not parsed.is_prerelease and not parsed.is_devrelease:
58+
published_versions.append(parsed)
59+
60+
pypi_ver = max(published_versions) if published_versions else version.parse("0.0.0")
61+
required_floor = max(main_ver, pypi_ver)
62+
63+
if pr_ver <= required_floor:
64+
print(
65+
f"❌ Version must be greater than main and PyPI! "
66+
f"Main: {main_ver}, PyPI: {pypi_ver}, PR: {pr_ver}"
67+
)
68+
raise SystemExit(1)
69+
70+
print(
71+
f"✅ Version properly incremented. "
72+
f"Main: {main_ver}, PyPI: {pypi_ver}, PR: {pr_ver}"
73+
)
74+
PY
4875
4976
- name: Require uv.lock update when pyproject changes
5077
run: |

.hooks/sync_version.py

Lines changed: 56 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@
1212

1313
VERSION_PATTERN = re.compile(r"__version__\s*=\s*['\"]([^'\"]+)['\"]")
1414
PYPROJECT_PATTERN = re.compile(r'^version\s*=\s*".*"$', re.MULTILINE)
15-
PYPI_API = "https://test.pypi.org/pypi/socketsecurity/json"
15+
STABLE_VERSION_PATTERN = re.compile(r"^\d+\.\d+\.\d+$")
16+
PYPI_PROD_API = "https://pypi.org/pypi/socketsecurity/json"
17+
PYPI_TEST_API = "https://test.pypi.org/pypi/socketsecurity/json"
1618

1719
def read_version_from_init(path: pathlib.Path) -> str:
1820
content = path.read_text()
@@ -39,24 +41,59 @@ def bump_patch_version(version: str) -> str:
3941
parts[-1] = str(int(parts[-1]) + 1)
4042
return ".".join(parts)
4143

42-
def fetch_existing_versions() -> set:
44+
def parse_stable_version(version: str):
45+
if not STABLE_VERSION_PATTERN.fullmatch(version):
46+
return None
47+
return tuple(int(part) for part in version.split("."))
48+
49+
50+
def format_stable_version(version_parts) -> str:
51+
return ".".join(str(part) for part in version_parts)
52+
53+
54+
def fetch_existing_versions(api_url: str) -> set:
4355
try:
44-
with urllib.request.urlopen(PYPI_API) as response:
56+
with urllib.request.urlopen(api_url) as response:
4557
data = json.load(response)
4658
return set(data.get("releases", {}).keys())
4759
except Exception as e:
48-
print(f"⚠️ Warning: Failed to fetch existing versions from Test PyPI: {e}")
60+
print(f"⚠️ Warning: Failed to fetch versions from {api_url}: {e}")
4961
return set()
5062

63+
64+
def fetch_latest_stable_pypi_version():
65+
versions = fetch_existing_versions(PYPI_PROD_API)
66+
stable_versions = []
67+
for ver in versions:
68+
parsed = parse_stable_version(ver)
69+
if parsed is not None:
70+
stable_versions.append(parsed)
71+
if not stable_versions:
72+
return None
73+
return max(stable_versions)
74+
5175
def find_next_available_dev_version(base_version: str) -> str:
52-
existing_versions = fetch_existing_versions()
76+
existing_versions = fetch_existing_versions(PYPI_TEST_API)
5377
for i in range(1, 100):
5478
candidate = f"{base_version}.dev{i}"
5579
if candidate not in existing_versions:
5680
return candidate
5781
print("❌ Could not find available .devN slot after 100 attempts.")
5882
sys.exit(1)
5983

84+
85+
def find_next_stable_patch_version(current_version: str) -> str:
86+
current_stable = current_version.split(".dev")[0] if ".dev" in current_version else current_version
87+
current_parts = parse_stable_version(current_stable)
88+
if current_parts is None:
89+
print(f"❌ Unsupported version format for stable bump: {current_version}")
90+
sys.exit(1)
91+
92+
latest_pypi_parts = fetch_latest_stable_pypi_version()
93+
base_parts = max([current_parts, latest_pypi_parts] if latest_pypi_parts else [current_parts])
94+
next_parts = (base_parts[0], base_parts[1], base_parts[2] + 1)
95+
return format_stable_version(next_parts)
96+
6097
def inject_version(version: str):
6198
print(f"🔁 Updating version to: {version}")
6299

@@ -105,13 +142,25 @@ def main():
105142
print(f"⚠️ Version was unchanged — auto-bumped. Please git add{lock_hint} + commit again.")
106143
sys.exit(0)
107144
else:
108-
new_version = bump_patch_version(current_version)
145+
new_version = find_next_stable_patch_version(current_version)
109146
inject_version(new_version)
110147
uv_lock_changed = run_uv_lock()
111148
lock_hint = " and uv.lock" if uv_lock_changed else ""
112-
print(f"⚠️ Version was unchanged — auto-bumped. Please git add{lock_hint} + commit again.")
149+
print(f"⚠️ Version was unchanged — auto-bumped to {new_version}. Please git add{lock_hint} + commit again.")
113150
sys.exit(1)
114151
else:
152+
if not dev_mode:
153+
current_parts = parse_stable_version(current_version)
154+
latest_pypi_parts = fetch_latest_stable_pypi_version()
155+
if current_parts is not None and latest_pypi_parts is not None and current_parts <= latest_pypi_parts:
156+
next_parts = (latest_pypi_parts[0], latest_pypi_parts[1], latest_pypi_parts[2] + 1)
157+
new_version = format_stable_version(next_parts)
158+
inject_version(new_version)
159+
uv_lock_changed = run_uv_lock()
160+
lock_hint = " and uv.lock" if uv_lock_changed else ""
161+
print(f"⚠️ Version {current_version} is already published on PyPI — auto-bumped to {new_version}. Please git add{lock_hint} + commit again.")
162+
sys.exit(1)
163+
115164
uv_lock_changed = run_uv_lock()
116165
if uv_lock_changed:
117166
print("⚠️ Version already bumped, but uv.lock was out of date and has been updated. Please git add uv.lock + commit again.")

README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ socketcli \
8484
| Use case | Recommended mode | Key flags |
8585
|:--|:--|:--|
8686
| Basic policy enforcement in CI | Diff-based policy check | `--strict-blocking` |
87+
| Legal/compliance artifact generation | Legal preset | `--legal` |
8788
| Reachable-focused SARIF for reporting | Full-scope grouped SARIF | `--reach --sarif-scope full --sarif-grouping alert --sarif-reachability reachable --sarif-file <path>` |
8889
| Detailed reachability export for investigations | Full-scope instance SARIF | `--reach --sarif-scope full --sarif-grouping instance --sarif-reachability all --sarif-file <path>` |
8990
| Net-new PR findings only | Diff-scope SARIF | `--reach --sarif-scope diff --sarif-reachability reachable --sarif-file <path>` |
@@ -134,6 +135,35 @@ Run:
134135
socketcli --config .socketcli.toml --target-path .
135136
```
136137

138+
Legal/compliance preset example:
139+
140+
```bash
141+
socketcli --legal --target-path .
142+
```
143+
144+
This preset enables license generation and writes default artifacts unless you override them:
145+
- `socket-report.json`
146+
- `socket-summary.txt`
147+
- `socket-report-link.txt`
148+
- `socket-sbom.json`
149+
- `socket-license.json`
150+
151+
FOSSA-compatibility shaped legal artifacts:
152+
153+
```bash
154+
socketcli --legal-format fossa --target-path .
155+
```
156+
157+
This switches the JSON report and legal artifact payloads to FOSSA-style compatibility shapes:
158+
- the analyze artifact becomes a `project` / `vulnerability` / `licensing` / `quality` report
159+
- the SBOM artifact becomes a FOSSA-attribution-style payload with `copyrightsByLicense`, `deepDependencies`, `directDependencies`, `licenses`, and `project` keys
160+
161+
When `--legal-format fossa` is used without explicit output paths, the defaults are closer to the FOSSA pipeline contract:
162+
- `fossa-analyze.json`
163+
- `fossa-test.txt`
164+
- `fossa-link.txt`
165+
- `fossa-sbom.json`
166+
137167
Reference sample configs:
138168

139169
TOML:

pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ build-backend = "hatchling.build"
66

77
[project]
88
name = "socketsecurity"
9-
version = "2.2.90"
9+
version = "2.2.91"
1010
requires-python = ">= 3.11"
1111
license = {"file" = "LICENSE"}
1212
dependencies = [
@@ -16,7 +16,7 @@ dependencies = [
1616
'GitPython',
1717
'packaging',
1818
'python-dotenv',
19-
"socketdev>=3.1.0,<4.0.0",
19+
"socketdev>=3.0.33,<4.0.0",
2020
"bs4>=0.0.2",
2121
"markdown>=3.10",
2222
]
@@ -57,7 +57,7 @@ socketcli = "socketsecurity.socketcli:cli"
5757
socketclidev = "socketsecurity.socketcli:cli"
5858

5959
[project.urls]
60-
Homepage = "https://github.com/SocketDev/socket-python-cli"
60+
Homepage = "https://socket.dev"
6161

6262
[tool.coverage.run]
6363
source = ["socketsecurity"]

socketsecurity/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
__author__ = 'socket.dev'
2-
__version__ = '2.2.90'
2+
__version__ = '2.2.91'
33
USER_AGENT = f'SocketPythonCLI/{__version__}'

socketsecurity/config.py

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,13 +79,16 @@ class CliConfig:
7979
enable_debug: bool = False
8080
allow_unverified: bool = False
8181
enable_json: bool = False
82+
json_file: Optional[str] = None
8283
enable_sarif: bool = False
8384
sarif_file: Optional[str] = None
8485
sarif_scope: str = "diff"
8586
sarif_grouping: str = "instance"
8687
sarif_reachability: str = "all"
8788
enable_gitlab_security: bool = False
8889
gitlab_security_file: Optional[str] = None
90+
summary_file: Optional[str] = None
91+
report_link_file: Optional[str] = None
8992
disable_overview: bool = False
9093
disable_security_issue: bool = False
9194
files: str = None
@@ -137,6 +140,8 @@ class CliConfig:
137140
reach_continue_on_no_source_files: bool = False
138141
max_purl_batch_size: int = 5000
139142
enable_commit_status: bool = False
143+
legal: bool = False
144+
legal_format: str = "socket"
140145
config_file: Optional[str] = None
141146

142147
@classmethod
@@ -194,13 +199,16 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig':
194199
'enable_diff': args.enable_diff,
195200
'allow_unverified': args.allow_unverified,
196201
'enable_json': args.enable_json,
202+
'json_file': args.json_file,
197203
'enable_sarif': args.enable_sarif,
198204
'sarif_file': args.sarif_file,
199205
'sarif_scope': args.sarif_scope,
200206
'sarif_grouping': args.sarif_grouping,
201207
'sarif_reachability': args.sarif_reachability,
202208
'enable_gitlab_security': args.enable_gitlab_security,
203209
'gitlab_security_file': args.gitlab_security_file,
210+
'summary_file': args.summary_file,
211+
'report_link_file': args.report_link_file,
204212
'disable_overview': args.disable_overview,
205213
'disable_security_issue': args.disable_security_issue,
206214
'files': args.files,
@@ -246,9 +254,40 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig':
246254
'reach_continue_on_no_source_files': args.reach_continue_on_no_source_files,
247255
'max_purl_batch_size': args.max_purl_batch_size,
248256
'enable_commit_status': args.enable_commit_status,
257+
'legal': args.legal or args.legal_format == "fossa",
258+
'legal_format': args.legal_format,
249259
'config_file': args.config_file,
250260
'version': __version__
251261
}
262+
263+
if config_args['legal']:
264+
config_args['generate_license'] = True
265+
if not config_args['json_file']:
266+
config_args['json_file'] = "socket-report.json"
267+
if not config_args['summary_file']:
268+
config_args['summary_file'] = "socket-summary.txt"
269+
if not config_args['report_link_file']:
270+
config_args['report_link_file'] = "socket-report-link.txt"
271+
if not config_args['sbom_file']:
272+
config_args['sbom_file'] = "socket-sbom.json"
273+
if config_args['license_file_name'] == "license_output.json":
274+
config_args['license_file_name'] = "socket-license.json"
275+
276+
if config_args['legal_format'] == "fossa":
277+
if not args.json_file:
278+
config_args['json_file'] = "fossa-analyze.json"
279+
if not args.summary_file:
280+
config_args['summary_file'] = "fossa-test.txt"
281+
if not args.report_link_file:
282+
config_args['report_link_file'] = "fossa-link.txt"
283+
if not args.license_file_name:
284+
# argparse always provides a default, so this branch is defensive only
285+
config_args['license_file_name'] = "fossa-sbom.json"
286+
elif args.license_file_name == "license_output.json":
287+
config_args['license_file_name'] = "fossa-sbom.json"
288+
if not args.sbom_file:
289+
# FOSSA's "SBOM" artifact is the attribution payload; suppress the extra Socket-only SBOM file by default.
290+
config_args['sbom_file'] = None
252291
excluded_ecosystems = config_args["excluded_ecosystems"]
253292
if isinstance(excluded_ecosystems, list):
254293
config_args["excluded_ecosystems"] = excluded_ecosystems
@@ -570,6 +609,12 @@ def create_argument_parser() -> argparse.ArgumentParser:
570609
action="store_true",
571610
help="Output in JSON format"
572611
)
612+
output_group.add_argument(
613+
"--json-file",
614+
dest="json_file",
615+
metavar="<path>",
616+
help="Output file path for JSON report"
617+
)
573618
output_group.add_argument(
574619
"--enable-sarif",
575620
dest="enable_sarif",
@@ -617,6 +662,18 @@ def create_argument_parser() -> argparse.ArgumentParser:
617662
default="gl-dependency-scanning-report.json",
618663
help="Output file path for GitLab Security report (default: gl-dependency-scanning-report.json)"
619664
)
665+
output_group.add_argument(
666+
"--summary-file",
667+
dest="summary_file",
668+
metavar="<path>",
669+
help="Output file path for a plain-text summary report"
670+
)
671+
output_group.add_argument(
672+
"--report-link-file",
673+
dest="report_link_file",
674+
metavar="<path>",
675+
help="Output file path for the Socket report link"
676+
)
620677
output_group.add_argument(
621678
"--disable-overview",
622679
dest="disable_overview",
@@ -750,6 +807,19 @@ def create_argument_parser() -> argparse.ArgumentParser:
750807
action="store_true",
751808
help="Disable SSL certificate verification for API requests"
752809
)
810+
advanced_group.add_argument(
811+
"--legal",
812+
dest="legal",
813+
action="store_true",
814+
help="Enable legal/compliance-friendly defaults and file outputs"
815+
)
816+
advanced_group.add_argument(
817+
"--legal-format",
818+
dest="legal_format",
819+
choices=["socket", "fossa"],
820+
default="socket",
821+
help="Select the legal artifact format. 'socket' keeps Socket-native outputs; 'fossa' emits compatibility-shaped JSON artifacts."
822+
)
753823
config_group.add_argument(
754824
"--include-module-folders",
755825
dest="include_module_folders",

0 commit comments

Comments
 (0)