Extract context size

[sathiraumesh]

##summary

• Populates Config.ContextSize during ExtractConfig for GGUF and safetensors models, so context size is baked into the OCI artifact at package time.

Fixes #398

[gemini-code-assist]

Code Review

This pull request adds support for extracting the maximum context size from model metadata in both GGUF and Safetensors formats. For GGUF, the context length is read directly from the architecture metadata, while for Safetensors, a new utility reads and parses the model's config.json file using a prioritized list of common keys. A security concern was identified regarding the Safetensors implementation, where reading the configuration file without a size limit could potentially lead to resource exhaustion; it is recommended to enforce a size limit during the file read process.

[gemini-code-assist]

The readContextSizeFromConfigJSON function reads the entire config.json file into memory without a size limit. While model configuration files are typically small, an attacker could provide a path to a very large file named config.json to cause an Out-Of-Memory (OOM) condition. For consistency and security, a size limit should be enforced, similar to the 100MB limit used in parseSafetensorsHeader at line 210.

Suggested change

	func readContextSizeFromConfigJSON(dir string) *int32 {
	data, err := os.ReadFile(filepath.Join(dir, "config.json"))
	if err != nil {
	return nil
	}
	func readContextSizeFromConfigJSON(dir string) *int32 {
	f, err := os.Open(filepath.Join(dir, "config.json"))
	if err != nil {
	return nil
	}
	defer f.Close()
	data, err := io.ReadAll(io.LimitReader(f, 10*1024*1024))
	if err != nil {
	return nil
	}

References

1. Repository Style Guide: Critical issues include security flaws and must be fixed before merge. ^(link)
2. Security rule: Input must be validated at system boundaries to prevent resource exhaustion (DoS).

[sourcery-ai]

Hey - I've left some high level feedback:

• In SafetensorsFormat.ExtractConfig, using filepath.Dir(paths[0]) assumes the first path is always in the directory that contains config.json; consider deriving the config directory more explicitly (e.g., from a known model root or by searching alongside all provided paths) to avoid brittle behavior when paths are reordered or sharded across directories.
• readContextSizeFromConfigJSON silently returns nil for malformed JSON or non-numeric values; if distinguishing between a missing file and a bad/unsupported config would be useful, consider returning an error or at least a boolean flag so callers can surface or log misconfigurations rather than treating everything as "no context size found".

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `SafetensorsFormat.ExtractConfig`, using `filepath.Dir(paths[0])` assumes the first path is always in the directory that contains `config.json`; consider deriving the config directory more explicitly (e.g., from a known model root or by searching alongside all provided paths) to avoid brittle behavior when paths are reordered or sharded across directories.
- `readContextSizeFromConfigJSON` silently returns `nil` for malformed JSON or non-numeric values; if distinguishing between a missing file and a bad/unsupported config would be useful, consider returning an error or at least a boolean flag so callers can surface or log misconfigurations rather than treating everything as "no context size found".

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}