Enhance advisory grouping mechanism

[TG1999]

We have started grouping advisories on basis of content like summary, packages, patches, severities and weaknesses. Which is a good start. But we need to have more heuristics like relating advisories with aliases and checking how much close two advisory summaries are.

[Dhirenderchoudhary]

@TG1999 I would like to take this and work on it.
And solve it asap.

Proposed changes :

1. Add new entries to Models.py(ISSUE_TYPE_CHOICES).
2. Migration.
3. Enhance Pipeline.
4. Add Tests for the changes.

Can I take this issue and work on it ?

[TG1999]

@Dhirenderchoudhary What are your plans to relate advisories ? What heuristics you are planning ?

[Dhirenderchoudhary]

@TG1999 I'm planning to add two heuristics:

1. Alias-based relating : Group advisories from different datasources when they share the same alias. If multiple sources report on the same alias, we flag them as POTENTIALLY_RELATED_BY_ALIASES.
2. Summary similarity : Within those alias groups, compare summaries using difflib.SequenceMatcher with a 0.8 threshold. If two advisories from different sources have near identical summaries, we flag them as SIMILAR_SUMMARIES.

For implementation, I'll follow the existing
detect_conflicting_advisories pattern add the new issue types to ISSUE_TYPE_CHOICES, write the pipeline steps and helper functions, create a migration, and add tests.

And if you want to add or suggest something on this you can add on I'll follow that structure.

[shivamshrma09]

Hello sir, I have also worked on this issue and submitted a PR implementing the same two heuristics:

• POTENTIALLY_RELATED_BY_ALIASES — flags advisories from different datasources sharing the same alias
• SIMILAR_SUMMARIES — uses difflib.SequenceMatcher with 0.8 threshold to flag near-identical summaries

Both steps follow the existing detect_conflicting_advisories pattern using LoopProgress, bulk_create_with_m2m, and advisories_checksum.

PR: #2228 [https://github.com//pull/2228]

Happy to incorporate any feedback or suggestions.

[shivamshrma09]

@Dhirenderchoudhary What are your plans to relate advisories ? What heuristics you are planning ?

Hi @TG1999, I have also worked on this. Here is my approach:

1. Alias-based relating — if advisories from different datasources share
   the same alias, flag them as POTENTIALLY_RELATED_BY_ALIASES. Same
   datasource pairs are skipped.

2. Summary similarity — compare cross-datasource advisory pairs using
   difflib.SequenceMatcher with a 0.8 threshold. If ratio >= 0.8, flag as
   SIMILAR_SUMMARIES. Empty summaries are excluded.

Both steps follow the existing detect_conflicting_advisories pattern.
Threshold is defined as a top-level constant for easy adjustment.

PR: #2228

[TG1999]

After discussing in status call we came to a conclusion that advisories should be grouped by both aliases and impacted packages of an advisory, using just one heuristic does not give confidence on the data quality. We still need to define the presentation of the data.

[shivamshrma09]

Sir, that means I have to now group on the basis of both Aliases and Impacted Packages together, and remove the previous Summary similarity logic. Is that correct? i can't understant your feedback