Skip to content

Fix/paraphrase transformers compat - #2234

Open
aspire488 wants to merge 2 commits into
NVIDIA:mainfrom
aspire488:fix/paraphrase-transformers-compat
Open

aspire488 wants to merge 2 commits into
NVIDIA:mainfrom
aspire488:fix/paraphrase-transformers-compat

Conversation

@aspire488

Copy link
Copy Markdown

Tell us what this change does. If you're fixing a bug, please mention
the github issue number.

Please ensure you are submitting from a unique branch in your repository to main upstream.

Verification

List the steps needed to make sure this thing works

  • Supporting configuration such as generator configuration file
{
    "huggingface": {
        "torch_type": "float32"
    }
}
  • garak -t <target_type> -n <model_name>
  • Run the tests and ensure they pass python -m pytest tests/
  • ...
  • Verify the thing does what it should
  • Verify the thing does not do what it should not
  • Document the thing and how it works (Example)

If you are opening a PR for a new plugin that targets a specific piece of hardware or requires a complex or hard-to-find testing environment, we recommend that you send us as much detail as possible.

Specific Hardware Examples:

  • GPU related
    • Specific support required cuda / mps ( Please not cuda via ROCm if related )
    • Minium GPU Memory

Complex Software Examples:

  • Expensive proprietary software
  • Software with an extensive installation process
  • Software without an English language UI
  • Summary

Fixes #1499.

The buffs.paraphrase.Fast buff currently relies on the external transformers-community/group-beam-search custom generation implementation and trust_remote_code. Newer Transformers versions provide the required group beam search functionality natively, making the custom generation dependency unnecessary and causing compatibility issues.

Changes

  • Removed the custom_generate dependency from Fast.
  • Removed the trust_remote_code requirement and argument.
  • Kept the existing native group beam search parameters:
    • num_beams
    • num_beam_groups
    • num_return_sequences
    • diversity_penalty
  • Added a regression test verifying that the deprecated custom-generation path is no longer configured.

Duplicate-work check

I checked the open garak PRs around paraphrase/custom generation and did not find an open PR addressing this specific compatibility fix.

Verification

Focused test:

python -m pytest tests/buffs/test_buffs.py -k "fast or native_group_beam_search"

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: update paraphrase buff to be compatible with recent transformers versions

1 participant