Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries

Setup • Usage • View Results • Data Pipeline

---

Overview

This repository contains the implementation of JARGON, a jailbreak attack that leverages domain‑specific contexts to bypass LLM safety mechanisms. By embedding harmful requests within legitimate academic, technical domains, JARGON exposes significant vulnerabilities in current alignment techniques. The framework supports multi‑round conversational attacks, prompt optimization, belief state tracking, and automatic extraction of harmful knowledge.

Authors: Ki Sen Hung, Xi Yang, Chang Liu, Haoran Li, Kejiang Chen, Changxuan Fan, Tsun On Kwok, Weiming Zhang, Xiaomeng Li, Yangqiu Song

---

Setup

1. Clone the Repository

git clone https://github.com/JerryHung1103/JARGON.git
cd JARGON

2. Create and activate the environment:

conda env create -f environment.yml
conda activate JARGON

3. Configure API keys and endpoints

Create a .env file in the project root (or copy from .env.example) with the following variables:

HUGGINGFACE_HUB_TOKEN=your_hf_token_here

# Attacker model
ATTACKER_API_KEY=your_api_key_here
ATTACKER_BASE_URL=your_base_url_here

# Optimizer model
OPTIMIZER_API_KEY=your_api_key_here
OPTIMIZER_BASE_URL=your_base_url_here

# Judge model
JUDGE_API_KEY=your_api_key_here
JUDGE_BASE_URL=your_base_url_here

# Summary helper model
SUMMARY_API_KEY=your_api_key_here
SUMMARY_BASE_URL=your_base_url_here

# Refusal checker model
REFUSAL_CHECKER_API_KEY=your_api_key_here
REFUSAL_CHECKER_BASE_URL=your_base_url_here

# Target model
TARGET_API_KEY=your_api_key_here
TARGET_BASE_URL=your_base_url_here

# Knowledge extractor (for purification)
KNOWLEDGE_EXTRACTOR_API_KEY=your_api_key_here
KNOWLEDGE_EXTRACTOR_BASE_URL=your_base_url_here

# Safeguard (optional, only if enable_safeguard=True)
SAFEGUARD_API_KEY=your_api_key_here
SAFEGUARD_BASE_URL=your_base_url_here

💡Note: All BASE_URL values should point to the API endpoint of your chosen LLM provider (e.g., OpenAI‑compatible endpoint). You can use the same API key for multiple roles if your provider allows.

4. Configure config/config.yml

The main settings are controlled through config.yml. Below are some example and explanations:

jailbreak_setting:
  mode: 'Attack' 
  # Mode can be 'Attack' or 'Distillation'
  # - 'Attack': Uses LLM judge to determine successful jailbreaks
  # - 'Distillation': No judge is used. Attack ends when max_rounds_per_attack is reached.
  # NOTE: Distillation mode cannot be used with early_stop enabled.

  max_retries: 3
  # Number of full retry attempts per test case (outer loop)

  max_trials_per_retry: 2
  # Number of trials (conversations) within each retry

  max_rounds_per_attack: 4
  # Maximum number of attack rounds (turns) in one conversation/trial

  example_injection_threshold: 2
  # After how many retries will successful jailbreak examples be injected into the attacker prompt.
  # Set to a low number to enable few-shot learning earlier.

  optimization_variants: 8
  # Number of prompt variants generated during each optimization step.
  # Higher = more diversity, but higher API cost.

  diverse_attack_ratio: 0.5
  # Ratio of variants that use diversity/role-playing techniques (0.0 ~ 1.0)
  # 0.5 means 50% diverse variants, 50% standard variants.

  early_stop: False
  # If True, stops the attack immediately once a successful jailbreak is found.
  # Greatly saves time and API cost in 'Attack' mode.
  # Cannot be used together with 'Distillation' mode.

  record_trajectories: False
  # If True, saves full attack trajectories (both harmful and safe) into separate folders.
  # Useful for analysis or collecting data for fine-tuning refusal models.

  num_workers: 5
  # Number of parallel worker processes.
  # Adjust according to your CPU cores and API rate limits.

  enable_safeguard: False
  # Enables dynamic safeguard simulation on the target model (for testing robustness against defenses).

# ...
target_model:                                  
  model: "deepseek-chat"    
  base_url: "${TARGET_BASE_URL}"   
  api_key: "${TARGET_API_KEY}"                              
  generate_kwargs:
    temperature: 0.0

attacker:           
  model: "deepseek-chat"                           
  base_url: "${ATTACKER_BASE_URL}" 
  api_key: "${ATTACKER_API_KEY}"          
  generate_kwargs:
    temperature: 0.3
# ... other models follow the same pattern

Usage

1. Paper Content Extraction (Optional)

To prepare a new research paper (e.g., as context for the Jargon attack), use the paper_content_extractor.py script. It processes a PDF file, extracts the core academic content (removing PDF artifacts, headers, footers, etc.), and generates a structured JSON summary containing the title, abstract, methodology, and full cleaned text.

python paper_content_extractor.py --pdf_file_path path_to_paper.pdf

2. Execute Attacks

Run the attacks:

python main.py

When you run an attack, the following components execute in a coordinated loop across retries, trials, and conversation rounds:

Component	Role
Jailbreak Engine (jailbreak_engine.py)	Orchestrates the entire attack: retries, trials, and multi‑round conversations. Injects successful examples from cache after example_injection_threshold retries (few‑shot prompting). Calls the attacker to generate a plan and the next query, then updates belief states. Optionally triggers the Prompt Optimizer when attackFlag == 1.
Attacker	Generates the next user query (nextPrompt), a plan (suggestedTactics), and an attackFlag. Updates belief states based on conversation history. Produces criticism and prompt notes after failed trials.
Prompt Optimizer (agents/optimizer.py)	Creates multiple variants of the current query: Paraphrase variants (human‑like rewrites). Diverse scenario attack variants (domain camouflage, e.g., film scripts, game design). For each variant, sends it to the target model and evaluates the response. Selects the variant with the highest evaluator score (preferring non‑refusal responses).
Target Model	The LLM under attack. Responds to each user message (attacker query).
Evaluator (agents/evaluator.py)	Scores each target response on a 1–5 scale (1 = no progress, 5 = full jailbreak). Uses a strict rubric based on the attack goal. Performs multiple inferences (default 3) and takes the majority score/reason.
Knowledge Extractor (knowledge_extractor_generate)	Extracts concise harmful claims from verbose target responses. Wraps extracted knowledge in <harm> tags and accumulates it across rounds. Caches successful extractions in harmful_trajectory_cache_file.
Refusal Checker (agents/refusal_checker_utils.py)	Detects whether the target model explicitly refused the request. Uses majority voting over 3 inferences. Helps the optimizer avoid selecting refused responses.
Judge	Performs final harmful content detection on the accumulated <harm> text. Returns a binary success flag and a rationale. If early_stop is enabled, a successful judge stops the attack immediately.
Safeguard (agents/safeguard.py) – optional (enabled via enable_safeguard)	Analyzes the current and previous queries to detect jailbreak patterns. Dynamically generates a safety system prompt for the target model.
Results & Caching	Final attack outcome is written to out_dir of config.yml. Successful conversation transcripts are cached in successfulExampleCachePath for future few‑shot injection. Harmful and safe trajectories are saved to separate cache files (enabled via record_trajectories in config.yml): harmful_trajectory_cache_file, safe_trajectory_cache_file.

Attack Results

After running the attack, You can calculate key metrics using the built-in script:

python metrics.py --result_json_file_path your_results.json

Processing Trajectories for Training

When record_trajectories: True, the system saves harmful and safe trajectories separately. Use process_data.py to clean harmful trajectories and generate training data for fine-tuning.

python process_data.py \
  --harmful_trajectories_path  your_harmful_trajectories_.json \
  --safe_trajectories_path your_safe_trajectories.json

It removes and refine the harmful elements from the last assistant response in harmful trajectories Merges the cleaned responses into the safe data then saves the result to ./training_data/