About BrowseSafe++
BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views.
Motivation
Browser agents act on behalf of users while reading untrusted web pages. A malicious instruction embedded in a page can redirect a legitimate task. Existing benchmarks establish the threat, but deployment needs consistent coverage of the web domains, languages and page views that agents actually encounter.
BrowseSafe++ extends BrowseSafe from 14,719 to 38,675 entries and starts from prompt injections observed in the wild, so detectors are measured on the attacks that real pages carry.
Construction
Real production injections
Collection covered 386 days, from 2025-09-01 to 2026-09-21. About 603,000 pages flagged by a deployed prompt-injection classifier, plus a daily sample of pages matched by regular-expression cues, gave 1,152,315 candidate pages, 1,092,957 after deduplication. Of these, 58,036 (5.3%) passed an injection gate of three models queried twice each (at least four of six votes). 32,187 pairs carry these production injections; 6,488 retain injections from BrowseSafe.
Attack/control pairs
Each attacked page has a control page that differs only by the injection. The original injection text is preserved. Each entry records the attacked page, the injection text, the control page, a legitimate task, the expected outcome, the attacker goal, whether the goal requires action tools, and the observation date.
Privacy
Sensitive portions of every page are rewritten: personal data such as names, contact details and identifiers is replaced with synthetic substitutes, while the injection itself is kept as observed.
Four views
Every page is provided as HTML, Markdown, accessibility tree and full-page screenshot. Detectors can be compared on the same pair across the representations agents consume.
Taxonomy
- domains
- 12 web domains, from news and blogs to checkout pages (table below).
- attack types
- 24 attack types.
- placements
- 18 placements of the injection within the page.
- goals
- 6 attacker goals.
- page tones
- 4 tones: formal, casual, marketing and legal.
- languages
- 80 languages: 75 appear as page languages and 49 as injection languages. 24.4% of pairs carry an injection that shares no language with its page.
- annotations
- Linguistic style and risk category.
Domain coverage
| Domain | Production injections | BrowseSafe injections | Train | Test | All pairs |
|---|---|---|---|---|---|
| News and blogs | 11,713 | 312 | 9,122 | 2,903 | 12,025 |
| Shared docs | 5,342 | 1,270 | 5,003 | 1,609 | 6,612 |
| Forums | 4,925 | 401 | 4,069 | 1,257 | 5,326 |
| Video sites | 2,106 | 1,701 | 3,004 | 803 | 3,807 |
| Web apps | 2,326 | 787 | 2,386 | 727 | 3,113 |
| Social profiles | 1,406 | 276 | 1,294 | 388 | 1,682 |
| Code repos | 1,389 | 177 | 1,177 | 389 | 1,566 |
| Marketplaces | 1,082 | 279 | 1,009 | 352 | 1,361 |
| 651 | 704 | 1,029 | 326 | 1,355 | |
| Social feeds | 624 | 323 | 728 | 219 | 947 |
| Search results | 554 | 144 | 542 | 156 | 698 |
| Checkout | 69 | 114 | 137 | 46 | 183 |
| Total | 32,187 | 6,488 | 29,500 | 9,175 | 38,675 |
Pairs per domain. One annotation per pair, taken from the attacked page; both pages of a pair share its split.
Splits
29,500 training pairs and 9,175 test pairs. Both pages of a pair always share a split, and no injection text appears in both splits. Classification results are reported on the 18,350 test pages.
The training split is available on Hugging Face. The test split is reserved for the leaderboard, and a submission endpoint for it is coming soon.
Comparison with other benchmarks
| Benchmark | Detection evaluation | Benign distractors | Contextual attacks | Multilingual cases | Web pages | Multi-modal page views | Outcome targets | Attack/control page pairs | Attacks observed in the wild |
|---|---|---|---|---|---|---|---|---|---|
| BIPIA | – | – | – | – | – | – | ✓ | – | – |
| InjecAgent | – | – | ✓ | – | – | – | ✓ | – | – |
| AgentDojo | – | – | ✓ | – | – | – | ✓ | – | – |
| ASB | ✓ | – | ✓ | – | – | – | ✓ | – | – |
| NotInject | ✓ | ✓ | – | ✓ | – | – | – | – | – |
| WASP | – | – | ✓ | – | ✓ | ✓ | ✓ | – | – |
| WAInjectBench | ✓ | – | ✓ | – | ✓ | ✓ | – | – | – |
| DTap-Bench | – | – | ✓ | – | ✓ | ✓ | ✓ | – | – |
| BrowseSafe | ✓ | ✓ | ✓ | ✓ | ✓ | – | – | – | – |
| BrowseSafe++ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Intended use and limitations
- Intended use: measuring and training prompt-injection detectors and evaluating the robustness of browser agents, for research and defence.
- Results compare models on page-level classification under one detection prompt; other prompts, thresholds or agent designs can rank models differently.
- No injection text appears in both splits, but injections from the same campaign or template can be similar, so test scores are not a full measure of generalisation to new attack styles.
- English dominates the page languages (29,999 pairs); many languages have few examples.
- Pages are rewritten for privacy, and HTML is a benchmark representation rather than a guaranteed copy of the original page.
- Screenshots establish what a page looks like, not whether an agent's task would succeed on a live site.
- The injections are real attacks. Do not use them against systems you are not authorised to test.
License
The dataset is released under CC BY 4.0 and the SDK under the MIT License.
Citation
@misc{chen2026browsesafepp,
title = {BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views},
author = {Chen, Yuetian and Zhang, Kaiyuan and Du, Yuntao and Li, Thomas and Tenenholtz, Mark and Polley, Kyle and Ma, Jerry and Yarats, Denis and Li, Ninghui},
year = {2026},
eprint = {2610.17457},
archivePrefix = {arXiv},
primaryClass = {cs.CR},
url = {https://arxiv.org/abs/2610.17457}
}