CADAR

Abstract

Augmented Reality (AR) enriches human perception by overlaying virtual elements onto the physical world. However, this tight coupling between virtual and real content makes AR vulnerable to cognitive attacks: manipulations that distort users' semantic understanding of the environment. Existing detection methods largely focus on visual inconsistencies at the pixel or image level, offering limited semantic reasoning or interpretability.

To address these limitations, we introduce CADAR, a neuro-symbolic framework for cognitive attack detection in AR that integrates neural and symbolic reasoning. CADAR fuses multimodal vision–language representations from pre-trained models into a perception graph that captures objects, relations, and temporal contextual salience. Building on this structure, a particle-filter-based statistical reasoning module infers anomalies in semantic dynamics to reveal cognitive attacks. This combination provides both the adaptability of modern vision–language models and the interpretability of probabilistic symbolic reasoning. Preliminary experiments on the CADAR-50K dataset demonstrate consistent advantages over existing approaches, achieving 74.6% accuracy and outperforming the next best baseline by 16%.

Neuro-Symbolic Framework

A hybrid architecture converting VLM outputs into symbolic perception graphs for explicit, interpretable reasoning in AR cognitive attack detection.

Perception Graph Schema

A spatio-temporal graph integrating multimodal signals from image/text encoders and detection modules to capture semantic scene evolution across time.

Particle-Filter Reasoning

Particle filtering on symbolic perception graphs improves robustness to noisy VLM outputs and enables principled statistical anomaly detection.

CADAR-50K Dataset

The first publicly available AR cognitive-attack dataset: 375 videos and 50K+ frames covering all seven attack and scene categories.

System Overview

CADAR processes sequential AR frames through two tightly coupled modules: a symbolic perception graph and a particle filter-based detector.

AR Video Sequential frames I_t

VLM OWLv2 + SAM2

Perception Graph G_t(V_t, E_t)

Particle Filter Match → Detect → Estimate

Attack Detection Type + localization

Temporal — persistent object identity maintained via NodeSet/EdgeSet across the full video sequence

Figure 2. A system overview of CADAR. Sequential video frames are first transformed into spatial-temporal symbolic perception graphs. The attack-detection module then analyzes these graphs to identify, classify, and localize adversarial attacks such as visual modification and injection attacks.

Method

Cognitive Attack Types

CADAR detects four classes of cognitive attack. Click any type to see how it is detected.

Text Modification

Visual Modification

Obstruction

Injection

Click an attack type above to see its description and detection mechanism

Figure 1. Illustrative AR/MR cognitive attacks. Top row: original scenes; bottom row: manipulated scenes. (a) Text Modification Attack — alters on-scene text to reverse meaning. (b) Visual Modification Attack — distorts object color or placement. (c) Obstruction Attack — occludes critical cues. (d) Injection Attack — inserts fictitious elements.

3.2 — Perception Graph Model

Given an AR frame I_t and task context c_t, a pre-trained VLM lifts the frame into a spatio-temporal perception graph G_t = (V_t, E_t). Nodes represent objects with multimodal attributes (name, description, visual features, probability, importance, reasonability). Edges encode pairwise semantic relationships. Across time, NodeSets and EdgeSets stitch object instances into a persistent GraphSet Ĝ_t.

Figure 3. Symbolic perception graph generation: given a video frame and its contextual description, VLMs generate the corresponding perception graph at time step t. Node and edge attributes are extracted via structured prompts to capture multimodal scene semantics.

3.3 — Particle Filter Model

Each graph entity is treated as a particle. The filter runs three stages at every frame:

1 — Prediction & Matching

Cosine similarity over name, description, and visual embeddings determines whether each new node belongs to an existing NodeSet or spawns a new identity. An Active Level A_i^t ∈ [0,3] tracks recency.

2 — Attack Detection

PCA projection + Mahalanobis distance flags Type 1/2 anomalies in embedding space. Type 3/4 use lightweight temporal heuristics: missing high-importance nodes → obstruction; new low-reasonability nodes → injection.

3 — Estimation

Weighted resampling over clean historical particles refines each non-attack observation, preventing noisy or missed-detection VLM outputs from corrupting the ground-truth reference set.

Figure 4. The particle filter-based attack detection framework. Nodes are matched to persistent NodeSets, statistically tested for attacks via PCA + Mahalanobis distance, and then refined via weighted resampling. Attacked particles are isolated from the clean reference set.

CADAR-50K Dataset

The first publicly available dataset for AR cognitive-attack analysis — 375 videos, 50K+ frames, five scene categories, four attack types.

375 Videos

8–12 second AR-captured videos recorded via Meta Quest 3 or Samsung Galaxy S24, with virtual content introduced at a specific frame to induce an attack.

50K+ Frames

Frame-wise annotations supporting variable-length multi-label detection and localization of simultaneous attacks within a single video.

5 Scene Categories

School & Office · Traffic & Mobility · Industrial & Infrastructure · Medical & Lab · Public & Home

Figure 5. CADAR-50K dataset overview by scene category (left) and attack type (right). Non-attack and hybrid/multiple-attack videos are included for realistic evaluation.

Data Collection Methods

Virtual Scene Method

Fully synthetic environments displayed on a monitor or VR headset, with AR content overlaid and captured via video recording. Enables rapid, controllable creation of diverse attack scenarios.

Real-World Method

AR headsets record first-person views in physical environments, introducing natural noise, lighting variation, and depth cues for realistic and challenging evaluation conditions.

Dataset Sample Examples

Figure 6. Visualization frame examples of CADAR outputs across five scene categories in CADAR-50K (375 videos). Rows show the Reference Scene, Detection Scene, and CADAR Output. Output attack types: I = Injection, V = Visual, T = Text, O = Obstruction, N = Non-attack. Some examples contain hybrid or multiple attacks.

Results

Cross-Scene Cognitive Attack Classification

CADAR achieves 74.6% overall accuracy and 84.2% F1-score, outperforming the next best baseline (GPT-5-mini) by 16% and attaining the highest class-specific accuracy and F1 in 23 out of 25 attack–scene blocks.

CADAR (Ours)

Text

72.0%

Visual

73.4%

Obstruction

80.7%

Injection

75.6%

Non-attack

71.5%

Overall Avg Acc / F174.6% / 84.2%

GPT-5-mini

Text

55.2%

Visual

53.7%

Obstruction

58.5%

Injection

60.9%

Non-attack

65.1%

Overall Avg Acc / F158.7% / 73.4%

Gemini-2.5-flash

Text

60.4%

Visual

53.9%

Obstruction

50.4%

Injection

60.3%

Non-attack

44.3%

Overall Avg Acc / F153.9% / 69.3%

Per-Attack Accuracy Comparison

Grouped bar chart of class-specific accuracy across all methods. CADAR consistently leads across all attack types.

Hybrid & Multiple Attack Detection

CADAR achieves the highest recall for both hybrid attack combinations and maintains the smallest performance drop as simultaneous attacks increase.

Hybrid Attack Type	GPT-5-mini	Grok-4-fast	Gemini-2.5-flash	CADAR (Ours)
Text + Visual	35.0%	41.2%	29.9%	76.4%
Obstruction + Injection	32.7%	26.1%	45.8%	80.4%

Method	1 Attack	2 Attacks	3 Attacks	4 Attacks	Avg
GPT-5-mini	66.4%	40.2%	31.5%	29.6%	41.9%
Grok-4-fast	69.7%	55.0%	51.3%	36.5%	53.1%
Gemini-2.5-flash	61.7%	43.6%	33.0%	31.5%	42.5%
CADAR (Ours)	81.0%	78.3%	75.2%	67.2%	75.4%

Recall (%) under different numbers of simultaneous attacks. CADAR maintains strong performance as complexity grows, whereas baselines degrade substantially.

BibTeX

@InProceedings{chen_2026_CVPR,
    author    = {Chen, Rongqian and Andreyev, Allison and Xiu, Yanming and Chilukuri, Joshua and Sen, Shunav and Imani, Mahdi and Li, Bin and Gorlatova, Maria and Tan, Gang and Lan, Tian},
    title     = {Cognitive Attack Detection in Augmented Reality (CADAR): A Neuro-Symbolic Approach with Particle Filtering on Perception Graphs},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
    month     = {June},
    year      = {2026},
    pages     = {799-808}
}