Fix matmul operation for concept attention computation

🔧 **Critical Fix**:

1. **Fixed Matmul Operation**:
   - Corrected similarity computation between concept embedding and attention output
   - Changed from [1, 1024, 1024] to [1, 1024] output
   - Proper dot product computation for each sequence position

2. **Corrected Tensor Operations**:
   - embedding: [1, 256], attention_flat: [1, 1024, 256]
   - Result: [1, 1024] - similarity scores for each position
   - Should resolve 'shape is invalid for input of size' error

3. **Improved Concept Map Generation**:
   - Proper spatial reshaping from 1024 to 32x32
   - Should finally generate valid concept maps

This should resolve the shape mismatch error and allow
concept map generation to work correctly.
This commit is contained in:
Junst
2025-09-24 13:58:45 +09:00
parent c7a0589ad4
commit 59be38dcab
+4 -5
View File
@@ -400,11 +400,10 @@ class ConceptAttention:
# Compute similarity between concept embedding and attention output
# Use einsum-like operation similar to original ConceptAttention
embedding_expanded = embedding.expand(attention_flat.shape[0], attention_flat.shape[1], -1)
# Compute concept-image attention map using einsum-like operation
concept_attn_output = torch.matmul(embedding_expanded, attention_flat.transpose(-2, -1))
concept_attn_output = concept_attn_output.squeeze(1) # [batch, seq_len]
# embedding: [1, 256], attention_flat: [1, 1024, 256]
# We want to compute similarity for each position in the sequence
concept_attn_output = torch.matmul(attention_flat, embedding.unsqueeze(-1)) # [1, 1024, 1]
concept_attn_output = concept_attn_output.squeeze(-1) # [1, 1024]
# Reshape to spatial dimensions
seq_len = concept_attn_output.shape[-1]