Added custom multihead attention code.

This commit is contained in:
HowToSD
2025-04-07 19:32:55 -07:00
parent 25edacf15a
commit 080a23778a
8 changed files with 317 additions and 1 deletions
+1
View File
@@ -176,6 +176,7 @@
| [Ptn Masked Mean Pooling](ptn_masked_mean_pooling.md) | Constructs a masked mean pooling layer. |
| [Ptn Max Pool 2d](ptn_max_pool2d.md) | A maxpool layer. |
| [Ptn Multihead Attention](ptn_multihead_attention.md) | A Multihead attention model. |
| [Ptn Multihead Attention Custom](ptn_multihead_attention_custom.md) | A Multihead attention model. |
| [Ptn Pre Add Channel Axis](ptn_pre_add_channel_axis.md) | Adds a channel axis after the batch axis if the input is rank 3 (bs, h, w) |
| [Ptn Pre Flatten](ptn_pre_flatten.md) | Flattens the input tensor before processing the tensor in the specified model. |
| [Ptn RNN](ptn_rnn.md) | A recurrent neural network (RNN) model consisting of one or more of a recurrent layer. |
@@ -0,0 +1,32 @@
# Ptn Multihead Attention Custom
A Multihead attention model.
Args:
embed_dim (int): Hidden dimension (d_model) for each token of the Transformer.
num_heads (int): Number of attention heads.
dropout (float): Dropout rate for output.
bias (bool): If True, bias will be added to the linear projections of query, key, value, and the final output.
kdim (int): Dimension of k.
vdim (int): Dimension of v.
batch_first (bool): Input shape format: True for (B, Seq, Feature), False for (Seq, B, Feature).
## Input
| Name | Data type |
|---|---|
| embed_dim | Int |
| num_heads | Int |
| dropout | Float |
| bias | Boolean |
| kdim | Int |
| vdim | Int |
| batch_first | Boolean |
## Output
| Data type |
|---|
| Ptmodel |
<HR>
Category: PyTorch wrapper - Model
ComfyUI Pt Wrapper Node Reference. © 2025 Hide Inada (HowToSD.com). All rights reserved.