Added custom multihead attention code.
This commit is contained in:
@@ -176,6 +176,7 @@
|
||||
| [Ptn Masked Mean Pooling](ptn_masked_mean_pooling.md) | Constructs a masked mean pooling layer. |
|
||||
| [Ptn Max Pool 2d](ptn_max_pool2d.md) | A maxpool layer. |
|
||||
| [Ptn Multihead Attention](ptn_multihead_attention.md) | A Multihead attention model. |
|
||||
| [Ptn Multihead Attention Custom](ptn_multihead_attention_custom.md) | A Multihead attention model. |
|
||||
| [Ptn Pre Add Channel Axis](ptn_pre_add_channel_axis.md) | Adds a channel axis after the batch axis if the input is rank 3 (bs, h, w) |
|
||||
| [Ptn Pre Flatten](ptn_pre_flatten.md) | Flattens the input tensor before processing the tensor in the specified model. |
|
||||
| [Ptn RNN](ptn_rnn.md) | A recurrent neural network (RNN) model consisting of one or more of a recurrent layer. |
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# Ptn Multihead Attention Custom
|
||||
A Multihead attention model.
|
||||
|
||||
Args:
|
||||
embed_dim (int): Hidden dimension (d_model) for each token of the Transformer.
|
||||
num_heads (int): Number of attention heads.
|
||||
dropout (float): Dropout rate for output.
|
||||
bias (bool): If True, bias will be added to the linear projections of query, key, value, and the final output.
|
||||
kdim (int): Dimension of k.
|
||||
vdim (int): Dimension of v.
|
||||
batch_first (bool): Input shape format: True for (B, Seq, Feature), False for (Seq, B, Feature).
|
||||
|
||||
## Input
|
||||
| Name | Data type |
|
||||
|---|---|
|
||||
| embed_dim | Int |
|
||||
| num_heads | Int |
|
||||
| dropout | Float |
|
||||
| bias | Boolean |
|
||||
| kdim | Int |
|
||||
| vdim | Int |
|
||||
| batch_first | Boolean |
|
||||
|
||||
## Output
|
||||
| Data type |
|
||||
|---|
|
||||
| Ptmodel |
|
||||
|
||||
<HR>
|
||||
Category: PyTorch wrapper - Model
|
||||
|
||||
ComfyUI Pt Wrapper Node Reference. © 2025 Hide Inada (HowToSD.com). All rights reserved.
|
||||
Reference in New Issue
Block a user