### 使用Transformer模型进行多输入预测 Transformer是一种基于自注意力机制的强大架构，在处理多模态数据和时间序列预测任务时表现出显著的优势[^1]。为了实现多输入预测，可以设计一种结构来整合多个输入源的信息，并利用变量选择网络进一步优化性能[^2]。 以下是使用PyTorch实现一个多输入Transformer预测模型的示例： #### 数据准备 假设我们有两类输入：静态特征（static features）和动态特征（time-dependent features）。这些特征可以通过预处理步骤转换为适合Transformer模型的形式。 ```python import torch import torch.nn as nn from torch.utils.data import DataLoader, TensorDataset # 假设我们有两个输入张量：X_static 和 X_time_series batch_size = 32 sequence_length = 50 num_features = 10 hidden_dim = 64 # 随机生成模拟数据 X_static = torch.randn(batch_size, num_features) # 静态特征 X_time_series = torch.randn(batch_size, sequence_length, num_features) # 时间序列特征 y = torch.randn(batch_size, 1) # 输出目标 dataset = TensorDataset(X_static, X_time_series, y) dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True) ``` #### Transformer模型定义 下面是一个简单的多输入Transformer模型实现，其中包含了静态特征的选择模块以及时间序列的编码器部分。 ```python class MultiInputTransformer(nn.Module): def __init__(self, input_dim, hidden_dim, output_dim, n_heads=8, n_layers=2, dropout=0.1): super(MultiInputTransformer, self).__init__() # 变量选择网络 (Variable Selection Network) self.static_selection_network = nn.Sequential( nn.Linear(input_dim, hidden_dim), nn.ReLU(), nn.Dropout(dropout), nn.Linear(hidden_dim, hidden_dim) ) # Time-series Encoder using Transformer layers encoder_layer = nn.TransformerEncoderLayer(d_model=input_dim, nhead=n_heads, dim_feedforward=hidden_dim, dropout=dropout) self.time_series_encoder = nn.TransformerEncoder(encoder_layer, num_layers=n_layers) # Final Prediction Layer self.final_projection = nn.Linear(hidden_dim + input_dim * sequence_length, output_dim) def forward(self, static_input, time_series_input): # Static Feature Processing selected_static = self.static_selection_network(static_input) # Time Series Encoding encoded_time_series = self.time_series_encoder(time_series_input.permute(1, 0, 2)).permute(1, 0, 2).flatten(start_dim=1) # Concatenate Selected Static Features and Encoded Time-Series Data combined_representation = torch.cat([selected_static, encoded_time_series], dim=-1) # Output Prediction output = self.final_projection(combined_representation) return output model = MultiInputTransformer(num_features, hidden_dim, 1) criterion = nn.MSELoss() optimizer = torch.optim.Adam(model.parameters(), lr=0.001) ``` #### 训练过程 训练过程中，我们将静态特征和时间序列特征分别传递给模型，并计算损失函数以更新参数。 ```python for epoch in range(10): # 运行10个epoch作为例子 model.train() total_loss = 0 for batch_idx, (static_batch, time_series_batch, target_batch) in enumerate(dataloader): optimizer.zero_grad() predictions = model(static_batch, time_series_batch) loss = criterion(predictions, target_batch) loss.backward() optimizer.step() total_loss += loss.item() avg_loss = total_loss / len(dataloader) print(f'Epoch {epoch+1}, Loss: {avg_loss:.4f}') ``` 以上代码展示了如何构建并训练一个支持多输入的时间序列预测模型。该模型结合了静态特征的选择能力与时间序列数据的建模能力，从而提高了整体表现[^2]。 ---