### NLP 主流模型应用开发实战 #### 中文关键词抽取 中文关键词抽取旨在从文档中自动提取最具代表性的词语或短语。常用方法包括TF-IDF、TextRank等传统算法，以及基于深度学习的方法。 对于基于BERT的关键词抽取，可以利用预训练好的BERT模型来获取文本的上下文敏感词嵌入表示，再通过特定层输出作为特征输入到下游任务中完成关键词预测[^1]： ```python from transformers import BertTokenizer, BertModel import torch tokenizer = BertTokenizer.from_pretrained('bert-base-chinese') model = BertModel.from_pretrained('bert-base-chinese') text = "这是一篇关于自然语言处理的文章" tokens = tokenizer(text, return_tensors='pt', max_length=512, truncation=True) with torch.no_grad(): outputs = model(**tokens) last_hidden_states = outputs.last_hidden_state ``` #### 命名实体识别 (NER) 及关系识别 命名实体识别是指识别并分类专有名词的任务；而关系识别则是进一步确定这些实体之间的关联。目前最先进的做法是采用双向编码器表征（BiLSTM-CRF），结合字符级和词级别信息，并引入外部知识库辅助建模。 使用BERT进行NER时，可以直接加载已经微调过的版本如`dbmdz/bert-large-cased-finetuned-conll03-english` 或者针对中文环境下的相应变体: ```python from transformers import pipeline nlp = pipeline("ner", model="dslim/bert-base-NER") result = nlp("Hugging Face Inc. is a company based in New York City.") print(result) ``` #### 机器翻译 神经网络架构如Transformer及其改进版广泛应用于现代机器翻译系统之中。这类模型能够捕捉源语言句子结构特性的同时生成目标语言表达形式。具体实现可借助开源工具包Fairseq或是HuggingFace提供的Transformers库快速搭建端到端解决方案。 以下是使用 MarianMT 进行英译中的例子: ```python from transformers import MarianMTModel, MarianTokenizer model_name = 'Helsinki-NLP/opus-mt-en-zh' tokenizer = MarianTokenizer.from_pretrained(model_name) model = MarianMTModel.from_pretrained(model_name) batch = tokenizer(['Hello world!'], return_tensors='pt') translated = model.generate(**batch) print([tokenizer.decode(t, skip_special_tokens=True) for t in translated]) ``` #### 文本分类 文本分类涉及将给定文本分配至预先定义的一组类别之一的过程。近年来，随着大规模预训练语言模型的发展，诸如BERT之类的框架被证明特别适用于此类任务。只需在基础之上附加简单的全连接层即可适应不同场景需求。 下面是一个简单的二元情感分类案例展示如何运用 BERT 实现这一功能: ```python from sklearn.model_selection import train_test_split from datasets import load_dataset from transformers import AutoTokenizer, Trainer, TrainingArguments, AutoModelForSequenceClassification dataset = load_dataset('glue', 'mrpc') train_texts, test_texts, train_labels, test_labels = train_test_split( dataset['train']['sentence1'] + '\n' + dataset['train']['sentence2'], dataset['train']['label'], test_size=.2, ) tokenizer = AutoTokenizer.from_pretrained('bert-base-chinese') tokenized_train = tokenizer(list(train_texts), padding=True, truncation=True, max_length=128) tokenized_test = tokenizer(list(test_texts), padding=True, truncation=True, max_length=128) training_args = TrainingArguments(output_dir='./results', num_train_epochs=3, per_device_train_batch_size=16, per_device_eval_batch_size=64, warmup_steps=500, weight_decay=0.01, logging_dir='./logs', ) model = AutoModelForSequenceClassification.from_pretrained('bert-base-chinese', num_labels=2) trainer = Trainer( model=model, args=training_args, compute_metrics=lambda p: {'accuracy': (p.predictions.argmax(-1) == p.label_ids).mean()}, train_dataset=tokenized_train, eval_dataset=tokenized_test, ) trainer.train() ``` #### LSTM 情感分析 长短时记忆网络(LSTM)是一种特殊的循环神经网络(RNN)，擅长处理序列化数据，在早期的情感分析研究中有广泛应用。尽管现在更多情况下会被更先进的 Transformer 结构所取代，但在某些资源受限环境下仍然不失为有效选择。 这里提供一段基本的 PyTorch 版本代码片段用于构建一个简易的情感分析器[^2]: ```python import torch.nn as nn class SentimentRNN(nn.Module): def __init__(self, vocab_size, output_size, embedding_dim, hidden_dim, n_layers, drop_prob=0.5): super(SentimentRNN, self).__init__() self.output_size = output_size self.n_layers = n_layers self.hidden_dim = hidden_dim self.embedding = nn.Embedding(vocab_size, embedding_dim) self.lstm = nn.LSTM(embedding_dim, hidden_dim, n_layers, dropout=drop_prob, batch_first=True) self.dropout = nn.Dropout(drop_prob) self.fc = nn.Linear(hidden_dim, output_size) self.sigmoid = nn.Sigmoid() def forward(self, x, hidden): batch_size = x.size(0) embeds = self.embedding(x) lstm_out, hidden = self.lstm(embeds, hidden) lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim) out = self.dropout(lstm_out) out = self.fc(out) sig_out = self.sigmoid(out) sig_out = sig_out.view(batch_size, -1) sig_out = sig_out[:, -1] return sig_out, hidden def init_hidden(self, batch_size): weight = next(self.parameters()).data hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_(), weight.new(self.n_layers, batch_size, self.hidden_dim).zero_()) return hidden ```