### Python 文本摘要和关键词提取库 #### Gensim 库 Gensim 是一个强大的自然语言处理工具包，特别适用于主题建模、文档相似度分析以及文本摘要。该库提供了多种算法来执行这些任务，其中包括 TF-IDF 和 LSI (潜在语义索引)[^1]。 ```python from gensim.summarization import summarize text = "Your long document goes here." print(summarize(text)) ``` #### NLTK 库 NLTK（Natural Language Toolkit）是一个广泛使用的Python平台，用于构建处理人类语言数据的程序。它包含了各种资源和接口，可以用来进行分词、标注、分类、解析等操作，并支持简单的基于频率统计的方法来进行关键词抽取[^2]。 ```python import nltk nltk.download('stopwords') from nltk.corpus import stopwords from nltk.tokenize import word_tokenize, sent_tokenize from collections import defaultdict from string import punctuation from heapq import nlargest def compute_frequencies(word_sent): freq = defaultdict(int) for s in word_sent: for word in s: if word not in stopwords.words('english') + list(punctuation): freq[word] += 1 max_freq = float(max(freq.values())) for w in freq.keys(): freq[w] /= max_freq return freq def summarize_text(text, n=3): sents = sent_tokenize(text) assert n <= len(sents) word_sent = [word_tokenize(s.lower()) for s in sents] freq = compute_frequencies(word_sent) ranking = defaultdict(int) for i,sent in enumerate(word_sent): for w in sent: if w in freq: ranking[i] += freq[w] sents_idx = nlargest(n, ranking, key=ranking.get) return [sents[j] for j in sorted(sents_idx)] document = """Your multi sentence paragraph.""" summary = summarize_text(document) for line in summary: print(line) ``` #### spaCy 库 spaCy 提供了一套工业级强度的功能集，可用于大规模NLP应用开发。通过其内置模型和服务，能够高效地完成命名实体识别、依存句法分析等功能；同时也可以利用 `TextRank` 算法实现较为精准的关键短语提取[^3]。 ```python import spacy from spacy.lang.en.stop_words import STOP_WORDS from sklearn.feature_extraction.text import CountVectorizer nlp = spacy.load("en_core_web_sm") def get_keywords_spacy(doc, num_keywords=5): doc = nlp(doc) tokens = [token.text for token in doc if not token.is_stop and not token.is_punct] cv = CountVectorizer(stop_words=list(STOP_WORDS), max_features=num_keywords) bow_matrix = cv.fit_transform([str(tokens)]) keywords = [] feature_array = np.array(cv.get_feature_names_out()) tfidf_sorting = np.argsort(bow_matrix.toarray()).flatten()[::-1] for idx in tfidf_sorting[:num_keywords]: keywords.append(feature_array[idx]) return keywords sample_doc = "Enter your sample text here" keywords = get_keywords_spacy(sample_doc) print(keywords) ```