### 使用 Python 编写爬虫抓取微博数据并进行分析和可视化的代码示例 以下是基于 Python 的完整流程，包括数据采集、分析以及可视化部分。 #### 1. 数据采集 使用 `requests` 和 `BeautifulSoup` 抓取微博数据。为了模拟真实浏览器访问，还可以结合 `Selenium` 来处理动态加载的内容[^3]。 ```python import requests from bs4 import BeautifulSoup import pandas as pd def fetch_weibo_data(url, headers): response = requests.get(url, headers=headers) soup = BeautifulSoup(response.text, 'html.parser') posts = [] for item in soup.find_all('div', class_='weibo-item'): text = item.find('span', class_='text').get_text(strip=True) likes = int(item.find('span', class_='like-count').get_text()) comments = int(item.find('span', class_='comment-count').get_text()) posts.append({ 'text': text, 'likes': likes, 'comments': comments }) return pd.DataFrame(posts) url = "https://example.weibo.com/search" headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" } data = fetch_weibo_data(url, headers) print(data.head()) ``` #### 2. 数据存储 将采集到的数据存入 MySQL 数据库以便后续查询和分析[^1]。 ```sql CREATE TABLE weibo_posts ( id INT AUTO_INCREMENT PRIMARY KEY, text TEXT NOT NULL, likes INT DEFAULT 0, comments INT DEFAULT 0 ); ``` ```python import pymysql connection = pymysql.connect( host='localhost', user='root', password='password', database='weibo_db' ) try: with connection.cursor() as cursor: for _, row in data.iterrows(): sql = """ INSERT INTO weibo_posts (text, likes, comments) VALUES (%s, %s, %s) """ cursor.execute(sql, (row['text'], row['likes'], row['comments'])) connection.commit() finally: connection.close() ``` #### 3. 数据分析 利用 `SnowNLP` 进行情感分析，并计算每条微博的情感得分[^1]。 ```python from snownlp import SnowNLP def analyze_sentiment(text): s = SnowNLP(text) return s.sentiments data['sentiment'] = data['text'].apply(analyze_sentiment) positive_count = len(data[data['sentiment'] >= 0.6]) negative_count = len(data[data['sentiment'] < 0.4]) print(f"Positive Posts: {positive_count}") print(f"Negative Posts: {negative_count}") ``` #### 4. 数据可视化 使用 ECharts 或 Matplotlib 将分析结果可视化[^1]。 ##### 方法一：Matplotlib 绘制柱状图 ```python import matplotlib.pyplot as plt plt.bar(['Positive', 'Negative'], [positive_count, negative_count], color=['green', 'red']) plt.title('Sentiment Analysis of Weibo Data') plt.ylabel('Number of Posts') plt.show() ``` ##### 方法二：ECharts 实现交互式图表 创建 HTML 文件嵌入 ECharts 图表。 ```javascript // index.html var chartDom = document.getElementById('main'); var myChart = echarts.init(chartDom); var option; option = { title: { text: 'Weibo Sentiment Analysis', subtext: 'Data from Web Scraping', left: 'center' }, tooltip: { trigger: 'item' }, series: [ { name: 'Access From', type: 'pie', radius: ['50%', '70%'], avoidLabelOverlap: false, label: { show: false, position: 'center' }, emphasis: { label: { show: true, fontSize: '40', fontWeight: 'bold' } }, labelLine: { show: false }, data: [ {value: positive_count, name: 'Positive'}, {value: negative_count, name: 'Negative'} ] } ] }; myChart.setOption(option); ``` --- ###