### Python 淘宝爬虫采集 10000 条数据并通过 Pandas 处理 为了实现这一目标，可以按照如下方式构建代码： #### 导入必要库 首先安装所需的第三方库 `requests` 和 `beautifulsoup4` 及其依赖项。可以通过 pip 工具完成这些操作。 ```bash pip install requests beautifulsoup4 lxml pandas openpyxl ``` #### 初始化设置 准备请求头信息模拟浏览器行为，并初始化存储商品信息列表。 ```python import requests from bs4 import BeautifulSoup as BS import time import random import pandas as pd headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " "(KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" } data_list = [] base_url = "https://s.taobao.com/search?q={}&s=" search_term = input("请输入要查询的商品名称:") max_items = 10000 current_item_count = 0 page_size = 48 # 默认每页显示数量 start_page = 0 ``` #### 抓取页面内容 循环遍历多页直到收集足够的项目数为止；每次加载新一页前稍作停顿防止被封禁IP地址。 ```python while current_item_count < max_items: url = base_url.format(search_term) + str(start_page * page_size) response = requests.get(url=url, headers=headers).content.decode() soup = BS(response,"lxml") items = soup.select(".items .item")[:min(page_size,max_items-current_item_count)] for item in items: try: title = item.select_one('.title').get_text(strip=True) price = float(item.select_one('.price')['data-value']) sales_volume = int(item.select_one('.deal-cnt').get_text().replace('人付款','')) data_dict = {"Title": title, "Price": price, "Sales Volume": sales_volume} data_list.append(data_dict) current_item_count += 1 if current_item_count >= max_items: break except Exception as e: print(f"Error occurred while processing an item: {e}") start_page += 1 sleep_time = round(random.uniform(1, 3), 2) time.sleep(sleep_time) ``` #### 数据处理与导出 将抓取下来的信息转换成 DataFrame 表格形式，并保存至 Excel 文件中以便后续分析使用。 ```python df = pd.DataFrame(data_list) output_file = f"{search_term}_taobao_data.xlsx" with pd.ExcelWriter(output_file, engine='openpyxl') as writer: df.to_excel(writer, index=False, sheet_name=search_term) print(f"\nData successfully saved to '{output_file}'") ``` 此段代码实现了从淘宝网站上自动检索指定关键词对应的产品详情，并最终汇总为电子表格文档供进一步研究之用[^1]。