### 编写Python爬虫程序抓取tianqihoubao.com的数据 #### 准备工作 为了成功从 `tianqihoubao.com` 抓取数据，首先要了解目标网站的robots协议，这有助于理解哪些页面可以被合法地访问和抓取[^1]。 #### 使用requests库发送HTTP请求 通过Python内置的`requests`库能够轻松发起GET或POST请求到指定URL地址。这里以获取某一天的具体天气情况为例： ```python import requests url = 'http://www.tianqihoubao.com/lishi/beijing/month/202311.html' headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" } response = requests.get(url, headers=headers) if response.status_code == 200: html_content = response.text else: print('Failed to fetch data') ``` 此段代码设置了自定义头部信息来模拟浏览器行为，从而提高成功率并遵循良好实践。 #### 解析HTML文档提取所需信息 一旦获得了完整的HTML源码，则需解析这些内容以便进一步处理。推荐使用BeautifulSoup库来进行DOM操作： ```python from bs4 import BeautifulSoup soup = BeautifulSoup(html_content, 'html.parser') table_rows = soup.find_all('tr')[1:] # 跳过表头行 for row in table_rows: columns = row.find_all('td') date_info = columns[0].get_text(strip=True) weather_condition = columns[1].get_text(strip=True) temperature_range = columns[2].get_text(strip=True).split('/') high_temperature = int(temperature_range[0][:-1]) low_temperature = int(temperature_range[-1][:-1]) wind_direction_and_speed = columns[3].get_text(strip=True) print(f'Date: {date_info}, Weather Condition: {weather_condition}, High Temp: {high_temperature}°C, Low Temp: {low_temperature}°C, Wind Info: {wind_direction_and_speed}') ``` 上述脚本遍历表格中的每一行记录，并从中抽取日期、天气状况、温度范围以及风向等字段的信息[^3]。 #### 数据存储 最后一步就是将收集来的资料妥善保存下来供后续分析之用。可以选择CSV文件作为简单的持久化方案之一： ```python import csv with open('weather_data.csv', mode='w', newline='', encoding='utf-8') as file: writer = csv.writer(file) header = ['Date', 'WeatherCondition', 'HighTemperature(C)', 'LowTemperature(C)', 'WindInfo'] writer.writerow(header) for row in table_rows: columns = row.find_all('td') date_info = columns[0].get_text(strip=True) weather_condition = columns[1].get_text(strip=True) temp_range = columns[2].get_text(strip=True).split('/') try: high_temp = int(temp_range[0][:-1]) low_temp = int(temp_range[-1][:-1]) wind_dir_spd = columns[3].get_text(strip=True) writer.writerow([date_info, weather_condition, high_temp, low_temp, wind_dir_spd]) except ValueError: continue ``` 这段代码创建了一个新的CSV文件并将之前整理好的每条日志逐行追加进去。