### 编写天气预报爬虫的最佳实践 使用Python编写天气预报爬虫时，需要结合网络爬虫的基本原理和实际需求进行设计。以下是实现这一目标的关键点和代码示例。 #### 1. 确定目标网站 选择一个提供天气预报数据的网站，并分析其页面结构。例如，可以使用`requests`库发送HTTP请求，使用`BeautifulSoup`解析HTML内容[^2]。 #### 2. 发送HTTP请求 通过`requests`库向目标网站发送请求，获取网页内容。以下是一个简单的代码示例： ```python import requests def fetch_weather_data(url): headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" } response = requests.get(url, headers=headers) if response.status_code == 200: return response.text else: raise Exception(f"Failed to fetch data: {response.status_code}") ``` #### 3. 解析网页内容 使用`BeautifulSoup`库解析HTML文档，提取所需的数据。例如，可以从页面中提取当前天气、温度和未来几天的天气预报信息[^1]。 ```python from bs4 import BeautifulSoup def parse_weather_data(html_content): soup = BeautifulSoup(html_content, 'html.parser') # 提取当前天气信息 current_weather = soup.find("div", class_="current-weather").text.strip() # 提取未来几天的天气预报 forecast_items = soup.find_all("div", class_="forecast-item") forecasts = [] for item in forecast_items: date = item.find("span", class_="date").text.strip() temperature = item.find("span", class_="temperature").text.strip() condition = item.find("span", class_="condition").text.strip() forecasts.append({ "date": date, "temperature": temperature, "condition": condition }) return {"current_weather": current_weather, "forecasts": forecasts} ``` #### 4. 数据存储与输出 将提取的数据保存到文件或数据库中，或者直接打印到控制台。以下是一个简单的输出示例： ```python def display_weather_data(weather_data): print(f"当前天气: {weather_data['current_weather']}") print("\n未来几天天气预报:") for forecast in weather_data["forecasts"]: print(f"日期: {forecast['date']}, 温度: {forecast['temperature']}, 天气状况: {forecast['condition']}") ``` #### 5. 完整代码示例 将上述步骤整合为一个完整的爬虫程序： ```python import requests from bs4 import BeautifulSoup def fetch_weather_data(url): headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" } response = requests.get(url, headers=headers) if response.status_code == 200: return response.text else: raise Exception(f"Failed to fetch data: {response.status_code}") def parse_weather_data(html_content): soup = BeautifulSoup(html_content, 'html.parser') current_weather = soup.find("div", class_="current-weather").text.strip() forecast_items = soup.find_all("div", class_="forecast-item") forecasts = [] for item in forecast_items: date = item.find("span", class_="date").text.strip() temperature = item.find("span", class_="temperature").text.strip() condition = item.find("span", class_="condition").text.strip() forecasts.append({ "date": date, "temperature": temperature, "condition": condition }) return {"current_weather": current_weather, "forecasts": forecasts} def display_weather_data(weather_data): print(f"当前天气: {weather_data['current_weather']}") print("\n未来几天天气预报:") for forecast in weather_data["forecasts"]: print(f"日期: {forecast['date']}, 温度: {forecast['temperature']}, 天气状况: {forecast['condition']}") if __name__ == "__main__": url = "https://example.com/weather" # 替换为目标网站的实际URL html_content = fetch_weather_data(url) weather_data = parse_weather_data(html_content) display_weather_data(weather_data) ``` ### 注意事项 - 在抓取数据时，需遵守目标网站的`robots.txt`文件规定，避免违反法律或道德规范[^3]。 - 使用代理或设置合理的请求间隔，以降低对目标服务器的压力。