### 使用Python编写猫眼电影数据爬虫 为了实现从猫眼电影网站获取所需的各种数据，可以采用`requests`库发送HTTP请求并利用`BeautifulSoup`或`lxml`解析HTML文档。考虑到反爬机制的存在，建议使用Selenium模拟浏览器行为或者通过API接口直接访问。 #### 准备工作 安装必要的依赖包： ```bash pip install requests beautifulsoup4 lxml selenium pandas openpyxl ``` #### 获取页面内容 构建函数用于发起GET请求，并处理异常情况： ```python import requests from bs4 import BeautifulSoup def get_page_content(url, headers=None): try: response = requests.get(url=url, headers=headers) if response.status_code == 200: return response.text else: print(f"Failed to fetch page content with status code {response.status_code}") return None except Exception as e: print(e) return None ``` 设置User-Agent伪装成正常浏览器访问： ```python headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 \ (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" } ``` #### 解析网页结构 根据目标站点的具体布局调整XPath表达式或其他定位方式提取信息。这里假设已经找到了合适的路径来匹配表格中的每一列数据。 ```python url = 'https://maoyan.com/films?showType=3' # 假设这是实时票房榜单URL html_text = get_page_content(url, headers) if html_text is not None: soup = BeautifulSoup(html_text, features="lxml") movie_list = [] movies = soup.find_all('div', class_='movie-item-info')[:10] # 取前十个作为例子 for index, item in enumerate(movies): title = item.select_one('.name').text.strip() detail_url = f"https://maoyan.com{item.a['href']}" details_html = get_page_content(detail_url, headers) if details_html is not None: details_soup = BeautifulSoup(details_html, features="lxml") release_date_tag = details_soup.select_one('.movie-brief-container > ul > li:nth-child(3)') release_days = int(release_date_tag.span.text.split()[0]) if release_date_tag and hasattr(release_date_tag,'span') else '' box_office_tags = details_soup.select('.stonefont') total_box_office = ''.join([char for char in box_office_tags[0].get_text(strip=True).replace('\n','')]) daily_box_office = ''.join([char for char in box_office_tags[1].get_text(strip=True).replace('\n','')]) occupancy_rate = float(box_office_tags[-1]['style'].split(':')[-1][:-1])/100 average_people_per_showing = float(details_soup.select_one('.averagePeoplePerShowing')['data-val']) box_office_ratio = float(details_soup.select_one('.boxOfficeRatio')['data-val']) screenings_count = int(details_soup.select_one('.screeningsCount')['data-val']) screening_ratio = float(details_soup.select_one('.screeningRatio')['data-val']) movie_info = [ str(index + 1), title, str(release_days), total_box_office, daily_box_office, "{:.2f}".format(screening_ratio * 100), str(screenings_count), "{:.2f}".format(box_office_ratio * 100), "{:.2f}".format(average_people_per_showing), "{:.2%}".format(occupancy_rate)] movie_list.append(movie_info) with open('./movies.csv', mode='w+', newline='', encoding='utf-8-sig') as file_obj: writer = csv.writer(file_obj) header = ['排名', '影片名', '上映时间(/天)', '影片总票房', '综合票房(/万)', '票房占比(%)', '排片场次', '排片占比(%)', '场均人次', '上座率(%)'] writer.writerow(header) writer.writerows(movie_list) ``` 上述代码片段展示了如何定义一个简单的爬虫流程，实际应用时还需要考虑更多细节问题，比如动态加载的内容、验证码验证以及更复杂的防爬策略等[^2]^。