### 使用Python爬取腾讯招聘网站指定页面的信息 为了实现从腾讯招聘网站第34页及其后的页面获取职位名称、地点、类别、职责以及所需的工作经验，可以采用基于`requests`库和`BeautifulSoup`库的方法来处理静态网页内容。如果遇到动态加载的内容，则可能需要用到Selenium这样的工具模拟浏览器行为。 对于本案例而言，在spiders文件夹下的`tencent.py`文件中编写相应的Spider逻辑[^1]： #### 安装依赖包 首先确保安装必要的Python库： ```bash pip install requests beautifulsoup4 lxml selenium ``` #### 编写爬虫脚本 下面是一个简单的示例代码片段用于说明如何构建一个基本的爬虫程序，该程序能够访问并解析特定编号之后的多页招聘信息列表： ```python import time from bs4 import BeautifulSoup import requests def get_job_info(page_num_start=34): base_url = "https://careers.tencent.com/tencentcareer/api/post/Query?" job_list = [] while True: url_params = { 'PageIndex': page_num_start, 'PageSize': 10, # 每页显示的数量可以根据实际情况调整 'Language': 'zh-cn' } try: response = requests.get(base_url, params=url_params) data = response.json() posts = data["Data"]["Posts"] if not posts or len(posts) == 0: break for post in posts: item = {} item['name'] = post.get('RecruitPostName', '') item['location'] = post.get('LocationName', '') item['category'] = post.get('CategoryName', '') detail_response = requests.get(f"https://careers.tencent.com/tencentcareer/detail/{post['PostId']}?lang=zh_cn") soup = BeautifulSoup(detail_response.content.decode(), features="lxml") responsibilities_section = soup.find("div", {"class": "recruit-detail__content"}) duties_text = '' requirements_text = '' if responsibilities_section is not None: sections = responsibilities_section.findAll("p") for section in sections: strong_tag = section.find("strong") if strong_tag and ("工作职责" in str(strong_tag)): next_sibling = section.nextSibling while next_sibling.name != 'p': if hasattr(next_sibling,'text'): duties_text += '\n' + next_sibling.text.strip() next_sibling = next_sibling.nextSibling elif strong_tag and ("任职要求" in str(strong_tag)): next_sibling = section.nextSibling while next_sibling.name != 'p': if hasattr(next_sibling,'text'): requirements_text += '\n' + next_sibling.text.strip() next_sibling = next_sibling.nextSibling item['responsibilities'] = duties_text.replace('\r\n','').strip() item['requirements'] = requirements_text.replace('\r\n','').strip() job_list.append(item) print(f"{page_num_start} 页面的数据已收集完毕.") page_num_start += 1 # 防止被封IP，适当延时 time.sleep(2) except Exception as e: print(e) continue return job_list ``` 此段代码实现了对腾讯招聘官网API接口调用以获得每一页上的岗位概要信息，并进一步通过详情链接抓取具体的工作描述与资格条件等内容。注意这里假设了API返回结构保持稳定不变；实际应用时需根据最新的HTML源码结构调整XPath表达式或CSS选择器路径[^2]。