### 使用Ansible自动化部署Docker和Hadoop集群 在使用Ansible自动化部署Docker和Hadoop集群时，需要确保所有节点的环境一致性，并通过Playbook完成Docker安装、Hadoop配置及集群启动等任务。以下是详细的实现方法。 #### 1. 准备工作 确保所有节点已正确配置SSH无密码登录，且Ansible主机能够访问所有目标节点。此外，需定义Inventory文件以指定目标主机及其分组信息[^1]。 ```ini [master] cluster-master ansible_host=172.18.0.2 [slaves] cluster-slave1 ansible_host=172.18.0.3 cluster-slave2 ansible_host=172.18.0.4 cluster-slave3 ansible_host=172.18.0.5 [all:vars] ansible_user=root ansible_python_interpreter=/usr/bin/python3 ``` #### 2. 安装Docker 编写一个Playbook用于在所有节点上安装Docker并启动服务。 ```yaml --- - name: Install Docker on all nodes hosts: all become: yes tasks: - name: Update package lists apt: update_cache: yes when: ansible_os_family == "Debian" - name: Install Docker dependencies yum: name: - yum-utils - device-mapper-persistent-data - lvm2 state: present when: ansible_os_family == "RedHat" - name: Add Docker repository yum_repository: name: docker-ce-stable description: Docker CE Stable Repository baseurl: https://download.docker.com/linux/centos/$releasever/$basearch/stable gpgcheck: yes enabled: yes when: ansible_os_family == "RedHat" - name: Install Docker yum: name: docker-ce state: present - name: Start and enable Docker service service: name: docker state: started enabled: yes ``` #### 3. 配置Hadoop集群 创建另一个Playbook用于部署Hadoop集群，包括解压安装包、配置核心文件（`core-site.xml`、`hdfs-site.xml`）以及格式化Namenode和启动服务。 ```yaml --- - name: Deploy Hadoop Cluster hosts: all become: yes vars: hadoop_version: "2.7.4" hadoop_tarball: "hadoop-{{ hadoop_version }}.tar.gz" hadoop_home: "/opt/hadoop-{{ hadoop_version }}" namenode_ip: "172.18.0.2" tasks: - name: Copy Hadoop tarball to master node copy: src: "{{ hadoop_tarball }}" dest: /tmp/ when: inventory_hostname == 'cluster-master' - name: Distribute Hadoop tarball to all nodes synchronize: src: "/tmp/{{ hadoop_tarball }}" dest: /tmp/ delegate_to: cluster-master - name: Extract Hadoop tarball unarchive: src: "/tmp/{{ hadoop_tarball }}" dest: /opt/ remote_src: yes - name: Configure core-site.xml template: src: templates/core-site.xml.j2 dest: "{{ hadoop_home }}/etc/hadoop/core-site.xml" - name: Configure hdfs-site.xml template: src: templates/hdfs-site.xml.j2 dest: "{{ hadoop_home }}/etc/hadoop/hdfs-site.xml" - name: Format Namenode command: "{{ hadoop_home }}/bin/hdfs namenode -format" args: chdir: "{{ hadoop_home }}" delegate_to: cluster-master run_once: true - name: Start Hadoop services shell: "{{ hadoop_home }}/sbin/start-dfs.sh && {{ hadoop_home }}/sbin/start-yarn.sh" args: chdir: "{{ hadoop_home }}" delegate_to: cluster-master run_once: true ``` #### 4. 验证部署结果 在完成上述步骤后，可以通过以下命令验证Hadoop集群是否正常运行： ```bash # 检查HDFS状态 [hadoop@cluster-master ~]$ hdfs dfsadmin -report # 检查YARN资源管理器 [hadoop@cluster-master ~]$ yarn node -list ``` --- ###