Provisioning quickly nodes in a SeaMicro chassis with Ansible

存储架构 2015-01-12

Recently I had to quickly test and deploy CentOS on 128 physical nodes, just to test hardware and that all currently "supported" CentOS releases could be installed quickly when needed. The interesting bit is that it was a completely new infra, without any traditional deployment setup in place, so obviously, as sysadmin, we directly think about pxe/kickstart, which is so trivial to setup. That was the first time I had to "play" with SeaMicro devices/chassis though, and so understanding how they work (the SeaMicro 15K fabric chassis
, to be precise). One thing to note is that those seamicro chassis don't provide remote VGA/KVM feature (but who cares, as we'll automate the whole thing, right ? ) but they instead provide either cli (ssh) or rest api access to the management interface, so that you can quickly reset/reconfigure a node, changing vlan assignement, and so on.

It's not a secret that I like to use Ansible
for ad-hoc tasks, and I thought that it would be (again) a good tool for that quick task. If you have used Ansible already, you know that you have to declare nodes and variables (not needed, but really useful) in the inventory (if you don't gather inventory from an external source). To configure my pxe setup (and so being able to reconfigure it when needed) I obviously needed to get mac addresses from all 64 nodes in each chassis, decide that hostnames will be n${slot-number}., etc .. (and yes in Seamicro slot 1 = 0/0, slot 2 = 1/0, and so on ...)

The following quick-and-dirty bash script let you do that quickly in 2 seconds (ssh into chassis, gather information, and fill some variables in my ansible host_vars/${hostname} file) :

ssh "enable ; show server summary | include Intel ; quit" | while read line ;  
  seamicrosrvid=$(echo $line |awk '{print $1}')  
  slot=$(echo $seamicrosrvid| cut -f 1 -d '/')  
  id=$(( $slot + 1)); ip=$id ; mac=$(echo $line |awk '{print $3}')  
  echo -e "name: n${id} nseamicro_chassis: hufty nseamicro_srvid: $seamicrosrvid nmac_address: $mac nip: 172.19.3.$ip ngateway: nnetmask: nnameserver: ncentos_dist: 6" >  inventory/n${id}  

Nice so we have all ~/ansible/hosts/host_vars/${inventory_hostname} files in one go (I let you add ${inventory_hostname} in the ~/ansible/hosts/hosts.cfg file with the same script, but modify to your needs

For the next step, we assume that we already have dnsmasq installed on the "head" node, and that we also have a httpd setup to provide the kickstart to the nodes during installation.

So our basic ansible playbook looks like this :

- hosts: ci-nodes  
  sudo: True  
  gather_facts: False

    seamicro_user_login: admin  
    seamicro_user_pass: obviously-hidden-and-changed  
    action: reset  
    using-pxe: "true"  
    username: "{{ seamicro_user_login }}"  
    password: "{{ seamicro_user_pass }}"

    - name: Generate kickstart file[s] for Seamicro node[s]  
      template: src=../templates/kickstarts/ci-centos-{{ centos_dist}}-ks.j2 dest=/var/www/html/ks/{{ inventory_hostname }}-ks.cfg mode=0755  
      delegate_to: "{{ deploy_node }}"

    - name: Adding the entry in DNS (dnsmasq)  
      lineinfile: dest=/etc/hosts regexp="^{{ ip }} {{ inventory_hostname }}" line="{{ ip }} {{ inventory_hostname }}"  
      delegate_to: "{{ deploy_node }}"  
      notify: reload_dnsmasq

    - name: Adding the DHCP entry in dnsmasq  
      template: src=../templates/dnsmasq-dhcp.j2 dest=/etc/dnsmasq.d/{{ inventory_hostname }}.conf  
      delegate_to: "{{ deploy_node }}"  
      register: dhcpdnsmasq

    - name: Reloading dnsmasq configuration  
      service: name=dnsmasq state=restarted  
      run_once: true  
      when: dhcpdnsmasq|changed  
      delegate_to: "{{ deploy_node }}"

    - name: Generating the tftp configuration boot file  
      template: src=../templates/pxeboot-ci dest=/var/lib/tftpboot/pxelinux.cfg/01-{{ mac_address | lower | replace(":","-") }} mode=0755  
      delegate_to: "{{ deploy_node }}"

    - name: Resetting the Seamicro node[s]  
      uri: url=https://{{ seamicro_chassis }}{{ seamicro_srvid }}  
           body='{{ seamicro_reset_body | to_json }}'  
      delegate_to: "{{ deploy_node }}"

    - name: Waiting for Seamicro node[s] to be available through ssh ...  
      action: wait_for port=22 host={{ inventory_hostname }} timeout=1200  
      delegate_to: "{{ deploy_node }}"

    - name: reload_dnsmasq  
      service: name=dnsmasq state=reloaded

The first thing to notice is that you can use Ansible to provision nodes that aren't already running : people think than ansible is just to interact with already provisioned and running nodes, but by providing useful informations in the inventory, and by delegating actions, we can already start "managing" those yet-to-come nodes.All the templates used in that playbook are really basic ones, so nothing "rocket science". For example the only diff for the kickstart.j2 template is that we inject ansible variables (for network and storage) :

network --bootproto=static --device=eth0 --gateway={{ gateway }}
--ip={{ ip }} --nameserver={{ nameserver }} --netmask={{ netmask }}
--ipv6=auto --activate  
network --hostname={{ inventory_hostname }}  
part /boot --fstype="ext4" --ondisk=sda --size=500  
part pv.14 --fstype="lvmpv" --ondisk=sda --size=10000 --grow  
volgroup vg_{{ inventory_hostname_short }} --pesize=4096 pv.14  
logvol /home --fstype="xfs" --size=2412 --name=home --vgname=vg_{{
inventory_hostname_short }} --grow --maxsize=100000  
logvol / --fstype="xfs" --size=8200 --name=root --vgname=vg_{{
inventory_hostname_short }} --grow --maxsize=1000000  
logvol swap --fstype="swap" --size=2136 --name=swap --vgname=vg_{{
inventory_hostname_short }}  

The dhcp step isn't mandatory, but at least in that subnet we only allow dhcp to "already known" mac address, retrieved from the ansible inventory (and previously fetched directly from the seamicro chassis) :

# {{ name }} ip assignement  
dhcp-host={{ mac_address }},{{ ip }}  

Same thing for the pxelinux tftp config file :

SERIAL 0 9600  
DEFAULT text  
ONTIMEOUT {{ inventory_hostname }}-deploy

LABEL local  
MENU LABEL (local)  

LABEL {{ inventory_hostname}}-deploy  
kernel CentOS/{{ centos_dist }}/{{ centos_arch}}/vmlinuz  
MENU LABEL CentOS {{ centos_dist }} {{ centos_arch }}- CI Kickstart
for {{ inventory_hostname }}  
{% if centos_dist == 7 -%}  
append initrd=CentOS/7/{{ centos_arch }}/initrd.img net.ifnames=0 biosdevname=0 ip=eth0:dhcp inst.ks={{ inventory_hostname }}-ks.cfg console=ttyS0,9600n8  
{% else -%}  
append initrd=CentOS/{{ centos_dist }}/{{ centos_arch }}/initrd.img ksdevice=eth0 ip=dhcp ks={{ inventory_hostname }}-ks.cfg console=ttyS0,9600n8  
{% endif %}  

The interesting part is the one on which I needed to spend more time : as said, it was the first time I had to play with SeaMicro hardware, so I had to dive into documentation (which I *always* do, RTFM FTW !) and understand how to use their Rest API
but once done, it was a breeze. Ansible by default doesn't provide a native resource for Seamicro, but that's why Rest exists, right and thanfully, Ansible has a native URI module
, which we use here . The only thing on which I had to spend more time was to understand how to properly construct the body, but declaring in the yaml file as a variable/list and then converting it on the fly to json (with the magical body='{{ seamicro_reset_body | to_json }}'
) was the way to go and is so self-explained when read now.

And here we go, calling that ansible playbook and suddenly 128 physical machines were being installed (and reinstalled with different CentOS versions - 5,6,7 - and arches i386,x86_64)

Hope this helps if you have to interact with Seamicro chassis from within an ansible playbook too

责编内容by:Arrfab's Blog (源链)。感谢您的支持!


How to Install Apache Cassandra on CentOS 7 Apache Cassandra is a free and open source distributed NoSQL database management...
CentOS 7下安装Tomcat到服务 环境: CentOS Linux release 7.3.1611 (Core) 3.10.0-514.16.1.el7.x86_64 1.安装J...
CentOS7使用DevStack快速搭建OpenStack实验环境... 安装环境:CentOS 7系统下安装DevStack 一、下载Ubuntu或者CentOS 7安装(实体机或者虚拟机都可以),建议选择最小安装镜像即可。...
Postgres Meltdown and Spectre Benchmarks You all probably heard about Meltdown and Spectre vulnerabilities that are expl...
Ansible, Part III: Playbooks Playbooks make Ansible even more powerful than before. To be quite honest, if ...