Contaminate AWS instances on ssh login

综合技术 2016-04-02

One of the principles of running large numbers of instances is that consistency is key. Config deviations cause oddities that’ll drain your time with investigations and nothing causes entropy on your hosts like an admin investigating an issue. In this post we’ll configure our instances to mark themselves as contaminated
when someone logs in. We can then use other tooling to query, collate and probably reap, machines corrupted by the keystrokes of humans.

While the example here is step-by-step and interactive, you’d normally bake this in to your AMI or deploy it very early in your config stage, possibly using cloud-init
. For our test we’ll spin up an instance and grant it an ec2 instance profile so it can alter its own tags.

In terms of moving parts we’ll install the awscli
package, add a short script that’ll tag the instance when run and configure PAM to invoke the script when an ssh session opens to the machine.

# install required dependency
sudo apt-get install awscli
cat > /usr/local/bin/add-dirty-tag <<EOF
#!/bin/bash

[ "$PAM_TYPE" = "open_session" ] || exit 0

INSTANCE_ID=$(ec2metadata --instance-id)

REGION=$(ec2metadata --availability-zone)
REGION=${REGION%?} # remove the last letter

aws ec2 --region $REGION create-tags --resources $INSTANCE_ID --tags Key=contaminated,Value=true
EOF
sudo chmod a+rx /usr/local/bin/add-dirty-tag

Now we have a script to add the ‘contaminated’ tag to our instance we’ll configure PAM to run it when a new ssh session starts. On a Ubuntu system the config should be placed in /etc/pam.d/sshd
.

# tag the machine as contaminated if anyone sshs in.
session    optional     pam_exec.so /usr/local/bin/add-dirty-tag

It’s worth opening another ssh session and logging in to confirm this works. That will leave you with an established connection in case you misconfigure PAM in some way. Once you’ve successfully logged in and caused the new tag to be added to the instance you can run a cli filter from outside the instance to show all hosts that have been interactively connected to:

aws --region eu-west-1 ec2 describe-instances             
    --filters "Name=tag:contaminated,Values=true"         
    --query 'Reservations[].Instances[].{id: InstanceId}'
[
    {
        "id": "i-x134x34x"
    }
]

If you decide to adopt an approach like this you can expand the values stored in the tag using the values PAM exposes, such as $PAM_USER
or $PAM_RUSER
and a time stamp. There’s also nothing stopping you from adding something more structured. A concise JSON dict maybe. Just be careful that you don’t overwrite the details on each successive login.

on UnixDaemon

责编内容by:on UnixDaemon (源链)。感谢您的支持!

您可能感兴趣的

Using ssh to connect to Linux for the SQL Server D... By: John Grover | | Related Tips:More >SQL Server on Linux Free...
Tricks to Optimize Your SSH Client Workflow. Here I'll describe some specific configuration options, I use to optimize my SSH...
iTerm2 结合 Linux expect 实现 ssh 自动登陆, 通过跳板机登录服务器内网功能... 痛点 一般情况下,公司所有的服务器都在内网,公网访问、管理服务器都要先通过登录一台跳板机,然后再由跳板机登录到相应的服务器进行操作,跳板机与服务器的连...
SSH框架之旅-hibernate(2) hibernate 1.主键生成策略 1.1 主键的两种类型 自然主键:把数据表中的某一业务字段作为表的主键。如一张用户表中,...
SaltStack: salt-ssh for agentless automation on Ub... Configuration Management tools like SaltStack are invaluable for managing ...