前陣子被客戶問到「在 Amazon EC2 用 boto3 都沒問題,為什麼本機開發就無法使用?」這類型的問題層出不窮,這問題在 boto3 Configuring credentials 章節內有提到讀取 credentials 的順序是:
- Passing credentials as parameters in the
boto.client()
method - Passing credentials as parameters when creating a
Session
object - Environment variables
- Shared credential file (
~/.aws/credentials
) - AWS config file (
~/.aws/config
) - Assume Role provider
- Boto2 config file (
/etc/boto.cfg
and~/.boto
) - Instance metadata service on an Amazon EC2 instance that has an IAM role configured.
而在 Amazon EC2 上通常用 IAM Role 來處理權限,也就是上方第 8 項所提到的 metadata service 這在每一個 Amazon EC2 上都會有自己的獨立參數,但是若客戶想要在本機也模擬同樣的情境時該怎麼處理?
AWS 其實有一個 Amazon EC2 Metadata Mock 的專案能在本機模擬出跟 Amazon EC2 上 87 分像的 metadata service,這篇作者將會針對一些常見功能來使用
How to use ec2-metadata-mock
Amazon EC2 Metadata Mock 支援 Linux, MacOS, Windows 甚至是用 Docker/Kubernetes 的形式安裝運行,以作者的 MacOS 為例:
$ brew tap aws/tap
$ brew install ec2-metadata-mock
然後就能拿到一個 ec2-metadata-mock
指令,第一次記得先用 -h
來看看支援的功能。
$ ec2-metadata-mock -h
不需要任何參數就能啟動 fake metadata service,預設會跑在 1338 port
$ ec2-metadata-mock
2022/07/03 17:04:50 Warning: Config File "aemm-config" Not Found in "[/Users/scottlwk]"
2022/07/03 17:04:50 Initiating ec2-metadata-mock for all mocks on port 1338
$ curl http://localhost:1338/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/ami
block-device-mapping/ebs0
block-device-mapping/ephemeral0
block-device-mapping/root
block-device-mapping/swap
elastic-inference/associations
elastic-inference/associations/eia-bfa21c7904f64a82a21b9f4540169ce1
events/maintenance/scheduled
events/recommendations/rebalance
hostname
iam/info
iam/security-credentials
iam/security-credentials/baskinc-role
instance-action
instance-id
instance-life-cycle
instance-type
kernel-id
latest
latest/api/token
local-hostname
local-ipv4
mac
network/interfaces/macs/0e:49:61:0f:c3:11/device-number
network/interfaces/macs/0e:49:61:0f:c3:11/interface-id
network/interfaces/macs/0e:49:61:0f:c3:11/ipv4-associations/192.0.2.54
network/interfaces/macs/0e:49:61:0f:c3:11/ipv6s
network/interfaces/macs/0e:49:61:0f:c3:11/local-hostname
network/interfaces/macs/0e:49:61:0f:c3:11/local-ipv4s
network/interfaces/macs/0e:49:61:0f:c3:11/mac
network/interfaces/macs/0e:49:61:0f:c3:11/network-card-index
network/interfaces/macs/0e:49:61:0f:c3:11/owner-id
network/interfaces/macs/0e:49:61:0f:c3:11/public-hostname
network/interfaces/macs/0e:49:61:0f:c3:11/public-ipv4s
network/interfaces/macs/0e:49:61:0f:c3:11/security-group-ids
network/interfaces/macs/0e:49:61:0f:c3:11/security-groups
network/interfaces/macs/0e:49:61:0f:c3:11/subnet-id
network/interfaces/macs/0e:49:61:0f:c3:11/subnet-ipv4-cidr-block
network/interfaces/macs/0e:49:61:0f:c3:11/subnet-ipv6-cidr-blocks
network/interfaces/macs/0e:49:61:0f:c3:11/vpc-id
network/interfaces/macs/0e:49:61:0f:c3:11/vpc-ipv4-cidr-block
network/interfaces/macs/0e:49:61:0f:c3:11/vpc-ipv4-cidr-blocks
network/interfaces/macs/0e:49:61:0f:c3:11/vpc-ipv6-cidr-blocks
placement/availability-zone
placement/availability-zone-id
placement/group-name
placement/host-id
placement/partition-number
placement/region
product-codes
public-hostname
public-ipv4
public-keys/0/openssh-key
ramdisk-id
reservation-id
security-groups
services/domain
services/partition
spot/instance-action
spot/termination-time
tags/instance/Name
tags/instance/Test
ec2-metadata-mock
預設模擬出了所有 Amazon EC2 上會有的所有 API 包含 iam/security-credentials
這裡就是 boto3 獲取 credentials 的地方。
如果你用了 spot instance 那麼 spot/termination-time
可以讓你測試 spot instance 即將被回收時的情境:
$ curl http://localhost:1338/latest/meta-data/spot/termination-time
2022-07-03T09:14:57Z
Sample data and Custom data
由於是 mock 資料理所當然都是假的,ec2-metadata-mock
可以一鍵 dump 所有 mock 資料到 ~/.ec2-metadata-mock/.aemm-config-used.json
:
$ ec2-metadata-mock -s
Successfully saved final configuration to local file /Users/scottlwk/.ec2-metadata-mock/.aemm-config-used.json
~/ec2-metadata-mock/.aemm-config-used.json
這包含所有設定參數以及 mock data 參數:
{
"server": {
"hostname": "0.0.0.0",
"port": "80"
}
}
再次引用修改過的參數就可以發現 port 改成 80 了!
$ ec2-metadata-mock -c ~/.ec2-metadata-mock/.aemm-config-used.json
$ curl curl http://localhost/latest/meta-data/spot/termination-time
但是在 ec2-metadata-mock 仍然無法做到與 Amazon EC2 完全一樣,原因是一般 PC/Mac 一般無法使用 169.254.x.x 這個保留段 IP 範圍,除此之外 ec2-metadata-mock
提供了與 Amazon EC2 metadata 幾乎一樣的功能。