この記事は freee Developers Advent Calendar の 14 日目です。
どうも id:renjikariです。
10月にEKSのingress controllerが2.0.0になり名前がAWS Load Balancer Controllerになって登場しました。
ぶっちゃけ私はingressにも、もとのalb-ingress-controllerにも全然詳しくないんですがこの機会に新旧のControllerを構築して試して(遊んで)見ようと思います。
できるだけだれでもできるように目指します。構築は基本的にはAWSの公式に則ります。
- EKSの構築: https://docs.aws.amazon.com/ja_jp/eks/latest/userguide/getting-started-eksctl.html
- ALB周り: https://docs.aws.amazon.com/ja_jp/eks/latest/userguide/alb-ingress.html
Contents
- aws-load-balancer-controllerの紹介
- aws-alb-ingress-controllerでingressを作ってみる
- aws-load-balancer-controllerでingressを作ってみる
- 触ってみてわかった差分とその所感
aws-load-balancer-controllerの紹介
aws-load-balancer-controllerの構成画像へのリンクを貼ってみます。これだけだと難しくてよくわからないので、解説のDocも読みました
https://aws.amazon.com/jp/blogs/containers/introducing-aws-load-balancer-controller/ 曰く目玉は以下の通り
- Kubernetesサービス用のネットワークロードバランサー(NLB)
- 複数のingress ruleでALBを共有することができるように
- 新しいTargetGroupBinding CRDの導入
- 完全にプライベートなクラスターのサポート
この記事ではALBを共有する話に触れます。
- 今まで1ingress = 1ALBだったのが複数のingressをまとめれるようになりました。
- 実際にどうなるかやってみます。
This project was formerly known as "AWS ALB Ingress Controller", we rebranded it to be "AWS Load Balancer Controller".
また、公式のDocやgithubのREADMEにこんなことが書いてあったのでやる気を感じました。
aws-alb-ingress-controllerでingressを作ってみる
まずはv1系(いままでのやつ)で構築してみます。
※注意 先述したように、https://docs.aws.amazon.com/ja_jp/eks/latest/userguide/alb-ingress.html を参考にしていて、これはaws-load-balancer-controllerの手順なので若干齟齬があり、最小の手順じゃなくなってる可能性が高いです。
- まずはclusterを作る
~ ❯ eksctl create cluster \ --name sandbox-renjikari-1 \ --version 1.18 \ --region ap-northeast-1 \ --nodegroup-name linux-nodes \ --nodes 3 \ --nodes-min 1 \ --nodes-max 4 \ --managed
- ALB用のIAMとServiceAccountを作る
# たぶんoidcはいらない(IRSAしてなさそう) ~ ❯ eksctl utils associate-iam-oidc-provider \ --region ap-northeast-1 \ --cluster sandbox-renjikari-1 \ --approve [ℹ] eksctl version 0.33.0 [ℹ] using region ap-northeast-1 [ℹ] will create IAM Open ID Connect provider for cluster "sandbox-renjikari-1" in "ap-northeast-1" [✔] created IAM Open ID Connect provider for cluster "sandbox-renjikari-1" in "ap-northeast-1" ~ ❯ curl -o iam-policy-ingress-controller.json https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/iam-policy.json ~ ❯ aws iam create-policy \ --policy-name LBControllerPolicyRenjikari-ingress \ --policy-document file://iam-policy-ingress-controller.json ~ ❯ aws iam attach-role-policy --region=ap-northeast-1 --role-name=$NODE_ROLE_NAME --policy-arn=LBControllerPolicyRenjikari-ingress ~ ❯ eksctl create iamserviceaccount \ --cluster=sandbox-renjikari-1 \ --namespace=kube-system \ --name=alb-ingress-controller \ --attach-policy-arn=arn:aws:iam::*******:policy/LBControllerPolicyRenjikari-ingress \ --override-existing-serviceaccounts \ --approve
- 上記だとclusterRoleとclusterRoleBindingが作られなかったので、sampleから作成
~ ❯ curl -sS "https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.0/docs/examples/rbac-role.yaml" > rbac-role.yaml k apply -f rbac-role.yaml
- ingress controllerのdeploy
- cluster nameとserviceAccountName/serviceAccountの項目を修正
~ ❯ curl -sS "https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.0/docs/examples/alb-ingress-controller.yaml" > alb-ingress-controller.yaml kubectl apply -f alb-ingress-controller.yaml
- 作られたリソースの確認
~ ❯ k get sa -n kube-system |grep ingress alb-ingress-controller 1 3d10h ~ ❯ ~ ❯ k describe sa alb-ingress-controller -n kube-system Name: alb-ingress-controller Namespace: kube-system Labels: <none> Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::**********:role/eksctl-sandbox-renjikari-1-addon-iamservicea-Role1-1IKRVPID7SU75 Image pull secrets: <none> Mountable secrets: alb-ingress-controller-token-wkwd2 Tokens: alb-ingress-controller-token-wkwd2 Events: <none> ~ ❯ k get pod -n kube-system NAME READY STATUS RESTARTS AGE alb-ingress-controller-55bd445656-jgk45 1/1 Running 0 7m51s
- アプリのデプロイ
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/2048/2048_full.yaml
作られたLBのDNS経由でアクセスすると2048が遊べた
http://*****-game2048-ingress2-*****-********.ap-northeast-1.elb.amazonaws.com/
logを確認しつつ作られたリソースの確認
- logからもv1.0.0である主張がめっちゃある
- LBを作成
- TargetGroupを作成
- LBのtagの変更(というか付与)
- (LBの)Listerの作成
- Listener Ruleの作成
- LB用のSGを作ってrule変えてLBに付与
- Instance(K8s Node)用のSGを作ってrule変えてnodeに付与
------------------------------------------------------------------------------- AWS ALB Ingress controller Release: v1.0.0 Build: git-c25bc6c5 Repository: https://github.com/kubernetes-sigs/aws-alb-ingress-controller ------------------------------------------------------------------------------- I1210 21:18:58.566632 1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource" "Controller"="alb-ingress-controller" "Source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"loadBalancer":{}}}} I1210 21:18:58.566859 1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource" "Controller"="alb-ingress-controller" "Source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"loadBalancer":{}}}} I1210 21:18:58.566984 1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource" "Controller"="alb-ingress-controller" "Source"={"Type":{"metadata":{"creationTimestamp":null}}} I1210 21:18:58.567219 1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource" "Controller"="alb-ingress-controller" "Source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"daemonEndpoints":{"kubeletEndpoint":{"Port":0}},"nodeInfo":{"machineID":"","systemUUID":"","bootID":"","kernelVersion":"","osImage":"","containerRuntimeVersion":"","kubeletVersion":"","kubeProxyVersion":"","operatingSystem":"","architecture":""}}}} I1210 21:18:58.567448 1 leaderelection.go:185] attempting to acquire leader lease kube-system/ingress-controller-leader-alb... I1210 21:50:20.000945 1 :0] kubebuilder/controller "level"=0 "msg"="Starting Controller" "Controller"="alb-ingress-controller" I1210 21:50:20.101096 1 :0] kubebuilder/controller "level"=0 "msg"="Starting workers" "Controller"="alb-ingress-controller" "WorkerCount"=1 I1210 22:12:18.387156 1 loadbalancer.go:185] game-2048/ingress-2048: creating LoadBalancer ********-game2048-ingress2-3738 I1210 22:12:19.442761 1 loadbalancer.go:201] game-2048/ingress-2048: LoadBalancer ********-game2048-ingress2-3738 created, ARN: arn:aws:elasticloadbalancing:ap-northeast-1:**********:loadbalancer/app/********-game2048-ingress2-3738/************ I1210 22:12:19.534395 1 targetgroup.go:108] game-2048/ingress-2048: creating target group ********-************ I1210 22:12:19.710479 1 targetgroup.go:127] game-2048/ingress-2048: target group ********-************ created: arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/********-************/************ I1210 22:12:19.733015 1 tags.go:43] game-2048/ingress-2048: modifying tags { kubernetes.io/cluster/sandbox-renjikari-1: "owned", kubernetes.io/namespace: "game-2048", kubernetes.io/ingress-name: "ingress-2048", kubernetes.io/service-name: "service-2048", kubernetes.io/service-port: "80"} on arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/********-************/************ I1210 22:12:19.785763 1 targets.go:73] game-2048/ingress-2048: Adding targets to arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/********-************/************: 192.168.35.42:80, 192.168.5.138:80, 192.168.58.15:80, 192.168.76.112:80, 192.168.88.182:80 I1210 22:12:20.045637 1 listener.go:83] game-2048/ingress-2048: creating listener 80 I1210 22:12:20.105254 1 rules.go:59] game-2048/ingress-2048: creating rule 1 on arn:aws:elasticloadbalancing:ap-northeast-1:**********:listener/app/********-game2048-ingress2-3738/************/97bba4db08bf92a9 I1210 22:12:20.154569 1 rules.go:76] game-2048/ingress-2048: rule 1 created with conditions [{ Field: "path-pattern", Values: ["/*"] }] I1210 22:12:20.243404 1 association.go:224] game-2048/ingress-2048: creating securityGroup ********-game2048-ingress2-3738:managed LoadBalancer securityGroup by ALB Ingress Controller I1210 22:12:20.368442 1 tags.go:69] game-2048/ingress-2048: modifying tags { kubernetes.io/ingress-name: "ingress-2048", kubernetes.io/cluster/sandbox-renjikari-1: "owned", kubernetes.io/namespace: "game-2048"} on sg-************ I1210 22:12:20.517525 1 security_group.go:50] game-2048/ingress-2048: granting inbound permissions to securityGroup sg-************: [{ FromPort: 80, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0", Description: "Allow ingress on port 80 from 0.0.0.0/0" }], ToPort: 80 }] I1210 22:12:20.792115 1 lb_attachment.go:30] game-2048/ingress-2048: modify securityGroup on LoadBalancer arn:aws:elasticloadbalancing:ap-northeast-1:**********:loadbalancer/app/********-game2048-ingress2-3738/************ to be [sg-************] I1210 22:12:21.207053 1 association.go:224] game-2048/ingress-2048: creating securityGroup instance-********-game2048-ingress2-3738:managed instance securityGroup by ALB Ingress Controller I1210 22:12:21.372176 1 tags.go:69] game-2048/ingress-2048: modifying tags { kubernetes.io/cluster/sandbox-renjikari-1: "owned", kubernetes.io/namespace: "game-2048", kubernetes.io/ingress-name: "ingress-2048"} on sg-************ I1210 22:12:21.498486 1 security_group.go:50] game-2048/ingress-2048: granting inbound permissions to securityGroup sg-************: [{ FromPort: 0, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-************" }] }] I1210 22:12:21.955031 1 instance_attachment.go:87] game-2048/ingress-2048: attaching securityGroup sg-************ to ENI eni-***************** I1210 22:12:22.425565 1 instance_attachment.go:87] game-2048/ingress-2048: attaching securityGroup sg-************ to ENI eni-***************** I1210 22:12:22.960393 1 instance_attachment.go:87] game-2048/ingress-2048: attaching securityGroup sg-************ to ENI eni-*****************
aws-load-balancer-controllerでingressを作ってみる
次に、新しく出たaws-load-balancer-controllerで構築してみます。 わかりやすさ重視のため別クラスタを立てています。構築については差分だけ書きます。
- ALB用のIAMとServiceAccountを作る
~ ❯ eksctl utils associate-iam-oidc-provider \ --region ap-northeast-1 \ --cluster sandbox-renjikari-2 \ --approve [ℹ] eksctl version 0.33.0 [ℹ] using region ap-northeast-1 [ℹ] will create IAM Open ID Connect provider for cluster "sandbox-renjikari-1" in "ap-northeast-1" [✔] created IAM Open ID Connect provider for cluster "sandbox-renjikari-1" in "ap-northeast-1" ~ ❯ curl -o iam-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json ~ ❯ aws iam create-policy \ --policy-name LBControllerPolicyRenjikari \ --policy-document file://iam-policy-ingress-controller.json ~ ❯ eksctl create iamserviceaccount \ --cluster=sandbox-renjikari-2 \ --namespace=kube-system \ --name=aws-load-balancer-controller \ --attach-policy-arn=arn:aws:iam::**********:policy/LBControllerPolicyRenjikari \ --override-existing-serviceaccounts \ --approve
- ingress controllerのdeploy
- helmでいれられるらしいのでhelmで
~ ❯ kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller//crds?ref=master" ~ ❯ helm3 repo add eks https://aws.github.io/eks-charts ~ ❯ helm3 upgrade -i aws-load-balancer-controller eks/aws-load-balancer-controller \ --set clusterName=sandbox-renjikari-2 \ --set serviceAccount.create=false \ --set serviceAccount.name=aws-load-balancer-controller \ -n kube-system ~ ❯ helm3 ls -n kube-system NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION ~ ❯ aws-load-balancer-controller kube-system 3 2020-12-11 08:18:31.666122 +0900 JST deployed aws-load-balancer-controller-1.1.0 v2.1.0
- サンプルアプリケーションのデプロイ
~ ❯ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/2048/2048_full.yaml
- log
- フォーマットがv1系と違っている
- やっていることの流れはほぼ同じっぽいが、targetgroupにIPをattachするところはTargetGroupBindigs経由でやっているのが相違点っぽい
{"level":"info","ts":1607642923.4134078,"msg":"version","GitVersion":"v2.1.0","GitCommit":"f95827224a66cae20a1af999d8ef1d46ec462547","BuildDate":"2020-12-01T19:27:50+0000"} {"level":"info","ts":1607642923.4581616,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"} {"level":"info","ts":1607642923.4603312,"logger":"setup","msg":"adding health check for controller"} {"level":"info","ts":1607642923.4604735,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-v1-pod"} {"level":"info","ts":1607642923.4605618,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-elbv2-k8s-aws-v1beta1-targetgroupbinding"} {"level":"info","ts":1607642923.460624,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-elbv2-k8s-aws-v1beta1-targetgroupbinding"} {"level":"info","ts":1607642923.460945,"logger":"setup","msg":"starting podInfo repo"} I1210 23:28:45.461095 1 leaderelection.go:242] attempting to acquire leader lease kube-system/aws-load-balancer-controller-leader... {"level":"info","ts":1607642925.4611657,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"} I1210 23:28:45.480647 1 leaderelection.go:252] successfully acquired lease kube-system/aws-load-balancer-controller-leader {"level":"info","ts":1607642925.5614917,"logger":"controller-runtime.webhook.webhooks","msg":"starting webhook server"} {"level":"info","ts":1607642925.5615087,"logger":"controller","msg":"Starting EventSource","controller":"service","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5615797,"logger":"controller","msg":"Starting Controller","controller":"service"} {"level":"info","ts":1607642925.5615995,"logger":"controller","msg":"Starting workers","controller":"service","worker count":3} {"level":"info","ts":1607642925.5616424,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"channel source: 0xc0002eaa00"} {"level":"info","ts":1607642925.561678,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"channel source: 0xc0002eaa50"} {"level":"info","ts":1607642925.5616996,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5617182,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.561738,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5614593,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5621867,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5622096,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.5628521,"logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"} {"level":"info","ts":1607642925.5629292,"logger":"controller-runtime.webhook","msg":"serving webhook server","host":"","port":9443} {"level":"info","ts":1607642925.5630434,"logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"} {"level":"info","ts":1607642925.661992,"logger":"controller","msg":"Starting Controller","controller":"ingress"} {"level":"info","ts":1607642925.6621237,"logger":"controller","msg":"Starting workers","controller":"ingress","worker count":3} -- ここまで立ち上がっときのlogっぽい {"level":"info","ts":1607642925.6624649,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="} {"level":"info","ts":1607642925.7627265,"logger":"controller","msg":"Starting Controller","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding"} {"level":"info","ts":1607642925.7627664,"logger":"controller","msg":"Starting workers","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","worker count":3} {"level":"info","ts":1607643258.6043127,"logger":"controllers.ingress","msg":"successfully built model","model":"{\"id\":\"game-2048/ingress-2048\",\"resources\":{\"AWS::EC2::SecurityGroup\":{\"ManagedLBSecurityGroup\":{\"spec\":{\"groupName\":\"k8s-game2048-ingress2-********\",\"description\":\"[k8s] Managed SecurityGroup for LoadBalancer\",\"ingress\":[{\"ipProtocol\":\"tcp\",\"fromPort\":80,\"toPort\":80,\"ipRanges\":[{\"cidrIP\":\"0.0.0.0/0\"}]}]}}},\"AWS::ElasticLoadBalancingV2::Listener\":{\"80\":{\"spec\":{\"loadBalancerARN\":{\"$ref\":\"#/resources/AWS::ElasticLoadBalancingV2::LoadBalancer/LoadBalancer/status/loadBalancerARN\"},\"port\":80,\"protocol\":\"HTTP\",\"defaultActions\":[{\"type\":\"fixed-response\",\"fixedResponseConfig\":{\"contentType\":\"text/plain\",\"statusCode\":\"404\"}}]}}},\"AWS::ElasticLoadBalancingV2::ListenerRule\":{\"80:1\":{\"spec\":{\"listenerARN\":{\"$ref\":\"#/resources/AWS::ElasticLoadBalancingV2::Listener/80/status/listenerARN\"},\"priority\":1,\"actions\":[{\"type\":\"forward\",\"forwardConfig\":{\"targetGroups\":[{\"targetGroupARN\":{\"$ref\":\"#/resources/AWS::ElasticLoadBalancingV2::TargetGroup/game-2048/ingress-2048-service-2048:80/status/targetGroupARN\"}}]}}],\"conditions\":[{\"field\":\"path-pattern\",\"pathPatternConfig\":{\"values\":[\"/*\"]}}]}}},\"AWS::ElasticLoadBalancingV2::LoadBalancer\":{\"LoadBalancer\":{\"spec\":{\"name\":\"k8s-game2048-ingress2-********\",\"type\":\"application\",\"scheme\":\"internet-facing\",\"ipAddressType\":\"ipv4\",\"subnetMapping\":[{\"subnetID\":\"subnet-0bedd9f3cc8fdd18c\"},{\"subnetID\":\"subnet-0242ee825e50cccc5\"},{\"subnetID\":\"subnet-********\"}],\"securityGroups\":[{\"$ref\":\"#/resources/AWS::EC2::SecurityGroup/ManagedLBSecurityGroup/status/groupID\"}]}}},\"AWS::ElasticLoadBalancingV2::TargetGroup\":{\"game-2048/ingress-2048-service-2048:80\":{\"spec\":{\"name\":\"k8s-game2048-service2-********\",\"targetType\":\"ip\",\"port\":80,\"protocol\":\"HTTP\",\"protocolVersion\":\"HTTP1\",\"healthCheckConfig\":{\"port\":\"traffic-port\",\"protocol\":\"HTTP\",\"path\":\"/\",\"matcher\":{\"httpCode\":\"200\"},\"intervalSeconds\":15,\"timeoutSeconds\":5,\"healthyThresholdCount\":2,\"unhealthyThresholdCount\":2}}}},\"K8S::ElasticLoadBalancingV2::TargetGroupBinding\":{\"game-2048/ingress-2048-service-2048:80\":{\"spec\":{\"template\":{\"metadata\":{\"name\":\"k8s-game2048-service2-********\",\"namespace\":\"game-2048\",\"creationTimestamp\":null},\"spec\":{\"targetGroupARN\":{\"$ref\":\"#/resources/AWS::ElasticLoadBalancingV2::TargetGroup/game-2048/ingress-2048-service-2048:80/status/targetGroupARN\"},\"targetType\":\"ip\",\"serviceRef\":{\"name\":\"service-2048\",\"port\":80},\"networking\":{\"ingress\":[{\"from\":[{\"securityGroup\":{\"groupID\":{\"$ref\":\"#/resources/AWS::EC2::SecurityGroup/ManagedLBSecurityGroup/status/groupID\"}}}],\"ports\":[{\"protocol\":\"TCP\"}]}]}}}}}}}}"} {"level":"info","ts":1607643259.5002263,"logger":"controllers.ingress","msg":"creating securityGroup","resourceID":"ManagedLBSecurityGroup"} {"level":"info","ts":1607643259.7226052,"logger":"controllers.ingress","msg":"created securityGroup","resourceID":"ManagedLBSecurityGroup","securityGroupID":"sg-********"} {"level":"info","ts":1607643259.9485986,"msg":"authorizing securityGroup ingress","securityGroupID":"sg-********","permission":[{"FromPort":80,"IpProtocol":"tcp","IpRanges":[{"CidrIp":"0.0.0.0/0","Description":""}],"Ipv6Ranges":null,"PrefixListIds":null,"ToPort":80,"UserIdGroupPairs":null}]} {"level":"info","ts":1607643260.110359,"msg":"authorized securityGroup ingress","securityGroupID":"sg-********"} {"level":"info","ts":1607643264.718656,"logger":"controllers.ingress","msg":"creating targetGroup","stackID":"game-2048/ingress-2048","resourceID":"game-2048/ingress-2048-service-2048:80"} {"level":"info","ts":1607643265.120149,"logger":"controllers.ingress","msg":"created targetGroup","stackID":"game-2048/ingress-2048","resourceID":"game-2048/ingress-2048-service-2048:80","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/k8s-game2048-service2-********/********"} {"level":"info","ts":1607643266.3804967,"logger":"controllers.ingress","msg":"creating loadBalancer","stackID":"game-2048/ingress-2048","resourceID":"LoadBalancer"} {"level":"info","ts":1607643267.251105,"logger":"controllers.ingress","msg":"created loadBalancer","stackID":"game-2048/ingress-2048","resourceID":"LoadBalancer","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:loadbalancer/app/k8s-game2048-ingress2-********/********"} {"level":"info","ts":1607643267.2861407,"logger":"controllers.ingress","msg":"creating listener","stackID":"game-2048/ingress-2048","resourceID":"80"} {"level":"info","ts":1607643267.3420944,"logger":"controllers.ingress","msg":"created listener","stackID":"game-2048/ingress-2048","resourceID":"80","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:listener/app/k8s-game2048-ingress2-********/********/********"} {"level":"info","ts":1607643267.3545017,"logger":"controllers.ingress","msg":"creating listener rule","stackID":"game-2048/ingress-2048","resourceID":"80:1"} {"level":"info","ts":1607643267.4122574,"logger":"controllers.ingress","msg":"created listener rule","stackID":"game-2048/ingress-2048","resourceID":"80:1","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:listener-rule/app/k8s-game2048-ingress2-********/********/********/********"} {"level":"info","ts":1607643267.4123323,"logger":"controllers.ingress","msg":"creating targetGroupBinding","stackID":"game-2048/ingress-2048","resourceID":"game-2048/ingress-2048-service-2048:80"} {"level":"info","ts":1607643267.4465287,"logger":"controllers.ingress","msg":"created targetGroupBinding","stackID":"game-2048/ingress-2048","resourceID":"game-2048/ingress-2048-service-2048:80","targetGroupBinding":{"namespace":"game-2048","name":"k8s-game2048-service2-********"}} {"level":"info","ts":1607643267.5947661,"logger":"controllers.ingress","msg":"successfully deployed model","ingressGroup":"game-2048/ingress-2048"} {"level":"info","ts":1607643267.7672768,"msg":"authorizing securityGroup ingress","securityGroupID":"sg-********","permission":[{"FromPort":0,"IpProtocol":"tcp","IpRanges":null,"Ipv6Ranges":null,"PrefixListIds":null,"ToPort":65535,"UserIdGroupPairs":[{"Description":"elbv2.k8s.aws/targetGroupBinding=shared","GroupId":"sg-********","GroupName":null,"PeeringStatus":null,"UserId":null,"VpcId":null,"VpcPeeringConnectionId":null}]}]} {"level":"info","ts":1607643267.9896986,"msg":"authorized securityGroup ingress","securityGroupID":"sg-********"} {"level":"info","ts":1607643268.2313712,"msg":"registering targets","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/k8s-game2048-service2-********/********","targets":[{"AvailabilityZone":null,"Id":"192.168.19.30","Port":80},{"AvailabilityZone":null,"Id":"192.168.4.120","Port":80},{"AvailabilityZone":null,"Id":"192.168.66.91","Port":80},{"AvailabilityZone":null,"Id":"192.168.9.200","Port":80},{"AvailabilityZone":null,"Id":"192.168.90.202","Port":80}]} {"level":"info","ts":1607643268.5573752,"msg":"registered targets","arn":"arn:aws:elasticloadbalancing:ap-northeast-1:**********:targetgroup/k8s-game2048-service2-********/********"}
2つ目のアプリをデプロイ
- aws-load-balancer-controller公式のdocに他のexampleが載ってたので、これを二つ目として構築してみる
- まずはそのまま構築
~ ❯ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.0.0/docs/examples/echoservice/echoserver-namespace.yaml &&\ ~ ❯ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.0.0/docs/examples/echoservice/echoserver-service.yaml &&\ ~ ❯ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.0.0/docs/examples/echoservice/echoserver-deployment.yaml # docには変更してapplyしろって書いてあったけどそのままで動いた ~ ❯ wget https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.0.0/docs/examples/echoservice/echoserver-ingress.yaml ~ ❯ kubectl apply -f echoserver-ingress.yaml # 疎通確認(Host指定しないとListenerがうまくいかんので超注意(1時間以上はまった)) curl -H "Host:echoserver.example.com" ${LBのDNS}
- aws-load-balancer-controllerの新機能の一つである1LBで複数サービスを共有する機能を試す
- 2つのingressリソースにおもむろに以下のannotationを付け足す
metadata: annotations: alb.ingress.kubernetes.io/group.name: test-group
- これだけでもともとのechoserverとgame-2048のLBが削除され、test-groupという名前のLBが生成されて、疎通できるようになった
- なにこれすごい
触ってみた差分とその所感
- 構築時
- aws-load-balancer-controllerの中で必要な権限はIRSAを利用しているので、Node Roleに直接policyをattachする必要がなくなっている
- helmでinstallできるようになっている
- updateの記事で、kubectlとawsコマンドを行き来する回数が減っている、みたいなことが書いてあって構築時もほとんどeksctlとhelmで完結している
aws iam create-policy
が唯一の残りっぽさ
- アプリのデプロイ
- user操作は変わっていない
- logの出力がjsonになった
- 結果的に少しわかりづらくはなった気がする
- 全体的な感想
- aws-load-balancer-controller、かなり良さそうに感じたので使っていきたい気持ちになった
明日の12月15日の記事はQAチームの(@mishizuka99)さんの担当です!