[이렇게 사용하세요!] 네이버 클라우드 플랫폼에서 terraform 활용[2]: Loadblancer 서비스와 ansible 연동
Published in
37 min readSep 20, 2020
※본 콘텐츠는 네이버 클라우드 플랫폼 Tech Evangelist 송창안 님이 작성하셨습니다.
안녕하세요 네이버 클라우드 플랫폼입니다! 이전 시간에는 네이버 클라우드 플랫폼에서 terraform과 ansible 을 연동하여 서비스를 생성해 봤습니다!
오늘은 네이버 클라우드 플랫폼에서 terraform 활용해 Loadblancer와ansible을 연동하는 방법을 알아보겠습니다!
load balancer 서비스란?
load balancer는 사용자들의 서비스에 대한 네트워크 트래픽을 분산해 서버의 부하를 낮추는 역할을 수행합니다. 로드밸런서에 연결된 서버 중 일부 서버에 장애가 발생하면 자동으로 다른 정상 서버로 부하를 배분해 무정지 서비스가 가능하게 해 서비스의 높은 안정성을 보장하고 있습니다.
예를 들어, http://www.csong.kr 이라는 도메인에 웹서비스를 하고 있는 경우, 서버가 1대인 경우에는 그 서버가 이슈가 발생하는 경우, 웹 서비스에 대해서 무중단이 불가능하지만, 서버를 2대 이상 구성 한 경우, 1개다 이슈가 발생해도 나머지 한 대는 서비스가 가능하게 되는 fault tolerance 보장할 수 있으며, 2대의 서비스에 대해서 부하 분산 도 가능하게 합니다.
필요한 준비 사항은 아래와 같습니다.
1. 네이버 클라우드 플랫폼 가입 (www.ncloud.com)
2. VM을 하나 생성하여 ansible 설치까지 확인(아래 내용 참조)
# yum install epel-release # yum install ansible -y
아래와 같이 네이버 클라우드 플랫폼에서 구성하는 예제를 소개하게 됩니다.
09.06 멀티존에서 생성할 때 확인된 에러 내용
테라폼에서 data ncloud_port_forwarding_rules를 사용하기 위해 기존에 사용 중인 ZONE에 서버가 없으면 ncloud_port_forwarding_rules의 zone에 정보를 가져오지 못하는 이슈가 있습니다. 각 멀티존 마다 아래와 같이 서버가 1대씩 포트포워딩만 되어 있으면, 에러는 발생하지 않습니다.
1. terraform 0.13.0 버전을 설치합니다.
# wget https://releases.hashicorp.com/terraform/0.13.0/terraform_0.13.0_linux_amd64.zip
# unzip terraform_0.13.0_linux_amd64.zip
Archive: terraform_0.13.0_linux_amd64.zip
inflating: terraform
# mv terraform /usr/bin/2. 아래와 같이 디렉토리를 생성합니다.
# mkdir ~/terraform_example
# mkdir ~/terraform_example/ncloud
# mkdir ~/terraform_example/ncloud/ansible-loadblacner3. main.tf 파일을 생성하고 아래의 내용을 구현합니다.
아래와 같이 github에 코드도 전체 코드를 업로드해두었습니다.
provider "ncloud" {
access_key = var.access_key
secret_key = var.secret_key
region = var.region
site = var.site
}data "ncloud_root_password" "rootpwd" {
count = "${var.instance_count}"
server_instance_no = ncloud_server.server[count.index].id
private_key = ncloud_login_key.key.private_key
}data "ncloud_port_forwarding_rules" "rules" {
count = "${var.instance_count}"
zone = ncloud_server.server[count.index].zone
}resource "random_id" "id" {
byte_length = 4
}resource "ncloud_login_key" "key" {
key_name = "${var.login_key_name}${random_id.id.hex}"
}
resource "ncloud_server" "server" {
count = "${var.instance_count}"
name = "ncloud-terraform-test-vm-${count.index+1}"
server_image_product_code = var.server_image_product_code
server_product_code = var.server_product_code
description = "ncloud-terraform-test-vm-${count.index+1} is best tip!!"
login_key_name = "${ncloud_login_key.key.key_name}"
zone = "${var.zones[count.index]}"
}
resource "ncloud_port_forwarding_rule" "forwarding" {
count = "${var.instance_count}"
port_forwarding_configuration_no = data.ncloud_port_forwarding_rules.rules[count.index].id
server_instance_no = ncloud_server.server[count.index].id
port_forwarding_external_port = "1100${count.index+1}"
port_forwarding_internal_port = "22"
}resource "ncloud_load_balancer" "lb" {
name = "ncloud-terraform-test-lb"
algorithm_type = "RR"
description = "ncloud-terraform-test-lb is best!!" rule_list {
protocol_type = "HTTP"
load_balancer_port = 80
server_port = 80
l7_health_check_path = "/"
} server_instance_no_list = [ncloud_server.server[0].id, ncloud_server.server[1].id]
internet_line_type = "PUBLC"
network_usage_type = "PBLIP"
region = "KR"
}#playbook for ansible
resource "null_resource" "ansible" {
provisioner "local-exec" {
command = <<EOF
echo "[ncloud]" > inventory
EOF }
}resource "null_resource" "ssh" {
count = "${var.instance_count}"
provisioner "local-exec" {
command = <<EOF
echo "${ncloud_server.server[count.index].name} ansible_host='${ncloud_port_forwarding_rule.forwarding[count.index].port_forwarding_public_ip}' ansible_port='${ncloud_port_forwarding_rule.forwarding[count.index].port_forwarding_external_port}' ansible_ssh_user=root ansible_ssh_pass='${data.ncloud_root_password.rootpwd[count.index].root_password}'" >> inventory
EOF } provisioner "local-exec" {
command = <<EOF
sleep 120; ANSIBLE_HOST_KEY_CHECKING=False \
ansible-playbook -i inventory playbook.yml
EOF }
}
4. output.tf 파일을 생성하고 아래의 내용을 구현합니다.
#output "load_balancer_instance_id" {
# value = ncloud_load_balancer.lb.id
#}output "server_name_list" {
value = join(",", ncloud_server.server.*.name)
}
5. variables.tf 파일을 생성 후, 인증 부분에 대해서 네이버 클라우드 플랫폼에서의 [ 마이페이지 -> 계정관리 -> 인증키 관리 ] 에서 없는 경우는 신규 API 인증키 생성을 하여 사용을 시작합니다. 이후 variables.tf 파일을 구현합니다.
*주의 : access_key 와 secret_key는 본인이 생성한 키를 입력해야 합니다. 아래의 기본값은 예제이며, 나중에 꼭 수정을 해야 사용이 가능합니다.
variable "access_key" { # export TF_VAR_access_key=...
default = ""
}variable "secret_key" { # export TF_VAR_secret_key=...
default = "SPSVRSTAND000072"
}variable "server_image_product_code" { # centos-7.8-64
default = "SPSW0LINUX000139" #SPSW0LINUX000046
}variable "server_product_code" { # vCPU 2EA, Memory 8GB, Disk 50GB[SSD]
default = "" #SPSVRSSD00000003
}variable "login_key_name" {
default = "tf-test-key"
}variable "instance_count" {
default = "2"
}variable "zones" {
type = list
default = ["KR-1", "KR-2"]
}variable "region" {
default = "KR"
}variable "site" {
default = "www.ncloud.com"
}
6. version.tf 파일을 생성하고, 구현합니다.
terraform {
required_version = ">= 0.13"
required_providers {
ncloud = {
source = "terraform-providers/ncloud"
}
null = {
source = "hashicorp/null"
}
random = {
source = "hashicorp/random"
}
}
}7. centos 7에는 nginx 관련 리파지토리가 없기 때문에 자동화로 생성될 서버에 포함을 시키기 위한 repo 파일을 생성합니다.
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=18. terraform destroy 명령 후, 불필요한 파일을 삭제하기 위한 스크립트입니다.
rm -rf inventory terraform.tfstate terraform.tfstate.backup9. playbook.yml 파일을 생성하여, 구현을 하며, git과 nginx 파일을 설치하기 위해 구성한 ansible playbook입니다.
---
- hosts: ncloud
become: true
tasks:
- name: ping
ping: - name: enable ngnix repo
copy:
src: nginx.repo
dest: /etc/yum.repos.d/nginx.repo
owner: root
group: root
mode: '0644'
backup: yes - name: install packages
yum:
name: "{{ packages }}"
state: latest
update_cache: yes
vars:
packages:
- git
- nginx - name: Make sure a service is running
systemd:
state: started
name: nginx
enabled: yes - name: Download the nginx package for backup version but do not install it
yum:
name: "{{ packages }}"
state: latest
download_only: true
vars:
packages:
- git
- nginx
10. 파일을 다 구성을 하면 디렉토리에는 아래와 같이 파일이 구성 되게 됩니다.
# ls
main.tf nginx.repo outputs.tf playbook.yml variables.tf versions.tf11. terraform init 명령을 통해, 네이버 클라우드 플랫폼에서 사용할 수 있도록 초기 화 및 플러그인들을 다운을 받아서 현재 디렉토리에서 진행을 합니다.
# terraform initInitializing the backend...Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/random...
- Finding latest version of terraform-providers/ncloud...
- Installing terraform-providers/ncloud v1.3.1...
- Installed terraform-providers/ncloud v1.3.1 (signed by HashiCorp)
- Installing hashicorp/null v2.1.2...
- Installed hashicorp/null v2.1.2 (signed by HashiCorp)
- Installing hashicorp/random v2.3.0...
- Installed hashicorp/random v2.3.0 (signed by HashiCorp)The following providers do not have any version constraints in configuration,
so the latest version was installed.To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.* hashicorp/null: version = "~> 2.1.2"
* hashicorp/random: version = "~> 2.3.0"
* terraform-providers/ncloud: version = "~> 1.3.1"
Warning: Additional provider information from registryThe remote registry returned warnings for
registry.terraform.io/terraform-providers/ncloud:
- For users on Terraform 0.13 or greater, this provider has moved to
NaverCloudPlatform/ncloud. Please update your source in required_providers.Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
12. terraform plan 명령을 통해 HCL로 구성된 내용과 네이버 클라우드 플랫폼 자원과의 적용이 가능한지 여부를 확인을 하며, 실제 적용은 현재 단계에서는 진행되지 않습니다.
# terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
<= read (data resources)Terraform will perform the following actions: # data.ncloud_port_forwarding_rules.rules[0] will be read during apply
# (config refers to values not yet known)
<= data "ncloud_port_forwarding_rules" "rules" {
+ id = "59497"
+ port_forwarding_configuration_no = "59497"
+ port_forwarding_rule_list = [
+ {
+ port_forwarding_external_port = "9987"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4544974"
},
+ {
+ port_forwarding_external_port = "9986"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4533453"
},
+ {
+ port_forwarding_external_port = "9981"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4087698"
},
+ {
+ port_forwarding_external_port = "2002"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2975238"
},
+ {
+ port_forwarding_external_port = "9980"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2952586"
},
]
+ zone = "KR-2"
} # data.ncloud_port_forwarding_rules.rules[1] will be read during apply
# (config refers to values not yet known)
<= data "ncloud_port_forwarding_rules" "rules" {
+ id = "59497"
+ port_forwarding_configuration_no = "59497"
+ port_forwarding_rule_list = [
+ {
+ port_forwarding_external_port = "9987"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4544974"
},
+ {
+ port_forwarding_external_port = "9986"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4533453"
},
+ {
+ port_forwarding_external_port = "9981"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4087698"
},
+ {
+ port_forwarding_external_port = "2002"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2975238"
},
+ {
+ port_forwarding_external_port = "9980"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2952586"
},
]
+ zone = "KR-2"
} # data.ncloud_root_password.rootpwd[0] will be read during apply
# (config refers to values not yet known)
<= data "ncloud_root_password" "rootpwd" {
+ id = (known after apply)
+ private_key = (sensitive value)
+ root_password = (sensitive value)
+ server_instance_no = (known after apply)
} # data.ncloud_root_password.rootpwd[1] will be read during apply
# (config refers to values not yet known)
<= data "ncloud_root_password" "rootpwd" {
+ id = (known after apply)
+ private_key = (sensitive value)
+ root_password = (sensitive value)
+ server_instance_no = (known after apply)
} # ncloud_load_balancer.lb will be created
+ resource "ncloud_load_balancer" "lb" {
+ algorithm_type = "RR"
+ certificate_name = (known after apply)
+ connection_timeout = (known after apply)
+ description = "ncloud-terraform-test-lb is best!!"
+ domain_name = (known after apply)
+ id = (known after
13. terraform apply -auto-approve 명령을 실행하여, 실제 네이버 클라우드 플랫폼에 HCL 내용을 반영합니다.
중간에 아래와 같은 에러가 나오지만, terrafirn, tfstate이 저장된 리소스를 알고 있기 때문에, terraform appy를 한 번 더 진행해도 멱등성이 지원 되는 것을 확인할 수 있으며, 이미 만들어진 리소스 외에 다음 과정을 수행하는 것을 확인할 수 있습니다.
Error: Status: 400 Bad Request, Body: {"responseError": {
"returnCode": "25033",
"returnMessage": "Currently port forwarding setting is being created/ modifying/ deleting. Please try again a few seconds later."
}}# terraform apply An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
<= read (data resources)Terraform will perform the following actions: # data.ncloud_port_forwarding_rules.rules[0] will be read during apply
# (config refers to values not yet known)
<= data "ncloud_port_forwarding_rules" "rules" {
+ id = "59497"
+ port_forwarding_configuration_no = "59497"
+ port_forwarding_rule_list = [
+ {
+ port_forwarding_external_port = "9987"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4544974"
},
+ {
+ port_forwarding_external_port = "9986"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4533453"
},
+ {
+ port_forwarding_external_port = "9981"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "4087698"
},
+ {
+ port_forwarding_external_port = "2002"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2975238"
},
+ {
+ port_forwarding_external_port = "9980"
+ port_forwarding_internal_port = "22"
+ port_forwarding_public_ip = "106.10.55.207"
+ server_instance_no = "2952586"
},
]
+ zone = "KR-2"
} ....null_resource.ssh[1] (local-exec): TASK [Make sure a service is running] ******************************************null_resource.ssh[0] (local-exec): TASK [Make sure a service is running] ******************************************
null_resource.ssh[0] (local-exec): changed: [ncloud-terraform-test-vm-2]
null_resource.ssh[1] (local-exec): changed: [ncloud-terraform-test-vm-2]
null_resource.ssh[1] (local-exec): changed: [ncloud-terraform-test-vm-1]
null_resource.ssh[0] (local-exec): changed: [ncloud-terraform-test-vm-1]null_resource.ssh[1] (local-exec): TASK [Download the nginx package for backup version but do not install it] *****null_resource.ssh[0] (local-exec): TASK [Download the nginx package for backup version but do not install it] *****
null_resource.ssh[0] (local-exec): ok: [ncloud-terraform-test-vm-1]
null_resource.ssh[0] (local-exec): ok: [ncloud-terraform-test-vm-2]null_resource.ssh[0] (local-exec): PLAY RECAP *********************************************************************
null_resource.ssh[0] (local-exec): ncloud-terraform-test-vm-1 : ok=6 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
null_resource.ssh[0] (local-exec): ncloud-terraform-test-vm-2 : ok=6 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0null_resource.ssh[0]: Creation complete after 2m23s [id=4042852432376986528]
null_resource.ssh[1] (local-exec): ok: [ncloud-terraform-test-vm-1]
null_resource.ssh[1] (local-exec): ok: [ncloud-terraform-test-vm-2]null_resource.ssh[1] (local-exec): PLAY RECAP *********************************************************************
null_resource.ssh[1] (local-exec): ncloud-terraform-test-vm-1 : ok=6 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
null_resource.ssh[1] (local-exec): ncloud-terraform-test-vm-2 : ok=6 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0null_resource.ssh[1]: Creation complete after 2m25s [id=5999409360354501284]Apply complete! Resources: 3 added, 1 changed, 0 destroyed.Outputs:server_name_list = ncloud-terraform-test-vm-1,ncloud-terraform-test-vm-2
14. loadbalancer 공인 ip를 확인 후, http 가 연결되는지 확인합니다.
15. 자원을 회수하기 위해서 terraform destroy -auto-approve 명령을 통하여, 네이버 클라우드 플랫폼 자원을 반환합니다.
# terraform destroy
random_id.id: Refreshing state... [id=ov5mPA]
ncloud_login_key.key: Refreshing state... [id=tf-test-keya2fe663c]
ncloud_server.server[0]: Refreshing state... [id=5021157]
data.ncloud_port_forwarding_rules.rules[1]: Refreshing state...
data.ncloud_root_password.rootpwd[0]: Refreshing state...
data.ncloud_port_forwarding_rules.rules[0]: Refreshing state...An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
- destroyTerraform will perform the following actions: # ncloud_login_key.key will be destroyed
- resource "ncloud_login_key" "key" {
- fingerprint = "74:52:bc:4a:41:b2:c4:2a:c7:76:96:c6:21:83:14:6c" -> null
- id = "tf-test-keya2fe663c" -> null
- key_name = "tf-test-keya2fe663c" -> null
- private_key = (sensitive value)
} # ncloud_server.server[0] will be destroyed
- resource "ncloud_server" "server" {
- base_block_storage_disk_type = "NET" -> null
- base_block_storage_size = 53687091200 -> null
- cpu_count = 2 -> null
- description = "ncloud-terraform-test-vm-1 is best tip!!" -> null
- id = "5021157" -> null
- instance_no = "5021157" -> null
- instance_operation = "NULL" -> null
- instance_status = "NSTOP" -> null
- instance_status_name = "stopped" -> null
- internet_line_type = "PUBLC" -> null
- is_fee_charging_monitoring = false -> null
- login_key_name = "tf-test-keya2fe663c" -> null
- memory_size = 4294967296 -> null
- name = "ncloud-terraform-test-vm-1" -> null
- platform_type = "LNX64" -> null
- port_forwarding_public_ip = "106.10.55.207" -> null
- private_ip = "10.41.169.99" -> null
- region = "KR" -> null
- server_image_name = "centos-7.3-64" -> null
- server_image_product_code = "SPSW0LINUX000046" -> null
- server_product_code = "SPSVRSSD00000003" -> null
- zone = "KR-2" -> null
} # random_id.id will be destroyed
- resource "random_id" "id" {
- b64 = "ov5mPA" -> null
- b64_std = "ov5mPA==" -> null
- b64_url = "ov5mPA" -> null
- byte_length = 4 -> null
- dec = "2734581308" -> null
- hex = "a2fe663c" -> null
- id = "ov5mPA" -> null
}Plan: 0 to add, 0 to change, 3 to destroy.
Warning: Interpolation-only expressions are deprecated on main.tf line 28, in resource "ncloud_server" "server":
28: count = "${var.instance_count}"Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.(and 4 more similar warnings elsewhere)
Warning: Quoted type constraints are deprecated on variables.tf line 26, in variable "zones":
26: type = "list"Terraform 0.11 and earlier required type constraints to be given in quotes,
but that form is now deprecated and will be removed in a future version of
Terraform. To silence this warning, remove the quotes around "list" and write
list(string) instead to explicitly indicate that the list elements are
strings.Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yesncloud_server.server[0]: Destroying... [id=5021157]
ncloud_server.server[0]: Still destroying... [id=5021157, 10s elapsed]
ncloud_server.server[0]: Destruction complete after 12s
ncloud_login_key.key: Destroying... [id=tf-test-keya2fe663c]
ncloud_login_key.key: Destruction complete after 3s
random_id.id: Destroying... [id=ov5mPA]
random_id.id: Destruction complete after 0sDestroy complete! Resources: 3 destroyed.
구성에 대한 자세한 설명은 아래의 링크를 참조해 주시기 바랍니다.
