YJWANG

Install OpenStack (with kolla-ansible, Ceph / without python-virtual-env) 본문

60.Cloud/60.OpenStack

Install OpenStack (with kolla-ansible, Ceph / without python-virtual-env)

왕영주 2021. 3. 22. 19:09

refer to :

Prerequisite

Ceph Node 사전 작업

위 포스팅을 참고하여 통해 우선 Ceph Cluster를 정상적으로 배포해준다.

이후 glance / ciner / nova를 위한 keyring과 pool을 생성한다.

pool create

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph osd pool create volumes
pool 'volumes' created
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph osd pool create images
pool 'images' created
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph osd pool create backups
pool 'backups' created
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph osd pool create vms
pool 'vms' created

pool init

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 rbd pool init volumes
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 rbd pool init images
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 rbd pool init backups
root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 rbd pool init vms

keyring 생성

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.glance mon 'profile rbd' osd 'profile rbd pool=images' mgr 'profile rbd pool=images'
[client.glance]
    key = AQDBClhgRbSRAhAAZzfMIQV5PN4j7zkH5Nyf9Q==

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.cinder mon 'profile rbd' osd 'profile rbd pool=volumes, profile rbd pool=vms, profile rbd-read-only pool=images' mgr 'profile rbd pool=volumes, profile rbd pool=vms'
[client.cinder]
    key = AQDCClhgCTaKGxAAVOGslUNfwQT6KBqw/Mh4wQ==

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.cinder-backup mon 'profile rbd' osd 'profile rbd pool=backups' mgr 'profile rbd pool=backups'
[client.cinder-backup]
    key = AQDEClhg26l6HRAASw4Fin8Vx9ocdEZj8DNM2A==

copy할 파일 준비

root@yjwang0-ceph-01:~/ceph-ansible# mkdir ceph-auth

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.glance |tee ceph-auth/ceph.client.glance.keyring
[client.glance]
    key = AQDBClhgRbSRAhAAZzfMIQV5PN4j7zkH5Nyf9Q==

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.cinder |tee ceph-auth/ceph.client.cinder.keyring
[client.cinder]
    key = AQDCClhgCTaKGxAAVOGslUNfwQT6KBqw/Mh4wQ==

root@yjwang0-ceph-01:~# docker exec ceph-mon-yjwang0-ceph-01 ceph auth get-or-create client.cinder-backup |tee ceph-auth/ceph.client.cinder-backup.keyring
[client.cinder-backup]
    key = AQDEClhg26l6HRAASw4Fin8Vx9ocdEZj8DNM2A==

root@yjwang0-ceph-01:~# cat /etc/ceph/ceph.conf  |tee ceph-auth/ceph.conf
[client.libvirt]
...

복사할 dir를 미리 생성한다.

Openstack Node에서 작업
root@yjwang0-stack-01:/etc/kolla# mkdir -p /etc/kolla/config/glance
root@yjwang0-stack-01:/etc/kolla# mkdir -p /etc/kolla/config/cinder/cinder-volume
root@yjwang0-stack-01:/etc/kolla# mkdir -p /etc/kolla/config/cinder/cinder-backup
root@yjwang0-stack-01:/etc/kolla# mkdir -p /etc/kolla/config/nova

해당 dir로 config를 복사한다.

Ceph Node에서 작업
root@yjwang0-ceph-01:~# scp ceph-auth/ceph.conf 10.99.99.20:/etc/kolla/config/glance/
root@yjwang0-ceph-01:~# scp ceph-auth/ceph.client.glance.keyring  10.99.99.20:/etc/kolla/config/glance/

root@yjwang0-ceph-01:~# scp ceph-auth/ceph.conf 10.99.99.20:/etc/kolla/config/cinder/
root@yjwang0-ceph-01:~# scp ceph-auth/ceph.client.cinder* 10.99.99.20:/etc/kolla/config/cinder/cinder-volume/
root@yjwang0-ceph-01:~# scp ceph-auth/ceph.client.cinder* 10.99.99.20:/etc/kolla/config/cinder/cinder-backup/

root@yjwang0-ceph-01:~# scp ceph-auth/ceph.conf 10.99.99.20:/etc/kolla/config/nova/
root@yjwang0-ceph-01:~# scp ceph-auth/ceph.client.cinder.keyring  10.99.99.20:/etc/kolla/config/nova/

Openstack Control Node 작업

Update package index

root@yjwang0-stack-01:~# apt update

Install python dependencies

root@yjwang0-stack-01:~# apt install -y python3-dev libffi-dev gcc libssl-dev

Install dependencies not using a virtual env

root@yjwang0-stack-01:~# apt install -y python3-pip

Install Ansible (Require at least 2.9 ver)

root@yjwang0-stack-01:~# apt -y install ansible
root@yjwang0-stack-01:~# ansible --version
ansible 2.9.6

Clone kolla , kolla-ansible git

root@yjwang0-stack-01:~# git clone https://github.com/openstack/kolla
root@yjwang0-stack-01:~# git clone https://github.com/openstack/kolla-ansible

Install requirements of kolla and kolla-ansible:

root@yjwang0-stack-01:~# pip3 install ./kolla
root@yjwang0-stack-01:~# pip3 install ./kolla-ansible

create /etc/koll dir

root@yjwang0-stack-01:~# mkdir -p /etc/kolla
root@yjwang0-stack-01:~# chown $USER:$USER /etc/kolla

root@yjwang0-stack-01:~# ls -ld /etc/kolla/
drwxr-xr-x 2 root root 4096 Mar  4 13:32 /etc/kolla/

copy file for installation

cp -r kolla-ansible/etc/kolla/* /etc/kolla
cp kolla-ansible/ansible/inventory/* .

 

Config Ansible

ansible config file

root@yjwang0-stack-01:~# cat ./ansible.cfg 
[defaults]
host_key_checking=False
pipelining=True
forks=100

check current config file path

root@yjwang0-stack-01:~# ansible --version
ansible 2.9.6
  config file = /root/ansible.cfg
...

inventory

root@yjwang0-stack-01:~# head -200 multinode 
# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[all]
yjwang0-stack-01 ansible_host=10.99.99.20
yjwang0-stack-02 ansible_host=10.99.99.21
yjwang0-stack-03 ansible_host=10.99.99.22
yjwang0-stack-04 ansible_host=10.99.99.23
yjwang0-stack-05 ansible_host=10.99.99.24
yjwang0-ceph-01 ansible_host=10.99.99.30
yjwang0-ceph-02 ansible_host=10.99.99.31
yjwang0-ceph-03 ansible_host=10.99.99.32

[control]
# These hostname must be resolvable from your deployment host
yjwang0-stack-01
yjwang0-stack-02
yjwang0-stack-03

# The above can also be specified as follows:
#control[01:03]     ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
yjwang0-stack-01
yjwang0-stack-02
yjwang0-stack-03

[compute]
yjwang0-stack-04
yjwang0-stack-05

[monitoring]

# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1

[storage:children]
control

[deployment]
localhost       ansible_connection=local

...(이하 default)

confirm connection

root@yjwang0-stack-01:~# apt install sshpass
or
root@yjwang0-stack-01:~# ssh-keygen -N "" -f /root/.ssh/id_rsa
root@yjwang0-stack-01:~# for i in 20 21 22 23 24 30 31 32;do ssh-copy-id 10.99.99.$i;done

root@yjwang0-stack-01:~# ansible -i multinode all -m ping

Gen kolla password

root@yjwang0-stack-01:~# python3 ./kolla-ansible/tools/generate_passwords.py

향후 horizon dashboard에서 사용될 암호이다.
root@yjwang0-stack-01:~# cat /etc/kolla/passwords.yml |grep keystone_admin_password
keystone_admin_password: pNXfHhvua8cC0ILxCuwaYimevCmIBLgPcSXQxKNP

kolla globals.yml

root@yjwang0-stack-01:~# vim /etc/kolla/globals.yml 
root@yjwang0-stack-01:~# grep -Ev '^$|^#' /etc/kolla/globals.yml
---
kolla_base_distro: "ubuntu"
kolla_install_type: "source"
openstack_release: "master"
kolla_internal_vip_address: "10.99.99.25"
network_interface: "ens2"
api_interface: "{{ network_interface }}"
storage_interface: "{{ network_interface }}"
tunnel_interface: "{{ network_interface }}"
neutron_external_interface: "ens5"
enable_openstack_core: "yes"
enable_cinder: "yes"
enable_cinder_backup: "yes"
external_ceph_cephx_enabled: "yes"
ceph_glance_keyring: "ceph.client.glance.keyring"
ceph_glance_user: "glance"
ceph_glance_pool_name: "images"
ceph_cinder_keyring: "ceph.client.cinder.keyring"
ceph_cinder_user: "cinder"
ceph_cinder_pool_name: "volumes"
ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring"
ceph_cinder_backup_user: "cinder-backup"
ceph_cinder_backup_pool_name: "backups"
ceph_nova_keyring: "{{ ceph_cinder_keyring }}"
ceph_nova_user: "cinder"
ceph_nova_pool_name: "vms"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
cinder_backup_driver: "ceph"
nova_backend_ceph: "yes"

 

Deploy

bootstrap servers

root@yjwang0-stack-01:~# cd kolla-ansible/tools/
root@yjwang0-stack-01:~/kolla-ansible/tools# ./kolla-ansible -i ../../multinode bootstrap-servers

Do pre-deployment checks for hosts:

root@yjwang0-stack-01:~/kolla-ansible/tools# ./kolla-ansible -i ../../multinode prechecks

Finally proceed to actual OpenStack deployment:

root@yjwang0-stack-01:~/kolla-ansible/tools# ./kolla-ansible -i ../../multinode deploy

 

Using Openstack

root@yjwang0-stack-01:~# pip3 install python-openstackclient

Create openrc file

root@yjwang0-stack-01:~# cd kolla-ansible/tools/
root@yjwang0-stack-01:~/kolla-ansible/tools# ./kolla-ansible post-deploy

Check

root@yjwang0-stack-01:~# . /etc/kolla/admin-openrc.sh 
root@yjwang0-stack-01:~# openstack network agent list
+--------------------------------------+--------------------+------------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host             | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+------------------+-------------------+-------+-------+---------------------------+
| 03571d28-0c8b-4c65-90c6-5b5a3ae17c8b | Open vSwitch agent | yjwang0-stack-05 | None              | :-)   | UP    | neutron-openvswitch-agent |
| 0909a247-10eb-45a7-9548-0b93f4db7e27 | L3 agent           | yjwang0-stack-01 | nova              | :-)   | UP    | neutron-l3-agent          |
| 13581e8d-4f2e-45b6-9b78-8221751f2798 | DHCP agent         | yjwang0-stack-03 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 339de114-9b30-400a-b42a-9d3e867d7c9d | L3 agent           | yjwang0-stack-03 | nova              | :-)   | UP    | neutron-l3-agent          |
| 3798bebc-5423-47a4-a889-f7e65bf0e14a | Metadata agent     | yjwang0-stack-02 | None              | :-)   | UP    | neutron-metadata-agent    |
| 383f8301-ce8e-4976-b815-583e35864321 | DHCP agent         | yjwang0-stack-02 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 3b2ffaa3-e908-4431-b2d0-4e8bebd357d7 | Open vSwitch agent | yjwang0-stack-01 | None              | :-)   | UP    | neutron-openvswitch-agent |
| 3ffae31f-0240-439c-957d-35d7f84ac05a | Metadata agent     | yjwang0-stack-03 | None              | :-)   | UP    | neutron-metadata-agent    |
| 407509af-64e5-47f5-9f70-041c1c2268de | Open vSwitch agent | yjwang0-stack-02 | None              | :-)   | UP    | neutron-openvswitch-agent |
| 8435ef0c-84e2-483e-8ea8-6f9d0ed75fb9 | Metadata agent     | yjwang0-stack-01 | None              | :-)   | UP    | neutron-metadata-agent    |
| a246c897-7fdc-4e6d-8b55-726f4633f4a0 | DHCP agent         | yjwang0-stack-01 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| a9d0be88-6da7-4501-8dba-986d0d6781ff | Open vSwitch agent | yjwang0-stack-03 | None              | :-)   | UP    | neutron-openvswitch-agent |
| b08acc32-2fbb-4275-9445-3206e9e27f87 | L3 agent           | yjwang0-stack-02 | nova              | :-)   | UP    | neutron-l3-agent          |
| ba97263b-ef18-4927-9f5a-3e295948fc3b | Open vSwitch agent | yjwang0-stack-04 | None              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+------------------+-------------------+-------+-------+---------------------------+

check hypervisor

root@yjwang0-stack-01:~/kolla-ansible/tools# openstack hypervisor list
+----+---------------------+-----------------+-------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP     | State |
+----+---------------------+-----------------+-------------+-------+
|  3 | yjwang0-stack-04    | QEMU            | 10.99.99.23 | up    |
|  6 | yjwang0-stack-05    | QEMU            | 10.99.99.24 | up    |
+----+---------------------+-----------------+-------------+-------+

[option] create example networks, images, etc

Edit Network Information

root@yjwang0-stack-01:~/kolla-ansible/tools# grep -iE '(ext_net).*=' init-runonce 
ENABLE_EXT_NET=${ENABLE_EXT_NET:-1}
EXT_NET_CIDR=${EXT_NET_CIDR:-'10.0.2.0/24'}
EXT_NET_RANGE=${EXT_NET_RANGE:-'start=10.0.2.150,end=10.0.2.199'}
EXT_NET_GATEWAY=${EXT_NET_GATEWAY:-'10.0.2.1'}

Run scripts

root@yjwang0-stack-01:~/kolla-ansible/tools# ./init-runonce

[option] bash completion

root@yjwang0-stack-01:~/kolla-ansible/tools# openstack complete > /etc/bash_completion.d/osc
[재접속]

Open V switch 확인

root@yjwang0-stack-01:~# docker exec -it openvswitch_db ovs-vsctl list-br
br-ex
br-int
br-tun

root@yjwang0-stack-01:~# docker exec -it openvswitch_db ovs-vsctl list-ports br-int
int-br-ex
patch-tun
tapb8f2242b-17

root@yjwang0-stack-01:~# docker exec -it openvswitch_db ovs-vsctl list-ports br-ex
ens2
phy-br-ex

root@yjwang0-stack-01:~# docker exec -it openvswitch_db ovs-vsctl list-ports br-tun
patch-int
vxlan-0a633a16
vxlan-0a633a17

VM을 생성하면 VM ID로 ceph rbd에 아래와 같이 image가 생성된다.
Instance 정보

root@yjwang0-stack-01:~# openstack server list
+--------------------------------------+-------+--------+------------------------------------+--------+---------+
| ID                                   | Name  | Status | Networks                           | Image  | Flavor  |
+--------------------------------------+-------+--------+------------------------------------+--------+---------+
| e1b4c952-a9af-418e-b9b2-e2deb762d9ce | demo1 | ACTIVE | demo-net=10.0.0.59                 | cirros | m1.tiny |
+--------------------------------------+-------+--------+------------------------------------+--------+---------+

rbd 정보

[root@yjwang0-ceph-01 /]# rbd ls vms
e1b4c952-a9af-418e-b9b2-e2deb762d9ce_disk

[root@yjwang0-ceph-01 /]# rbd info vms/e1b4c952-a9af-418e-b9b2-e2deb762d9ce_disk
rbd image 'e1b4c952-a9af-418e-b9b2-e2deb762d9ce_disk':
    size 1 GiB in 256 objects
    order 22 (4 MiB objects)
    snapshot_count: 0
    id: 179127eb8cce7
    block_name_prefix: rbd_data.179127eb8cce7
    format: 2
    features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
    op_features: 
    flags: 
    create_timestamp: Thu Mar 25 10:20:47 2021
    access_timestamp: Thu Mar 25 10:22:45 2021
    modify_timestamp: Thu Mar 25 10:24:31 2021
반응형
저작자표시 변경금지
'60.Cloud/60.OpenStack' Related Articles more