kolla安装Ceph（all in one）

 云计算  Add comments
Mar 012017
 

默认安装的all in one，是没有安装cinder服务的，也是没有ceph。如何在一个已经装好的环境下，增加Ceph？那么这个其实比我直接装Ceph更加有挑战。

下面的文档，就是在All in one的基础上，继续安装Ceph，并且实现nova，glance，cinder整合，还支持swift api。

需要强调的是，我们需要把all in one环境里的虚拟机，镜像都删除掉。

 

增加磁盘

我们给虚拟机增加一块磁盘，100G。重启虚拟机，应该就可以看到 sdb

parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1

这条命令，就是对磁盘进行打标签。表示是用来做kolla的ceph。

查看命令的结果。

#  parted /dev/sdb print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 107GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End    Size   File system  Name                      Flags
 1      1049kB  107GB  107GB               KOLLA_CEPH_OSD_BOOTSTRAP

配置Ceph

由于是单节点，all in one的部署，ceph数据只存放1份

创建 /etc/kolla/config/ceph.conf

[global]
osd pool default size = 1
osd pool default min size = 1

全局设置

编辑 /etc/kolla/globals.yml  文件

enable_cinder: "yes"
enable_ceph: "yes"
enable_ceph_rgw: "yes"
enable_ceph_rgw_keystone: "yes"
glance_backend_ceph: "yes"

安装

这个时候，你就可以重复运行kolla安装

kolla-ansible deploy -i /home/all-in-one

目前的版本，装完后，需要把相关的容器重启一遍，或者你把虚拟机重启也是可以的。

镜像格式

由于Ceph要求使用Raw的格式的镜像。所以要把qcow2的镜像进行格式转换。

yum install qemu-img

对于ceph，我们需要上传raw格式，需要对镜像进行转换。

# qemu-img info  cirros-0.3.4-x86_64-disk.img   
image: cirros-0.3.4-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 13M
cluster_size: 65536
Format specific information:
    compat: 0.10

转成raw格式

qemu-img convert -f qcow2 -O raw \
cirros-0.3.4-x86_64-disk.img cirros-0.3.4-x86_64-disk.raw

这时候查看镜像信息

# qemu-img info cirros-0.3.4-x86_64-disk.raw 
image: cirros-0.3.4-x86_64-disk.raw
file format: raw
virtual size: 39M (41126400 bytes)
disk size: 18M

上传镜像

openstack image create --disk-format raw --container-format bare \
--public true --file ./cirros-0.3.4-x86_64-disk.raw

这个时候，你就可以登录Dashboard，进行创建虚拟机。

参考文档

http://www.ibm.com/developerworks/cn/linux/1409_qiaoly_qemuimgages/

附上我同事给我的ceph的管理命令

docker exec -it ceph_mon ceph osd pool ls
docker exec -it ceph_mon ceph osd pool set vms size 1
docker exec -it ceph_mon ceph osd pool set images size 1
docker exec -it ceph_mon ceph osd pool set volumes size 1
docker exec -it ceph_mon ceph osd pool set backups size 1
docker exec -it ceph_mon ceph osd pool set rbd size 1
docker exec -it ceph_mon ceph osd pool set rbd min_size 1
docker exec -it ceph_mon ceph osd pool set backups min_size 1
docker exec -it ceph_mon ceph osd pool set volumes min_size 1
docker exec -it ceph_mon ceph osd pool set images min_size 1
docker exec -it ceph_mon ceph osd pool set vms min_size 1
docker exec -it ceph_mon ceph osd pool ls
docker exec -it ceph_mon ceph -s

  10 Responses to “kolla安装Ceph（all in one）”

  1. watson says:
    2017/03/03 at 3:48 PM

    看样子kolla离fuel还有段距离。

    Reply
  2. neil says:
    2017/03/06 at 10:46 AM

    cinder 后端能接多个不同ceph吗？ 比如不同的物理机组群，安装ceph的版本也不同。

    Reply
  3. weidafei says:
    2017/03/28 at 12:39 PM

    我搭了个实验环境，一台服务器做计算和存储节点，并配置了3个RAD盘分别做CEPH(ceph_osd_2 ceph_osd_1 ceph_osd_0 )共运行3个docker。在实际虚机运行过程中，发现，运行一段时间就会无法访问，类似死机状态，但有的能ping通。
    后查看到ceph_osd_0的docker log日志，发现以下告警：
    {“log”:”2017-03-28 11:16:23.415012 7f94399f8700 0 log_channel(cluster) log [WRN] : slow request 480.153826 seconds old, received at 2017-03-28 11:08:23.261092: osd_op(client.100899.0:1 1.85fca992 (undecoded) ack+read+known_if_redirected e166) currently waiting for active\r\n”,”stream”:”stdout”,”time”:”2017-03-28T03:16:23.415474491Z”}
    {“log”:”2017-03-28 11:17:20.421442 7f94399f8700 0 log_channel(cluster) log [WRN] : 16 slow requests, 1 included below; oldest blocked for \u003e 4561.414427 secs\r\n”,”stream”:”stdout”,”time”:”2017-03-28T03:17:20.421898894Z”}
    后分析，只要 log [WRN] : 16 slow requests,中的16计数告警累加到100时虚机就连不上。这种情况下，我重启 ceph_osd_0这个docker虚机就立刻恢复正常。
    通过iotop工具查了disk IO情况，基本虚机正常开启，不操作状态下disk突发写和读最多100KB/S，没发现什么异常，想请教是否遇见类似情况，如何查找问题和解决，非常感谢

    Reply
  4. nww says:
    2017/06/19 at 3:15 PM

    您好，我的服务器只有一块磁盘，请问有办法安装ceph吗？

    Reply
  5. ftong says:
    2017/10/27 at 4:21 PM

    陈兄，

    安装后发现一个问题：容器cinder_api一直处于restarting状态，日志如下：

    INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
    INFO:__main__:Validating config file
    INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
    INFO:__main__:Copying service configuration files
    INFO:__main__:Deleting file /etc/cinder/cinder.conf
    INFO:__main__:Coping file from /var/lib/kolla/config_files/cinder.conf to /etc/cinder/cinder.conf
    INFO:__main__:Setting file /etc/cinder/cinder.conf owner to cinder:cinder
    INFO:__main__:Setting file /etc/cinder/cinder.conf permission to 0600
    INFO:__main__:Deleting file /etc/httpd/conf.d/cinder-wsgi.conf
    INFO:__main__:Coping file from /var/lib/kolla/config_files/cinder-wsgi.conf to /etc/httpd/conf.d/cinder-wsgi.conf
    INFO:__main__:Setting file /etc/httpd/conf.d/cinder-wsgi.conf owner to cinder:cinder
    INFO:__main__:Setting file /etc/httpd/conf.d/cinder-wsgi.conf permission to 0600
    INFO:__main__:Writing out command to execute
    INFO:__main__:Setting permission for /var/lib/cinder
    INFO:__main__:Setting permission for /var/log/kolla/cinder
    INFO:__main__:Setting permission for /var/log/kolla/cinder/cinder-manage.log
    INFO:__main__:Setting permission for /var/log/kolla/cinder/cinder-scheduler.log
    INFO:__main__:Setting permission for /var/log/kolla/cinder/cinder-volume.log
    INFO:__main__:Setting permission for /var/log/kolla/cinder/cinder-backup.log
    Running command: ‘httpd -DFOREGROUND’
    (13)Permission denied: AH00058: Error retrieving pid file /run/httpd/httpd.pid
    AH00059: Remove it before continuing if it is corrupted.

    这个正常吗？我的是4.0.3。

    Reply

 Leave a Reply

(required)

(required)

This site uses Akismet to reduce spam. Learn how your comment data is processed.