07/26/18 by Christian Zunker and Daniel Marks
No Comments
We use OpenStack-Ansible to set up our OpenStack cluster and Ceph’s Rados Gateway (radosgw) as object store backend. Unfortunately, the telemetry (and in consequence accounting) for radosgw will not work out of the box. You need to change different parts of your OpenStack and Ceph setup to get it up and running. This blog post will explain what to change and where to change it.
Ceilometer needs additional pip packages to talk to Ceph Rados Gateway. To install it, edit the default ceilometer_pip_packages in your user_variables.yml file:
ceilometer_pip_packages:
- ceilometer
- ceilometermiddleware
- cryptography
- gnocchiclient
- libvirt-python
- PyMySQL
- pymongo
- python-ceilometerclient
- python-memcached
- tooz
- warlock
- requests-aws>=0.1.4 #https://github.com/openstack/ceilometer/blob/stable/pike/test-requirements.txt |
ceilometer_pip_packages: - ceilometer - ceilometermiddleware - cryptography - gnocchiclient - libvirt-python - PyMySQL - pymongo - python-ceilometerclient - python-memcached - tooz - warlock - requests-aws>=0.1.4 #https://github.com/openstack/ceilometer/blob/stable/pike/test-requirements.txt
You also have to configure Ceilometer to actually query radosgw. When your ceilometer isn’t configured to poll everything, add these pollsters to your polling.yml file:
- name: radosgw_pollsters
interval: 1200
meters:
- rgw.containers.objects
- rgw.containers.objects.size
- rgw.objects
- rgw.objects.size
- rgw.objects.containers
- rgw.usage |
- name: radosgw_pollsters interval: 1200 meters: - rgw.containers.objects - rgw.containers.objects.size - rgw.objects - rgw.objects.size - rgw.objects.containers - rgw.usage
Be aware, these names changed with the Queens release. Add them also to your pipeline:
- name: radosgw_source
interval: 60
meters:
- "rgw.objects"
- "rgw.objects.size"
- "rgw.objects.containers"
- "rgw.api.request"
- "rgw.containers.objects"
- "rgw.containers.objects.size"
sinks:
- meter_sink |
- name: radosgw_source interval: 60 meters: - "rgw.objects" - "rgw.objects.size" - "rgw.objects.containers" - "rgw.api.request" - "rgw.containers.objects" - "rgw.containers.objects.size" sinks: - meter_sink
Declare Ceph Rados Gateway as object-store in your ceilometer.conf file:
[service_types]
radosgw = object-store
[rgw_admin_credentials]
access_key = XXX
secret_key = XXX |
[service_types] radosgw = object-store [rgw_admin_credentials] access_key = XXX secret_key = XXX
This can be done with OpenStack-Ansible’s user_variables.yml file:
ceilometer_ceilometer_conf_overrides:
service_types:
radosgw: object-store
rgw_admin_credentials:
access_key: XXX
secret_key: XXX |
ceilometer_ceilometer_conf_overrides: service_types: radosgw: object-store rgw_admin_credentials: access_key: XXX secret_key: XXX
The required credentials are created with these commands:
radosgw-admin user create --uid admin --display-name "admin user" --caps "usage=read,write;metadata=read,write;users=read,write;buckets=read,write" |
radosgw-admin user create --uid admin --display-name "admin user" --caps "usage=read,write;metadata=read,write;users=read,write;buckets=read,write"
To get your credentials, execute:
radosgw-admin user info --uid admin | jq '.keys' |
radosgw-admin user info --uid admin | jq '.keys'
That was all for the Ceilometer part, but you also need to change some configs for radosgw. The required changes are described in the documentation of Ceilometer.
These are the additions to your ceph.conf file on your radosgw servers:
[client.radosgw.gateway]
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024
rgw usage max shards = 32
rgw usage max user shards = 1 |
[client.radosgw.gateway] rgw enable usage log = true rgw usage log tick interval = 30 rgw usage log flush threshold = 1024 rgw usage max shards = 32 rgw usage max user shards = 1
After some polling, you should see radosgw metrics coming in:
# openstack metric measures show --resource-id XXX radosgw.objects.size
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2018-06-07T12:00:00+00:00 | 300.0 | 1.0 |
| 2018-06-07T12:05:00+00:00 | 300.0 | 1.0 |
| 2018-06-07T12:10:00+00:00 | 300.0 | 1.0 |
... |
# openstack metric measures show --resource-id XXX radosgw.objects.size +---------------------------+-------------+-------+ | timestamp | granularity | value | +---------------------------+-------------+-------+ | 2018-06-07T12:00:00+00:00 | 300.0 | 1.0 | | 2018-06-07T12:05:00+00:00 | 300.0 | 1.0 | | 2018-06-07T12:10:00+00:00 | 300.0 | 1.0 | ...
I hope this helped set up your ceilometer accounting for radosgw.
