Measure your radosgw usage with OpenStack-Ansible
07/26/18
We use OpenStack-Ansible to set up our OpenStack cluster and Ceph’s Rados Gateway (radosgw) as object store backend. Unfortunately, the telemetry (and in consequence accounting) for radosgw will not work out of the box. You need to change different parts of your OpenStack and Ceph setup to get it up and running. This blog post will explain what to change and where to change it.
Ceilometer needs additional pip packages to talk to Ceph Rados Gateway. To install it, edit the default ceilometer_pip_packages in your user_variables.yml file:
ceilometer_pip_packages: - ceilometer - ceilometermiddleware - cryptography - gnocchiclient - libvirt-python - PyMySQL - pymongo - python-ceilometerclient - python-memcached - tooz - warlock - requests-aws>=0.1.4 #https://github.com/openstack/ceilometer/blob/stable/pike/test-requirements.txt |
You also have to configure Ceilometer to actually query radosgw. When your ceilometer isn’t configured to poll everything, add these pollsters to your polling.yml file:
- name: radosgw_pollsters interval: 1200 meters: - rgw.containers.objects - rgw.containers.objects.size - rgw.objects - rgw.objects.size - rgw.objects.containers - rgw.usage |
Be aware, these names changed with the Queens release. Add them also to your pipeline:
- name: radosgw_source interval: 60 meters: - "rgw.objects" - "rgw.objects.size" - "rgw.objects.containers" - "rgw.api.request" - "rgw.containers.objects" - "rgw.containers.objects.size" sinks: - meter_sink |
Declare Ceph Rados Gateway as object-store in your ceilometer.conf file:
[service_types] radosgw = object-store [rgw_admin_credentials] access_key = XXX secret_key = XXX |
This can be done with OpenStack-Ansible’s user_variables.yml file:
ceilometer_ceilometer_conf_overrides: service_types: radosgw: object-store rgw_admin_credentials: access_key: XXX secret_key: XXX |
The required credentials are created with these commands:
radosgw-admin user create --uid admin --display-name "admin user" --caps "usage=read,write;metadata=read,write;users=read,write;buckets=read,write" |
To get your credentials, execute:
radosgw-admin user info --uid admin | jq '.keys' |
That was all for the Ceilometer part, but you also need to change some configs for radosgw. The required changes are described in the documentation of Ceilometer.
These are the additions to your ceph.conf file on your radosgw servers:
[client.radosgw.gateway] rgw enable usage log = true rgw usage log tick interval = 30 rgw usage log flush threshold = 1024 rgw usage max shards = 32 rgw usage max user shards = 1 |
After some polling, you should see radosgw metrics coming in:
# openstack metric measures show --resource-id XXX radosgw.objects.size +---------------------------+-------------+-------+ | timestamp | granularity | value | +---------------------------+-------------+-------+ | 2018-06-07T12:00:00+00:00 | 300.0 | 1.0 | | 2018-06-07T12:05:00+00:00 | 300.0 | 1.0 | | 2018-06-07T12:10:00+00:00 | 300.0 | 1.0 | ... |
I hope this helped set up your ceilometer accounting for radosgw.
