Measure your radosgw usage with OpenStack-Ansible

No Comments

We use OpenStack-Ansible to set up our OpenStack cluster and Ceph’s Rados Gateway (radosgw) as object store backend. Unfortunately, the telemetry (and in consequence accounting) for radosgw will not work out of the box. You need to change different parts of your OpenStack and Ceph setup to get it up and running. This blog post will explain what to change and where to change it.

Ceilometer needs additional pip packages to talk to Ceph Rados Gateway. To install it, edit the default ceilometer_pip_packages in your user_variables.yml file:

ceilometer_pip_packages:
- ceilometer
- ceilometermiddleware
- cryptography
- gnocchiclient
- libvirt-python
- PyMySQL
- pymongo
- python-ceilometerclient
- python-memcached
- tooz
- warlock
- requests-aws>=0.1.4 #https://github.com/openstack/ceilometer/blob/stable/pike/test-requirements.txt

You also have to configure Ceilometer to actually query radosgw. When your ceilometer isn’t configured to poll everything, add these pollsters to your polling.yml file:

- name: radosgw_pollsters
    interval: 1200
    meters:
    -  rgw.containers.objects
    -  rgw.containers.objects.size
    -  rgw.objects
    -  rgw.objects.size
    -  rgw.objects.containers
    -  rgw.usage

Be aware, these names changed with the Queens release. Add them also to your pipeline:

- name: radosgw_source
    interval: 60
    meters:
        - "rgw.objects"
        - "rgw.objects.size"
        - "rgw.objects.containers"
        - "rgw.api.request"
        - "rgw.containers.objects"
        - "rgw.containers.objects.size"
    sinks:
        - meter_sink

Declare Ceph Rados Gateway as object-store in your ceilometer.conf file:

[service_types]
radosgw = object-store
 
[rgw_admin_credentials]
access_key = XXX
secret_key = XXX

This can be done with OpenStack-Ansible’s user_variables.yml file:

ceilometer_ceilometer_conf_overrides:
 service_types:
   radosgw: object-store
rgw_admin_credentials:
   access_key: XXX
   secret_key: XXX

The required credentials are created with these commands:

radosgw-admin user create --uid admin --display-name "admin user" --caps "usage=read,write;metadata=read,write;users=read,write;buckets=read,write"

To get your credentials, execute:

radosgw-admin user info --uid admin | jq '.keys'

That was all for the Ceilometer part, but you also need to change some configs for radosgw. The required changes are described in the documentation of Ceilometer.

These are the additions to your ceph.conf file on your radosgw servers:

[client.radosgw.gateway]
   rgw enable usage log = true
   rgw usage log tick interval = 30
   rgw usage log flush threshold = 1024
   rgw usage max shards = 32
   rgw usage max user shards = 1

After some polling, you should see radosgw metrics coming in:

# openstack metric measures show --resource-id XXX radosgw.objects.size
+---------------------------+-------------+-------+
| timestamp                 | granularity | value |
+---------------------------+-------------+-------+
| 2018-06-07T12:00:00+00:00 |       300.0 |   1.0 |
| 2018-06-07T12:05:00+00:00 |       300.0 |   1.0 |
| 2018-06-07T12:10:00+00:00 |       300.0 |   1.0 |
...

I hope this helped set up your ceilometer accounting for radosgw.

Christian Zunker

Christian Zunker is creating and operating distributed software systems and infrastructures. In his current role as a System Engineer, he builds OpenStack based clouds for cc cloud GmbH and its customers.

Daniel Marks is creating and operating complex distributed software systems and infrastructures. In his current role as Senior Cloud Engineer, he builds OpenStack-based clouds for codecentric and its customers.

Comment

Your email address will not be published. Required fields are marked *