Ceph¶
RBD
RADOS Block Device (RBD)¶
The Ceph RBD driver registers a driver named rbd
with the libStorage
driver
manager and is used to connect and mount RADOS Block Devices from a Ceph
cluster.
Requirements¶
- The
ceph
andrbd
binary executables must be installed on the host - The
rbd
kernel module must be installed - A
ceph.conf
file must be present in its default location (/etc/ceph/ceph.conf
) - The ceph
admin
key must be present in/etc/ceph/
Configuration¶
The following is an example with all possible fields configured. For a running example see the Examples section.
rbd:
defaultPool: rbd
testModule: true
Configuration Notes¶
- The
defaultPool
parameter is optional, and defaults to "rbd". When set, all volume requests that do not reference a specific pool will use thedefaultPool
value as the destination storage pool. - The
testModule
parameter is optional, and defaults to "true". This setting indicates whether the libStorage client should test if therbd
kernel module is loaded, with the side-effect of loading it if is not already loaded. This setting should be disabled when the driver is executing inside of a container.
Runtime behavior¶
The Ceph RBD driver only works when the client and server are on the same node.
There is no way for a centralized libStorage
server to attach volumes to
clients, therefore the libStorage
server must be running on each node that
wishes to mount RBD volumes.
The RBD driver uses the format of <pool>.<name>
for the volume ID. This allows
for the use of multiple pools by the driver. During a volume create, if the
volume ID is given as <pool>.<name>
, a volume named name will be created in
the pool storage pool. If no pool is referenced, the defaultPool
will be
used.
Both pool and name may only contain alphanumeric characters, underscores, and dashes.
When querying volumes, the driver will return all RBDs present in all pools in
the cluster, prefixing each volume with the appropriate <pool>.
value.
All RBD creates are done using the default 4MB object size, and using the "layering" feature bit to ensure greatest compatibility with the kernel clients.
Activating the Driver¶
To activate the Ceph RBD driver please follow the instructions for
activating storage drivers, using rbd
as the
driver name.
Troubleshooting¶
- Make sure that
ceph
andrbd
commands work without extra parameters for ID, key, and monitors. All configuration must come fromceph.conf
. - Check status of the ceph cluster with
ceph -s
command.
Examples¶
Below is a full config.yml
that works with RBD
libstorage:
# The libstorage.service property directs a libStorage client to direct its
# requests to the given service by default. It is not used by the server.
service: rbd
server:
services:
rbd:
driver: rbd
rbd:
defaultPool: rbd
Caveats¶
- Snapshot and copy functionality is not yet implemented
- libStorage Server must be running on each host to mount/attach RBD volumes
- There is not yet options for using non-admin cephx keys or changing RBD create features
- Volume pre-emption is not supported. Ceph does not provide a method to forcefully detach a volume from a remote host -- only a host can attach and detach volumes from itself.
- RBD advisory locks are not yet in use. A volume is returned as "unavailable" if it has a watcher other than the requesting client. Until advisory locks are in place, it may be possible for a client to attach a volume that is already attached to another node. Mounting and writing to such a volume could lead to data corruption.