Archive for May, 2013

OpenLDAP: A comparison of back-mdb and back-hdb performance

May 16, 2013

One of the biggest changes to OpenLDAP in years has made its way into the latest OpenLDAP 2.4 releases, and that is a brand new backend named “back-mdb”.  This new backend leverages the Lightning Memory-Mapped Database from Symas.  To see why this new backend was introduced, it is useful to look at the differences in performance and resource utilization between old BDB based back-hdb and the new LMDB based back-mdb.

Hardware details

  • Dell PowerEdge R710
  • 36GB of RAM
  • ESXi 5.1 hypervisor
  • 2 CPU, 4 cores per CPU, with hyperthreading (16 vCPUs)
  • 1.2 TB RAID array from 4x SEAGATE ST9300603SS 300GB 10kRPM drives
  • Ubuntu12 64-bit OS (3.5.0-28-generic #48~precise1-Ubuntu SMP kernel)
  • LDAP data is stored on its own /ldap partition, using ext2 as the filesystem type
  • ext2 options: noatime,defaults

Software details

  • OpenLDAP 2.4 Engineering from 5/10/2013
  • Berkeley DB 5.2.36 for the back-hdb backend
  • For read tests, slamd 2.0.1 was used to generate results

Data description

For this test, I examined the databases of several of Zimbra’s clients to get an idea of what a typical user entry looked like.  I took the larger average of the common user entries I found between the customer databases to create a template from which to generate purely random data.  Using this template, I generated a 5,000,000 random user database.  The resulting LDIF file was 10GB in size.  By comparison, a large client with 6.3 million entries has a 4.6 GB LDIF, so my user entries are larger than average.

OpenLDAP tuning

Frontend (slapd) specific tuning

  • 16 server threads
  • 2 tool threads (mdb)
  • 16 tool threads (hdb)

MDB tuning

  • writemap
  • nometasync

HDB tuning

  • 31 GB Shared Memory BDB cache for loading the db (slapadd)
  • 22 GB Shared Memory BDB cache for searching (slapd)
  • DN cachesize: unlimited
  • Entry cachesize: 50,000
  • IDL cachesize: 50,000
  • Cachefree: 500

Database Indexing

  • objectClass eq
  • entryUUID eq
  • entryCSN eq
  • cn pres,eq,sub
  • uid pres,eq
  • zimbraForeignPrincipal eq
  • zimbraYahooId eq
  • zimbraId eq
  • zimbraVirtualHostname eq
  • zimbraVirtualIPAddress eq
  • zimbraMailDeliveryAddress eq,sub
  • zimbraAuthKerberos5Realm eq
  • zimbraMailForwardingAddress eq
  • zimbraMailCatchAllAddress eq,sub
  • zimbraMailHost eq
  • zimbraShareInfo sub
  • zimbraMailTransport eq
  • zimbraMailAlias eq,sub
  • zimbraACE sub
  • zimbraDomainName eq,sub
  • mail pres,eq,sub
  • zimbraCalResSite eq,sub
  • givenName pres,eq,sub
  • displayName pres,eq,sub
  • sn pres,eq,sub
  • zimbraCalResRoom eq,sub
  • zimbraCalResCapacity eq
  • zimbraCalResBuilding eq,sub
  • zimbraCalResFloor eq,sub

Notes

You will note that the entry cache and idl cache for HDB does not cover all entries in the database.  To do so would have required more RAM than the server has available.  My best calculation is it would take at least 64GB of total RAM to hold the entry cache and idl cache in full.  Part of the point of this benchmark is to detail the differences in resource requirements between the two backends, and this is where the first immediate difference becomes apparent.  This does not affect load time via slapadd, but may affect read rate.

Database resource usage and load times

MDB load time

  • Total time to load the database using slapadd -q: 59m3.377s
  • Database size: 26GB

HDB load time

  • Total time to load the database using slapadd -q: 108m57.655s
  • Database size: 30GB in .bdb files, plus shared memory key (55GB or 61GB depending on “run” vs “load” configuration)

MDB is able to load the DB 1.847 times faster than HDB.  MDB consumes 2.11 times fewer resources than HDB.

Read rates

SLAMD was used for generating load against the different backends.  For the purposes of this benchmark, there were 16 slamd clients configured to generate load against the LDAP server.  The slamd clients would start with a single thread generating load for 20 minutes, logging the total read rate.  The number of threads per client would then be incremented by 1, and the same 20 minute interval measured again.  After 3 intervals of no increase in read rate, the test run would stop.  Prior to running an optimizing search job, the full database was iterated through in a slamd job to ensure all data was loaded into memory and off disk so that comparisons were fair between the two backends.  The following two search filters were used in an even split for searches.  I chose these search filters as they are what our postfix service uses to find users for mail delivery, so reflect real-world usage.

  • (&(zimbraMailDeliveryAddress=user.[1-5000000]@zre-ldap001.eng.vmware.com)(zimbraMailStatus=enabled))
  • (&(zimbraMailAlias=user.[1-5000000]@zre-ldap001.eng.vmware.com)(zimbraMailStatus=enabled))

MDB read rates

  • Optimal search rate reached with 16 slamd clients, 23 threads per client, for a total of 368 querying clients
  • Read rate is 61,433 searches/second
  • Slamd report

HDB read rates

  • Optimal search rate was reached with 16 slamd clients, 4 threads per client, for a total of 64 querying clients
  • Read rate is 20,818.336 searches/second
  • Slamd report

MDB is 295% faster than HDB.  In addition, MDB scales with high read rates for a greater amount of querying clients than HDB.

Conclusions

MDB outperforms HDB in all areas tested.  MDB consumes significantly fewer resources than HDB, allowing for better utilization of server resources.  MDB scales better than HDB, allowing it to server more clients per server.