The Artirix team have recently undertaken a number of tests to compare a commercial search engine technology we have in place for various customers, versus Lucene (Solr build 1.4.0) with some interesting results.

The objective was to gauge the QPS and search latency rates on the same hardware ( 2x Dell 2950s Intel Quad Core Xenon 2.33GHz, 20GB Mem), with the same record sets (a set of 873179 real-estate property records), similar architecture setup – against a sample set of real search requests taken from 1 hr of traffic.

Lucene results

1 thread: 17.9 QPS peak, median response 23.6ms

6 threads: 83.7 QPS peak, median response 23.4ms

10 threads: 79.2 QPS, median response 124.9ms

“Commercial search engine” results

1 thread: 6.1 QPS, median response 162.6ms

6 threads: 23.6 QPS, median response 252.9ms

10 threads: 27.4 QPS, median response 363.5ms

A few conclusions…

• Lucene can perform 3x better under load, in particular latency is impressive with a 10x increase!
• Robust testing with simultaneous index updates, and querying showed consistent results on Lucene