Benchmarks
The following tables and charts provide the results of comparing
- The BaseX XQJ Remote Driver
- The BaseX XQJ Remote Driver + XQPool (Connection Pooling)
Single Threaded. Connect, Query and Close.
In this test, the
XQDataSource.getConnection() method is invoked n times.
Each time, an XQuery is executed via the XQConnection, which is then
closed. Everything executes sequentially and is not run in parallel.
| DataSource Type | Runs | Total (seconds) | Average (millis) | Memory Used (KB) |
|---|---|---|---|---|
| BaseX XQJ Remote | 100 | 0.249 | 2.49 | 7,962 |
| BaseX XQJ Remote + Pooling | 100 | 0.041 | 0.41 | 3,313 |
| BaseX XQJ Remote | 1,000 | 0.854 | 0.85 | 46,080 |
| BaseX XQJ Remote + Pooling | 1,000 | 0.266 | 0.27 | 5,896 |
| BaseX XQJ Remote | 10,000 | 5.992 | 0.6 | 40,960 |
| BaseX XQJ Remote + Pooling | 10,000 | 2.01 | 0.2 | 27,704 |
| BaseX XQJ Remote | 100,000 | 47.12 | 0.47 | 17,920 |
| BaseX XQJ Remote + Pooling | 100,000 | 17.484 | 0.17 | 10,532 |
Single Threaded. Connect and Close.
In this test, the
XQDataSource.getConnection() method is invoked n times.
The connection is then immediately closed.
Everything executes sequentially and is not run in parallel.
This test primarily tries to show the overhead required in creating a connection in a single threaded environment.
| DataSource Type | Runs | Total (seconds) | Average (millis) | Memory Used (KB) |
|---|---|---|---|---|
| BaseX XQJ Remote | 100 | 0.104 | 1.04 | 5,300 |
| BaseX XQJ Remote + Pooling | 100 | 0.01 | 0.1 | 2,652 |
| BaseX XQJ Remote | 1,000 | 0.581 | 0.58 | 43,008 |
| BaseX XQJ Remote + Pooling | 1,000 | 0.025 | 0.02 | 3,290 |
| BaseX XQJ Remote | 10,000 | 4.192 | 0.42 | 26,048 |
| BaseX XQJ Remote + Pooling | 10,000 | 0.103 | 0.01 | 11,370 |
| BaseX XQJ Remote | 100,000 | * FAILED | ||
| BaseX XQJ Remote + Pooling | 100,000 | 0.293 | 0 | 7,183 |
Multi-Threaded. Connect, Query and Close.
This test is similar to the Single Threaded (Connect, Query and Close) test. The difference being that this test is executed 6 times in parallel. So "100 runs" is actually "6 x 100" runs, where each of the 6 tests run in parallel.
| DataSource Type | Runs | Total (seconds) | Average (millis) | Memory Used (KB) |
|---|---|---|---|---|
| BaseX XQJ Remote | 100 | 0.327 | 0.55 | 32,768 |
| BaseX XQJ Remote + Pooling | 100 | 0.07 | 0.12 | 15,913 |
| BaseX XQJ Remote | 1,000 | 1.532 | 0.26 | 15,752 |
| BaseX XQJ Remote + Pooling | 1,000 | 0.529 | 0.09 | 29,062 |
| BaseX XQJ Remote | 10,000 | * FAILED | ||
| BaseX XQJ Remote + Pooling | 10,000 | 5.037 | 0.08 | 29,800 |
| BaseX XQJ Remote | 100,000 | * FAILED | ||
| BaseX XQJ Remote + Pooling | 100,000 | 49.871 | 0.08 | 18,588 |
Multi-Threaded. Connect and Close.
This test is similar to the Single Threaded (Connect and Close) test. The difference being that this test is executed 6 times in parallel. So "100 runs" is actually "6 x 100" runs, where each of the 6 tests run in parallel.
This test primarily tries to show the overhead required in creating a connection in a multi-threaded environment.
| DataSource Type | Runs | Total (seconds) | Average (millis) | Memory Used (KB) |
|---|---|---|---|---|
| BaseX XQJ Remote | 100 | 0.215 | 0.36 | 31,856 |
| BaseX XQJ Remote + Pooling | 100 | 0.018 | 0.03 | 15,913 |
| BaseX XQJ Remote | 1,000 | 1.234 | 0.21 | 4,277 |
| BaseX XQJ Remote + Pooling | 1,000 | 0.048 | 0.01 | 10,258 |
| BaseX XQJ Remote | 10,000 | * FAILED | ||
| BaseX XQJ Remote + Pooling | 10,000 | 0.181 | 0 | 55,296 |
| BaseX XQJ Remote | 100,000 | * FAILED | ||
| BaseX XQJ Remote + Pooling | 100,000 | 0.272 | 0 | 26,004 |
* FAILED:
java.net.SocketTimeoutException thrown with message connect timed out, this is almost certainly due to far too many Socket connections being created by the Operating System.
Environment used while testing
- Using XQJ implementation
net.xqj.basex.BaseXXQDataSource
. - Connecting to BaseX running on localhost (no network latency).
- OS: Mac OS X 10.9.1
- Hardware: 2.7 GHz Intel Core i7. 16 GB 1600 MHz DDR3 memory.
- JVM: Java 1.7.0_45
Conclusion
Using an XQJ Connection Pool Mechanism in conjunction with (in this case) the BaseX XQJ Remote driver increases performance and scalability. It is ideal for high through-put environments where fast response times are critical.
With an XQJ Connection Pool, the overhead for creating connections is reduced. In fact, the harder you push it, the faster it gets.
Running your own tests
The source files used to find these statistics can be are located on GitHub.