Quantcast
Channel: Blog | Dell
Viewing all articles
Browse latest Browse all 17822

Benchmarking Stream Performance With Spring XD 1.2 and Apache Kafka

$
0
0
EMC logo

sfeatured-XD-KafkaOne of goals for the Spring XD 1.2 release was to obtain the baseline performance metrics on a typical cluster of machines and then optimize stream performance where necessary. Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. Our testing drove several optimizations to increase streaming performance. The benchmarks found that a single, single threaded Spring XD stream can handle over 2 million (100 byte) events a second, using Apache Kafka as a transport. If the business case allows the stream to run on one container, XD can handle ~13 million (100 byte) events a second using direct (in-memory) binding.
To gain a perspective on these results, the graph below shows XD’s performance in megabytes per second, compared to Kafka’s own performance benchmarks with various message sizes.

image03

image01

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This graph shows that a customer will get all of the benefits of Spring XD with near-native performance for the underlying transport. Spring XD 1.2 uses the new Spring Integration Kafka adapter, which offers a richer feature set than the standard Kafka client library. The other takeaway from our tests is that these results were obtained using Spring XD out of the box, requiring  minimal configuration to set up the Kafka batch size, mode, and the connectivity to the Kafka transport.

For anyone interested in reproducing these numbers, please refer to the engineering blog, which describes in detail the tests performed and infrastructure used.

Learn more:


Viewing all articles
Browse latest Browse all 17822

Trending Articles