In today’s data-driven world, businesses face a constant struggle between scaling analytics and controlling costs. For our most recent customer, a lean company dealing with massive data volumes, the challenge was real and costly. Using traditional HDFS and Elasticsearch clusters, they were hitting the limits of both management complexity and hardware budgets. That’s when Quantrail and Bare-Metal.io stepped in, turning a daunting migration into a seamless, cost-saving success story.
The Status Quo: Complex, Costly, and Hard to Scale
Customer’s setup looked impressive on paper:
- ~20 Hadoop nodes with 24 physical cores/48 Logical cores, 128 GB RAM, and 150 TB storage.
- 50 Elasticsearch nodes with 16 physical cores/32 Logical Cores, 128 GB RAM, and ~15 TB storage.
The data pipeline periodically aggregated records from HDFS into Elasticsearch, which then powered consumer-facing applications and custom reporting.
But beneath the surface, cracks were showing:
- Rising hardware costs – With data projected to grow 5x, the infrastructure costs were spiraling.
- High complexity – Managing 70 nodes across Hadoop and Elasticsearch was resource-intensive.
- Fragile batch jobs – Aggregations often ran over weekends, and failures meant lost time and wasted compute.
- Blocked innovation – The in-house team was overloaded, leaving no bandwidth to explore modern technologies.
clickhouse - Long migration timelines – A rough internal estimate suggested ~6 months just to plan and execute a move.
The writing was on the wall: Customer needed a simpler, faster, and more cost-effective solution.
Enter Quantrail: Specialists Who Make open-source ClickHouse® Easy
Enter Quantrail, a team of open-source ClickHouse® specialists. Instead of incremental patches, we offered Customer a clear migration roadmap with guaranteed timelines.
Here’s what Quantrail delivered:
- End-to-end ownership – From open-source ClickHouse® installation to schema migration, indexing, optimizations, query benchmarking, cluster topology, and monitoring.
- Hands-free migration – Customer’s team stayed focused on their daily tasks while Quantrail executed the transition.
- Faster timelines — The migration finished in just 4 months-two months ahead of schedule.
- Robust documentation – Everything was tracked in Confluence, SQL queries were version-controlled in Git.
- Workflow automation – Over 50 Airflow DAGs automated reporting pipelines, replacing fragile weekend batch jobs.
- Training & knowledge transfer – Customized sessions equipped the Customer’s team to manage and extend the new setup.
- Future-proof scaling – Load testing confirmed the cluster could handle 12x ingestion, ensuring years of runway.
With open-source ClickHouse® at the core, Customer suddenly had a leaner, more reliable, and high-performance analytics stack.
The bare-metal.io Advantage: Bigger, Faster, Cheaper
The next step was infrastructure. Running open-source ClickHouse® on AWS would have been costly:
- AWS m6a.metal instance (192 vCPU, 768GB RAM, 84 TB EBS (SSD – GP3) storage with 16,000 IOPS) costs $10773 per month per node. (1 year commitment)
- A 4-node production cluster would total ~ $43,000 per month.
- On top of that, AWS charges ~$50 per TB for outbound data transfer (egress costs).
Cost Calculator – https://calculator.aws/#/estimate?id=9ccbfa7a2ddceeacb40d9db184b147e1456e771d
Instead, Customer went with bare-metal.io, who provided enterprise-grade servers at a fraction of the cost:
- 128 physical cores/ 256 Logical cores, 1 TB RAM, 84 TB SSD storage per node with unlimited IOPS.
- 4 nodes (2 shards + 2 replicas), designed for 12x ingestion rates.
- All at ~40% of AWS’s cost.
- And with zero egress costs.
Even better, open-source ClickHouse®’s advanced compression meant storage requirements shrank significantly, allowing the Customer to store more data with less hardware.
The result? 55-60% cost savings compared to AWS-while actually gaining performance headroom.
The Outcome: Leaner, Faster, Future-Proof
By partnering with Quantrail and bare-metal.io, Customer achieved what once seemed impossible:
- From 70+ nodes to 4 nodes – A radical simplification of infrastructure.
- Massive cost savings – Up to 60% lower spend compared to AWS.
- Fuss-free migration – Completed ahead of schedule with minimal disruption.
- Automation-first reporting – Airflow DAGs replaced brittle weekend jobs.
- Future-ready architecture – Load tested for 12x growth in ingestion.
Key Takeaways
- Quantrail takes the stress out of open-source ClickHouse® transition: Full-service migration, optimization, and training.
- Simplified architecture: Multi-node Hadoop + Elasticsearch clusters replaced by a 4-node open-source ClickHouse® cluster.
- Major cost reduction: ~55-60% savings versus equivalent AWS infrastructure.
- Future-ready: Cluster load-tested for 12x ingestion, ensuring long-term scalability.
Fully automated reports: Airflow-based reporting with ~50 DAGs for seamless operations.
Conclusion
Customer’s story is proof that moving to open-source ClickHouse® doesn’t have to be risky, slow, or expensive. With Quantrail’s open-source ClickHouse® expertise and bare-metal.io’s cost-effective infrastructure, the transition was seamless, stress-free, and future-ready.
For companies struggling with rising hardware costs, operational complexity, and technology adoption barriers, Quantrail + Bare-Metal.io provides a seamless, cost-effective, and future-proof open-source ClickHouse® solution.
This case study proves that even underdogs can outmaneuver the industry giants with the right strategy, expertise, and infrastructure.
Disclaimer: The customer name has been withheld due to NDA. A reference can be provided at the time of customer verification before signing up with us, upon request.
Useful Links
Home page – bare-metal.io
Services – https://bare-metal.io/services/
ClickHouse® Managed Services – https://bare-metal.io/services/clickhouse-consulting-managed-services/
Quantrail Services – https://quantrail-data.com/clickhouse-services/
Contact – https://quantrail-data.com/contact/