What is Qbeast
Qbeast is a way to accelerate lakehouses by optimizing the data layout, significantly impacting the performance of analytics queries. Incorporating various aspects from advanced algorithms to cloud integration, Qbeast leverages advanced indexing and sophisticated sampling techniques to organize data, targeting the speed-up of analytics and AI workloads while significantly reducing computing costs.
Performance Benefits
Qbeast delivers substantial performance improvements:
- Data Ingestion: 58% faster
- Query Execution: 31% faster
Qbeast enables queries to process 70% less data, significantly accelerating execution times and reducing both computing and storage costs, especially beneficial for large-scale cloud data operations.
Key Features
- Effective Sampling: Provides quick, approximate query results without loading entire datasets.
- Multi-dimensional Indexing: Uses a tree structure to efficiently organize and query data.
- Automatic Layout Optimization: Continuously monitors and optimizes data distribution and indexing.
- Format-agnostic: Works with various table formats, enhancing Delta Lake performance and planning support for other formats like Snowflake, Iceberg, and Hudi.
- Open-source and Cloud Support: Available as an open-source project with initial deployments on AWS and plans for Google Cloud support.
Qbeast Core Technology
Qbeast integrates with Apache Spark through “qbeast-spark” to enhance data management and query performance. It partitions data into smaller cubes organized into a tree structure, ensuring efficient queries by targeting relevant data segments. Automatic indexing and integration of new data streamline data management, reducing the administrative overhead for data engineers.
Getting Started
Open-source
At Qbeast, we embrace open source for its transparency and collaborative benefits. Sharing our code invites developers and data scientists from around the world to contribute, driving continuous innovation and better security.
Stay ahead with the Qbeast community!
Connect on Slack
Subscribe to Our YouTube Channel