Barcelona Supercomputing Center spinout Qbeast has created an open-source solution to better align data lakes — server-side repositories of raw, unprocessed data — so they can be deployed later on for operational machine learning.
Despite dealing with raw data, data lake standards diverge a fair bit in terms of the other protocols
On the same side of the process, clients may look at several data warehouses that in some cases would meet their needs, so long as they need to action immediate business intelligence. They'll need a solution for raw data if they want to rapidly store live information for as yet unforeseen ML workloads, in which case a warehouse mightn't fit the bill.
As a result, engineering teams are faced with a crowded marketplace of data lake and warehousing solutions, with the unfortunate knowledge that further changes will exhaust further operational resource.
Qbeast CEO Cesare Cugnasco commented: "Companies dealing with data have little choice: if they want to know what is happening in their business, they need to use a data warehouse. While if they want to predict the future, optimize its processes and use machine learning, they need a data lake.
"They end up using different technologies and needing different people, but also with double the cloud bill and double the time to develop, which is a huge problem."
Qbeast's open source licence is claimed to be the most advanced for building and managing data lakes.
It's hoped that engendering a development community can garner trust among data engineers and lead to a single, reliable format for building data lake infrastructure. With a single format, perhaps less friction changing the stack provider.
In terms of data execution, Qbeast cites a 68% improvement around timeframes, and up to 50 times enhanced sampling workloads. An unnamed cybersecurity company has already paid for a subscription, while Qbeast's ongoing collaborations also include US language learning platform Preply which hopes to support its machine learning model training.
For its €2.5 million seed round of funding, Qbeast drew funds from Uber's founding CTO Oscar Salazar, Inveready and Spanish bank Banco Sabadell's Venture Capital unit. The round was led by French VC firm Elaia.
Banco Sabadell's accelerator-stage venture fund, BStartup Banco Sabadell, and unnamed angel investors had backed Qbeast in earlier funding rounds.