Snowflake Data Cloud is more than just a piece of technology, Snowflake senior vice president of product Christian Kleinerman told iTWire.
The company describes it as "an ecosystem where thousands of Snowflake customers, partners, data providers, and data service providers can break down data silos, derive insights, and deliver value from rapidly growing data sets in secure, governed, compliant, and seamless ways."
It's about accessing your data in context, Kleinerman explained. For instance, information about the growth of your business in various countries is more meaningful when GDP data from those countries is included in the analysis.
Data Cloud makes it simple to enrich your data with data sets from other sources for "better data-driven insights," he said.
While that's not a new idea, the friction involved in the process meant it rarely happened in practice. But Data Cloud removes that friction by providing the technology that enables data exchange and a data marketplace open to all customers as buyers and sellers, plus making provision for customers to operate their own data exchanges, he added.
"Organisations have historically struggled to fully mobilise data in the service of their business," Snowflake CEO Frank Slootman said.
"Snowflake was specifically architected to leverage the incredible scale and computing power of the cloud. As part of the Data Cloud, organisations fully mobilise their data by blending and joining data with broader context, giving them the power to achieve crucial insights beyond what has previously been possible."
Snowflake also announced new features for the Snowflake Cloud Data Platform, which is used by directly by customers as well as underpinning the Data Cloud.
Snowsight is a revamped analyst experience within Snowflake, Kleinerman said. It includes SQL auto-completion, schema browsing, the ability to share queries with colleagues, simple visualisations in the context of a query, and dashboards for a streamlined experience.
Search Optimization Service carries out precomputations to accelerate columnar lookups. This gives "a massive speed boost" for tasks such as log analysis, Kleinerman said.
Dynamic data masking provides policy-based masking, so appropriately authorised users might see complete email addresses or credit card numbers, while others might only see the last five digits or the email domain.
A related feature is integration with third-party tokenisation solutions for increased protection of sensitive data.
External functions allow Snowflake to call external services for richer query support and to build robust data pipelines that integrate with third-party libraries or services such as the AWS Lambda serverless platform.
Java UDFs allow Snowflake to run business logic developed in Java. Support for Python and other languages is also under development.
Other changes include provision for centrally operating and managing multiple Snowflake accounts (even across multiple clouds), private data exchanges to share live, governed data with others, and Snowflake Data Marketplace to discover and access third-party data sets without copying or moving the data.
Some of these new features are available immediately, some will be rolled out over the next week or two, and some are in public preview. The exception is support for business logic and libraries written in Java, Python and other languages – this is in early preview and is not expected to be generally available for a few months.
New compute cluster sizes have been added. The 5XL is twice the capacity of 4XL (previously the largest size), and 6XL is twice as big again. The new sizes are only needed for workloads processing hundreds of terabytes of data in short periods of time, according to Kleinerman.
"The feature enhancements we're announcing today will help companies unify, integrate, analyse, and share virtually any amount of data, eliminating the complexity and friction of alternative solutions," he said.
"In addition, the Snowflake Cloud Data Platform gives our customers access to the Data Cloud, so they can achieve new levels of success as members of a global ecosystem of data consumers, providers, and service providers."