Tuesday, October 15, 2024
Home » Building an autonomous & self-driving data platform

Building an autonomous & self-driving data platform

Enterprises have now embraced the cloud model for IT services. The benefits of on-demand access, agility, and the reduced capital expenditures of public clouds have driven broad acceptance. This extends from public cloud services from the hyperscale vendors along with smaller, more specialized and regional players, as well as enterprise private cloud deployments. Most large enterprises will claim that they deploy a cloud infrastructure within their own data centers, based on VMware virtualization and cloud stacks, OpenStack or other cloud frameworks. Over the last decade, we see that businesses have gained enough experience with private and public clouds that they are now ready for an optimal mix of advantages in a hybrid cloud model; they look forward to full multi-cloud freedom and flexibility. This opens the door to the use of agile and on-demand public cloud services where it is best suited (for burst requirements or specialized computing services, for example), but allows companies to retain control and performance advantages of local private cloud services in their enterprise data centers.

As enterprises transform digitally, we see the emergence of two new trends shaping the future of IT infrastructure: the deployment of IT services as containers, and edge computing.  For new cloud-native applications, containerization (like those services based on Docker) has sparked a new model for deployment of both the apps and the underlying infrastructure as lightweight, distributed (and often stateless) services. Containers and Kubernetes are becoming mainstream for deployment and orchestration models. This is the next wave after virtual machines and cloud orchestration frameworks. The emergence of edge devices in a massive variety of shapes and forms will further increase the demands on IT to support both compute and data requirements coming from the edge. As IDC predicts in its 2019 forecast, communities of smart/connected edge devices (eg, scanners and IoT in factories and hospitals, mobile devices in stadiums, kiosks in airports, entertainment devices in cruise ships, and so on) will create a new tier of “mini clouds” to provide local, dedicated IT services. This will include the need to replicate and stage data from enterprise hybrid cloud environments, and also to cache and store data from edge devices. IDC predicts the emergence of millions of these “service edge” data centers, with the need for localized-scale, compute and data management services, that will leverage new container-based services using Kubernetes for automation.

Finally, we see that businesses will need to manage data to fit their unique processes and requirements from their own data workflows. This gets at the heart of the complexity of data management: to ensure data protection, meet regulatory and data sovereignty requirements, maintain control over data mobility, and face an overarching “data governance” burden on the CIO and the new Chief Data Officers of corporations. This will require new, simplified ways to define and orchestrate business-specific, intelligent, metadata-driven data management policies as part of any new data platform.

To address these trends, Scality is creating a new generation of autonomous self-driving Kubernetes-orchestrated data platforms that can scale seamlessly from a few Gigabyte to Exabytes. With simple customizable data management policies, these solutions will serve a wide variety of data-centric use cases that span from edge devices such as those performing video capture & processing, IoT device logging, healthcare image capture, geo-location tagging, and high-performance AI/ML applications, as well as in private, hybrid, public and multi-cloud architectures.

As enterprise finds new needs in IT services, from edge and IoT processes to containerized solutions, Scality is available to help meet those requirements where and how they’re needed most.

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

About Us

Solved is a digital magazine exploring the latest innovations in Cloud Data Management and other topics related to Scality.

Editors' Picks

Newsletter

Challenges solved, insights delivered, straight to your inbox.

Receive hand-picked articles, case studies, and expert opinions. Keep up with industry innovations and get actionable insights to optimize your strategy.

All Right Reserved. Designed by Scality.com