The Convergence of MLOps and DevOps: A New Era in AI Deployment
Written on
Chapter 1: Understanding the AI/ML Trend
The rapid evolution of AI and machine learning (ML) is transforming various applications at an unprecedented pace. Unlike past technological advancements, such as blockchain, which developed tools and infrastructure alongside their applications, AI/ML has leapt directly into practical use. This swift transition has left the supporting tools and infrastructure somewhat behind.
While numerous tools for model development and tuning have emerged, the transition of these models into production is a relatively recent challenge. The limited availability of MLOps tools, many of which were created before the surge in AI/ML popularity, raises a crucial question: Are we crafting sustainable solutions or merely improvising our way to production? It seems we have reached a pivotal point—one that merges the realms of DevOps and MLOps.
Currently, MLOps, akin to DevOps a decade ago, is a hot topic, with technology firms eager to adopt these methodologies. However, as I delve deeper, I increasingly view MLOps as a fleeting trend. I struggle to envision a future where we further fragment operational practices; instead, I advocate for a return to the integrated methodologies rooted in established DevOps practices.
Section 1.1: The Challenge of Model Handoff
One significant hurdle in the AI/ML workflow is the model handoff process. Transitioning a model from its development phase—often in a Jupyter Notebook—through various ML tools, to a development server, and ultimately to a production environment like Kubernetes can be quite challenging. Each transition demands that the model be repackaged to suit its new environment.
This complexity burdens the teams, particularly DevOps, responsible for everything post-training and introduces considerable risks and delays into the deployment cycle. Moreover, the lack of standardized packaging methods exacerbates these challenges, complicating what should be a straightforward process.
To tackle this issue, we conceived a bold idea: why not package models using a format that we’ve already standardized?
Section 1.2: Standardizing Model Packaging
In 2013, Docker revolutionized application packaging by simplifying the use of containers. Their success hinged on the Open Container Initiative (OCI), which established industry-wide standards for application packaging. Thanks to these standards, a multitude of tools now exists for managing and deploying applications. So, why not apply this approach to package Notebooks and AI models? By doing so, we can simplify deployment, sharing, and management for everyone involved.
Subsection 1.2.1: Introducing Kit
In response to the challenges of model packaging, we developed Kit, an open-source MLOps project that encapsulates your model, datasets, code, and configuration. This allows data scientists and developers to utilize their preferred tools while collaborating seamlessly.
Our mission is to standardize the packaging, reproduction, deployment, and tracking of AI/ML models, ensuring they can be managed with the same agility as application code. Kit simplifies the deployment process, allowing the DevOps team to deploy a model with minimal friction—no need for them to learn how to write a Docker or Kubernetes file.
The Kit CLI compiles all assets/resources from your Notebook into a ModelKit, facilitating easy transitions between ML engineers and application teams. This lets both teams leverage the tools they are already familiar with.
Chapter 2: Enhancing Collaboration and Traceability
KitOps emphasizes improving collaboration without necessitating a complete overhaul of existing workflows. By adhering to industry standards, we encourage cross-team involvement, extending beyond just data scientists, in the model development process. The compatibility of ModelKits and Kitfiles with existing tools means minimal disruption to established deployment pipelines.
Section 2.1: Model Traceability and Reproducibility
KitOps's strategy for model traceability and reproducibility diverges from the common practice of merely using Docker files. While Docker files have their place, including hyperparameters and datasets in a container is often impractical. Kit offers a comprehensive package that encompasses everything needed to reproduce, integrate with, or test a model throughout its lifecycle. Users can flexibly extract specific components of a ModelKit or access the entire package as needed.
Additionally, storing ModelKits within an organization's registry maintains a detailed log of state changes, essential for auditing. Since ModelKits comply with OCI standards, users can utilize their existing registries (like Docker Hub, Amazon ECR, Artifactory, and Quay). Furthermore, ModelKits align perfectly with secure bill-of-materials (SBOM) initiatives, addressing the growing need for security and compliance in AI/ML model deployment.
Section 2.2: The Future of DevOps Integration
We believe that KitOps marks a crucial juncture for the integration of AI/MLOps and DevOps. By resolving model deployment and collaboration issues, KitOps paves the way for a future where the boundaries between MLOps and DevOps become increasingly blurred. This isn't to say that current MLOps practices lack value; rather, I assert that the foundational tools and methodologies established in our DevOps workflows are adaptable enough to meet the needs of AI/ML deployment.
As we continue to develop, we are eager to gather feedback from the community.