Backend.AI 25.15 (LTS)
Released 25.10.02
Release Policy Update: New Backend.AI Release Schedule Effective 2025
Beginning in 2025, new release rules will apply to Backend.AI. Long Term Support (LTS) versions will be released at the end of the first and third quarters each year. These LTS versions will focus on stability, receiving ongoing maintenance and bug fixes throughout the year. New features and enhancements will be introduced in separate releases, ensuring that LTS versions remain stable while revisions allowing for continued improvement in the platform.
About this release
The Long Term Support (LTS) version of Backend.AI has officially arrived with version 25.15, introducing significant enhancements to user experience and system performance. This update includes internal optimizations and multiple bug fixes, ensuring greater stability and efficiency. Backend.AI 25.15 offers more intuitive and reliable platform for building, training, and deploying AI models at scale.
Simplify your AI journey with Backend.AI.
Major Highlights
This release is a Long Term Support (LTS) version that includes all improvements since 25.06.
Improved Setting Management System
Pydantic-based Configuration Schema: Pydantic-based configuration has been applied across Manager, Agent, Storage-proxy, and Webserver, enabling more robust validation and systematic management.
ServiceConfig GraphQL API: New GraphQL interfaces allow querying and updating system configuration status in real-time.
Configuration Management CLI: CLI commands with sample generators are now available for Agent, Storage, and Webserver components.
Scheduling and Orchestration Improvements
Sokovan Orchestrator: A new scheduler architecture has been introduced with scheduling prioritizers, validators, and controller components.
Distributed Lock-based Scheduling: Centralized distributed locking ensures consistency in scheduling operations.
Session Sweeping: Automatic cleanup of stuck pulling/creating sessions has been implemented.
Monitoring and Observability Improvements
OpenTelemetry Integration: Distributed tracing is now supported across Manager, Storage-proxy, Webserver, and AppProxy.
etcd Service Discovery: etcd and HTTP-based service discovery for Prometheus has been added.
Event Log System: System events can be systematically recorded and queried.
Health Check Endpoints: Simple health check APIs returning component status, version, and name have been added.
Artifact & Reservoir Registry
Artifact Registry: GraphQL types and REST APIs for model deployment and revision management have been added.
HuggingFace Registry: Database tables and GraphQL schemas for HuggingFace model management are now supported.
Reservoir Registry: Sync APIs between Managers and remote registry scan functionality are provided.
enable_reservoir Option: Reservoir features can be activated in webserver settings.
RBAC (Role-Based Access Control)
RBAC Database Schema: Database structure for role-based access control has been added.
VFolder RBAC Migration: Existing VFolder permissions are migrated to RBAC tables.
permission_groups Table: Permissions can be organized and managed by target groups.
Model Service Improvements
AppProxy-based Health Check: Model service health checks have been offloaded to AppProxy for improved scalability.
Redis-based Route Management: Real-time endpoint monitoring and health information synchronization are now available.
service-definition.toml Support: Service definition file handling is supported during model service creation.
runtime_variant Parameter: Runtime variants can be specified via SDK and CLI.
Caution Health check capability is temporarily unavailable on OSS AppProxy due to architectural changes. Users must disable health check in
model-definition.yamlfor Open Source deployments. OSS AppProxy support will be restored in future releases.
Session and Agent Management
Session Status History API: GET /status_history endpoint allows querying session status change history.
Session Dependency Graph: Dependency graph retrieval enables validation of relationships between sessions.
Unlimited Concurrent Sessions: Session limits can be set to None or 0 to allow unlimited concurrent sessions.
Automatic Port Release: Unused host ports are periodically monitored and automatically released.
AgentStats GraphQL: A new GraphQL type for retrieving agent resource utilization has been added.
API Integration and Improvements
Apollo Router Supergraph: Separate GraphQL subgraph endpoints have been unified into a single supergraph for simplified client access.
Generalized Pagination: Consistent pagination has been applied across the Strawberry GraphQL API.
Global ID Node Resolver: Clients can now query sessions and agents directly via global ID.
Security Enhancements
PBKDF2-SHA3-256 Password Hashing: A new password hashing algorithm has been implemented.
TOTP Registration Support: Anonymous users can now register TOTP for two-factor authentication.
VFolder RBAC Validation: RBAC-based permission validation has been added for VFolder operations.
Performance Improvements
15% Startup Latency Reduction: Deferred imports in CLI and server entrypoints reduce startup time.
NUMA-aware Affinity Allocation: NUMA allocation has been improved to find the largest connected components with remaining capacity.
Resource Preset Cache: A cache layer has been applied to resource presets for improved query performance.
Login History Expiration: Expiration times on Redis keys reduce memory usage.
Developer Tools
all-smi Utility: A GPU monitoring utility is now available within session containers.
backend.ai events --wait: A wait option has been added for automation purposes.
TRACE Log Level: A more detailed log level is now supported for debugging.
System Stability Improvements
Various patches have been applied to enhance system stability. Improvements span session management, network/Docker operations, API permissions, proxy health checks, and model service token handling.
Make your AI accessible with Backend.AI!
Check out the datailed changelog in our GitHub.
Last updated