Nginx Ingress Migration Guide

Editor's Note: The retirement of Ingress NGINX has caused strong concern among developers. The public account article "Regretfully, Ingress NGINX is Retiring" has received over 20,000 reads.

The official response provides comprehensive solutions for migrating to Gateway API and over 20 Ingress controllers. However, during the migration process, enterprises will want to understand whether the new Ingress controllers are compatible with Ingress NGINX annotations, how to perform gradual traffic switching during the migration, and how to quickly roll back in case of traffic loss to ensure a smooth migration process without affecting online business.

Therefore, this article will provide operationally-based response solutions, using Alibaba Cloud's cloud-native API gateway (Higress Enterprise Edition) as an example, detailing the migration process step by step.

Overview

As Nginx Ingress gradually stops maintenance, users need to migrate to a new gateway solution. The cloud-native API gateway is a sub-product of Alibaba Cloud's API gateway, unifying traffic gateway, microservice gateway, and security gateway, providing Nginx Ingress users with a smooth migration path and powerful feature upgrades.

The cloud-native API gateway offers two core configuration modes to suit different management needs and usage scenarios:

1. Listening to K8s Ingress (Ingress Mode): The gateway operates as an APIG Ingress Controller, compatible with K8s Ingress resources and Nginx Ingress annotations, suitable for teams looking to maintain K8s native workflows (such as GitOps).

2. Console Configuration API (API Management Mode): Configuration is done through the Alibaba Cloud console or API, providing complete API lifecycle management, advanced security policies, and API operation capabilities, suitable for scenarios requiring centralized governance and refined management.

This document will detail the comparison of the features, advantages, and applicable scenarios of these two modes to help you choose the most suitable configuration path.

Mode One: Listening to K8s Ingress (Ingress Mode)

This mode deploys the cloud-native API gateway as the Ingress Controller for the Kubernetes cluster, managing the north-south traffic of the cluster.

1.1 Core Advantages and Applicable Scenarios

• Smooth Migration: Provides Nginx Ingress users with an one-click migration tool to minimize migration costs and risks of business interruption.

• Maintaining K8s Native Workflows: Fully compatible with K8s Ingress resources and annotations, teams can continue to use kubectl apply, GitOps, and other existing workflows to manage routing rules.

• Feature Enhancement: Builds on compatibility with Nginx Ingress to provide more robust governance capabilities, such as global rate limiting, etc.

Applicable Scenarios:

• Existing users of Nginx Ingress.

• Teams centered around K8s that rely on GitOps processes to manage application releases.

• Development and operations teams that need to quickly implement cluster traffic routing and basic governance.

1.2 Functional Details

For the complete Ingress capabilities supported by the APIG Ingress Controller, please refer to:

1. "Annotations Supported by APIG Ingress"

2. "Advanced Usage of APIG Ingress"

1.2.1 Highly Compatible with Nginx Ingress Annotations

APIG Ingress (the Ingress Controller of the cloud-native API gateway) supports the vast majority of Nginx Ingress annotations (statistically, it supports 51 types, covering 90% of user scenarios). This means existing K8s Ingress YAML files can be migrated without significant modification.

Key Compatible Annotation Examples:

1.2.2 Unique Feature Enhancements (Higress Annotations)

This mode not only is compatible with Nginx but also provides advanced features not available in Nginx through higress.ingress.kubernetes.io/ prefixed annotations, such as:

Traffic Warm-Up

• Issues with Nginx: This capability cannot be achieved.

• APIG Ingress Solution: Provides a native higress.ingress.kubernetes.io/warmup annotation that ensures when new nodes come online, traffic gradually increases within the specified warm-up window, thoroughly ensuring the new nodes complete their warm-up.

Global Rate Limiting

• Issues with Nginx: nginx.ingress.kubernetes.io/limit-rps implements Pod-level rate limiting, where the total limit is equal to "limit value x number of Pods," making precise control difficult.

• APIG Ingress Solution: higress.ingress.kubernetes.io/rate-limit provides a global rate limiting across all gateway instances, allowing for precise control of total QPS.

Global Concurrency Control

• Issues with Nginx: Lacks simple and effective global concurrency control.

• APIG Ingress Solution: higress.ingress.kubernetes.io/concurrency-limit provides global concurrency limits, protecting backend services from sudden traffic spikes.

Traffic Mirroring

• Issues with Nginx: Lacks traffic mirroring capabilities; requires running Lua scripts.

• APIG Ingress Solution: Provides a native higress.ingress.kubernetes.io/mirror-target-service annotation for easily copying traffic to testing services, useful for shadow testing in production environments.

  
  

Mode Two: Console Configuration API (API Management Mode)

This mode positions the cloud-native API gateway as a centralized API management platform. Users define and manage APIs through the Alibaba Cloud console (or API/Terraform), achieving a comprehensive upgrade from routing forwarding to API governance.

2.1 Core Advantages and Applicable Scenarios

• Centralized Governance: Allows platform teams, architects, or security teams to manage all APIs from a unified view, enforcing security, compliance, and traffic policies.

• Full Lifecycle Management: Supports the complete lifecycle of APIs from design, development, testing, deployment to decommissioning, including version control, release auditing, and one-click rollback.

• Advanced Security Capabilities: Natively integrates complex authentication mechanisms (such as OIDC, JWT, custom authentication and authorization).

• API Operations and Ecology: Supports management of API consumers, subscription relationships, and call quotas, empowering the API economy.

Applicable Scenarios:

• Enterprises needing refined, centralized governance of APIs.

• Businesses with high requirements for API security and identity authentication.

• Teams needing to manage API versions, conduct canary releases, and perform audits.

• Building an open platform, requiring management of third-party developers (consumers) and their call quotas.

2.2 Functional Details

2.2.1 Complete API Lifecycle Management

Supports full-cycle management of API design, development, testing, deployment, and decommissioning. Key features include:

• Version Management: Supports multiple versions of APIs (e.g. v1, v2) online simultaneously, managing their release statuses.

• Publishing and Rollback: Provides historical records of API releases, supporting one-click rollback to any historical version.

2.2.2 Advanced Enterprise Security

Offers foundational security capabilities far beyond those of the Ingress mode, decoupling complex authentication logic from backend services:

• Rich Authentication and Authorization: Natively supports JWT, OIDC, and can integrate with Alibaba Cloud IDaaS (Identity as a Service).

• Multi-layer Defense: Deeply integrated WAF (Web Application Firewall), supports mTLS mutual authentication, IP black and white lists, and custom security plugins.

2.2.3 Powerful Scalability

• Plugin Marketplace: Provides a wealth of official plugins (covering authentication, security, traffic, etc.) and supports users uploading custom plugins.

• Hot Updates: The gateway supports hot updates of plugins and configurations without restarting instances, ensuring high availability of services.

2.2.4 API Operations and Multi-source Service Discovery

• API Ecology: Provides "consumer management" capabilities to manage API call quotas and subscription rules.

• Multi-source Discovery: Backend services are not limited to K8s clusters, but also support discovering services from Nacos, Function Computing (FC), and fixed addresses/domains.

Mode Comparison Summary

The following table summarizes the differences between the two configuration modes across key dimensions:

How to Choose: Recommended Migration and Evolution Paths

Scenario One: Smooth Migration

• Target Audience: Teams prioritizing migration speed and wishing to maintain existing K8s workflows.

• Recommended Solution: Use Mode One: K8s Ingress Mode.

• Implementation:

1. Use the official migration tool to migrate Nginx Ingress configurations to the cloud-native API gateway.

2. Review the migration report and handle any incompatible annotations (support tickets can be submitted for consultation).

3. (Optional) Replace existing configurations with higress.ingress.kubernetes.io/ annotations to enable global rate limiting and other advanced features.

Scenario Two: New Business Architecture

• Target Audience: Enterprises building a brand-new API platform or those with high demands for security and governance.

• Recommended Solution: Use Mode Two: Console API Mode.

• Implementation:

1. Define APIs in the console, configure security policies (like OIDC/JWT) and rate limiting policies.

2. Utilize the gateway's service discovery capabilities, pointing API backends to Service in ACK clusters or other service sources.

Scenario Three: Progressive Evolution (Recommended Strategy)

• Target Audience: The vast majority of organizations needing to address existing migration issues while gradually enhancing governance capabilities.

• Recommended Solution: Start with Mode One, progressively evolve to Mode Two.

• Implementation:

1. Step One (Migration): Start with Mode One (Ingress), completing a smooth migration of all Nginx Ingress, quickly resolving Nginx EOL issues.

2. Step Two (Governance): Identify core APIs within the organization (for example: external-facing, high-security-level, and finely managed APIs).

3. Step Three (Evolution): Gradually incorporate these core APIs into Mode Two (Console). You can configure JWT authentication, WAF protection, consumer quotas, and other advanced strategies for these APIs in the console, while other non-core APIs can continue to run in Mode One.

Routing Priority Explanation:

For routing with the same domain name and path, the API created in the console will have a higher priority over the routes synchronized via Ingress mode, thus allowing you to configure them individually in the console during the migration process. If issues arise, you can immediately revert to Ingress mode by deleting the console configuration.

Note: Priorities are based on individual routing granularity, not the entire domain name. This means:

• Certain paths under a domain can use console configuration while others continue to use Ingress.

• Routes configured in the console will only override Ingress routes that match the same conditions.

• It is recommended to migrate by path gradually, rather than migrating all routes of an entire domain at once.

This priority mechanism can be better understood through examples:

Scenario: You have a domain example.com and need to gradually migrate from Ingress to console configuration.

1. Initial State (only Ingress configuration)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service-v1
            port:
              number: 8080
      - path: /web
        pathType: Prefix
        backend:
          service:
            name: web-service-v1
            port:
              number: 80

The automatically generated routes by the API gateway at this point are:

• /api → api-service-v1:8080

• /web → web-service-v1:80

2. During Migration (console configuration for /api path)

A route is created in the console for example.com, pointing /api to the new version service api-service-v2:8080.

The actual merged route order now is:

/api → api-service-v2:8080 (configured in console, takes priority) ✅
/api → api-service-v1:8080 (Ingress configuration, will NOT be matched)
/web → web-service-v1:80 (Ingress configuration, works normally)

Effect:

• Accessing example.com/api/* → routes to api-service-v2 (console configuration takes effect)

• Accessing example.com/web/* → routes to web-service-v1 (Ingress configuration takes effect)

  
  

3. Issue Found, Quick Rollback

If issues arise with api-service-v2, simply delete the /api route configuration in the console.

After deletion, the routing order becomes:

/api → api-service-v1:8080 (Ingress configuration, restored immediately) ✅
/web → web-service-v1:80 (Ingress configuration)

Effect: Traffic immediately reverts to the Ingress configuration for api-service-v1, without needing to modify Ingress or restart any services.

4. Complete Migration (console configuration for all paths)

After continuing to configure the /web path in the console:

/api → api-service-v2:8080 (console configuration) ✅
/web → web-service-v2:80 (console configuration) ✅
/api → api-service-v1:8080 (Ingress configuration, will NOT be matched)
/web → web-service-v1:80 (Ingress configuration, will NOT be matched)

At this point, all traffic is controlled by the console configuration, allowing for safe removal of the corresponding Ingress configuration.