About AI Gateway
About AI Gateway
About AI Gateway
AI Gateway: Specifically designed for AI scenarios, the AI Gateway serves as a unified proxy for LLM APIs and MCP Server, while offering comprehensive integration and governance capabilities.
AI Gateway: Specifically designed for AI scenarios, the AI Gateway serves as a unified proxy for LLM APIs and MCP Server, while offering comprehensive integration and governance capabilities.
A Unified Entry Point for AI Applications Accessing LLMs
For LLMs, it provides a unified proxy for various mainstream and self-hosted large model services, supporting OpenAI-compatible access methods. It also offers secondary API key issuance, rate limiting, security protection, observability, and other governance features.
High Availability:
Supports multi-model grey release, failover between multiple models, token quota management, and traffic control.
High Security:
Offers content security inspection for large models, supports routing configuration authentication and consumer authorization, ensuring API access control, security, and policy management.
Reduced Model Hallucination:
Provides web search capabilities, automatic fusion of search results, and question intent recognition.Observability:
Enables OT (Observability and Tracing) services at the application, gateway, and backend LLM service levels for end-to-end tracking. TraceId is used to correlate logs, request parameters, and other information across all components.
A Unified Entry Point for AI Applications Accessing LLMs
For LLMs, it provides a unified proxy for various mainstream and self-hosted large model services, supporting OpenAI-compatible access methods. It also offers secondary API key issuance, rate limiting, security protection, observability, and other governance features.
High Availability:
Supports multi-model grey release, failover between multiple models, token quota management, and traffic control.
High Security:
Offers content security inspection for large models, supports routing configuration authentication and consumer authorization, ensuring API access control, security, and policy management.
Reduced Model Hallucination:
Provides web search capabilities, automatic fusion of search results, and question intent recognition.Observability:
Enables OT (Observability and Tracing) services at the application, gateway, and backend LLM service levels for end-to-end tracking. TraceId is used to correlate logs, request parameters, and other information across all components.
A Unified Entry Point for AI Applications Accessing LLMs
For LLMs, it provides a unified proxy for various mainstream and self-hosted large model services, supporting OpenAI-compatible access methods. It also offers secondary API key issuance, rate limiting, security protection, observability, and other governance features.
High Availability:
Supports multi-model grey release, failover between multiple models, token quota management, and traffic control.
High Security:
Offers content security inspection for large models, supports routing configuration authentication and consumer authorization, ensuring API access control, security, and policy management.
Reduced Model Hallucination:
Provides web search capabilities, automatic fusion of search results, and question intent recognition.Observability:
Enables OT (Observability and Tracing) services at the application, gateway, and backend LLM service levels for end-to-end tracking. TraceId is used to correlate logs, request parameters, and other information across all components.
A Unified Entry Point for AI Applications Accessing LLMs
For LLMs, it provides a unified proxy for various mainstream and self-hosted large model services, supporting OpenAI-compatible access methods. It also offers secondary API key issuance, rate limiting, security protection, observability, and other governance features.
High Availability:
Supports multi-model grey release, failover between multiple models, token quota management, and traffic control.
High Security:
Offers content security inspection for large models, supports routing configuration authentication and consumer authorization, ensuring API access control, security, and policy management.
Reduced Model Hallucination:
Provides web search capabilities, automatic fusion of search results, and question intent recognition.Observability:
Enables OT (Observability and Tracing) services at the application, gateway, and backend LLM service levels for end-to-end tracking. TraceId is used to correlate logs, request parameters, and other information across all components.
Unified Management and Access for MCP Server
For MCP Server, it provides proxying, security authentication, as well as unified observability, rate limiting, and other governance capabilities.
API-to-MCP:
Supports direct transformation of REST APIs into MCP Server, avoiding redundant effort in rebuilding and maintaining MCP Server instances.Protocol Offloading:
Seamlessly supports the latest official MCP protocol, reducing upgrade costs. For example, it supports converting SSE to Streamable HTTP, so that stateless applications do not need to use SSE.MCP Marketplace:
Provides an officially maintained MCP marketplace to ensure MCP servers are usable, user-friendly, and secure.
Unified Management and Access for MCP Server
For MCP Server, it provides proxying, security authentication, as well as unified observability, rate limiting, and other governance capabilities.
API-to-MCP:
Supports direct transformation of REST APIs into MCP Server, avoiding redundant effort in rebuilding and maintaining MCP Server instances.Protocol Offloading:
Seamlessly supports the latest official MCP protocol, reducing upgrade costs. For example, it supports converting SSE to Streamable HTTP, so that stateless applications do not need to use SSE.MCP Marketplace:
Provides an officially maintained MCP marketplace to ensure MCP servers are usable, user-friendly, and secure.
Unified Management and Access for MCP Server
For MCP Server, it provides proxying, security authentication, as well as unified observability, rate limiting, and other governance capabilities.
API-to-MCP:
Supports direct transformation of REST APIs into MCP Server, avoiding redundant effort in rebuilding and maintaining MCP Server instances.Protocol Offloading:
Seamlessly supports the latest official MCP protocol, reducing upgrade costs. For example, it supports converting SSE to Streamable HTTP, so that stateless applications do not need to use SSE.MCP Marketplace:
Provides an officially maintained MCP marketplace to ensure MCP servers are usable, user-friendly, and secure.
Unified Management and Access for MCP Server
For MCP Server, it provides proxying, security authentication, as well as unified observability, rate limiting, and other governance capabilities.
API-to-MCP:
Supports direct transformation of REST APIs into MCP Server, avoiding redundant effort in rebuilding and maintaining MCP Server instances.Protocol Offloading:
Seamlessly supports the latest official MCP protocol, reducing upgrade costs. For example, it supports converting SSE to Streamable HTTP, so that stateless applications do not need to use SSE.MCP Marketplace:
Provides an officially maintained MCP marketplace to ensure MCP servers are usable, user-friendly, and secure.
Contact
Follow and engage with us through the following channels to stay updated on the latest developments from higress.ai.
Contact
Follow and engage with us through the following channels to stay updated on the latest developments from higress.ai.
Contact
Follow and engage with us through the following channels to stay updated on the latest developments from higress.ai.
Contact
Follow and engage with us through the following channels to stay updated on the latest developments from higress.ai.