docs: add kserve integration guide

[Lasse4]

This PR is part of kgateway-dev/kgateway.dev#606

[npolshakova]

We might want to rephrase the agent-aware policy to be more specific to policies that can be applied when routing to inference models:

Suggested change

	[KServe](https://kserve.github.io/website/) is a Kubernetes-native model inference platform. By fronting KServe with agentgateway, you can apply agent-aware policies, including token-based rate limiting, to your model serving endpoints without modifying your inference services.
	[KServe](https://kserve.github.io/website/) is a Kubernetes-native platform for serving machine learning models. With agentgateway in front of KServe, you can enforce traffic management policies, such as token-based rate limiting, for inference requests without modifying your inference services.

[npolshakova]

I think we want Latest Version here. Experimental is a little confusing since it refers to experimental apis in Gateway API. Maybe rephrase to be version v1.2+ ?

[artberger]

I am thinking we can remove this note because this will only show up in main, which is for the 1.2 version.

[npolshakova]

Do we want to try https://kserve.github.io/website/docs/getting-started/genai-first-llmisvc?_highlight=inferenceservice instead of httpbun here?

[artberger]

Start steps with imperative verbs.

Suggested change

	1. KServe requires cert-manager for webhook certificates.
	1. Install cert-manager, which KServe requires for webhook certificates.

[artberger]

Use step formatting for these subsequent checks.

Suggested change

	Wait for the `InferenceService` to become ready.
	3. Wait for the `InferenceService` to become ready.

[artberger]

Suggested change

	Once `READY` is `True`, KServe creates an `HTTPRoute` that attaches to the agentgateway. Verify it. The route attaches to `kserve/kserve-ingress-gateway` with hostname `mock-llm-kserve-test.example.com`.
	4. Verify that KServe created an HTTPRoute after the Gateway becomes `READY`. The route attaches to `kserve/kserve-ingress-gateway` with hostname `mock-llm-kserve-test.example.com`.

[artberger]

The second sentence is a bit complex, maybe split it up and rephrase for readability.

Also, combine with the note later on so you can remove it there, since they say similar things. In this rephrase, I also would remove the language about agentgateway supporting responses natively in AgentgatewayPolicy because we typically do not commit to or imply support for future releases in the docs.

Suggested change

KServe generates the `HTTPRoute` with a plain Kubernetes `Service` as the `backendRef`. Agentgateway only applies token-based rate limiting to traffic that flows through an `AgentgatewayBackend` with `spec.ai.provider` configured, because that is what signals to the proxy that the backend is an LLM and that response bodies contain a `usage.total_tokens` field to count against the rate limit bucket.

KServe generates the `HTTPRoute` with a plain Kubernetes `Service` as the `backendRef`. However, to apply a token-based rate limiting policy, agentgateway needs the backend to be an AgentgatewayBackend. This way, agentgateway knows that the backend is an LLM that has a response body with the `usage.total_tokens` field to count against the rate limit bucket. In the following steps, you create an AgentgatewayBackend and a second HTTPRoute to route to it as a workaround to the KServe-created, Service-based setup.

[artberger]

I am not sure what predictor means, is it needed? maybe just

Suggested change

	1. Create an `AgentgatewayBackend` that points at the httpbun predictor service.
	1. Create an `AgentgatewayBackend` that points at the httpbun service.

[artberger]

Maybe move this to the shortdesc and delete here, see previous comment.

Suggested change

	{{< callout type="info" >}}
	This extra `HTTPRoute` is a current workaround. Token-based rate limiting requires traffic to flow through an `AgentgatewayBackend` so the proxy knows to inspect the response body for `usage.total_tokens`. A future agentgateway release may support activating LLM-aware response parsing directly on an `AgentgatewayPolicy`, which would remove the need for this step.
	{{< /callout >}}

[artberger]

Suggested change

	1. Apply an `AgentgatewayPolicy` that caps requests at **100 tokens per minute**. The policy targets the `mock-llm-ai` route that flows through the `AgentgatewayBackend`.
	1. Apply an `AgentgatewayPolicy` that caps requests at **100 tokens per minute**. The policy targets the `mock-llm-ai` route that selects the `AgentgatewayBackend`.

[npolshakova]

I wonder if it's more confusing if we highlight this route is created by kserve but then don't use it? I think there's a couple options here:

• Remove the call out to kubectl get httproute mock-llm -n kserve-test -o yaml section and only focus on the new HTTPRoute that references the AgentgatewayBackend
• Add an example policy that you can attach to the HTTPRoute that's generated by KServe (maybe tracing or a transformation?)

[npolshakova]

For the transformation it could be something like this:

  traffic:
    transformation:
      response:
        set:
        - name: x-requested-model
          value: 'string(json(request.body).model)'
        - name: x-actual-model
          value: 'string(json(response.body).model)'