ADK import overhead (8-20s) causes slow cold starts

[pentium10]

When you deploy a basic agent to Cloud Run, somehow the framework core is so slow to boot that is unusable.

25s + cold starts constantly, using the bare minimum examples from Tutorial.

Improvements need to be made here, to reduce boilerplate for cold starts.

Dockerfile

FROM python:3.13-slim
WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

RUN adduser --disabled-password --gecos "" myuser && \
    chown -R myuser:myuser /app

COPY . .

USER myuser

ENV PATH="/home/myuser/.local/bin:$PATH"

CMD ["sh", "-c", "uvicorn main:app --host 0.0.0.0 --port $PORT"]

main.py

import os

import uvicorn
from google.adk.cli.fast_api import get_fast_api_app

# Get the directory where main.py is located
AGENT_DIR = os.path.dirname(os.path.abspath(__file__))
# Example session service URI (e.g., SQLite)
SESSION_SERVICE_URI = "sqlite:///./sessions.db"
# Example allowed origins for CORS
ALLOWED_ORIGINS = ["http://localhost", "http://localhost:8080", "*"]
# Set web=True if you intend to serve a web interface, False otherwise
SERVE_WEB_INTERFACE = True

# Call the function to get the FastAPI app instance
# Ensure the agent directory name ('capital_agent') matches your agent folder
app = get_fast_api_app(
    agents_dir=AGENT_DIR,
    session_service_uri=SESSION_SERVICE_URI,
    allow_origins=ALLOWED_ORIGINS,
    web=SERVE_WEB_INTERFACE,
)

# You can add more FastAPI routes or configurations below if needed
# Example:
# @app.get("/hello")
# async def read_root():
#     return {"Hello": "World"}

if __name__ == "__main__":
    # Use the PORT environment variable provided by Cloud Run, defaulting to 8080
    uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))

my cold start command is to create a session

curl -X POST 
    "$APP_URL/apps/demo_agent/users/api_user/sessions/api_session_1" \
    -H "Content-Type: application/json" \
    -d '{"state": {}}' # Optional initial state

[Jacksunwei]

@pentium10 Is that because the cloud run instance is not up? What if you set minimum 1 instance for that cloud run app?

[pentium10]

@Jacksunwei Its the cold start. So the initial boot of the container and app. To boot one instance takes 20 seconds more. To boot additional instances another 20 seconds. Minimum instance doesn't help this world as its a serverless environment, you pay per use. So you want to have scale to zero.

[Jacksunwei]

I think that's the general limitation for all container-based pay-as-you-use cloud service, incl. Cloud Run.

ADK as a software has no control over that.

[pentium10]

I cannot agree on the generalities.
I deliver talks on Cloud Run since the begining I know Cloud Run service entirely, crafted workshops and so on. Been deep involved so I know something is off with ADK here.
ADK has some issue on loading of libraries. It loads too much than needs to run the agent. 20 seconds is huge. At most it should be 2-3 seconds (even that is slow), but a <sub 1second would be ideal.

[calganaygun]

Could agree @pentium10, it is a performance issue regarding ADK. So, it takes too long to load and start to serve it causes massive cold start latencies. Cold start is the end result of slow library in this case not about the serverless architecture.

[calganaygun]

I might be wrong, but I suspect the cold start delay is caused by eager imports in __init__.py. From reviewing the source, it seems the ADK loads most lower-level modules in these init files.

In my tests, even importing a specific submodule still triggers unrelated components to load because the parent __init__.py is executed.

[arryon]

I agree with the comments above. I seems a general pattern that has been applied through the entirety of ADK is to initialize global objects inside a module's __init__ scope or individual Python file scope.
While I could understand the rationale from Python's lack of dependency injection, it creates various fundamental problems on resource allocation in memory and also on testability:

An example is the AgentLoader functionality [1]. It expects to be pointed to a python module, where it will look for the export of root_agent inside an agent directory. But before we can construct a functional Agent, it might have to consume configuration such as VertexAI Engine ID, GCS bucket name, etc. This forces us to also design our configuration as a global singleton as well, otherwise agents can not import and share underlying configuration

When writing unittests for an Agent, we now have to be very careful which modules are imported.
Even something as simple as @patch("my_package.agent_location.my_agent.main.some_function") will trigger a full import of the agent, its configuration, any sub-agents, and any configuration they might have. Let's say we use a GmailApiToolset, we already require access to GCP credentials in order to convert the OIDC spec to OpenAPI spec [2]

This forces all users of ADK to essentially write a lot of boilerplate around global imports, or discard the AgentLoader altogether and create factory patterns to generate agents elsewhere.

I'd like to hear if other users are also encountering the same issues.

[1] https://github.com/google/adk-python/blob/main/src/google/adk/cli/utils/agent_loader.py#L37
[2] https://github.com/google/adk-python/blob/main/src/google/adk/tools/google_api_tool/googleapi_to_openapi_converter.py#L63

[yshuolu]

I might be wrong, but I suspect the cold start delay is caused by eager imports in __init__.py. From reviewing the source, it seems the ADK loads most lower-level modules in these init files.

In my tests, even importing a specific submodule still triggers unrelated components to load because the parent __init__.py is executed.

yes this is what happens when I profile the adk web command, 50% of trace time is spent on adk/__init__.py, where aiplatform/__init__ contributes the most. This is quite a blocker to use adk for prod.

[crw2998]

For what it's worth: ADK 1.13.0 takes about 8 seconds just to import on my linux VM. This is a real outlier for "heavyweight" Python libraries, let alone one that doesn't perform compute-heavy tasks.

$ time python -c "from google.adk.agents import BaseAgent"

real    0m8.022s
user    0m7.492s
sys     0m0.523s

And for comparison:

$ time python -c "import numpy as np"

real    0m0.162s
user    0m0.283s
sys     0m0.194s
$ time python -c "import pandas as pd"

real    0m0.552s
user    0m0.609s
sys     0m0.256s
$ time python -c "from pydantic_ai import Agent"

real    0m0.695s
user    0m0.626s
sys     0m0.066s

For us, this long import time has two significant effects:

• Developer Experience: As a "code-first" library, the 8-second delay after each code/config edit significantly slows down the development and debugging cycle.
• Production Environments: It poses a major challenge for serverless platforms like AWS Lambda or Google Cloud Functions, where cold start times are critical. An 8-second import consumes a large portion of a typical default timeout (e.g., Lambda's is 10 seconds).

[Jacksunwei]

Thanks, @crw2998 ! I think this aligns with our observe. We could take a look at this.

---

(edit) here is the colab that I think we can target for optimize. For anyone, feel free to comment below for your use case

https://colab.research.google.com/drive/17K11VWBu5jMpvFaFFpr6A38Gq9i9D2V5?usp=sharing