`pyrefly report` ignores function declared in stub file

[MarcoGorelli]

I have these two files:

(tmp-repo) marcogorelli@DESKTOP-U8OKFP3:~/tmp-repo$ cat foo/__init__.py
def one():
    return None

(tmp-repo) marcogorelli@DESKTOP-U8OKFP3:~/tmp-repo$ cat foo/__init__.pyi
from typing import Callable

one: Callable[[], None]

If I run pyrefly report foo, then I get:

{
  "/home/marcogorelli/tmp-repo/foo/__init__.pyi": {
    "line_count": 5,
    "functions": [],
    "suppressions": []
  },
  "/home/marcogorelli/tmp-repo/foo/__init__.py": {
    "line_count": 4,
    "functions": [
      {
        "name": "foo.one",
        "return_annotation": null,
        "parameters": [],
        "location": {
          "start": {
            "line": 1,
            "column": 1
          },
          "end": {
            "line": 2,
            "column": 16
          }
        }
      }
    ],
    "suppressions": []
  }
}

Note how we don't get any information for the function one in the stub file

pyright verifytypes includes:

        "symbols": [
            {
                "category": "function",
                "name": "foo.one",
                "referenceCount": 1,
                "isExported": true,
                "isTypeKnown": true,
                "isTypeAmbiguous": false,
                "diagnostics": []
            }
        ]

[tsembp]

Hello! I am new to open source and would like to take this.

I’ll make pyrefly report include functions declared in .pyi (stubs) as well as .py, and add a regression test for the minimal repro in the issue.

Let me know how this sounds! :)
@MarcoGorelli

[yangdanny97]

From discussion on discord, we don't report variables annotated with a callable type ATM. The parameters are not named and they're always considered "fully annotated", unless we want to start flagging Callable[..., _].

Should the output format be the same as def functions except with placeholder/empty/unknown names for the parameters?

[tsembp]

Thanks, this makes sense.
In .pyi a name like one: Callable[[], None] is technically a variable alias to a callable type, but users likely expect it to appear under functions in pyrefly report.

To unblock, which behavior do we want?

1. Parse .pyi exports and find symbols with Callable type (or typing.Callable / collections.abc.Callable).
2. Treat x: Callable[[...], R] in stubs as a function symbol in report, emitting a functions entry (name foo.one) with synthetic parameter names (e.g. arg0, arg1) and return annotation R.
3. Keep functions for real def only, and introduce a new bucket like callables / variables in report output for Callable-typed symbols.
4. Do nothing (current behavior), and document that report only surfaces def-declared functions.>

If we choose (1): should we also handle Callable[..., R] and mark params as unknown/variadic?
I’m happy to implement whichever option you prefer + add a regression test for the minimal repro.

[MarcoGorelli]

thanks for your response!

always considered "fully annotated"

Sure, it would still be good to include it in the report though.

Otherwise, if I'm matching .py symbols with .pyi, then the current report makes it look like the one function is completely unannotated, whereas the reverse is true

[yangdanny97]

Maybe we should put variables into a separate toplevel field alongside functions and classes, since the format would be different.

Something like this

{
  "variables": {
    "x": { "annotation": null },
    "y": { "annotation": "Callable[[int], int]" }
  },
  "classes": ...
  "functions": ...
  "line_count": ...
  "suppressions": ...
}