Type Python projection outputs as non-null by default

.github/workflows/build.yml

@@ -552,6 +552,23 @@ jobs:
             --test implementation_naming_test `
             --test python_overload_legacy_compat_test `
             --test python_released_implementation_test
+      - name: Test generated nullable collection values
+        shell: pwsh
+        run: |
+          $env:DYNWINRT_TEST_PYTHON = (Resolve-Path .\bindings\py\.venv\Scripts\python.exe).Path
+          $env:DYNWINRT_REQUIRE_IMPLEMENTATION_RUNTIME = '1'
+          $env:DYNWINRT_REQUIRE_MYPY = '1'
+          cargo test -p dynwinrt-codegen --test python_consumer_typing_test `
+            mutable_collection_mutators_accept_none -- --exact
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          cargo test -p dynwinrt-codegen --test python_consumer_typing_test `
+            reference_array_results_preserve_null_elements -- --exact
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          cargo test -p dynwinrt-codegen --test python_collection_helper_collision_test `
+            collection_helper_aliases_execute_for_packaged_and_standalone_outputs -- --exact
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          cargo test -p dynwinrt-codegen --test python_delegate_callback_test `
+            nested_delegate_callback_argument_preserves_native_null -- --exact --nocapture
       - name: Run E2E tests
         run: .\tests\e2e\e2e_test.ps1 -SkipBuild -Codegen $env:DYNWINRT_CODEGEN
       # This optional-SDK behavioral smoke is separate from generated coverage

bindings/py/README.md

@@ -22,6 +22,56 @@ interface overloads fail explicitly when a required native guard is missing.
 Generated `IReference<T>` values are projected as `T | None`; native values,
 `None`, and generated `IReference_*` wrappers are accepted as inputs.
 
+### Nullability in type stubs
+
+WinRT metadata does not record which values can be null, and most APIs raise
+an exception instead of returning null. The generated `.pyi` stubs therefore
+type the values you receive as non-null by default: method and property
+results, async results and out values. For example,
+`StorageFolder.create_file_async()` returns `WinRTCoroutine[StorageFile]`.
+
+These values keep `| None`:
+
+- `IReference<T>` values, projected as `T | None` everywhere;
+- results of `Try*` members, such as `try_get_item_async()` or
+  `JsonObject.try_parse()`, where null means "not found";
+- results of Windows SDK members whose documentation says they can return
+  null, such as `Accelerometer.get_default()`,
+  `DispatcherQueue.get_for_current_thread()` or
+  `StorageFolder.get_parent_async()`. The codegen embeds this list, derived
+  from the Windows SDK API reference; it does not cover Windows App SDK
+  (`Microsoft.*`) APIs;
+- `Object`/`IInspectable` values (`DynWinRTValue | None`) and delegate-typed
+  values, which are often null.
+
+Reference-type elements read from WinRT collection interfaces are always typed
+`T | None`, including vectors, views, iterables, iterators, map keys and values,
+and key-value-pair keys and values. Here reference means the projection's
+supported COM-pointer shapes: `Object`, interfaces, runtime classes, delegates,
+and parameterized interfaces. Async wrappers are not collection element
+shapes. Reference-type elements of returned WinRT arrays are also `T | None`;
+the array value itself remains non-null. String, GUID, scalar, enum, and struct
+array elements and keys remain non-null. A view or iterator obtained from a
+mutable collection can expose a null slot, and WinRT collection interfaces do
+not retain enough provenance for the stubs to distinguish that case. For
+example, a `JsonArray` holds
+`IJsonValue | None`, and `get_files_async()` returns
+`WinRTCoroutine[Sequence[StorageFile | None]]`. Value-type elements remain
+non-null.
+
+Mutable collections accept `None` when their element, map-key, or map-value
+type is a WinRT reference type and store a real null WinRT value. This includes
+`append()`, `insert()`, index and slice assignment, `extend()`, `update()` and
+`setdefault()`. String, GUID, scalar, enum, and struct keys and value-type
+elements reject `None` with `TypeError`.
+
+Other arguments keep accepting `None` where they did before. The stubs are
+optimistic, like the generated TypeScript declarations: the runtime still
+returns `None` when a WinRT API returns null, so check the API documentation
+when a result can legitimately be absent. The inline annotations of the
+generated `.py` modules, which `typing.get_type_hints()` and `--no-pyi` output
+expose, still mark every object result `| None`.
+
 ## Async WinRT operations
 
 Generated async methods return typed, asyncio-compatible operation objects:
@@ -109,7 +159,9 @@ parameter such as `ThreadPool.run_async(handler)`, or a delegate-typed
 property) receives the delegate's arguments as projected Python values, typed
 from the delegate's `Invoke` signature. WinRT `Object` arguments stay
 `DynWinRTValue | None`, and `IReference<T>` arguments are native values or
-`None`. Async-operation arguments stay raw `DynWinRTValue` objects so a
+`None`. Delegate-typed callback arguments are raw `DynWinRTValue | None`
+because a null native delegate is passed to the callable as `None`.
+Async-operation arguments stay raw `DynWinRTValue` objects so a
 callback projection cannot take over or cancel the operation's completion.
 For example, `map_changed` handlers of `PropertySet`, `StringMap`,
 `ValueSet`, and other `IObservableMap<K, V>` implementations receive the
@@ -127,10 +179,11 @@ value passed to a runtime class is reserved for wrapping an existing native
 instance before constructor overload dispatch. Keep the delegate object for a
 constructor, or pass the raw delegate to a named factory/method instead.
 
-Callback parameter annotations are non-null by default, matching generated
-method-output typing. This is an intentionally optimistic typing policy, not a
-guarantee from the `Invoke` metadata: WinMD carries no nullability information,
-and the runtime still passes `None` when WinRT supplies a null reference.
+Callback parameter annotations are non-null by default except for `Object`,
+`IReference<T>`, and delegate-typed arguments. This is an intentionally
+optimistic typing policy, not a guarantee from the `Invoke` metadata: WinMD
+carries no nullability information, and the runtime still passes `None` when
+WinRT supplies a null reference.
 Precise callback signatures live in the generated `.pyi` contract. Executable
 `.py` methods use the cycle-safe runtime annotation
 `Callable[..., object] | DynWinRTValue | DynWinRtDelegate`, so

eng/ci/test_extract_null_results.py

@@ -0,0 +1,111 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+import importlib.util
+from pathlib import Path
+import unittest
+
+
+ROOT = Path(__file__).resolve().parents[2]
+SCRIPT = (
+    ROOT
+    / "tools"
+    / "dynwinrt-codegen"
+    / "scripts"
+    / "extract-null-results.py"
+)
+SPEC = importlib.util.spec_from_file_location("extract_null_results", SCRIPT)
+assert SPEC is not None and SPEC.loader is not None
+extractor = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(extractor)
+
+
+def method_doc(
+    api_id: str,
+    *,
+    returns: str = "The current reading.",
+    remarks: str = "",
+) -> str:
+    return f"""---
+-api-id: {api_id}
+-api-type: winrt method
+---
+## -returns
+{returns}
+## -remarks
+{remarks}
+"""
+
+
+class NullResultExtractionTests(unittest.TestCase):
+    def test_required_return_null_check_is_nullable(self):
+        wording = (
+            "Before using the return value from this method, the application "
+            "must first check that the value is not null. (If the value is "
+            "null and you attempt to retrieve it, Windows will generate an "
+            "exception.)"
+        )
+        for sensor in (
+            "Accelerometer",
+            "Compass",
+            "Gyrometer",
+            "Inclinometer",
+            "LightSensor",
+            "OrientationSensor",
+        ):
+            api_id = f"M:Windows.Devices.Sensors.{sensor}.GetCurrentReading"
+            with self.subTest(sensor=sensor):
+                self.assertEqual(
+                    extractor.classify_document(
+                        method_doc(api_id, remarks=wording)
+                    ),
+                    (api_id, "remarks"),
+                )
+
+    def test_negations_arguments_and_null_holders_are_not_nullable(self):
+        cases = (
+            "This method always returns a value that is not null.",
+            (
+                "Before using this method, check that the input parameter is "
+                "not null."
+            ),
+            "The returned object contains a JSON null value.",
+        )
+        api_id = "M:Contoso.Sensor.GetCurrentReading"
+        for remarks in cases:
+            with self.subTest(remarks=remarks):
+                self.assertEqual(
+                    extractor.classify_document(
+                        method_doc(api_id, remarks=remarks)
+                    ),
+                    (api_id, None),
+                )
+
+    def test_direct_result_nullability_still_uses_result_sections(self):
+        method_id = "M:Contoso.Sensor.GetDefault"
+        self.assertEqual(
+            extractor.classify_document(
+                method_doc(
+                    method_id,
+                    returns="The default sensor, or null if none is installed.",
+                )
+            ),
+            (method_id, "returns"),
+        )
+
+        property_id = "P:Contoso.Reading.OptionalValue"
+        property_doc = f"""---
+-api-id: {property_id}
+-api-type: winrt property
+---
+## -property-value
+The current value, or null when no value is available.
+"""
+        self.assertEqual(
+            extractor.classify_document(property_doc),
+            (property_id, "property-value"),
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

samples/python/async-file-io/app.py

@@ -15,12 +15,7 @@ async def run() -> None:
     with tempfile.TemporaryDirectory(prefix="dynwinrt-python-") as directory:
         with RoApartment(), projected_lifetime_scope():
             folder = await StorageFolder.get_folder_from_path_async(directory)
-            if folder is None:
-                raise RuntimeError("StorageFolder returned no temporary folder")
-
             file = await folder.create_file_async("sample.txt")
-            if file is None:
-                raise RuntimeError("StorageFolder returned no file")
             await FileIO.write_text_async(file, "Hello from dynwinrt.")
             await FileIO.append_text_async(
                 file,

samples/python/cryptography/app.py

@@ -9,12 +9,8 @@
 def sha256(text: str) -> str:
     with RoApartment(), projected_lifetime_scope():
         provider = HashAlgorithmProvider.open_algorithm("SHA256")
-        if provider is None:
-            raise RuntimeError("SHA256 provider is unavailable")
         data = IBuffer.from_bytes(text.encode("utf-8"))
         digest = provider.hash_data(data)
-        if digest is None:
-            raise RuntimeError("HashAlgorithmProvider returned no digest")
         copied_digest = digest.to_bytes()
         expected_length = provider.hash_length
         if len(copied_digest) != expected_length:

samples/python/device-watcher/app.py

@@ -15,8 +15,6 @@
 async def enumerate_devices(timeout: int, show_names: bool) -> None:
     with RoApartment(), projected_lifetime_scope():
         watcher = DeviceInformation.create_watcher()
-        if watcher is None:
-            raise RuntimeError("DeviceInformation returned no watcher")
 
         loop = asyncio.get_running_loop()
         enumeration_completed = asyncio.Event()

samples/python/ocr-image/app.py

@@ -17,30 +17,21 @@ def normalized_words(value: str) -> set[str]:
 async def recognize(path: Path) -> str:
     with RoApartment(), projected_lifetime_scope():
         file = await StorageFile.get_file_from_path_async(str(path.resolve()))
-        if file is None:
-            raise RuntimeError("StorageFile returned no image file")
         stream = await file.open_read_async()
-        if stream is None:
-            raise RuntimeError("StorageFile returned no image stream")
 
         decoder = await BitmapDecoder.create_async(
             stream.as_interface(IRandomAccessStream)
         )
-        if decoder is None:
-            raise RuntimeError("BitmapDecoder returned no decoder")
         bitmap = await decoder.get_software_bitmap_async()
-        if bitmap is None:
-            raise RuntimeError("BitmapDecoder returned no SoftwareBitmap")
 
         with bitmap:
+            # Try* members return None instead of raising when nothing matches.
             engine = OcrEngine.try_create_from_user_profile_languages()
             if engine is None:
                 raise RuntimeError(
                     "No OCR engine is available for the user profile languages"
                 )
             result = await engine.recognize_async(bitmap)
-            if result is None:
-                raise RuntimeError("OcrEngine returned no result")
             return result.text
 
 

samples/python/text-to-speech/app.py

@@ -13,8 +13,6 @@ async def speak(text: str, smoke: bool) -> None:
     with RoApartment(), projected_lifetime_scope():
         with SpeechSynthesizer() as synthesizer:
             stream = await synthesizer.synthesize_text_to_stream_async(text)
-            if stream is None:
-                raise RuntimeError("SpeechSynthesizer returned no stream")
 
             with stream:
                 if smoke: