[Bug]: import TypedDict from typing does not work for python<3.12

[juncgu]

Your current environment

The output of python collect_env.py

INFO 05-11 18:34:35 [__init__.py:248] Automatically detected platform tpu.
Collecting environment information...

CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 4.0.0
Libc version: glibc-2.35

Python version: 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-6.5.0-1013-gcp-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             224
On-line CPU(s) list:                0-223
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7B13
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 56
Socket(s):                          2
Stepping:                           0
BogoMIPS:                           4899.99
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr arat npt nrip_save umip vaes vpclmulqdq rdpid fsrm
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          3.5 MiB (112 instances)
L1i cache:                          3.5 MiB (112 instances)
L2 cache:                           56 MiB (112 instances)
L3 cache:                           448 MiB (14 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-55,112-167
NUMA node1 CPU(s):                  56-111,168-223
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==2.2.5
[pip3] pyzmq==26.4.0
[pip3] torch==2.8.0.dev20250430+cpu
[pip3] torch-xla==2.8.0+gita7c3e85
[pip3] torchvision==0.22.0.dev20250430+cpu
[pip3] transformers==4.51.3
[pip3] triton==3.3.0
[conda] numpy                     2.2.5                    pypi_0    pypi
[conda] pyzmq                     26.4.0                   pypi_0    pypi
[conda] torch                     2.8.0.dev20250430+cpu          pypi_0    pypi
[conda] torch-xla                 2.8.0+gita7c3e85          pypi_0    pypi
[conda] torchvision               0.22.0.dev20250430+cpu          pypi_0    pypi
[conda] transformers              4.51.3                   pypi_0    pypi
[conda] triton                    3.3.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.1.dev6496+g4d66549.d20250511 (git sha: 4d66549, date: 20250511)
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect

NCCL_CUMEM_ENABLE=0
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1

🐛 Describe the bug

Launching openai serving engine will face the issue:

...
File "/home/xxx/rh/vllm/vllm/entrypoints/cli/main.py", line 7, in <module>
    import vllm.entrypoints.cli.benchmark.main
  File "/home/xxx/rh/vllm/vllm/entrypoints/cli/benchmark/main.py", line 6, in <module>
    import vllm.entrypoints.cli.benchmark.throughput
  File "/home/xxx/rh/vllm/vllm/entrypoints/cli/benchmark/throughput.py", line 4, in <module>
    from vllm.benchmarks.throughput import add_cli_args, main
  File "/home/xxx/rh/vllm/vllm/benchmarks/throughput.py", line 26, in <module>
    from vllm.entrypoints.openai.api_server import (
  File "/home/xxx/rh/vllm/vllm/entrypoints/openai/api_server.py", line 75, in <module>
    from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
  File "/home/xxx/rh/vllm/vllm/entrypoints/openai/serving_chat.py", line 29, in <module>
    from vllm.entrypoints.openai.serving_engine import (OpenAIServing,
  File "/home/xxx/rh/vllm/vllm/entrypoints/openai/serving_engine.py", line 102, in <module>
   class RequestProcessingMixin(BaseModel):
...

pydantic.errors.PydanticUserError: Please use typing_extensions.TypedDict instead of typing.TypedDict on Python < 3.12.

For further information visit https://errors.pydantic.dev/2.11/u/typed-dict-version

I am using Python 3.10.16, and it causes the issue above. The issue was imported by #17032.
TypedDict should be imported from typing for python >= 3.12 or typing_extensions for python < 3.12.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[juncgu]

@robertgshaw2-redhat

[juncgu]

@frieda-huang

[frieda-huang]

@frieda-huang

Thanks for reporting the issue, @juncgu ! I believe another PR (#17962) addressed it as well.