Is there any order for ms-swift and vllm installation?

[chzhyang]

Before install ms-swift, i have env:
python3.10, vllm==0.6.6.post1, torch==2.5.1, cuda_version is 12

Then in installed ms-swift: pip install ms-swift[all] -U, it showed an error:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. cnstd 1.2.3 requires opencv-python>=4.0.0, which is not installed. transparent-background 1.3.2 requires opencv-python>=4.6.0.66, which is not installed.

It seems not related to ms-swift and vllm, and pip show ms-swift represent the ms-swift==3.1.0, so i just passed this error.

Then i run inference.py(using ms-swift and vllm to load model)

from swift.llm import ModelType, get_vllm_engine
model_type = ModelType.llama3_70b_instruct
vllm_engine = get_vllm_engine(model_type=model_type,
                                  model_id_or_path=model_path,
                                  tensor_parallel_size=args.tensor_parallel_size,
                                  max_num_seqs=args.max_num_seqs,
                                  engine_kwargs={
                                      'enable_prefix_caching': args.enable_prefix_caching,
                                  })

it showed an error:
ImportError: cannot import name 'get_vllm_engine' from 'swift.llm'

After that, i reinstalled vllm pip install vllm -U, it installed the latest vllm ==0.7.1, but the upon ImportError also exist.

Could you guide the best practice of ms-swift and vllm installation???

I checked related issues: not work.

• ImportError: cannot import name 'get_vllm_engine' from 'swift.llm' #1557
• ImportError: cannot import name 'get_vllm_engine' from 'swift.llm' #1557

[Jintao-Huang]

vllm的推理代码请查看examples/infer