Skip to content

Commit 0d16a58

Browse files
Add GLM-Edge-V notebook (#2572)
1 parent c04fcfb commit 0d16a58

File tree

5 files changed

+1160
-0
lines changed

5 files changed

+1160
-0
lines changed

.ci/skipped_notebooks.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -558,3 +558,7 @@
558558
- ubuntu-20.04
559559
- ubuntu-22.04
560560
- windows-2019
561+
- notebook: notebooks/glm-edge-v/glm-edge-v.ipynb
562+
skips:
563+
- os:
564+
- macos-13

notebooks/glm-edge-v/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
## Visual-language assistant with GLM-Edge-V and OpenVINO
2+
3+
The [GLM-Edge](https://huggingface.co/collections/THUDM/glm-edge-6743283c5809de4a7b9e0b8b) series is [Zhipu](https://huggingface.co/THUDM)'s attempt to meet real-world deployment scenarios for edge devices. It consists of two sizes of large language dialogue models and multimodal understanding models (GLM-Edge-1.5B-Chat, GLM-Edge-4B-Chat, GLM-Edge-V-2B, GLM-Edge-V-5B). Among them, the 1.5B / 2B models are mainly targeted at platforms like mobile phones and car machines, while the 4B / 5B models are aimed at platforms like PCs. Based on the technological advancements of the GLM-4 series, some targeted adjustments have been made to the model structure and size, balancing model performance, real-world inference efficiency, and deployment convenience. Through deep collaboration with partner enterprises and relentless efforts in inference optimization, the GLM-Edge series models can run at extremely high speeds on some edge platforms.
4+
5+
In this tutorial we consider how to launch multimodal model GLM-Edge-V using OpenVINO for creation multimodal chatbot. Additionally, we optimize model to low precision using [NNCF](https://github.com/openvinotoolkit/nncf)
6+
7+
![image](https://github.com/user-attachments/assets/06c51867-0580-4434-962e-31b6068c2001)
8+
9+
## Notebook contents
10+
The tutorial consists from following steps:
11+
12+
- Install requirements
13+
- Convert and Optimize model
14+
- Run OpenVINO model inference
15+
- Launch Interactive demo
16+
17+
In this demonstration, you'll create interactive chatbot that can answer questions about provided image's content.
18+
19+
20+
## Installation instructions
21+
This is a self-contained example that relies solely on its own code.</br>
22+
We recommend running the notebook in a virtual environment. You only need a Jupyter server to start.
23+
For details, please refer to [Installation Guide](../../README.md).
24+
25+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5b5a4db0-7875-4bfb-bdbd-01698b5b1a77&file=notebooks/phi-3-vision/README.md" />
26+
27+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5b5a4db0-7875-4bfb-bdbd-01698b5b1a77&file=notebooks/glm-edge-v/README.md" />

notebooks/glm-edge-v/glm-edge-v.ipynb

Lines changed: 472 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)