本地模型部署工具:OpenVINO™
OpenVINO™ 是一个深度学习模型的部署工具,可实现针对特定模型结构的优化与推理任务提升,目前 OpenVINO™ 已适配x86,ARM 在内的异构硬件终端,支持 CNN,Transformer 等多种网络结构,提供 C/C++/Python 等多类 API 接口语言,不管是传统的机器视觉 NLP 模型,或是最新的 LLM 多模态模型,都可以利用 OpenVINO™ 工具包中提供的丰富接口进行部署。
安装方式:
OpenVINO runtime:pip install openvino
OpenVINO with GenAI: pip install openvino-genai
更多安装方式可以参考:
https://docs.openvino.ai/2024/get-started/install-openvino.html
官方文档:
Documents:https://docs.openvino.ai/
Github:https://github.com/openvinotoolkit
Gitee:https://gitee.com/openvinotoolkit-prc
Notebooks示例:
https://openvinotoolkit.github.io/openvino_notebook****r/>
视频教程:
https://space.bilibili.com/38566875
文字版教程可关注 微信公众号
Python示例:
YOLO示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov11-optimization
https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/yolov10-optimization/yolov10-optimization.ipynb
SAM分割示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sam2-image-segmentation
文生图示例:
1)性能最优:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/latent-consistency-models-image-generation
https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/sdxl-turbo/sdxl-turbo.ipynb
2)效果最佳:
https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/flux.1-image-generation/flux.1-image-generation.ipynb
https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/stable-diffusion-v3/stable-diffusion-v3.ipynb
Whisper语音识别示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/distil-whisper-asR
Chatbot示例(支持多种模型):
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot
RAG示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-llamaindex
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-langchain
多模态示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/qwen2-vl
https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/qwen2-audio/qwen2-audio.ipynb
Agent示例:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-agent-functioncall
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-agent-react
更多实践示例可以参考:
https://github.com/openvinotoolkit/openvino_notebook***r/>
C/C++示例:
基础用法:
https://github.com/openvinotoolkit/openvino/tree/master/sample***r/>
GenAI示例:
https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/cpp
英特尔 AI PC
介绍及使用案例:
https://www.intel.com/content/www/us/en/products/docs/processors/core-ultra/ai-pc.html
哪吒开发套件
哪吒开发板及英特尔边缘开发套件使用案例**: https://inteldevkit.csdn.net/