Hugging Face's logo

microsoft
/
Phi-3.5-MoE-instruct

Text Generation
Transformers
Safetensors
multilingual
phimoe
nlp
code
conversational
custom_code
Model card Files Community
43

CPU support by ONNX as phi3?

#1
by alierenak - opened
Discussion
alierenak

As far as I know, this model currently doesn't support running on a CPU due to flash_attn constraints. Are there any plans to release an ONNX version, similar to what was done with phi3

LeeStott
 Microsoft org

3.5 versions will ship too https://huggingface.co./onnx-community

At present there is a 3.5 mini instruct onnx model available https://huggingface.co./onnx-community/Phi-3.5-mini-instruct-web/tree/main

phi0112358

the second link leads to 404 ('-onnx' ist missing)

this link does work:
https://huggingface.co./onnx-community/Phi-3.5-mini-instruct-onnx-web/tree/main

Srrp

How to run onnx model

Sign up or log in to comment