microsoft

/

Phi-3.5-MoE-instruct

like 474

Hi, it's not clear to me how much VRAM I need to run this model, as it have 6.6B of active parameters it should fit in 24 GB of VRAM, or I'm wrong?

I have tried using vLLM.

Last question, is possible to change the number of experts?