Cutting a person out of a photo, boxing every car in a street scene or reading the text on a sign used to take three different models in ComfyUI.
WeVisOne-VL, Tencent’s new WeChat Vision model, does all of it from one prompt, and as of 9 November it runs in ComfyUI core with no custom node pack.
You type what you want (“the woman in the red coat”, “all cars”, “read the sign”) and it returns boxes, points, soft alpha mattes or text. This guide covers the files, the folder, the one-node setup, every task mode, a background-removal workflow, the benchmarks and the limits you should know before relying on it.
- What: Tencent WeVisOne-VL, 2B and 4B vision models (Qwen3-VL fine-tunes plus a 9.6M-parameter mask decoder) for detection, segmentation with alpha mattes, pointing, counting and OCR.
- ComfyUI: native node
WeVisOne-VL Detect(added in PR 17632). Needs ComfyUI master: tag v0.44.2 does not have it yet. - Files: from Comfy-Org/WeVisOne-VL into
ComfyUI/models/text_encoders/: 4B int8 is 5.3 GB, 2B int8 is 2.9 GB. - Load it with:
CLIPLoader, typestable_diffusion, into the node’sclipinput. - Licence: Apache-2.0 on the ComfyUI repack.
- Limits: single images only (no video tracking), fine edges can drift on transparent or hidden subjects, one label per line.
What WeVisOne-VL Is
WeVisOne-VL comes from Tencent’s WeChat Vision team. It is built on Qwen3-VL, the vision-language model family, and adds a small 9.6M-parameter mask decoder so it can output pixel masks, not just words.
That combination is the point. A normal vision chatbot can tell you there is a dog in a photo. WeVisOne-VL can tell you where it is, draw a box round it, give you a soft-edged cut-out of it, and count how many there are, all from a plain-English request.
There are two sizes:
- 4B: the stronger model, and the one behind the benchmark numbers below.
- 2B: smaller and lighter, for 8 GB cards or quick previews.
The original weights are at tencent/WeVisOne-VL-4B and tencent/WeVisOne-VL-2B, but ComfyUI needs the Comfy-Org repack, which is the version that loads in the native node.
Files, Sizes and Folder
All four files go in the same folder: ComfyUI/models/text_encoders/. You only need one.
| File | Size | Pick it when |
|---|---|---|
wevisone_vl_4b_bf16.safetensors |
9.7 GB | 16 GB+ card, best quality |
wevisone_vl_4b_int8_convrot.safetensors |
5.3 GB | The sensible default for 8 to 12 GB cards |
wevisone_vl_2b_bf16.safetensors |
4.9 GB | Smaller model at full precision |
wevisone_vl_2b_int8_convrot.safetensors |
2.9 GB | Low-VRAM cards, or running next to a big image model |
Neither Tencent nor Comfy-Org publishes a VRAM figure. My “pick it when” column is a sizing rule from the file sizes, not a measured result: leave a few GB above the file size for the image and the decoder.
If you plan to run it in the same graph as a large image or video model, the 2B int8 file is the safe choice. Our ComfyUI VRAM guide explains how models share the card.
Setup in ComfyUI
- Update ComfyUI to master. The node landed on 9 November. The latest tagged release at the time (v0.44.2) does not include it, so the desktop app and Comfy Cloud will get it with the next stable release. On a git install, run
git pullin the ComfyUI folder. On the portable build, run the update script in theupdatefolder. See our ComfyUI install guide if you are unsure which build you have. - Download one model file from Comfy-Org/WeVisOne-VL into
ComfyUI/models/text_encoders/. - Restart ComfyUI and refresh the browser.
- Add a Load Image node with your photo.
- Add a CLIPLoader node. Set type to
stable_diffusionand pick the WeVisOne file. - Add WeVisOne-VL Detect (search for it, or find it under
image/detection). Connect the image and connect the CLIPLoader output to itsclipinput. - Pick a task, type your query, and queue.
The node has four outputs:
masks: the soft alpha mattes.bboxes: the boxes, as JSON in pixel coordinates.points: points, as JSON in pixel coordinates.- the raw answer text, useful for OCR and counting.
Turn on individual_masks if you want one mask per box. Leave it off for a single combined mask of everything that matched.
Every Task Mode Explained
| Task | What it returns | Example query |
|---|---|---|
segmentation |
Boxes plus soft alpha mattes | person |
detection |
Boxes for each label | car (new line) bicycle |
referring |
The one object a phrase describes | the woman holding the blue umbrella |
point |
A point on each match | door handle |
count_box / count_point |
A count, with boxes or points | apples |
ocr |
The text in the image | (no query needed) |
ocr_grounding |
Text plus where each line sits | (no query needed) |
visual_prompt |
Matches guided by a visual hint | depends on your hint |
freeform |
A free text answer | what is the person doing? |
One detail catches people out: detection and point expect one label per line. If you type “car, bicycle, bus” on one line, the model reads it as a single strange phrase and may find nothing.
The model is also good at saying “not there”. On WeRef-Bench, which tests whether a model wrongly finds objects that are absent, the 4B scored 96.4 against 73.0 for the best open baseline. So if it returns nothing, the object most likely is not in the image or your phrase does not match it.
Workflow: Background Removal With a Soft Matte
This is the job most people will want. Because WeVisOne-VL outputs a soft alpha matte rather than a hard mask, hair and fuzzy edges come out cleaner than a simple threshold mask.
- Load Image with your photo.
- CLIPLoader (type
stable_diffusion) with the WeVisOne file. - WeVisOne-VL Detect, task
segmentation(orreferringfor one specific subject), query “person” or a precise phrase. - Connect
masksto an InvertMask node. - Feed the image and the inverted mask into JoinImageWithAlpha.
- Save Image. ComfyUI saves it as a PNG with transparency.
The InvertMask step is needed because JoinImageWithAlpha treats the mask as the area to make transparent. The page that announced the node gives exactly this InvertMask plus JoinImageWithAlpha chain for cut-outs.
For anime art that you generate rather than cut out, Anima RGBA makes images with a real alpha channel from the start. For a CPU-only option outside ComfyUI, see our local background removal guide.
More Workflow Ideas
- Inpainting masks from text: use
referringwith “the man’s jacket” and send the mask into your inpaint workflow, instead of painting the mask by hand. - Product photos:
segmentationwith the product name, then composite onto a new background. - Auto-crop: take a box from
bboxesand crop to it before upscaling or editing. - Read text before editing it: run
ocron a sign, then use the exact text in an edit prompt for Qwen-Image 2.1 edit. - Counting:
count_boxon stock shelves, crowds or parking spaces, with the boxes to check the count.
Benchmarks
These are the 4B model’s scores against the strongest open baseline, as reported with the ComfyUI release.
| Task | Benchmark | Best open baseline | WeVisOne-VL-4B |
|---|---|---|---|
| Detection | LVIS (F1) | 50.7 | 55.3 |
| Rejecting absent targets | WeRef-Bench | 73.0 | 96.4 |
| OCR | TotalText (F1) | 43.3 | 52.0 |
| Referring segmentation | ReasonSeg (gIoU) | 65.0 | 71.2 |
| Box matting | P3M-500-NP (MAD, lower is better) | 15.4 | 7.3 |
The matting result stands out: its error is less than half the baseline’s. That lines up with the cut-out use case above.
WeVisOne-VL vs Other Masking Tools
| Tool | Strength | Weak spot |
|---|---|---|
| WeVisOne-VL | One model for boxes, mattes, points, counts and OCR from text; native in ComfyUI | Single images only |
| SAM 3 (Meta) | Strong promptable masks and video tracking | No OCR or counting answers; a separate model to set up |
| A general vision LLM (for example Qwen3-VL) | Rich descriptions and reasoning | No pixel masks on its own |
| Classic background removers | Very fast, tiny | Cannot pick “which” subject from a phrase |
If you need to track an object across video frames, WeVisOne-VL is not the tool, because its pixel decoder rejects video input. For picking out one described thing in a still image and getting a clean matte, it is now the simplest option in ComfyUI.
For general vision chat, see our list of local vision LLMs.
Limits
- Still images only. The pixel decoder rejects video grids, so it is not a tracking node.
- Hard edges. Fine alpha edges can drift on heavily hidden or transparent subjects, such as glass or veils.
- Wording matters. Different phrasing can change the result. If a query misses, try a simpler or more specific phrase.
- Label format. One label per line for
detectionandpoint.
Troubleshooting
I cannot find the WeVisOne-VL Detect node
Your ComfyUI is older than the 9 November master build. Update to the latest master. A tagged stable release (v0.44.2 or earlier) does not include it. If the node is still missing after updating, check the console for an import error.
The CLIPLoader does not list the file
The file must be in ComfyUI/models/text_encoders/, not checkpoints or diffusion_models. Press R in ComfyUI to refresh the model lists.
Error or nonsense output after loading
Check the CLIPLoader type is set to stable_diffusion, as the release notes say, and that you downloaded the Comfy-Org repack rather than the original Tencent files.
The node finds nothing
Put each label on its own line, use simple nouns, and check the object really is in the image. The model is tuned to reject absent objects rather than guess.
Hair or glass edges look wrong
That is a known limit. Try the 4B bf16 file, a tighter query, or a cleaner source image. For hard cases, touch up the mask in an editor.
Out of memory
Switch to the int8 file or the 2B model, and do not load it in the same run as a large video model.
FAQ
Is WeVisOne-VL free?
Yes. The Comfy-Org repack is listed under Apache-2.0.
Do I need a custom node pack?
No. It runs on ComfyUI core with the WeVisOne-VL Detect node, as long as you are on a recent master build. There are also community node packs, but you do not need them.
2B or 4B?
4B for quality, 2B for low VRAM or when it shares the card with a big model. The published benchmarks are for the 4B.
Can it remove backgrounds from video?
Not directly. It works on single images and rejects video input, so it cannot track a subject through a clip.
Does it read text in other languages?
The release lists OCR and OCR grounding but does not list languages. It comes from WeChat’s team, so test it on your own text before relying on it.
Why is it loaded as a text encoder?
ComfyUI loads vision-language models through the CLIP loader path. The file goes in text_encoders and plugs into the node’s clip input, even though it does vision work.
Sources
- ComfyUI Wiki: WeVisOne-VL in ComfyUI (files, node, tasks, benchmarks, limits)
- ComfyUI PR 17632
- Comfy-Org/WeVisOne-VL (licence, files)
- tencent/WeVisOne-VL-4B and tencent/WeVisOne-VL-2B
