Leaderboard

• LVLM: ARC Prize, Chatbot, LLM Benchmark, OmniSpatial
• Detection: Zero-Shot COCO, Open-Voc COCO, Open-Voc LVIS
• Text-encoder: MTEB, SuperGLUE
• 6D Object Pose: BOP
• other: KITTI, Mip-NeRF 360

Installation

Install the basic requirements:

pip install -r requirements.txt

Select preferences and run the command to install PyTorch >= 2.5.1 locally.

• CLIP

pip install open_clip_torch

• CoDet

pip install mss git+https://github.com/facebookresearch/detectron2.git git+https://github.com/openai/CLIP.git

• Contact-GraspNet

pip install provider pyrender torch-cluster

• Depth Anything V3

pip install xformers==0.0.29    # for torch 2.5.1
pip install --no-deps git+https://github.com/ByteDance-Seed/Depth-Anything-3.git
pip install e3nn evo "moviepy<2" pillow-heif plyfile pycolmap

• Grounding DINO

pip install git+https://github.com/IDEA-Research/GroundingDINO.git

• Segment Anything V2

pip install git+https://github.com/facebookresearch/sam2.git

• Tag2Text

pip install fairscale git+https://github.com/xinyu1205/recognize-anything.git
wget https://huggingface.co/spaces/xinyu1205/recognize-anything/resolve/main/tag2text_swin_14m.pth?download=true
mv tag2text_swin_14m.pth?download=true checkpoints/tag2text_swin_14m.pth

• Zero123++

pip install diffusers

Web

Install the following packages to run the web server:

pip install fastapi uvicorn

You need to create a Python file that stores the API in a dictionary format within a variable named FUNCTIONS, and set the file path in server.py.

Then, run the server using the command below:

uvicorn server:app --port 8000