Farm3D

Farm3D is a tool developed by the University of Oxford that creates articulated 3D animal models from a single image. It uses a unique approach by leveraging a 2D diffusion-based image generator, like Stable Diffusion, to train a network that reconstructs detailed 3D shapes without needing real 3D data. This method allows Farm3D to generate 3D models with fine details such as legs and ears, even though it was not trained on real images. The tool supports controllable 3D synthesis, enabling users to adjust lighting, swap textures between models of the same animal category, and animate the shapes. It works with both real images and images generated by Stable Diffusion, producing 3D assets in seconds. Farm3D factors input images into components like articulated shape, appearance, viewpoint, and light direction, giving users control over the final model. Farm3D also introduces the Animodel dataset, a collection of textured 3D meshes of articulated animals such as horses, cows, and sheep, with realistic poses. This dataset serves as a benchmark to evaluate the quality of single-view 3D reconstruction for articulated animals. This tool is ideal for researchers, digital artists, and developers interested in 3D animal modeling, animation, and synthesis without requiring extensive 3D training data. Its novel use of diffusion models for virtual supervision and scoring sets it apart from traditional 3D reconstruction methods. Farm3D's approach reduces the need for costly 3D annotations by using synthetic views generated and critiqued by diffusion models during training. This results in a monocular reconstruction network that is fast, flexible, and capable of producing high-quality 3D animal models with controllable features.

Key Features

• 🐾 Single-image 3D reconstruction creates detailed animal models quickly

• 🎨 Texture swapping lets you customize appearances between models

• 💡 Adjustable lighting enhances realism with relighting controls

• 🎬 Animation support adds movement to 3D animal shapes

• 🖼️ Uses synthetic views from Stable Diffusion for training without real 3D data

Pros

• Generates detailed 3D animal models from a single image without real 3D data

• Supports controllable synthesis including animation, texture swapping, and relighting

• Trains using synthetic views from diffusion models, reducing data collection costs

• Produces results quickly, suitable for creative and research applications

• Includes a benchmark dataset for evaluating articulated animal reconstruction

Cons

• Currently focused on articulated animals, limiting use for other object categories

• Advanced features like animation and texture swapping may require Pro plan access

FAQ

How does Farm3D create 3D models from a single image?

Farm3D uses a network trained with synthetic views generated by Stable Diffusion to reconstruct detailed 3D animal shapes from one input image.

Can I animate the 3D animal models created with Farm3D?

Yes, Farm3D supports animation of the articulated 3D animal shapes, allowing you to bring your models to life.

Is it possible to change the texture or lighting of the 3D models?

Farm3D lets you swap textures between models of the same category and adjust lighting through relighting controls for realistic effects.

Do I need real 3D data to train Farm3D's reconstruction network?

No, Farm3D trains its network using virtual supervision from a 2D diffusion model, eliminating the need for real 3D annotations.

What types of animals can Farm3D reconstruct?

Farm3D focuses on articulated animals like horses, cows, and sheep, capturing fine details such as legs and ears.

How fast can Farm3D generate a 3D model from an image?

The system can produce controllable 3D assets from a single image in a matter of seconds.