Automated Annotation with Auto Label
Auto-label images for use in training models.
Roboflow Auto Label lets you use large foundation vision models (i.e. Grounding DINO) or Roboflow trained models to automatically label images.
Roboflow Auto Label will try to use the following models to identify the objects you specify:
Grounding DINO (Object detection)
Grounded SAM (Segmentation)
Models trained in Roboflow (Train a Model in Roboflow)
Auto Label is powered by Autodistill, an open source framework for auto-labeling image datasets developed by Roboflow.
Auto Label has been used to label millions of images for use in training computer vision models.
When to Use Roboflow Auto Label
You should use Roboflow Auto Label if you need to annotate common objects such as vehicles (i.e. forklifts), people, generic defects (i.e. cracks), and generic products (i.e. vinyl records, bread).
You should not use foundation models in Autodistill if you need to identify specific variants of an object. For example, Autodistill cannot distinguish between different types of crack, or identify unique defects in electronics.
Label Data with Roboflow Auto Label
The Roboflow platform lets you preview how Autodistill will perform on labeling classes of data in your dataset. Then, Roboflow will share a code snippet that you can use to auto-label images on your own hardware. You can upload your labeled dataset back to Roboflow for quality assurance (recommended) and to train a model.
Step #1: Upload Data
First, upload data to Roboflow. See our Upload Data instructions for more information.
Step #2: Select Automated Labeling
Once you have uploaded all of your images, you will be asked how you want to label your images. Select "Auto Label".
Step #3: Choose a Prompt
The Auto Label labeling interface will appear in which you can configure your auto labeling job.
On this page, you can choose one or more prompts for use in labeling your image. This is called an "ontology". You can map prompts, which are sent to your chosen foundation model (Grounding DINO by default) for use in labeling data, to a class name which will be used to save labels to your dataset.
Auto Label works best when labeling common objects. For example, Auto Label will be able to identify the location of a forklift in a manufacturing facility. But, Auto Label will be unable to label images according to specific requirements, such as distinguishing the brand of a forklift.
Once you have configured Auto Label, click "Generate Test Results" to test your prompts on a small subset of your dataset. By default, four images are selected.
Step #4: Evaluate Roboflow Auto Label Labels
Here are the results from Autodistill when prompted with the prompt "forklift" on an example image:
Auto Label uses Grounding DINO to label data by default. Grounding DINO returns bounding boxes. If you want to label segmentation masks, click on "Grounding DINO" in the Auto Label interface and choose "Grounded SAM". Grounded SAM uses Grounding DINO to detect objects, then Segment Anything to generate segmentation masks for objects of interest.
To review labels on different images from the test set used to preview Auto Label on your dataset, click on an image in the "Test images" section off the interface.
You can adjust the confidence threshold to use for labeling.
If Auto Label labels your images as you expect, click "Auto Label with This Model".
If Auto Label doesn't label images as expected, try a different prompt. Otherwise, Auto Label may not presently be able to help label your dataset.
When you click "Auto Label with This Model", Auto Labeling will start. It should take a few minutes to label a thousand images.
Last updated
