In our previous posts, we discussed the critical foundations of reliable automotive AI: setting up vehicles for high-quality data collection and leveraging auto-annotation to scale ADAS/AD development and validation. The performance and safety of these systems depend directly on the quality, diversity, and correctness of the training and validation data. However, as data lakes grow into petabyte-scale oceans, a new challenge emerges. For an AI engineer, the question is no longer "Do we have the data?" but rather: "How do we find samples for a rainy night in Tokyo with a pedestrian crossing near a construction site?" Thus, find the data that matters and build diverse datasets for a variety of use-cases. 

At aiMotive, we’ve developed a multi-layered retrieval ecosystem in aiData designed to turn massive datasets into highly targeted training and validation sets, leveraging the dedicated data curation component, the aiData Annotation Center. By combining proven curation methods, such as tagging, with advanced AI-powered retrieval, we dramatically increase the speed, precision, and efficiency of data preparation. While the aiData Annotation Center already supports several external clients, it has become just as essential within our organization, particularly in advancing the development of our ADAS/AD stack, aiDrive. To demonstrate its practical strengths, we walk through an internal case study where a researcher leveraged the tool to build a training dataset. 

The context: preparing a diverse roadwork dataset

Our researcher was tasked with assembling a diverse dataset for a binary image classifier designed to distinguish roadwork from non-roadwork scenes. In our definition, roadwork refers to any construction or maintenance activity that includes cues such as traffic cones, temporary lane markings, warning barriers, partial closures, lane‑narrowing structures, and temporary signage or other traffic‑control elements.

Although the task may sound straightforward, the scale of the challenge quickly became apparent. Our internal dataset contains millions of driving clips, yet none of them carry a reliable or consistently applied roadwork tag. Because roadworks are temporary by nature, static map‑based labels - such as those derived from OpenStreetMap - were unreliable. The researcher needed a way to efficiently surface both positive and negative samples without manually combing through hundreds of millions of frames.

To ensure the classifier would generalize well across real‑world driving scenarios, the dataset needed to reflect a broad spectrum of conditions. This meant sourcing recordings from multiple continents and countries, spanning multiple road categories, including urban, rural, and highways, and covering a wide range of weather and lighting conditions. 

To address this, we adopted a two‑stage strategy. First, we performed large‑scale retrieval using Deja Vu, our iterative tool for fine-tuning image- and text-based searching, to narrow the search space and surface candidate recordings. Then, through iterative, targeted data mining and refinement, we narrowed the data to high‑quality video clips that met the project’s diversity requirements. This combination allowed us to move from an unstructured dataset to a curated collection suitable for training a robust classifier.  

1.  Initial dataset curation with Deja Vu, the iterative tool to fine-tune image and text-based searching

To guide the initial search, we defined a prompt describing a typical roadwork scene: “Road scene showing active construction or maintenance activity, with traffic cones, temporary lane markings, warning barriers, construction vehicles, workers, or equipment. The roadway includes lane shifts, narrowed lanes, blocked sections, or diverted traffic paths, along with roadworks signs or temporary traffic control markings.” Using this description as a reference, we collected an initial set of candidate samples, including at least 100 positive and 100 negative samples. 

Figure 1. The Deja Vu selector is used to mark positive and negative frames. Positive samples are outlined in teal, while negative samples are outlined in red.  

As a next step, we began re-ranking the retrieved examples by defining clear thresholds for positive and negative samples, along with minimum counts for each category. From the filtered set, we generated a metadata file and labeled the selected recordings as potential “roadwork” or “non‑roadwork.” Finally, we performed a manual quality check to ensure the samples met the standards needed for the subsequent refinement stages. 

Figure 2. The Deja Vu review shows the retrieved candidate frames for quick visual check and refinement. 

2. Targeted data mining and enrichment with tag- and text-based search

Once the initial dataset from the first stage had passed quality review, a few remaining gaps became clear. To address these, we conducted targeted data mining and enrichment to fill in missing recordings and ensure the diversity required for a reliable training set. 

2.1 Domain expansion to rural recordings 

We quickly noticed that rural environments were underrepresented in the dataset, as revealed by the statistical analysis of the metadata file generated by Deja Vu. To address this, we applied an auto‑tag for rural scenes to ensure they were included in the search space. We then combined this new tag with text-based search, using the prompt from stage one, which allowed us to surface a more balanced set of samples and properly include rural roadwork scenarios in the dataset.

Figure 3. The Dataset tab combines tag-based filtering with LLM-based text search to narrow down relevant clips.
  
As we continued the review, we also discovered that recordings from Japan were missing, leaving an important regional gap. To address this, we incorporated Japan‑specific samples by applying map‑based filtering and combining them with the roadwork prompt used earlier in stage one. This approach enabled us to gather the necessary region‑specific data and to strengthen the dataset's overall diversity.

Figure 4. The Map view enables location-based filtering and helps select clips from specific geographic areas. 

Figure 5. Map-filtered results are further refined with text search to find more relevant samples.  

2.2 Filtering false positives 

During the validation process, we also noticed a recurring source of false positives: construction vehicles appearing on their own were often flagged as roadwork, even when no active roadwork zone was present. To correct this, we turned again to text‑based search. We gathered examples of construction vehicles in ordinary traffic settings using a prompt describing "Large utility or heavy-duty service trucks driving or parked in ordinary road scenes, with regular permanent lanes, open traffic flow, and typical roadside surroundings. The truck may resemble a dump truck, cement mixer, crane truck, excavator carrier, or maintenance-style vehicle". We complemented this with an image-based search, providing a picture of a TMA truck to further refine the results. These recordings were then labeled as hard negatives to ensure that scenes containing construction vehicles without any active roadwork context would not be misclassified in later stages.

Figure 6. Image-based search retrieves visually similar clips from the dataset using a reference image.

Moreover, highway ramps, retroreflective turn signs, poles, and red-and-white-striped fences also fooled the classifier, leading to more false positives in the dataset. To overcome this, we used prompt-based searching again to collect these hard negatives.

2.3 Dataset re-balancing

As a result of the targeted data-mining phase, the dataset became dominated by positive samples, resulting in a clear class imbalance. To counter this, we conducted focused negative sample mining. We began by clustering the existing roadwork embeddings, then searched outside those clusters by enforcing a minimum distance threshold to surface scenes unlikely to contain roadwork.

To further diversify the negative set, we applied tag‑based filtering across road categories and weather conditions, ensuring that the negative samples reflected a similarly wide range of conditions as the positive ones.

The result: a diverse and well-balanced dataset for training

This case study demonstrated how a high‑quality dataset can be built in just a few focused steps.  At the end of this data-curation process - combining Deja Vu-based retrieval for the initial dataset with targeted mining through tag‑, image‑, text‑, and map-based filtering - we arrived at a final collection of 10,000 frames for training and 3,000 for testing. This set, containing both “roadwork” and “non‑roadwork” sequences, achieved the required diversity across all key dimensions: road category (highway, urban, rural), geographic region, and weather conditions. The aiData platform provides a rich toolkit for data curation, enabling the creation of intuitive, efficient training and validation datasets.

The Future: Scenario-Based Retrieval

While this case study centered on object‑level search, our next frontier for 2026 is scenario‑level search. Leveraging our latest research, we are integrating components that understand the intent and relationship between actors. Instead of searching for "car" and "bicycle," we will search for "bicycle overtaking a car from the right at an intersection." The input can either be a video, or you can leverage one of our most exciting developments, enabling searching for real-world video clips based on drawn sketches without any simulation or image data. Users can simply draw bounding boxes and define how these actors move within a scene. Using this “drawn scenario” as input, the system can then search through real recordings and retrieve matching situations. For example, you can sketch a cyclist’s bounding box passing in front of the ego vehicle, and the search engine will return real‑world recordings that exhibit the same interaction. Keep reading our blogs to learn more about this new feature soon! 

Data is the fuel for AI, but searchability is the engine. By combining traditional tagging, geospatial filtering, and cutting-edge vision-language models, aiMotive’s Data Pipeline enables developers to spend less time searching and more time training the next generation of autonomous intelligence.