***This feature is currently in a closed beta. It will be generally available soon***
OpenAsset AI Visual Search enables finding images based on the visual content they depict, without the need for keywords. This means you can find images that match concepts like “Construction at Sunset”, “Gardens with People”, or “Bridges at Night” based on the visual content of the images. This also means that if your company is making good use of visual search there is much less need to tag images with File Keywords.
Visual Search Mode
Using visual search requires you to switch to visual search mode. You will notice a toggle in the search bar:
Switching to the “sparkle” AI icon will put you into visual search mode:
You can then try a visual search. Type the words or phrase that describe the images that you are looking for and hit return:
Hopefully this gave you great results but if not, try adjusting the search term to see if alternative phrasing gets you to what you need. It’s also worth scrolling down or paging through your search results to see if what you are looking for is not at the top of the screen.
Using Visual Search in combination with Regular Search
You are able to combine visual searches with regular searches in order to find exactly what you need. For example you might want to find “construction at sunset” images but for projects in the “healthcare” sector, in “Chicago”. To do this:
Switch to visual search mode:
Type your visual search and hit enter:
Switch to regular search mode:
Add more search criteria in the usual way. For example project keywords that describe the type of project:
Sort Order with Visual Search
When you have a visual search set, the search results that you see will be fixed at “Visual Relevance”. This is because the relevance ordering is a fundamental part of the visual search functionality. We want to get you the most relevant results to the top of the screen.
You may have a scenario where you apply a visual search and you then want to order those results by another criteria such as date created. To do this, make a Selection of the files that you want to order, then order the Selection.
How does the AI work?
OpenAsset’s visual search is powered by a third party AI model. The model is run within OpenAsset and within AWS so your data is not being sent to a third party. Your data remains within the secure boundary of OpenAsset.
The AI model is pre-trained on 400 million images. OpenAsset has not trained the model, nor is the model being trained as you use it. In that sense the AI is a black box that we do not have influence over. That said, OpenAsset will be reviewing the performance of this feature and the value that it delivers to you in order to take opportunities to deliver improved models in the future.
In order for this feature to work, OpenAsset automatically “indexes” your images using the AI model. This index provides much richer information than traditional keywords. This means you can search with a much wider vocabulary whilst saving your company many hours of manual tagging.
Search Performance
Visual search performance is variable and subjective so you are therefore not guaranteed to get great results every time. This could be for a number of reasons:
Good search results will depend on the images you have. You simply may not have what you are looking for.
The model will have a better understanding of some terms than others. It’s therefore worth trying other terms that might get you to the results that you are looking for.
With some searches you might find that relevant images are returned but the relevance performance isn’t perfect. So it’s always worth scrolling/paging through your results.
The model has not been trained on AEC specific terms. That said, it should perform well on describing common terminology relating to buildings, engineering or construction. For example, words like rendering, aerial shot, excavation, crane, bridge, tunnel, stairs and kitchen should all work well. With very specific technical terms it’s less likely to work.
Searches are likely to work better with more context. So if you can describe what you are looking for with a phrase, you are likely to get closer to what you are looking for than a single word search.
Visual Search Permissions
AI Visual Search is made available to all users by default. However, you are able to control who has access if you wish. The feature can be controlled by permissions at group level. To do this, go:
Settings
System Settings
Users > Groups > [the group you would like to control]
Advanced > Turn “Use AI Visual Search” on
Click “Save Changes”