How to search photos by what's in them

You know the photo exists.

You know the photo exists. A dog on a beach, a sketch on a napkin, a product shot with teal packaging. You took it, saved it, maybe even edited it. But when you open your photo library and try to find it, you're left scrolling through thousands of thumbnails, hunting by date or folder name, hoping your past self was organised enough to help your present self out.

This is the central frustration of working with images. Text files have words you can search. Spreadsheets have columns you can filter. But photos? Most tools treat them as opaque blobs of pixels, searchable only by the thin layer of metadata wrapped around them: the date, the filename, the folder. If you didn't name the file descriptively or sort it into the right place at the time, it might as well not exist.

It doesn't have to work this way. Semantic image search lets you describe what's in a photo and find it, the same way you'd describe it to another person. And when you pair that with colour search and similar search, you get something closer to how visual memory works: not by dates and labels, but by what things looked like.

Why filename and date search falls short

Traditional photo search relies on metadata. That means the data attached to the image file rather than the image itself: the date it was taken, the camera settings, GPS coordinates if your device recorded them, and whatever you chose to call the file. Some tools index folder names too, so if you dragged a batch of holiday photos into a folder called "Crete 2024", you can find them that way.

This works well enough when you remember roughly when you took something, or when you've been disciplined about naming conventions. But it breaks down quickly in practice. A freelance photographer might shoot thousands of images in a week. A design team might accumulate tens of thousands of assets across projects. Nobody names every file descriptively, and nobody remembers the exact date of every shot.

The deeper problem is that metadata describes the circumstances of a photo, not its contents. The filename "IMG_4823.heif" tells you nothing. Even a well-named file like "team-offsite-march.jpg" doesn't help you find the specific frame where everyone was laughing, or the one with the whiteboard visible in the background.

If you want to understand what metadata can and can't do for your photo library, the guide on what photo metadata is covers the details.

How semantic image search works

Semantic search bridges the gap between what you're looking for and what's stored in your files. Rather than matching keywords against filenames, it understands the visual content of your images: the objects, scenes, colours, textures, and spatial relationships within them.

The technology behind this involves machine learning models trained on vast datasets of images and their descriptions. These models learn to represent both images and text in a shared mathematical space, so that a photo of a sunset over the ocean and the phrase "sunset over water" end up close together. When you type a search query, the system converts your words into this shared representation and finds the images nearest to it.

This means you can search in natural, descriptive language. "Red bicycle leaning against a wall." "Group of people around a conference table." "Close-up of handwritten notes." The search doesn't need exact keyword matches because it understands meaning. A search for "dog on a beach" will find your photo even if no one ever tagged it or named it anything related to dogs or beaches.

Fabric's search works this way across your entire library. Every image you store is analysed and indexed by its visual content, so the moment you save a file, it becomes searchable by what's in it. There's no manual tagging step, no waiting for someone to organise things first. Because Fabric serves as a persistent indexing layer across all your connected tools, images from Google Drive, Dropbox, or any other source are included alongside files stored in your cloud.

Searching by colour

Sometimes you don't remember the subject of a photo, but you remember its dominant colour. The presentation slide had a deep navy background. The product mockup was predominantly coral. The landscape shot was all golden hour warmth.

Colour search lets you find images this way. You can use a colour picker to select a precise shade, or describe it in natural language: "forest green", "pastel pink", "warm orange". The system scans your library for images where that colour is prominent and returns them ranked by relevance.

This is particularly useful for designers and brand teams who work with colour palettes daily. If you need to find every asset that uses a specific brand colour, or you're looking for stock imagery that matches a mood board, colour search gets you there without scrolling. The guide on searching images by colour goes deeper into how this works in practice.

Finding visually similar content

There's a third mode of searching that's less about describing what you want and more about pointing at something and saying "more like this." Similar search takes an image you already have and finds others that share visual characteristics: similar composition, subject matter, colour palette, or style.

For photographers culling a shoot, this is invaluable. Find one frame you like, then pull up every visually related image from the session. For designers maintaining a library of visual assets, it helps surface alternatives you might have forgotten about. For anyone managing a large image collection, it's a way to discover connections you didn't know existed.

Similar search works because the same underlying representation that powers semantic search also captures visual similarity. Images that look alike end up near each other in that mathematical space, so finding related content is a matter of looking at nearby points.

You can read more about this approach in the guide on finding files that look similar to one you have.

Automatic organisation and tagging

Search is only half the picture. The other half is how your photos get organised in the first place. If every image sits in a single unsorted heap, even good search can feel overwhelming when it returns dozens of results.

Smart organisation addresses this by automatically tagging and categorising your images as they're added to your library. The same visual understanding that powers search also powers organisation: the system recognises what's in each image and groups related content together without you lifting a finger.

This means your photo library develops structure over time, even if you never manually sort a single file. Screenshots cluster with screenshots. Landscapes group with landscapes. Product photos sit alongside other product photos. The system handles common image formats across the board, so it doesn't matter whether your files are JPEG, PNG, HEIF, or RAW. The guide on image file formats explains the differences between these if you're deciding what to shoot or export in.

For those building a more deliberate organisational system, the guide on how to organise a photo library covers strategies that work well alongside automatic tagging.

Who benefits most from visual search

Anyone with more than a few hundred photos benefits from being able to search by content rather than metadata. But some groups find it transformative.

Photographers deal with enormous volumes of images. A single wedding shoot might produce 2,000 frames. A commercial photographer working across multiple clients might accumulate tens of thousands of images in a year. Being able to search for "bride holding bouquet" or "close-up of product on marble surface" turns a needle-in-a-haystack problem into a five-second task.

Designers and creative teams maintain asset libraries that grow constantly. Finding the right visual reference, the right texture, the right mood image for a presentation shouldn't require remembering where you filed it or what you called it. Semantic search and similar search let you work by visual intent rather than organisational memory.

Researchers and archivists working with historical or documentary image collections can surface specific content across thousands of unsorted files. A query like "handwritten letter" or "aerial view of coastline" can cut through decades of unorganised material.

Anyone managing photos across devices and services faces the problem of fragmentation. Images scatter across phones, cameras, cloud services, and hard drives. Fabric's memory engine remembers context about your images as they accumulate, so the system's understanding of your library deepens over time. When everything lives in one searchable library, the question shifts from "where did I put that?" to "what was in it?" And the second question is almost always easier to answer.

Combining search methods

The most effective way to find photos isn't to rely on a single search method but to use several in sequence. Start with a semantic search to narrow down the general area. Refine by colour if the palette matters. Use similar search to fan out from a promising result.

This layered approach mirrors how visual memory works. You might remember a photo as "that outdoor shot from the trip, the one with all the blue." Semantic search handles "outdoor shot." Colour search handles "all the blue." Between them, you get to the right image far faster than scrolling or filtering by date.

Fabric's explorer supports this kind of fluid, iterative searching. You're not locked into a single query; you can move between search modes, refine as you go, and follow visual threads wherever they lead.

Practical tips for better photo searching

A few habits make visual search even more effective. First, keep your images in their original quality where possible. Heavily compressed thumbnails lose visual detail that search models use to understand content. The comparison between RAW, JPEG, and HEIF formats is worth reading if you're deciding how to store your originals.

Second, don't delete your "almost" shots too aggressively. A photo you think is a throwaway today might be exactly the reference image you need in six months. With good search, the cost of keeping more images drops significantly because finding things doesn't depend on keeping collections small.

Third, let automatic organisation do its work. Resist the urge to manually sort everything into rigid folder hierarchies. Folders impose a single organisational axis, and that axis is rarely the one you need when you're searching later. Tags and smart categorisation are more flexible because a single image can belong to many categories at once.

The guide on photo storage and organisation covers broader strategies for managing growing collections.

Frequently asked questions

Can semantic search find specific people in photos?

Semantic search can identify that a photo contains a person or group of people, and it can often distinguish characteristics like clothing, setting, or activity. It works best when you describe the scene rather than naming individuals. A search for "person in red jacket standing by a lake" will return relevant results, while searching for a specific person's name would require face recognition, which is a different technology.

Does image search work with screenshots and digital artwork?

Yes. Semantic search analyses the visual content of any image, regardless of how it was created. Screenshots of websites, digital illustrations, UI mockups, and scanned documents are all searchable by their content. A search for "bar chart" will find screenshots of presentations containing bar charts, for instance.

How accurate is colour search compared to picking a hex code?

Colour search supports both approaches. You can select a precise colour using a picker, which is useful when you know the exact shade you need. Natural language descriptions like "teal" or "burnt orange" are slightly less precise but far more convenient for exploratory searching. The system interprets colour terms broadly enough to be useful while staying specific enough to filter out irrelevant results.

Will search find photos stored in RAW format?

Semantic search works across all common image formats, including RAW files from major camera manufacturers. The visual analysis processes the image data regardless of the container format, so a RAW file is just as searchable as a JPEG or HEIF. The guide on image file formats explains the practical differences between these formats.

How is this different from Google Photos search?

Google Photos offers some content-based search capabilities, but it's limited to the Google ecosystem and works primarily with photos synced through Google's services. Fabric's semantic search works across files from any source, in any supported format, stored in your own cloud library. It also integrates colour search and similar search into a single workflow, which Google Photos doesn't offer in the same way.

Can I search for text that appears inside a photo?

If a photo contains legible text, such as a whiteboard, a sign, a printed page, or a screenshot, semantic search can often find it based on the textual content visible in the image. This is particularly useful for finding photos of whiteboards from meetings, screenshots of error messages, or images of documents.

Does the search improve over time as I add more photos?

The underlying search models don't train on your personal images, so the quality of understanding stays consistent regardless of library size. What does improve is the usefulness of features like similar search: the more images you have, the richer the connections between them, and the more likely you are to find relevant alternatives when searching for visually similar content.

How do I search for photos when I only remember one detail about them?

Start with whatever you remember, even if it's vague. "Blue background" or "two people outdoors" or "close-up of food" are all valid starting points. Semantic search is designed to work with partial descriptions. Once you find a result that's close to what you're looking for, use similar search to fan out and find the exact image. Searching iteratively, starting broad and narrowing down, is often faster than trying to compose the perfect query on the first attempt.

What happens when I search and get too many results?

Combine search methods to narrow things down. If a semantic search for "landscape" returns hundreds of images, add colour constraints or refine your description: "landscape with snow-covered mountains" or "landscape at golden hour." You can also use Fabric's explorer to filter and sort results, applying additional criteria until you reach a manageable set.

Is my photo library analysed locally or in the cloud?

Visual analysis happens in the cloud when images are added to your Fabric library. This allows the system to use powerful models that would be impractical to run on a local device. Your images are processed securely, and the resulting search index is tied to your account. The guide on finding old photos without remembering when they were taken covers more about how this works in practice for large historical collections.