Medialake uses several specialist AI and machine-learning components rather than relying on one model for every task. Each component has a defined purpose, such as finding visually related assets, searching by meaning, recognising known public figures, matching approved talent references, or answering questions in natural language.
This page explains the technical flow, the role of each component, and the boundaries to keep in mind when interpreting AI-generated results.
At a glance
Component | Purpose | Input | Output | Medialake capability |
HyperOne | Visual fingerprinting | Images and representative portions of videos | Compact visual fingerprints and similarity scores | Precise Scan, visual matching, content mapping |
Google Cloud Vertex AI | Semantic retrieval | Library assets, text searches, or a reference image | Multimodal vector embeddings and relevance rankings | AI/Vector Search and reverse image search |
Amazon Web Services (AWS) | Celebrity recognition | Visible faces in images and videos | Candidate celebrity names, confidence information, and video timestamps where available | Celebrity detections and celebrity-based search |
Medialake proprietary OCR model | Optical character recognition (OCR) | Visible text in images and video frames | Recognised text, confidence information, positions, and video timestamps where available | Character Recognition and text-based search |
Medialake proprietary segment-detection model | Video segment detection | Video assets | Time-based segment boundaries and associated detection information | Segment detections and timeline navigation |
Medialake proprietary facial-embedding model | Comparison with approved talent references | Detected faces in images and videos | A numerical facial representation for each detected face | Talent matching and grouping |
Google Gemini | Natural-language and visual reasoning | User questions, approved business data, and images when a feature requires visual analysis | Generated answers, summaries, structured results, and explanations | Medialake Intelligence and other enabled generative-AI features |
End-to-end processing flow
An asset enters Medialake. The asset is uploaded or synchronised from a connected source. Medialake records the asset and prepares the viewable media needed by enabled analysis services.
Eligible analysis is queued. The file type, feature configuration, and processing status determine which specialist components run. Processing happens asynchronously, so an asset can remain available while analysis is under way.
The relevant component analyses the content. Depending on the feature, the component receives a time-limited view of the asset, a representative video segment, a search phrase, or a reference image.
Medialake stores derived results. Fingerprints, embeddings, detections, confidence information, and generated responses are linked to the original asset and its team. These derived records can be recalculated if the processing configuration changes.
Product features use the results. Medialake compares stored representations, ranks relevant assets, or presents structured detections and generated answers. Normal team and asset access controls still apply.
The components do not form a single decision-making pipeline. For example, HyperOne and Vertex AI both represent visual content numerically, but they solve different problems and their results are not interchangeable.
HyperOne visual fingerprinting
HyperOne produces a compact visual fingerprint for an image or video. Medialake compares these fingerprints using vector similarity to find assets that contain the same or closely related visual material.
For video, multiple visual fingerprints can represent different portions of the asset. Medialake can use an individual portion for a precise comparison or combine the results to represent the video more broadly.
HyperOne powers capabilities such as:
Precise Scan, which surfaces visually matching assets and a similarity score
matching an image against images or videos that contain related visual material
finding reuse of creative assets across connected media and campaign sources
content mapping between library assets and published or paid-media assets
A visual fingerprint match is different from an identical-file match. Identical files can be detected using a standard file hash. HyperOne is useful when two files are visually related even though their encoding or other technical file properties differ.
Vertex AI semantic search
Medialake uses Google Cloud Vertex AI multimodal embeddings for semantic and reverse image search. An embedding is a numerical representation of the meaning and visual characteristics of content.
When assets are processed:
an image is represented by an image embedding
a video is divided into representative segments, with an embedding stored for each processed segment
each embedding remains associated with its asset and team
When a user performs a search:
a natural-language query, such as
videos showing a product on a beach, is converted into a text embedding, oran uploaded reference image is converted into an image embedding
Medialake compares the search embedding with the library embeddings using cosine similarity. Results are ranked from the closest semantic match to the least similar match within the result set.
Semantic search is designed to understand concepts rather than only exact words in filenames or metadata. Reverse image search uses the same retrieval approach but starts with the visual meaning of the supplied image.
AWS celebrity recognition
Celebrity recognition uses an AWS-managed recognition service to compare visible faces with its catalogue of well-known public figures.
For each candidate match, Medialake can store:
the recognised name
the provider's confidence information
the position or time at which the person appears, where available
supporting descriptive information returned or resolved for the detection
These records power the Celebrities area, asset-level detections, timeline navigation for video, and celebrity-based filtering or search.
Celebrity recognition and talent matching are separate capabilities. AWS celebrity recognition uses a provider-managed catalogue of public figures. It is not used to identify a client's own talent library.
Medialake proprietary OCR model
Medialake uses its own proprietary optical character recognition model to detect visible text in images and video frames. Eligible assets are sent to Nataero for processing by the model. OCR can identify text such as captions, product labels, packaging, signs, logos, and text added as part of a creative.
For each detection, Medialake can store:
the recognised word or line of text
the provider's confidence information
the text's position within the image or video frame
the time at which the text appears in a video, where available
These records power Character Recognition, text-based asset search, and timeline navigation to points where the text appears.
OCR reads visible characters; it does not confirm that the text is factually correct or infer the intended meaning. Small text, stylised fonts, motion blur, low contrast, unusual orientation, occlusion, and unsupported character sets can reduce recognition quality.
Medialake proprietary segment-detection model
Medialake uses its own proprietary segment-detection model to identify the temporal structure of a video. Eligible videos are sent to Nataero for processing by the model. Each detection represents a time-bound portion of the video identified from changes or transitions in its visual content.
For each detected segment, Medialake can store:
the start and end time
the segment duration
associated confidence or technical information, where available
Segment detections support timeline navigation and give other Medialake workflows time-based sections of a video to work with. They describe detected visual boundaries rather than the meaning of the content.
Segment detections are also distinct from the representative video segments used by Vertex AI semantic search. The proprietary model detects time-based boundaries; Vertex AI embeddings represent selected portions of a video for relevance matching.
Medialake proprietary facial-embedding model
Talent matching uses Medialake's proprietary facial-embedding model. It detects distinct faces and converts each face into a numerical representation based on visual facial characteristics.
The derived result can include:
a facial embedding
the face's location within the source image or frame
a representative face crop
technical information used to assess and compare the detection
The embedding provides the technical basis for comparing a detected face with an approved talent reference or grouping repeated appearances of the same person. The embedding does not provide a person's name by itself. A named association must come from a separately managed and approved reference.
Facial similarity should be treated as supporting evidence rather than definitive identity proof. Image quality, pose, lighting, occlusion, age, and the quality of the reference material can all affect the result.
Gemini large language model
Medialake uses Google Gemini for enabled large-language-model and visual-reasoning features. Gemini is the generative component of the architecture. It is distinct from the embedding and recognition services described above.
Depending on the enabled Medialake feature, Gemini can:
interpret a user's natural-language question
work with approved, team-scoped business and asset data
summarise analytical results
format results as text, tables, or chart-ready data
analyse an image against a supplied question or brand rule
return structured answers with a reason, explanation, and confidence value where the workflow requires them
Medialake controls the context and tools made available for each workflow. A model does not receive unrestricted access to the application or database. Analytical access is limited to approved data views and is scoped to the relevant team.
Generative answers are produced from the available context at the time of the request. They can be incomplete or incorrect and should be reviewed before they are used for material business, legal, rights, or compliance decisions.
How the components differ
Question | Component |
Is this the same or closely related visual creative? | HyperOne |
Which assets are conceptually relevant to this phrase or reference image? | Vertex AI semantic search |
Does this asset contain a well-known public figure? | AWS celebrity recognition |
What visible text appears in this asset, and where? | Medialake proprietary OCR model |
Where do detected visual segments begin and end in this video? | Medialake proprietary segment-detection model |
Does this detected face resemble an approved talent reference? | Medialake proprietary facial-embedding model |
Can the system explain, summarise, analyse, or converse about available information? | Gemini |
Using separate components keeps each result tied to the type of evidence that produced it. A semantic-search result is a relevance ranking, a fingerprint result is a visual similarity measurement, a facial match is a comparison, and an LLM response is generated content.
Data handling and access boundaries
AI processing is initiated by Medialake's server-side services, not directly from the browser to a model provider.
For OCR and segment detection, eligible media is sent to Nataero for processing by Medialake's proprietary models.
Where a processing service needs asset content, Medialake uses authenticated service connections and time-limited access to the required media.
Temporary processing copies are separate from the original source asset.
Derived results are linked to the asset and team that produced them.
Search and retrieval remain subject to the user's normal Medialake access.
Features can be enabled or disabled according to the customer's deployment and agreed configuration.
Provider-specific processing locations, retention commitments, and contractual controls depend on the customer's deployment. These should be confirmed in the environment-specific data-processing and service documentation.
Facial matching should be configured and used in line with the customer's privacy, consent, and access-control policies.
Accuracy and operational considerations
Similarity scores express mathematical closeness. They are not a percentage probability that two assets or people are the same.
Celebrity and talent results can contain false positives or miss a valid match.
OCR can misread or miss text, especially when the source is small, stylised, blurred, obstructed, or low contrast.
Segment boundaries can be missed or placed differently from a human editor's interpretation, particularly around fades and gradual transitions.
Semantic search ranks the closest available results, so lower-ranked results can be weakly related to the query.
Generative AI can produce plausible but incorrect statements.
Poor image quality, very short video segments, obstructed faces, or limited visual detail can reduce result quality.
Newly added assets may not appear in AI-powered results until their background processing has completed.
Human review remains important whenever a result affects rights, compliance, identity, or a significant business decision.