Dataplex in Google Cloud: The Complete Intermediate Guide

Dataplex in Google Cloud: The Complete Intermediate Guide

Learn how Google Cloud finds, describes, checks, and protects data across your whole organization, from catalogs and glossaries to quality scans, lineage, and governance, explained step by step in plain English.

Imagine a gigantic library that has grown for twenty years. Books sit in many buildings, some on shelves, some in boxes, some in a basement nobody visits. There is no catalog. A researcher who needs one specific report must wander for days, and even after finding something, nobody can say who wrote it, whether it is still correct, or who may read it. Most companies treat their data exactly like that library. Dataplex is the librarian, the catalog, the quality inspector, and the security desk, all in one service.

This guide explains Dataplex from the first idea to production-grade design choices. You do not need any data governance background. Every new word is explained the first time it appears, with an everyday comparison. By the end you will know what the service does, how its catalog is organized, how metadata flows in, how quality checks and lineage work, how access is controlled, and which mistakes to avoid.

1What Dataplex Is and Why It Exists

Dataplex is Google Cloud’s service for discovering, understanding, governing, and trusting data that is scattered across many systems.

The problem: data everywhere, trust nowhere

A modern company stores data in many places at once. Sales figures sit in a data warehouse. Raw logs sit in storage buckets. Customer records live in operational databases. Machine learning features sit in yet another system. Each team builds its own tables, names its own columns, and fixes its own errors. Soon nobody knows what data exists, which copy is correct, where a number came from, or who is allowed to see it.

This causes real damage. Analysts spend more time hunting for data than analyzing it. Two dashboards show different revenue figures, and nobody can explain why. A privacy officer cannot answer which tables contain personal information. An engineer changes a column and silently breaks twelve reports. The root cause is the same each time: the company has data but lacks reliable information about its data.

Simple Analogy

Think of a supermarket where products have no labels, no price tags, no expiry dates, and no aisle signs. The food might be excellent, but shoppers cannot find it, cannot trust it, and cannot tell whether it is safe. Dataplex puts labels, signs, and inspection stickers on your data.

Information about data is called metadata

Metadata means data about data. The name of a table, the meaning of a column, the owner of a dataset, the date it was last updated, and the quality score of its contents are all metadata. Managing metadata well is the foundation of data governance, because you cannot protect, fix, or share what you cannot describe.

A note about names and history

Names in this area have changed several times, so it helps to know the story. Google first offered a service called Data Catalog for tagging and searching data. It then introduced Dataplex, which organized data into lakes, zones, and assets and added quality checks and discovery. Later the catalog features were presented as Dataplex Universal Catalog. According to Google’s documentation, on April 10, 2026 Dataplex Universal Catalog was renamed Knowledge Catalog. Google states that the API, client library, command line tool, and permission role names did not change, so the technical name Dataplex still appears in endpoints and roles.

Because many engineers, tutorials, and tools still say Dataplex, this guide uses Dataplex as the main name and mentions Knowledge Catalog where it matters. The ideas are the same under either name.

Find
discover and search data assets across systems
Trust
check quality and trace where data came from
Govern
control who sees what, with consistent rules
i
Good to Know

Dataplex does not usually move or copy your data. It builds an index and a layer of knowledge on top of data that stays where it is, such as BigQuery tables or Cloud Storage files.

2Core Concepts You Must Know First

A handful of data governance ideas explain nearly everything in Dataplex.

Governance vocabulary can sound heavy, but each idea is simple. Learn these eight terms and the rest of the guide will feel natural.

Term 1

Data Asset

Anything that holds or represents data: a table, a file, a topic, a dashboard, or a machine learning model.

Term 2

Metadata

Facts about an asset, such as its name, owner, schema, description, and last update time.

Term 3

Data Catalog

A searchable inventory of assets and their metadata, like a library catalog for data.

Term 4

Business Glossary

An agreed list of business terms, such as “active customer”, with clear definitions linked to the data that represents them.

Term 5

Data Quality

How accurate, complete, fresh, and consistent data is. Quality rules turn those ideas into automatic checks.

Term 6

Data Lineage

The recorded journey of data from its sources, through each transformation, to the places it is used.

Term 7

Data Governance

The rules and habits that decide who owns data, who may use it, and how it must be protected.

Term 8

Data Product

A curated bundle of data assets packaged for reuse, with an owner and clear meaning, like a product on a shelf.

Technical metadata and business metadata

There are two flavors of metadata. Technical metadata describes the machine side: column names, data types, sizes, and update times. Much of it can be collected automatically. Business metadata describes the human side: what a table means, who is responsible for it, how sensitive it is, and which business term it represents. People usually have to add it, though modern tools can help suggest it. A good catalog holds both, because analysts need meaning as much as structure.

Simple Analogy

A library record has technical details such as page count and size, and business details such as the topic, the author’s reputation, and which shelf it belongs to. A catalog with only page counts would be nearly useless for a researcher looking for the best book on a subject.

Governance is more than blocking access

Many people hear “governance” and picture a locked door. Good governance is closer to a well-run airport. It makes sure the right passengers reach the right gates quickly, with clear signs and safety checks that people trust. The aim is not to stop data use but to make safe, confident data use easy.

!
Common Misconception

Many teams believe a catalog is a one-time documentation project. A catalog is a living system. If metadata is not updated as data changes, people stop trusting it, and the tool quietly dies.

3The Pieces of the Platform

Dataplex bundles several capabilities that work together around one shared metadata layer.

It helps to see Dataplex as a toolbox rather than a single tool. Each capability solves one problem, and they reinforce each other because they share the same catalog. Google’s use-case documentation groups them under search and retrieval, enrichment and metadata management, aggregation, governance, data quality and profiling, and data lineage.

Capability 1

Catalog and Search

An index of data and AI assets across Google Cloud and other environments, searchable by keyword or natural language.

Capability 2

Discovery

Automatic scans that find data, such as files in Cloud Storage, and register what they contain.

Capability 3

Business Glossary

A taxonomy of business terms linked to columns and assets, so people find data by meaning.

Capability 4

Data Profiling

Statistics about columns, such as value ranges and null counts, that reveal anomalies and suggest quality rules.

Capability 5

Auto Data Quality

Scheduled scans that test tables against rules and produce quality scores and alerts.

Capability 6

Data Lineage

Automatic and imported records of how data flows between sources, jobs, and tables.

Capability 7

Data Products

Curated bundles of assets with owners and descriptions, published for others to find and reuse.

Capability 8

Governance Controls

Centralized policies, such as column-level security, applied consistently across the data estate.

flowchart TD
  SRC["Data sources: BigQuery, Cloud Storage, databases, streams"] --> CAT["Shared metadata catalog"]
  CAT --> SEARCH["Search and discovery"]
  CAT --> GLOS["Business glossary"]
  CAT --> DQ["Profiling and data quality"]
  CAT --> LIN["Lineage"]
  CAT --> GOV["Governance policies"]
  CAT --> DP["Data products"]
        
FIG 1 — One shared catalog feeding many capabilities

Read the diagram from the top. Information about your sources flows into one shared catalog, and every other capability reads from and writes to that catalog. This is why a quality score can appear on a table’s catalog page, why lineage can show glossary terms, and why a governance rule can follow a table wherever it appears.

Simple Analogy

A hospital has one patient record that doctors, nurses, labs, and pharmacists all read and update. Each department does a different job, yet they all trust the same record. The shared catalog is the patient record for your data.

How it relates to other Google Cloud services

Dataplex does not replace storage or processing tools. BigQuery still stores and queries tables. Cloud Storage still holds files. Dataproc and Dataflow still process data. Dataplex sits across them, describing and watching what they do. Google’s documentation lists metadata sources such as BigQuery, Vertex AI, Bigtable, Spanner, Pub/Sub, and Cloud Storage, along with connectors for third-party systems.

i
Start Small

You do not need every capability on day one. Many teams begin with search and automatic metadata, add a glossary for key terms, then introduce quality scans on their most important tables.

4How the Catalog Is Organized

The catalog uses entries, aspects, and groups, which work like index cards, fill-in forms, and drawers.

Entries

An entry is the catalog’s record for one data asset. Google’s documentation gives the example of a BigQuery table, which becomes an entry identified by its project, dataset, and table name. Columns appear on the entry when a schema aspect is attached to it. Every searchable thing in the catalog is an entry.

Aspects and aspect types

An aspect is a group of related metadata fields attached to an entry, for example an “ownership” aspect with fields for team and contact, or a “sensitivity” aspect with a classification level. An aspect type is the reusable template that defines which fields an aspect has. Aspects are stored inside the entry they describe rather than as separate objects. Google states that aspect types succeed the tag templates of the older Data Catalog service.

Simple Analogy

An entry is an index card for a book. An aspect type is a blank form such as “Condition report” with set boxes. An aspect is that form filled in and stapled to the card. Every card can carry several different filled forms.

Entry types and entry groups

An entry type is a template saying which aspect types an entry of that kind must carry, so every BigQuery table entry looks consistent. An entry group is a container for entries. According to Google’s documentation, the entry group is where you set access control, project attribution, and location for the entries inside it. Think of it as a labeled drawer with its own lock.

Entry links

Assets do not live alone, so the catalog also records relationships through entry links. The documented link types are synonym, related, definition, and schema-join. A synonym link says two terms mean the same thing. A definition link connects a glossary term to the asset or column it defines. A schema-join link says two tables can be joined on certain columns. Links can be symmetric or directional, and many can point to a specific column.

flowchart TD
  EG["Entry group: a drawer with its own access rules"] --> E1["Entry: sales orders table"]
  EG --> E2["Entry: customers table"]
  ET["Entry type: required aspect types"] -.-> E1
  E1 --> A1["Aspect: schema"]
  E1 --> A2["Aspect: ownership"]
  E1 --> A3["Aspect: sensitivity"]
  E1 -- "schema-join link" --> E2
  GT["Glossary term: Customer"] -- "definition link" --> E2
        
FIG 2 — Entries, aspects, groups, and links

The diagram shows one drawer holding two entries. The orders entry carries three filled forms and a join relationship to the customers entry. A business term links to the customers entry as its definition. Together, these pieces describe not only what exists but how things connect.

Building blockWhat it isEveryday comparison
EntryCatalog record for one assetIndex card
Aspect typeTemplate for a set of fieldsBlank form
AspectFilled-in set of fields on an entryCompleted form stapled to the card
Entry typeTemplate listing required aspect typesCard layout for a kind of item
Entry groupContainer with access, project, and location settingsLocked drawer
Entry linkRelationship between entriesString between two cards on a pinboard
!
Common Mistake

Creating aspect types casually, with overlapping fields and unclear names. Once many entries carry them, changing them is painful. Design a small, clear set of aspect types first and reuse them everywhere.

5How Metadata Gets Into the Catalog

A catalog is only as good as what flows into it, so Dataplex offers automatic, connector-based, and custom ways to load metadata.

Automatic metadata from Google Cloud

For many Google Cloud services, technical metadata arrives without you doing anything. Google’s documentation lists sources including BigQuery (with related items such as sharing listings and Dataform), Dataproc Metastore, Iceberg REST catalogs, Vertex AI, Bigtable, Spanner, and Pub/Sub, with Cloud Storage handled through discovery. AlloyDB and Cloud SQL can be added after enabling the integration, and Looker support is listed as a preview. Because this metadata is collected automatically, new tables appear in the catalog soon after they are created.

Discovery scans for files

Files in Cloud Storage have no built-in table definitions, so Dataplex can run discovery scans that look inside buckets, recognize structured data such as common file formats, work out the columns, and register the results as catalog entries. This turns a pile of files into something people can search and even query.

Third-party and custom sources

Few companies keep everything in Google Cloud. Google’s documentation mentions connectors for third-party systems such as dbt Core, Oracle, MySQL, PostgreSQL, and SQL Server, plus managed connectivity pipelines and custom ingestion through open APIs. A custom import lets you load metadata from an in-house system, so the catalog becomes a single view across the whole company rather than just one cloud.

flowchart LR
  A["Google Cloud services: BigQuery, Pub/Sub, Spanner, and more"] --> C["Knowledge Catalog index"]
  B["Cloud Storage discovery scans"] --> C
  D["Third-party connectors: databases, dbt, and others"] --> C
  E["Custom ingestion through APIs"] --> C
  C --> F["Search, glossary, quality, lineage, governance"]
  C -. "Change feed to Pub/Sub" .-> G["Downstream tools and automation"]
        
FIG 3 — Four routes for metadata to enter the catalog

Change feeds keep other tools in sync

Catalog information is useful beyond the catalog itself. Google’s documentation describes change feeds that send near real-time notifications of metadata changes to a Pub/Sub topic. A security tool could react when a table’s sensitivity label changes, or a data platform could refresh its documentation site whenever descriptions are edited.

Simple Analogy

Metadata ingestion is like a library accepting new books. Some arrive through the front door automatically from the main publisher. Some come in unlabeled boxes that a librarian must open and register. Some arrive from partner libraries. A well-run library welcomes all of them and records each in the same catalog format.

i
Automatic First, Human Second

Let machines collect technical metadata and put human effort into business meaning. People are slow at copying column names but irreplaceable when explaining what a column means to the business.

!
Common Mistake

Forgetting that date-sharded tables, such as one table per day, are indexed as separate tables. A catalog can fill up with thousands of near-identical entries. Prefer partitioned tables where possible, and plan for the entry count.

6Search and Discovery

Search is the front door of the catalog. If people cannot find data quickly, nothing else matters.

What search covers

Search looks across entries and their metadata, so a query can match a table name, a column name, a description, or a business term. Google’s documentation says search requires one of the catalog roles, such as Viewer, Editor, or Admin, and that results are limited to the same organization and the same VPC Service Controls perimeter. It also notes that the search API does not search across organizations. Creating and managing catalog resources, search calls, and console search are described as free of charge, while metadata storage is billed.

Keyword and natural language search

Traditional search matches words. Newer capabilities also let people ask questions in plain language, such as “Which tables hold monthly subscription revenue?”, and use AI to find relevant assets. Google’s documentation describes use cases where an agent answers natural-language queries about data assets through the API, and where a single request returns context about assets that a language model can use. These capabilities make the catalog useful not only to people but also to AI applications that need to know which data to trust.

Simple Analogy

Keyword search is asking a librarian for a book by its exact title. Natural language search is describing what you need and letting the librarian suggest the best three books, with a short note on why each fits.

Making search results trustworthy

Finding an asset is only half the job. A good result page tells the searcher what the asset is, who owns it, how fresh it is, whether it has passed quality checks, which business terms it matches, and where it came from. Without these signals, people find ten similar tables and have no idea which one to use. This is why the catalog connects search with quality scores, lineage, and glossary links.

Signal 1

Description

A plain sentence explaining what the asset contains and how it should be used.

Signal 2

Owner

A named person or team who answers questions and fixes problems.

Signal 3

Quality Score

The share of quality rules passed, so people see at a glance if the data is healthy.

Signal 4

Freshness

When the data was last updated, so nobody builds a report on a stale table.

Signal 5

Sensitivity

A label showing whether the data contains personal or restricted information.

Signal 6

Lineage

The upstream sources and downstream users of the asset, revealing how trustworthy and how important it is.

i
Measure Search Success

Ask a few analysts which data they could not find last month, and use their answers to improve descriptions and terms. The best search improvements come from real searches that failed.

!
Common Mistake

Assuming people can see everything in search. Results respect permissions and organizational boundaries. If someone says an asset is missing, check their role and the perimeter before suspecting the index.

7Business Glossary and Enrichment

A glossary gives the whole company one shared language, which is often the biggest source of confusion in data work.

Why a glossary matters

Ask three departments to define “active customer” and you may get three answers. Sales counts anyone who bought this year. Support counts anyone with an open account. Finance counts anyone who paid in the last ninety days. All three are reasonable, yet reports using different meanings under one name cause endless arguments. A business glossary records agreed definitions, so everyone knows which meaning a given number uses.

Google’s documentation describes a glossary as a taxonomy of business terms that you can link to columns and assets, and use to find assets by term. Terms can be grouped into categories, and links such as synonym and definition show how terms relate to each other and to the data.

Simple Analogy

A glossary is a shared dictionary for a team of travelers from different countries. Without it, the word “football” might mean two different sports, and the team plans the wrong evening. With it, everyone knows exactly what is meant.

Linking terms to data

A glossary becomes powerful when terms point to real columns. If the term “Customer ID” links to the matching columns in five tables, a person searching for the term finds all five at once, and a data steward can see where a definition is actually in use. If a definition changes, the people responsible can find every affected asset.

Enrichment: adding meaning at scale

Adding descriptions to thousands of tables by hand is unrealistic. Google’s use-case page describes enrichment, including an example agent that generates AI-powered overviews of data assets at scale. Generated descriptions give people a useful starting point, which human owners then review and correct. Treat generated text as a draft, not a final fact, particularly for sensitive or business-critical assets.

1

Pick the critical terms

Start with the dozen terms that cause the most arguments, such as customer, revenue, and active user.

2

Agree on definitions

Bring owners from each department together and record one clear definition, with examples of what is included and excluded.

3

Link terms to columns

Connect each term to the actual columns and assets that represent it.

4

Name stewards

Give each term an owner who approves changes and answers questions.

5

Review on a schedule

Revisit definitions regularly, because the business changes and glossaries must change with it.

i
Keep It Small and Useful

A glossary of fifty well-linked, well-owned terms beats a glossary of five thousand terms that nobody reads. Grow it in response to real confusion.

!
Common Mistake

Writing glossary definitions in vague language, such as “a customer is someone who uses our service”. Good definitions are precise enough that two people applying them to the same record reach the same answer.

8Automatic Data Quality

Quality scans act like inspectors who test your tables against written rules, on a schedule, and report what they find.

What data quality means

Good data is accurate, complete, fresh, consistent, and unique. Bad data looks normal until someone makes a decision with it. A missing country code, a negative price, a duplicated order, or a table that stopped updating three days ago can quietly spoil every report built on top. Automatic data quality lets you turn those worries into checks that run without anyone remembering to look.

How a scan works

Google’s documentation explains that an auto data quality scan targets one table and tests it against rules you define. Scans run on a schedule or on demand, produce quality scores, and log alerts when records fail. Supported table types include BigQuery tables, Iceberg REST catalog tables, SAP Business Data Cloud Delta Lake tables, and Apache Hive tables. The scan runs in a Google-managed tenant project, so you do not provision or maintain any machines for it.

Simple Analogy

A quality scan is a food inspector who visits a factory every night with a checklist. Is every box labeled? Are the weights within range? Are any items expired? The inspector writes a score on a certificate and raises an alarm if the failures pile up.

The kinds of rules

Rule kindWhat it checksExample
Non-nullA column has no empty valuesEvery order has a customer ID
RangeValues fall between a minimum and a maximumPrices are never negative
SetValues belong to an allowed listStatus is one of a few known codes
Regular expressionText matches a patternEmail addresses look like email addresses
UniquenessNo duplicate values in a columnOrder numbers are not repeated
Statistic rangeAn average, minimum, or maximum stays in rangeAverage basket value stays reasonable
Custom SQLAny condition you can express in SQLShipping date is never before order date

Google describes row-level rules, which test each row against a passing threshold percentage, and aggregate rules, which test one value across the whole table. Custom SQL rules can be a row condition, a table condition, or an assertion that returns the rows that break a rule. Every rule is assigned to a quality dimension, with the predefined set including freshness, volume, completeness, validity, consistency, accuracy, and uniqueness. A scan can hold up to one thousand rules.

flowchart LR
  T["BigQuery table"] --> S["Quality scan with rules"]
  S --> R["Scores by job, dimension, column"]
  R --> BQ["Results exported to BigQuery"]
  R --> CAT["Quality scorecard on catalog page"]
  R --> AL["Logs and email alerts when score drops"]
  BQ --> L["Looker dashboards"]
        
FIG 4 — From a scan to scores, dashboards, and alerts

Controlling cost and scope

Scanning huge tables repeatedly can be expensive, so scans offer controls. You can sample a percentage of records, apply row filters to limit what is scanned, and run incremental jobs that look only at new data using a date or timestamp column as a marker. Running jobs can be canceled. Google also notes that Private Google Access must be enabled on the subnet used by the scan.

i
Start With a Few Meaningful Rules

Ten rules on your most important table beat five hundred rules nobody reads. Begin with checks tied to real past incidents, such as a feed that once arrived empty.

!
Common Mistake

Running scans and ignoring the results. A score nobody reviews is decoration. Decide who receives each alert, what they must do when it fires, and how quickly.

9Profiling, Scores, and Acting on Results

Profiling shows what your data really looks like, and results turn quality from a feeling into numbers people can act on.

Profiling first, rules second

Before writing rules you need to know what is normal. Data profiling computes statistics for each column of a table, such as the smallest and largest values, the share of empty values, the number of distinct values, and common patterns. Google’s use-case page describes profiling a BigQuery table for column statistics and anomalies, then configuring and running a quality scan. The documentation also lists profiling-based recommendations as one source of rules, so the profile can suggest sensible starting checks.

Simple Analogy

Profiling is a doctor’s check-up that records your normal pulse, weight, and blood pressure. Quality rules are the alarms set against those normal values. You cannot set a good alarm without first knowing the normal reading.

Reading scores at different levels

Scores show the percentage of rules passed. According to Google’s documentation, they can be viewed at the level of a whole job, of each quality dimension, and of each column, although column-level scores are available through the API. A table might score ninety-eight percent overall yet only seventy percent on freshness, which tells you the contents are fine but the pipeline is late. Looking at dimensions points you to the kind of problem rather than just its existence.

Finding the bad rows

A failing score is only the start. People then need to know which records broke the rule so they can repair the source. Google describes a query that retrieves failed rows, returning all columns of the table. Teams commonly route failed rows to a repair queue or quarantine table, so that downstream users see only trusted records while owners fix the rest.

1

Profile the table

Look at ranges, null rates, and distinct counts to learn what normal looks like.

2

Draft rules from the profile

Turn clear patterns into rules, and have the data owner confirm each one makes business sense.

3

Schedule the scan

Run it after the pipeline that updates the table, not at a random clock time.

4

Route alerts to owners

Send email or log-based alerts when scores fall below targets or jobs fail.

5

Fix the cause, then tighten rules

Repair the upstream problem, and add a rule so the same failure is caught earlier next time.

Publishing scores where people look

Google’s documentation says results can be published as catalog metadata, shown on the table’s data quality tab, and exported to BigQuery for dashboards in Looker. Publishing scores on the catalog page lets an analyst judge trustworthiness at the very moment of choosing a table.

i
Reusable Rules

Google describes rule templates for reuse. If twenty tables need the same non-empty email check, define it once and apply it everywhere, so a change to the rule updates all of them.

!
Common Mistake

Setting every threshold to one hundred percent. Real data has small, acceptable flaws. Overly strict rules fire constantly, and people learn to ignore them. Pick thresholds that separate real trouble from normal noise.

10Data Lineage

Lineage is the family tree of your data, showing where each table came from and what depends on it.

What lineage records

When a job reads a table, transforms it, and writes another table, lineage records the link. Over time these links form a graph that shows, for any asset, which sources fed it and which reports, models, or tables rely on it. Google describes lineage as automatically tracking data flows, including how sensitive information moves through pipelines.

Simple Analogy

Lineage is a food label that also traces ingredients back to farms and forward to every store that sold the product. If a farm reports contamination, the company can instantly find every affected package.

How lineage is captured

Google’s documentation says lineage from BigQuery, Managed Airflow (Cloud Composer), Cloud Data Fusion, and Managed Service for Apache Spark (Dataproc) appears in the same interface. For other tools, an open standard called OpenLineage lets pipelines send events describing which datasets a job read and wrote. Dataplex can import these events, and some services, such as Spark and Airflow, have built-in producers that send events automatically. The documentation also mentions client libraries and an OpenLineage transport for tools such as Airflow, Spark, and Flink.

sequenceDiagram
  participant J as Pipeline job
  participant P as OpenLineage producer
  participant L as Lineage API
  participant U as Lineage graph in console
  J->>P: Job reads table A and writes table B
  P->>L: Send run event with inputs and outputs
  L->>L: Store process, run, and lineage event
  U->>L: Analyst opens table B
  L-->>U: Show table A upstream and reports downstream
        
FIG 5 — How a pipeline’s activity becomes a lineage graph

What lineage is good for

Impact Analysis

Before changing or deleting a column, see every downstream table and report that depends on it. This turns risky guesses into informed decisions and prevents surprise breakages.

Root Cause Analysis

When a dashboard shows a wrong number, trace upstream to find the table or job where the error entered, instead of searching blindly.

Privacy and Compliance

Google’s use-case page mentions tracing the origin of sensitive data leaks from trusted to untrusted locations. Lineage helps show regulators where personal data travels.

Cost Cleanup

Assets with no downstream users may be unused. Lineage helps find them, which can reduce storage costs after a careful review.

Know the limits

The lineage graph shows inputs and outputs for each run, but not every low-level step, such as individual Spark stages. Google documents limits on imported events, including a maximum message size of five megabytes and caps on the number of links per event. The imported API supports the first major version of the OpenLineage standard. Check current limits when you design large pipelines.

i
Check Lineage Before Every Big Change

Make “open the lineage graph” a standard step in your change checklist. Five minutes of looking can prevent a long night of repairing broken reports.

!
Common Mistake

Assuming lineage is complete. Jobs that copy data through tools that do not report events leave gaps. Treat the graph as strong evidence, not as proof, and keep important dependencies documented.

11Data Products and Domains

A data product packages related assets with an owner and a purpose so other teams can adopt them with confidence.

From loose tables to products

A shared warehouse often contains hundreds of tables, and nobody is sure which are meant for general use. A data product changes that mindset. Instead of exposing raw tables, a team publishes a curated bundle, such as “Customer 360”, that includes the right tables, clear descriptions, an owner, quality expectations, and information about how to request access. Consumers search for the product, not for a half-remembered table name.

Simple Analogy

Raw tables are loose ingredients in a warehouse. A data product is a boxed meal kit with a recipe card, a freshness date, and the name of the chef to call if something is wrong. Customers choose the kit, not individual ingredients.

Domains give ownership a home

Large companies group data by business area, called domains, such as sales, marketing, supply chain, and finance. Each domain’s team knows its data best and is best placed to publish and maintain products. Google’s documentation lists data products and data domains within the service, and third-party summaries describe data products as generally available in 2026. Because this area is evolving, check the current status and limits in the documentation before building a large program on it.

Element 1

Clear Purpose

A one-line statement of the question the product answers.

Element 2

Named Owner

A team accountable for quality, changes, and support.

Element 3

Quality Promise

Published expectations, backed by quality scans, about freshness and accuracy.

Element 4

Access Path

A simple way to request and receive permission, with rules the owner controls.

i
Product Thinking

Ask consumers what they need before building. A data product that nobody asked for is just another table with a nicer name.

!
Common Mistake

Declaring everything a data product. If every table is a product, the word means nothing. Publish a small number of well-maintained products and retire those that are no longer supported.

12Security and Governance

Governance means the right people see the right data, with rules that are written down once and applied everywhere.

Layers of control

Dataplex works alongside Google Cloud’s normal security tools rather than replacing them. Access to catalog resources is controlled through Identity and Access Management, and Google’s documentation notes that roles such as catalog admin, editor, and viewer decide who can search and manage metadata. Entry groups carry their own access control, so a sensitive drawer of entries can be restricted even when other groups are open.

Layer 1

IAM Roles

Decide who can view, edit, or administer catalog resources. Give the smallest role that gets the job done.

Layer 2

Entry Group Permissions

Restrict groups of entries separately, so sensitive metadata stays in a locked drawer.

Layer 3

VPC Service Controls

A boundary around services and data. Search results stay within the same organization and perimeter.

Layer 4

Policy Tags

Labels on columns that drive column-level security, so only approved people can read sensitive columns.

Layer 5

Centralized Policies

Rules applied once and enforced across the lakehouse, as described in Google’s governance use case for Iceberg tables.

Layer 6

Audit Logs

Records of who changed metadata or ran scans, supporting investigations and compliance reviews.

Simple Analogy

Think of a company building. IAM roles are the badges. Entry group permissions are locks on individual rooms. The service perimeter is the fence around the property. Policy tags are the extra locked cabinets inside the rooms holding the most sensitive papers.

Classify first, then protect

You cannot protect sensitive data you have not identified. A common approach is to tag columns that hold personal information, financial details, or health data, and then attach security rules to those tags. When a new table arrives with a matching column, the same protection can follow. Lineage then adds a safety net by showing where tagged data flows, so you can spot copies that escaped the rules.

Separate who manages from who reads

Managing metadata and reading data are different powers. A person who edits glossary terms does not automatically need permission to read the underlying tables. Likewise, an analyst who reads tables should not be able to rewrite catalog policies. Separating duties limits the damage from mistakes and compromised accounts.

i
Policy as Code

Keep governance settings, such as aspect types, glossary structure, and scan definitions, in version control and deploy them through reviewed automation. Changes become visible and reversible instead of clicks nobody remembers.

!
Common Mistake

Granting broad catalog administrator roles to many people “to avoid access tickets”. Administrators can change shared structure that everybody relies on. Keep that role to a small, accountable group.

13Lakes, Zones, and Lakehouse Foundations

Dataplex began with lakes and zones, ideas that still help explain how to organize data stored in files.

The original model

When Dataplex was introduced, it organized data into three layers. A lake was a logical home for related data, such as everything belonging to a business area. A zone divided a lake by readiness, commonly a raw zone for data exactly as it arrived and a curated zone for cleaned, trusted data. An asset was a Cloud Storage bucket or a BigQuery dataset attached to a zone. This structure let teams apply rules, discovery, and security consistently to data spread across many buckets and datasets.

Simple Analogy

A lake is a neighborhood, zones are districts such as the loading docks and the showroom, and assets are the individual buildings. Rules for the loading docks, where goods arrive messy, differ from rules for the showroom, where everything must be spotless.

Where this fits today

Google’s current catalog documentation focuses on entries, aspects, glossaries, lineage, quality, and data products, and it notes that lakes, zones, assets, and entities cannot be registered as catalog entries. That means the older lake model and the newer catalog model are distinct. If your organization has existing lakes and zones, check the current documentation to see which features still apply to them and how Google recommends transitioning. For new work, designing around the catalog, data products, and standard Google Cloud resources is the safer direction.

The raw, refined, curated idea lives on

Even when the specific lake and zone objects are not used, the layered thinking remains valuable. Land data raw and untouched, so you can always replay it. Refine it by cleaning and standardizing. Curate it into trusted, well-described products. Each layer has different owners, different quality expectations, and different access rules, and the catalog can record those differences through aspects and entry groups.

flowchart LR
  R["Raw layer: data exactly as received"] --> F["Refined layer: cleaned and standardized"]
  F --> C["Curated layer: trusted products"]
  C --> U["Analysts, dashboards, ML models"]
  Q["Quality scans"] -.-> F
  Q -.-> C
        
FIG 6 — Layered data organization with quality checks between layers

Open table formats and the lakehouse

Modern lakehouse designs store data as files in open table formats such as Apache Iceberg, which add reliable tables on top of storage. Google’s documentation mentions Iceberg REST catalogs as a metadata source and as a supported target for quality scans, and describes a governance use case of creating Iceberg tables with centralized column-level security and visualizing automated lineage. The catalog is the shared source of truth that lets many engines understand the same tables.

i
Keep Layers Visible

Name datasets and buckets so the layer is obvious, and record the layer as an aspect on each entry. Analysts should never wonder whether they are looking at raw or trusted data.

!
Common Mistake

Letting analysts query the raw layer directly because it is convenient. Raw data contains duplicates, errors, and sensitive fields that later layers are meant to remove or protect.

14Operating, Monitoring, and Troubleshooting

A governance platform needs its own care, with alerts, ownership, and a routine for fixing the problems it reports.

What to watch

Dataplex generates operational signals you can observe like any other service. Google’s documentation says quality scans write entries to Cloud Logging that can drive alerts, and that email notifications can be sent when a score falls below a target, a job fails, or a job finishes. It also notes that scan costs can be tracked through billing export using labels, and that scans do not emit time-series metrics to Cloud Monitoring, so plan to use logs and billing data rather than expecting standard metric charts.

SignalWhere to see itWhat to do
Quality score dropScan results, email, logsOpen failed rows, find the upstream cause
Failed scan jobLogs and notificationsCheck permissions, network settings, and table access
Metadata gapsCatalog coverage reports you buildAssign owners and request descriptions
Lineage gapsMissing upstream or downstream linksEnable event producers on the tools that lack them
Metadata changesChange feed on a Pub/Sub topicTrigger reviews or sync downstream tools
SpendBilling export labelsReview scan frequency, sampling, and scope
Simple Analogy

An inspection service also needs inspecting. If the inspectors stop turning up or their reports pile up unread, the certificates on the products become meaningless. Monitoring the governance platform keeps its stamps trustworthy.

Troubleshooting routine

1

Is the asset in the catalog?

If not, check the ingestion route, the integration setting, and whether the asset type is supported.

2

Can the person see it?

Check their role, the entry group permissions, and the service perimeter before blaming the index.

3

Did the scan run?

Check the service account, table permissions, and network settings such as Private Google Access on the subnet.

4

Is the lineage missing?

Confirm the tool sends events, the message size is within limits, and the event format is supported.

5

Is the cost surprising?

Look at scan frequency, full-table versus incremental runs, and whether sampling is enabled.

Ownership is an operational duty

Every glossary term, aspect type, scan, and data product needs a named owner. When an owner leaves, the item should pass to someone else rather than rot. Many governance programs fail not because the tools are weak, but because nobody feels responsible. Regular review meetings, where owners confirm their items are current, keep the system alive.

i
Coverage Metric

Track the percentage of important tables that have an owner, a description, and at least one quality scan. A simple rising number motivates teams more than any policy document.

!
Common Mistake

Waiting for a perfect catalog before announcing it. Launch with the important tables, show quick wins, and expand. People adopt tools that already help them, not tools promised for the future.

15Advantages, Trade-offs, and Cost Thinking

Dataplex brings real strengths, but like any platform it has limits and a bill worth understanding.

Advantages

  • Automatic technical metadata from many Google Cloud services, with no scripts to maintain.
  • One shared metadata layer that connects search, glossary, quality, lineage, and governance.
  • Serverless scans, so there are no machines to size, patch, or monitor.
  • Reach beyond Google Cloud through connectors and open APIs for third-party and custom sources.
  • Open standards such as OpenLineage and Iceberg, which reduce lock-in.
  • Useful context for AI applications that need to find and trust data.

Disadvantages / Trade-offs

  • Tools do not create governance. People must define owners, terms, and rules.
  • Names, features, and launch stages have changed quickly, so documentation and tutorials can disagree.
  • Some automated extraction and features may be limited to built-in Google Cloud services.
  • Documented limits apply, such as maximum rules per scan and event sizes for lineage.
  • Scans and processing add cost, especially on very large tables scanned often.
  • Migration from the older catalog service takes planning and has gaps to check.

How the bill is built

Google’s documentation states that storing metadata is billed, while creating and managing catalog resources, search calls, and console search are free. Processing-heavy features, such as quality scans, profiling, discovery, and lineage, consume processing capacity measured in Data Compute Units. A third-party summary of Google’s pricing, published in September 2026, reports different rates for standard processing such as discovery and for premium processing such as quality, profiling, and lineage, a metadata storage rate, and a small monthly free allowance. It also reports a new charge for data insights features starting in late October 2026. Because these figures come from a secondary source and prices change, always confirm current rates on Google’s pricing page and test with your own data volumes.

Simple Analogy

Using the catalog is like visiting a public library: browsing and asking questions are free. Storing a large private collection on the library’s shelves has a rent, and hiring inspectors to test every book has a labor charge. The more often and more thoroughly you inspect, the higher the labor bill.

Practical ways to keep cost sensible

Tip 1

Scan What Matters

Put quality scans on critical tables first, not on every table in the warehouse.

Tip 2

Sample Large Tables

Scanning a percentage of records often gives the same signal at a fraction of the cost.

Tip 3

Use Incremental Runs

Check only new data using a date or timestamp column, instead of re-reading history every night.

Tip 4

Match Frequency to Change

A table that updates weekly does not need a daily scan. Align scans with the pipeline schedule.

Tip 5

Label and Track Spend

Use billing export labels to see which scans cost the most, then review whether each earns its keep.

Tip 6

Mind Entry Counts

Date-sharded tables and thousands of temporary tables multiply entries. Clean up or restructure them.

Dataplex compared with other approaches

ApproachStrengthLimitationFits when
DataplexNative Google Cloud integration, automatic metadata, serverless scansEvolving features and namingYour data is mostly on Google Cloud
Third-party catalogRich workflows and wide connector librariesExtra product to buy, integrate, and runData is spread across many clouds and tools
Spreadsheet or wiki documentationSimple and freeGoes out of date quickly, no automationVery small teams and few datasets
Custom-built metadata storeExactly fits your needsHigh build and maintenance effortUnusual requirements nothing else meets
i
Decision Helper

If most of your important data already lives in BigQuery and Cloud Storage, Dataplex is usually the natural starting point. If your data estate is deeply multi-cloud, compare it with dedicated catalog tools and consider combining them through open APIs.

16Design Patterns and Popular Use Cases

A handful of tested patterns cover most governance needs in the real world.

Enterprise Data Discovery Portal

All analysts search one catalog for tables, files, and dashboards. Each result shows description, owner, freshness, sensitivity, and quality score. Time spent hunting for data drops, and duplicate tables become easier to spot and retire.

Quality Gates in Pipelines

A pipeline writes to a staging table, a quality scan checks it, and only passing data is published to the curated layer. Failed data goes to a quarantine area for repair. Consumers see only trusted data.

Regulatory and Privacy Reporting

Columns holding personal data are tagged, column-level security is applied, and lineage shows where tagged data flows. When a regulator or customer asks where personal data goes, the answer comes from the graph instead of weeks of manual research.

Safe Change Management

Before altering a table, engineers open the lineage graph to list every downstream consumer, notify the owners, and schedule the change. Surprise breakages become rare.

Data Mesh by Domain

Each business domain owns, documents, and publishes its own data products, while a small central team provides shared standards, aspect types, and platform tools. Ownership sits close to knowledge.

Context for AI Applications

Chat assistants and agents query the catalog to find the right tables, learn what columns mean, and check quality before generating answers. Google’s use-case page describes discovery and enrichment agents built on the catalog API, with the catalog acting as shared context for data assets.

Lakehouse Governance

Iceberg tables on Cloud Storage are governed with centralized column-level policies, automatic lineage, and quality scans, so a single set of rules protects the data whichever engine reads it.

flowchart TD
  P["Pipeline writes to staging table"] --> Q{"Quality scan passes?"}
  Q -- "Yes" --> C["Publish to curated layer"]
  Q -- "No" --> X["Quarantine failed rows and alert owner"]
  C --> CAT["Catalog shows fresh score and lineage"]
  X --> FIX["Owner fixes source, rerun pipeline"]
  FIX --> P
        
FIG 7 — A quality gate pattern protecting the curated layer

The diagram shows the pattern’s heart: bad data never reaches consumers, owners are alerted, and the cycle repeats until the problem is fixed. This one pattern prevents a large share of the “who changed my numbers?” incidents.

“A catalog tells you what data exists, but trust is earned only when quality, lineage, and ownership sit on the same page.”

17Anti-Patterns to Avoid

Most governance programs stumble over a short list of repeated mistakes.

ANTI-PATTERN-01 Avoid
Problem

The company buys or enables a catalog, loads it once, and assumes the governance problem is solved.

Why It’s Harmful

Metadata decays as data changes. Within months descriptions are stale, owners have left, and people return to asking colleagues. The catalog becomes shelfware.

Correct Approach

Automate technical metadata, assign owners for business metadata, and review coverage regularly as a normal operating routine.

ANTI-PATTERN-02 Avoid
Problem

Hundreds of quality rules are created in bulk, with every threshold set to one hundred percent.

Why It’s Harmful

Alerts fire constantly for trivial flaws. Owners stop reading them, and real problems hide in the noise.

Correct Approach

Start with a few rules tied to real business risk, choose realistic thresholds, and add rules after real incidents.

ANTI-PATTERN-03 Avoid
Problem

Every table, including scratch and temporary ones, is scanned fully every day.

Why It’s Harmful

Processing charges grow quickly, with little benefit, because most tables are not critical and most do not change daily.

Correct Approach

Scan critical tables, use sampling and incremental runs, and align schedules with pipeline timing.

ANTI-PATTERN-04 Avoid
Problem

A central team tries to describe and own all data for the whole company by itself.

Why It’s Harmful

The central team lacks business knowledge, becomes a bottleneck, and produces generic descriptions that help nobody.

Correct Approach

Let domain teams own and describe their data, while the central team sets standards, templates, and tools.

ANTI-PATTERN-05 Avoid
Problem

Governance is treated as a way to block access, with long approval queues for every dataset.

Why It’s Harmful

Teams create shadow copies and unofficial extracts to get around delays, which spreads ungoverned data far more widely than the original.

Correct Approach

Make approved access fast and easy through data products, clear owners, and policy tags, while blocking only what must be blocked.

ANTI-PATTERN-06 Avoid
Problem

Teams trust AI-generated descriptions and lineage as complete facts without review.

Why It’s Harmful

Generated text can be wrong or vague, and lineage may miss tools that send no events. Decisions built on gaps can mislead people.

Correct Approach

Treat generated content as drafts for owners to review, and document critical dependencies independently of automatic graphs.

18Best Practices, Migration, and Common Mistakes

A short checklist of habits separates thriving governance programs from abandoned ones.

Habits that pay off

1

Start with a business problem

Pick a painful, visible problem, such as conflicting revenue numbers, and show it solved before expanding.

2

Name owners everywhere

Every table, term, scan, and product has an accountable owner with a backup.

3

Design a small set of aspect types

Define clear, reusable templates for ownership, sensitivity, layer, and domain, and use them consistently.

4

Automate what machines do well

Use automatic metadata, discovery, profiling, and lineage, and reserve human time for meaning and decisions.

5

Put checks inside pipelines

Gate publication on quality results rather than hoping someone looks at a dashboard.

6

Manage configuration as code

Keep glossary structure, scans, and policies under version control with review.

7

Measure adoption

Track searches, coverage of owners and descriptions, and the share of key tables with quality scans.

Moving from the older Data Catalog

Many organizations still have tags and templates from the older Data Catalog service. Google publishes a transition guide, and notes that aspect types succeed tag templates. A third-party summary from September 2026 reports that the legacy service began shutting down in June 2026, and lists migration gaps, such as private tag templates having no direct counterpart, regional tag templates becoming global aspect types so name collisions matter, and limits in managing certain custom items through infrastructure tooling. Because the transition details can change and sources describe the timeline differently, read Google’s current guide before planning a migration, inventory your existing tags, and test with a small subset first.

1

Inventory what you have

List existing tag templates, tags, and the teams that depend on them.

2

Map to the new model

Decide which templates become aspect types, how to avoid name clashes, and which entry groups will hold the results.

3

Pilot with one domain

Migrate one team’s metadata, check search and permissions, and collect feedback.

4

Migrate in waves and retire the old

Move remaining teams in planned waves, confirm access rules carried over, then retire old tags and processes.

Mistakes that appear again and again

Mistake 1

No named owner

Items without owners decay first, and nobody notices until an audit.

Mistake 2

Ignoring permissions in testing

Everything works for the admin who built it, then analysts cannot see the data they need.

Mistake 3

Skipping the glossary

Technical metadata alone cannot settle arguments about what a number means.

Mistake 4

Scans with no follow-up

Scores drift down while nobody investigates, and trust in the program drops.

Mistake 5

Relying on preview features

Preview capabilities can change. Check launch status before depending on them for critical processes.

Mistake 6

Forgetting the people side

Training, communication, and visible wins matter as much as configuration.

i
Pre-Launch Checklist

Before announcing the catalog, confirm key tables have owners and descriptions, search roles work for ordinary analysts, quality alerts reach a real person, and a quick-start guide explains how to find and request data.

19Real-World Scenarios

Walking through realistic situations shows how the ideas fit together.

The Two Revenue Numbers

Finance and marketing present different revenue totals at a leadership meeting. Using the catalog, an analyst finds two similar tables, sees from lineage that one includes refunds and the other does not, and links both to a new glossary term that defines “net revenue”. The definition is agreed, the old table is marked deprecated, and the argument ends.

The Empty Feed on Monday Morning

A partner’s file arrives empty, and the weekly sales dashboard shows zero. A quality scan with a volume rule on the staging table fails, an email alert reaches the data owner at six, and the pipeline holds back publication. The owner requests a resend from the partner, and executives never see a wrong dashboard.

The Privacy Request

A customer asks a company to delete their personal data. The privacy team searches for the sensitivity tag, finds forty tables, and uses lineage to discover three downstream copies in an analytics project. All are cleaned within the deadline, with an audit trail showing how they were found.

The Column Nobody Dared to Delete

An engineer wants to remove an old column. The lineage graph shows seven downstream tables and two dashboards using it. The engineer messages their owners, schedules a coordinated change, and avoids a silent outage.

The New Analyst’s First Week

A new analyst needs customer churn data. Instead of asking ten colleagues, she searches the catalog, finds a data product with a clear description, a named owner, a recent quality score, and a request button. She starts working in hours instead of weeks.

A Ninety-Day Rollout Plan for a New Team

Teams often ask how to turn all these ideas into a calendar. A realistic first quarter has three phases, each with a small, visible result that builds confidence for the next one.

During the first month, focus on visibility. Enable the catalog in the projects that hold your most important warehouse data, and confirm that automatic metadata appears for tables and files. Choose one business domain, such as sales, and ask its owners to review descriptions for the ten tables people use most. Create a short list of glossary terms for the metrics that cause the most arguments. Teach analysts how to search, and collect their first complaints about missing or confusing results. The goal of this phase is simple: people can find the important data and see who owns it.

During the second month, focus on trust. Run profiling on the critical tables to learn what normal looks like, then add a few quality rules for completeness, uniqueness, and valid ranges. Send alerts to a real mailbox or channel that somebody watches, and agree on who responds and how fast. Connect scan results to the catalog so consumers see scores beside the table description. Review lineage for the most important reports and note any gaps where a tool sends no events. The goal of this phase is that people trust the data more, and problems are noticed before executives notice them.

During the third month, focus on scale and protection. Add sensitivity tags and column-level access rules for personal data, document the approval path for requesting access, and publish the first data products for the domain that went first. Measure adoption through search activity, the share of key tables with owners and descriptions, and the number of quality incidents caught early. Present these numbers to leadership, then use the results to bring the next domain on board. The goal of this phase is a repeatable pattern that other teams can copy without starting from scratch.

!
Keep The Scope Small

Resist the urge to onboard every team in the first quarter. A small program that visibly works earns trust and budget. A giant program that touches everything but fixes nothing earns skepticism and is hard to rescue.

Roles That Make The Program Work

Governance succeeds when responsibilities are clear. A data owner is accountable for a dataset, approves access, and answers questions about meaning. A data steward looks after daily quality, descriptions, and glossary terms, acting as the practical caretaker. A platform engineer maintains the technical setup, such as permissions, scans, and ingestion. A consumer, such as an analyst or scientist, uses the data and reports problems back. When a catalog entry names all four roles, nobody has to guess whom to contact, and problems travel to the right person quickly instead of circulating in chat threads.

Findable
data that people locate in minutes
Trusted
data with scores, owners, and lineage
Protected
data with rules that follow it everywhere
Simple Analogy

Each story is the same well-run library helping a different visitor. One needs the right edition of a book, one needs to know a shipment was damaged, one needs every copy of a recalled title removed. A good catalog, inspector, and security desk solve all three.

20Frequently Asked Questions

Quick answers to the questions that come up most often.

Q1Is Dataplex a database or storage service?

No. It does not store your business data. It catalogs, describes, checks, and governs data that lives in services such as BigQuery and Cloud Storage.

Q2I see the name Knowledge Catalog. Is that the same thing?

Yes. Google’s documentation says Dataplex Universal Catalog was renamed Knowledge Catalog on April 10, 2026, and that API, library, command line, and role names did not change, which is why Dataplex still appears in technical names.

Q3What is the difference between metadata and data?

Data is the content itself, such as customer orders. Metadata is information about that content, such as the table name, owner, column meanings, and quality score.

Q4What are aspects and aspect types?

An aspect type is a reusable template of metadata fields. An aspect is that template filled in and attached to an entry. Aspect types succeed the tag templates of the older Data Catalog.

Q5Is searching the catalog free?

Google’s documentation says search calls and console search are free, and that creating and managing catalog resources is free, while metadata storage is billed. Processing features such as scans consume billable processing capacity.

Q6Which tables can quality scans check?

According to Google’s documentation, supported table types include BigQuery tables, Iceberg REST catalog tables, SAP Business Data Cloud Delta Lake tables, and Apache Hive tables. Each scan targets one table.

Q7How many rules can one scan have?

Google documents a limit of one thousand rules per scan. Start with a few meaningful rules rather than approaching the limit.

Q8How does lineage get captured?

Automatically for several Google Cloud services such as BigQuery, Managed Airflow, Data Fusion, and Dataproc, and through OpenLineage events imported from other tools that produce them.

Q9Can it catalog data outside Google Cloud?

Yes, through connectors for several third-party systems, managed connectivity pipelines, and custom ingestion through open APIs, though some automated features may be limited to built-in Google Cloud services.

Q10Who can search the catalog?

People with a catalog role such as viewer, editor, or admin. Results are limited to the same organization and service perimeter, and they respect permissions on entry groups.

Q11Do lakes and zones still matter?

They were central to the original Dataplex. Google’s catalog documentation says lakes, zones, assets, and entities cannot be registered as catalog entries, so check current guidance if you use them. The raw, refined, and curated layering idea remains useful either way.

Q12Where should a team begin?

Pick one important domain, let automatic metadata fill the catalog, add owners and descriptions to the key tables, define a handful of glossary terms, and add quality scans to the two or three most critical tables.

Q13Can the catalog help AI applications?

Yes. Google’s use-case page describes agents that search the catalog with natural language and retrieve context about data assets, so AI applications can find and understand the right data.

21Summary and Key Takeaways

Dataplex is the librarian, inspector, and security desk for your data. It builds a searchable catalog on top of data that stays where it is, adds meaning through glossaries and aspects, checks health through profiling and quality scans, records history through lineage, and applies protection through consistent policies. The platform and its names have evolved, but the goal is stable: make data easy to find, safe to trust, and safe to share. Tools help, but results come from habits: named owners, small clear templates, meaningful checks, and routines that keep metadata alive.

Key Takeaways

  • Dataplex governs data it does not store — it catalogs and checks data in BigQuery, Cloud Storage, and other systems, and Google now presents the catalog as Knowledge Catalog while technical names stay the same.
  • Learn the catalog building blocks — entries, aspect types, aspects, entry types, entry groups, and entry links describe what exists and how things connect.
  • Automate technical metadata, own business metadata — machines collect schemas and updates, while people supply meaning through owners, glossary terms, and descriptions.
  • Profile first, then write quality rules — scans with sensible thresholds, sampling, and alerts turn data quality into measurable, actionable scores.
  • Use lineage for confidence and safety — impact analysis, root cause tracing, and privacy investigations all depend on knowing how data flows.
  • Protect data with layered, centralized controls — IAM roles, entry group permissions, service perimeters, and policy tags keep access consistent and auditable.
  • Treat governance as a living practice — assign owners, review coverage, watch cost, check launch status of newer features, and plan carefully when migrating from older catalog tools.